% IMPORTANT: The following is UTF-8 encoded.  This means that in the presence
% of non-ASCII characters, it will not work with BibTeX 0.99 or older.
% Instead, you should use an up-to-date BibTeX implementation like “bibtex8” or
% “biber”.

@ARTICLE{Lejaeghere:807084,
      author       = {Lejaeghere, K. and Bihlmayer, G. and Bjorkman, T. and
                      Blaha, P. and Blugel, S. and Blum, V. and Caliste, D. and
                      Castelli, I. E. and Clark, S. J. and Dal Corso, A. and de
                      Gironcoli, S. and Deutsch, T. and Dewhurst, J. K. and Di
                      Marco, I. and Draxl, C. and Du ak, M. and Eriksson, O. and
                      Flores-Livas, J. A. and Garrity, K. F. and Genovese, L. and
                      Giannozzi, P. and Giantomassi, M. and Goedecker, S. and
                      Gonze, X. and Granas, O. and Gross, E. K. U. and Gulans, A.
                      and Gygi, F. and Hamann, D. R. and Hasnip, P. J. and
                      Holzwarth, N. A. W. and Iu an, D. and Jochym, D. B. and
                      Jollet, F. and Jones, D. and Kresse, G. and Koepernik, K.
                      and Kucukbenli, E. and Kvashnin, Y. O. and Locht, I. L. M.
                      and Lubeck, S. and Marsman, M. and Marzari, N. and Nitzsche,
                      U. and Nordstrom, L. and Ozaki, T. and Paulatto, L. and
                      Pickard, C. J. and Poelmans, W. and Probert, M. I. J. and
                      Refson, K. and Richter, M. and Rignanese, G.-M. and Saha, S.
                      and Scheffler, M. and Schlipf, M. and Schwarz, K. and
                      Sharma, S. and Tavazza, F. and Thunstrom, P. and Tkatchenko,
                      A. and Torrent, M. and Vanderbilt, D. and van Setten, M. J.
                      and Van Speybroeck, V. and Wills, J. M. and Yates, J. R. and
                      Zhang, G.-X. and Cottenier, S.},
      title        = {{R}eproducibility in density functional theory calculations
                      of solids},
      journal      = {Science},
      volume       = {351},
      issn         = {1095-9203},
      address      = {Washington, DC [u.a.]},
      publisher    = {American Association for the Advancement of Science64196},
      reportid     = {FZJ-2016-02113},
      pages        = {6280},
      year         = {2016},
      abstract     = {INTRODUCTIONThe reproducibility of results is one of the
                      underlying principles of science. An observation can only be
                      accepted by the scientific community when it can be
                      confirmed by independent studies. However, reproducibility
                      does not come easily. Recent works have painfully exposed
                      cases where previous conclusions were not upheld. The
                      scrutiny of the scientific community has also turned to
                      research involving computer programs, finding that
                      reproducibility depends more strongly on implementation than
                      commonly thought. These problems are especially relevant for
                      property predictions of crystals and molecules, which hinge
                      on precise computer implementations of the governing
                      equation of quantum physics.RATIONALEThis work focuses on
                      density functional theory (DFT), a particularly popular
                      quantum method for both academic and industrial
                      applications. More than 15,000 DFT papers are published each
                      year, and DFT is now increasingly used in an automated
                      fashion to build large databases or apply multiscale
                      techniques with limited human supervision. Therefore, the
                      reproducibility of DFT results underlies the scientific
                      credibility of a substantial fraction of current work in the
                      natural and engineering sciences. A plethora of DFT computer
                      codes are available, many of them differing considerably in
                      their details of implementation, and each yielding a certain
                      “precision” relative to other codes. How is one to
                      decide for more than a few simple cases which code predicts
                      the correct result, and which does not? We devised a
                      procedure to assess the precision of DFT methods and used
                      this to demonstrate reproducibility among many of the most
                      widely used DFT codes. The essential part of this assessment
                      is a pairwise comparison of a wide range of methods with
                      respect to their predictions of the equations of state of
                      the elemental crystals. This effort required the combined
                      expertise of a large group of code developers and expert
                      users.RESULTSWe calculated equation-of-state data for four
                      classes of DFT implementations, totaling 40 methods. Most
                      codes agree very well, with pairwise differences that are
                      comparable to those between different high-precision
                      experiments. Even in the case of pseudization approaches,
                      which largely depend on the atomic potentials used, a
                      similar precision can be obtained as when using the full
                      potential. The remaining deviations are due to subtle
                      effects, such as specific numerical implementations or the
                      treatment of relativistic terms.CONCLUSIONOur work
                      demonstrates that the precision of DFT implementations can
                      be determined, even in the absence of one absolute reference
                      code. Although this was not the case 5 to 10 years ago, most
                      of the commonly used codes and methods are now found to
                      predict essentially identical results. The established
                      precision of DFT codes not only ensures the reproducibility
                      of DFT predictions but also puts several past and future
                      developments on a firmer footing. Any newly developed
                      methodology can now be tested against the benchmark to
                      verify whether it reaches the same level of precision. New
                      DFT applications can be shown to have used a sufficiently
                      precise method. Moreover, high-precision DFT calculations
                      are essential for developing improvements to DFT
                      methodology, such as new density functionals, which may
                      further increase the predictive power of the simulations.},
      cin          = {IAS-1 / PGI-1 / JARA-HPC / JARA-FIT},
      ddc          = {500},
      cid          = {I:(DE-Juel1)IAS-1-20090406 / I:(DE-Juel1)PGI-1-20110106 /
                      $I:(DE-82)080012_20140620$ / $I:(DE-82)080009_20140620$},
      pnm          = {142 - Controlling Spin-Based Phenomena (POF3-142) / 143 -
                      Controlling Configuration-Based Phenomena (POF3-143)},
      pid          = {G:(DE-HGF)POF3-142 / G:(DE-HGF)POF3-143},
      typ          = {PUB:(DE-HGF)16},
      UT           = {WOS:000372756200038},
      pubmed       = {pmid:27013736},
      doi          = {10.1126/science.aad3000},
      url          = {https://juser.fz-juelich.de/record/807084},
}