001018113 001__ 1018113
001018113 005__ 20250728202331.0
001018113 0247_ $$2doi$$a10.5281/ZENODO.10091431
001018113 037__ $$aFZJ-2023-04557
001018113 041__ $$aEnglish
001018113 1001_ $$0P:(DE-Juel1)174151$$aClausen, Alexander$$b0$$eCorresponding author
001018113 245__ $$aLiberTEM/LiberTEM: 0.13.0
001018113 250__ $$a0.13.0
001018113 260__ $$c2023
001018113 3367_ $$2DCMI$$aSoftware
001018113 3367_ $$0PUB:(DE-HGF)33$$2PUB:(DE-HGF)$$aSoftware$$bsware$$msware$$s1753698750_3710
001018113 3367_ $$2BibTeX$$aMISC
001018113 3367_ $$06$$2EndNote$$aComputer Program
001018113 3367_ $$2ORCID$$aOTHER
001018113 3367_ $$2DataCite$$aSoftware
001018113 520__ $$aHomepage: https://libertem.github.io/LiberTEM/GitHub repository: https://github.com/LiberTEM/LiberTEM/PyPI: https://pypi.org/project/libertem/ LiberTEM is an open source platform for high-throughput distributed processing of large-scale binary data sets and live data streams using a modified MapReduce programming model. The current focus is pixelated scanning transmission electron microscopy (STEM) and scanning electron beam diffraction data. MapReduce-like processing allows to specify an algorithm through two functions: One function that is mapped on portions of the input data, and another function that merges (reduces) a partial result from this mapping step into the complete result. A wide range of TEM and 4D STEM processing tasks can be expressed in this fashion, see Applications. The UDF interface of LiberTEM offers a standardized, versatile API to decouple the mathematical core of an algorithm from details of data source, parallelism, and use of results. Mapping and merging can be performed in any order and with different subdivisions of the input data, including running parts of the calculation concurrently. That means the same implementation can be used in a wide range of modalities, including massive scaling on clusters. Since each merge step produces an intermediate result, this style of processing is suitable for displaying live results from a running calculation in a GUI application and for processing live data streams. A closed-loop feedback between processing and instrument control can be realized as well. See User-defined functions for more details on the LiberTEM UDF interface. The LiberTEM back-end offers high throughput and scalability on PCs, single server nodes, clusters and cloud services. On clusters it can use fast distributed local storage on high-performance SSDs. That way it achieves very high aggregate IO performance on a compact and cost-efficient system built from stock components. All CPU cores and CUDA devices in a system can be used in parallel. LiberTEM is supported on Linux, Mac OS X and Windows. Other platforms that allow installation of Python 3.7+ and the required packages will likely work as well. The GUI is running in a web browser. Installation The short version: $ virtualenv -p python3 ~/libertem-venv/ $ source ~/libertem-venv/bin/activate (libertem-venv) $ python -m pip install 'libertem[torch]' # optional for GPU support # See also https://docs.cupy.dev/en/stable/install.html (libertem-venv) $ python -m pip install cupy Please see our documentation for details! Alternatively, to run the LiberTEM Docker image: $ docker run -p localhost:9000:9000 --mount type=bind,source=/path/to/your/data/,dst=/data/,ro libertem/libertem or $ singularity exec docker://libertem/libertem /venv/bin/libertem-server Deployment for offline data processing on a single-node system for a local user is thoroughly tested and can be considered stable. Deployment on a cluster is experimental and still requires some additional work, see Issue #105. Back-end support for live data processing is still experimental as well, see https://github.com/LiberTEM/LiberTEM-live. Applications Since LiberTEM is programmable through user-defined functions (UDFs), it can be used for a wide range of processing tasks on array-like data and data streams. The following applications have been implemented already: Virtual detectors (virtual bright field, virtual HAADF, center of mass , custom shapes via masks) Analysis of amorphous materials Strain mapping Off-axis electron holography reconstruction Single Side Band ptychography Some of these applications are available through an interactive web GUI. Please see the applications section of our documentation for details! The Python API and user-defined functions (UDFs) can be used for complex operations such as arbitrary linear operations and other features like data export. Example Jupyter notebooks are available in the examples directory. If you are having trouble running the examples, please let us know by filing an issue or by joining our Gitter chat. LiberTEM is suitable as a high-performance processing backend for other applications, including live data streams. Contact us if you are interested! LiberTEM is evolving rapidly and prioritizes features following user demand and contributions. Currently we are working on live data processing, improving application support for sparse data and event-based detectors, performance improvements for GPU processing, and implementing analysis methods for various applications of pixelated STEM and other large-scale detector data. If you like to influence the direction this project is taking, or if you'd like to contribute, please join our gitter chat and our general mailing list. File formats LiberTEM currently opens most file formats used for pixelated STEM. See our general information on loading data and format-specific documentation for more information! Raw binary files NumPy .npy binary files Thermo Fisher EMPAD detector files Quantum Detectors MIB format Nanomegas .blo block files Direct Electron DE5 files (HDF5-based) and Norpix SEQ files for DE-Series detectors Gatan K2 IS raw format Stacks of Gatan DM3 and DM4 files (via openNCEM) Single-file Gatan DM4 scans when saved using C-ordering FRMS6 from PNDetector pnCCD cameras (currently alpha, gain correction still needs UI changes) FEI SER files (via openNCEM) MRC (via openNCEM) HDF5-based formats such as HyperSpy files, NeXus and EMD TVIPS binary files Sparse data in Raw CSR (compressed sparse row) format, as is possible to generate from event-based detectors Please contact us if you are interested in support for an additional format! Live processing and detectors (experimental) See LiberTEM-live! License LiberTEM is licensed under GPLv3. The I/O parts are also available under the MIT license, please see LICENSE files in the subdirectories for details. Acknowledgements We are very grateful for your continuing support for LiberTEM! See the acknowledgement page for a list of authors and contributors to LiberTEM and its subprojects. See also our info on funding and industry partners.
001018113 536__ $$0G:(DE-HGF)POF4-5351$$a5351 - Platform for Correlative, In Situ and Operando Characterization (POF4-535)$$cPOF4-535$$fPOF IV$$x0
001018113 536__ $$0G:(DE-HGF)POF4-5353$$a5353 - Understanding the Structural and Functional Behavior of Solid State Systems (POF4-535)$$cPOF4-535$$fPOF IV$$x1
001018113 536__ $$0G:(EU-Grant)780487$$aVIDEO - Versatile and Innovative Detector for Electron Optics (780487)$$c780487$$fERC-2017-PoC$$x2
001018113 536__ $$0G:(EU-Grant)686053$$aCritCat - Towards Replacement of Critical Catalyst Materials by Improved Nanoparticle Control and Rational Design (686053)$$c686053$$fH2020-NMP-2015-two-stage$$x3
001018113 536__ $$0G:(EU-Grant)823717$$aESTEEM3 - Enabling Science and Technology through European Electron Microscopy (823717)$$c823717$$fH2020-INFRAIA-2018-1$$x4
001018113 536__ $$0G:(EU-Grant)856538$$a3D MAGiC - Three-dimensional magnetization textures: Discovery and control on the nanoscale (856538)$$c856538$$fERC-2019-SyG$$x5
001018113 536__ $$0G:(DE-HGF)VH-NG-1317$$amoreSTEM - Momentum-resolved Scanning Transmission Electron Microscopy (VH-NG-1317)$$cVH-NG-1317$$x6
001018113 536__ $$0G:(DE-HGF)ZT-I-0025$$aPtychography 4.0 - Proposal for a pilot project "Information & Data Science" (ZT-I-0025)$$cZT-I-0025$$x7
001018113 588__ $$aDataset connected to DataCite
001018113 650_7 $$2Other$$aSTEM
001018113 650_7 $$2Other$$aTEM
001018113 650_7 $$2Other$$apixelated STEM
001018113 650_7 $$2Other$$a4D STEM
001018113 650_7 $$2Other$$ahigh-throughput
001018113 650_7 $$2Other$$aelectron microscopy
001018113 7001_ $$0P:(DE-Juel1)171370$$aWeber, Dieter$$b1$$eCorresponding author
001018113 7001_ $$00000-0001-9134-384X$$aBryan, Matthew$$b2
001018113 7001_ $$0P:(DE-Juel1)180323$$aRuzaeva, Karina$$b3
001018113 7001_ $$0P:(DE-Juel1)159136$$aMigunov, Vadim$$b4
001018113 7001_ $$00000-0002-2870-366X$$aBaburajan, Anand$$b5
001018113 7001_ $$00000-0001-5045-5650$$aBahuleyan, Abijith$$b6
001018113 7001_ $$0P:(DE-Juel1)157760$$aCaron, Jan$$b7
001018113 7001_ $$00000-0003-2079-5368$$aChandra, Rahul$$b8
001018113 7001_ $$0P:(DE-HGF)0$$aDey, Shankhadeep$$b9
001018113 7001_ $$00000-0003-0224-3746$$aHalder, Sayandip$$b10
001018113 7001_ $$00000-0001-5934-7525$$aKatz, Daniel S.$$b11
001018113 7001_ $$00000-0003-1010-0366$$aLevin, Barnaby D. A.$$b12
001018113 7001_ $$00000-0001-7981-5293$$aNord, Magnus$$b13
001018113 7001_ $$00000-0003-2348-8558$$aOphus, Colin$$b14
001018113 7001_ $$0P:(DE-HGF)0$$aPeter, Simon$$b15
001018113 7001_ $$0P:(DE-HGF)0$$aPuskás, Levente$$b16
001018113 7001_ $$0P:(DE-HGF)0$$aSchyndel van, Jay$$b17
001018113 7001_ $$0P:(DE-HGF)0$$aShin, Jaeweon$$b18
001018113 7001_ $$0P:(DE-HGF)0$$aSunku, Sai$$b19
001018113 7001_ $$00000-0002-1213-2911$$aÅnes, Håkon W.$$b20
001018113 7001_ $$0P:(DE-Juel1)165314$$aMüller-Caspary, Knut$$b21
001018113 7001_ $$0P:(DE-Juel1)144121$$aDunin-Borkowski, Rafal E.$$b22
001018113 7001_ $$0P:(DE-Juel1)130525$$aBarthel, Juri$$b23
001018113 7001_ $$0P:(DE-Juel1)139077$$aBauer, Reimar$$b24
001018113 7001_ $$0P:(DE-HGF)0$$aBecker, Julian$$b25
001018113 7001_ $$0P:(DE-Juel1)157750$$aBeckmann, Andreas$$b26
001018113 7001_ $$0P:(DE-HGF)0$$aBücker, Robert$$b27
001018113 7001_ $$0P:(DE-HGF)0$$azu Castell, Wolfgang$$b28
001018113 7001_ $$00000-0003-4806-9413$$aFedorov, Andrey$$b29
001018113 7001_ $$00000-0001-5215-101X$$aJensen, Mark$$b30
001018113 7001_ $$0P:(DE-HGF)0$$aKrajnak, Matus$$b31
001018113 7001_ $$0P:(DE-HGF)0$$aCrout, Phillip$$b32
001018113 7001_ $$0P:(DE-HGF)0$$aEhrig, Simeon$$b33
001018113 7001_ $$0P:(DE-HGF)0$$aEljarrat Ascunce, Alberto$$b34
001018113 7001_ $$0P:(DE-HGF)0$$aErcius, Peter$$b35
001018113 7001_ $$0P:(DE-HGF)0$$aFuery, Caroline$$b36
001018113 7001_ $$0P:(DE-HGF)0$$aFurhmann, Patrick$$b37
001018113 7001_ $$0P:(DE-HGF)0$$aGaida, John$$b38
001018113 7001_ $$0P:(DE-HGF)0$$aGoscinski, Wojtek James$$b39
001018113 7001_ $$0P:(DE-HGF)0$$aGuzzinati, Giulio$$b40
001018113 7001_ $$0P:(DE-HGF)0$$aHaas, Benedikt$$b41
001018113 7001_ $$0P:(DE-HGF)0$$aHines, Chris$$b42
001018113 7001_ $$0P:(DE-Juel1)130723$$aHouben, Lothar$$b43
001018113 7001_ $$0P:(DE-Juel1)130724$$aHuang, Chien-Cheng$$b44
001018113 7001_ $$0P:(DE-HGF)0$$aHuth, Martin$$b45
001018113 7001_ $$0P:(DE-HGF)0$$aJemian, Pete$$b46
001018113 7001_ $$0P:(DE-HGF)0$$aJohnstone, Duncan N.$$b47
001018113 7001_ $$0P:(DE-HGF)0$$aKoch, Christoph$$b48
001018113 7001_ $$0P:(DE-Juel1)159421$$aKrings, Alexander$$b49
001018113 7001_ $$0P:(DE-HGF)0$$aLam, Siu Kwan$$b50
001018113 7001_ $$0P:(DE-HGF)0$$aLei, Weng I$$b51
001018113 7001_ $$0P:(DE-HGF)0$$aLesnichaia, Anastasiia$$b52
001018113 7001_ $$0P:(DE-HGF)0$$aLiberti, Emanuela$$b53
001018113 7001_ $$0P:(DE-Juel1)167381$$aLu, Penghan$$b54
001018113 7001_ $$0P:(DE-HGF)0$$aMacLaren, Ian$$b55
001018113 7001_ $$0P:(DE-HGF)0$$aMahr, Christoph$$b56
001018113 7001_ $$0P:(DE-HGF)0$$aMcCartan, Shane$$b57
001018113 7001_ $$0P:(DE-HGF)0$$aMeissner, Heide$$b58
001018113 7001_ $$0P:(DE-HGF)0$$aMittelberger, Andreas$$b59
001018113 7001_ $$0P:(DE-HGF)0$$aMoldovan, Grigore$$b60
001018113 7001_ $$0P:(DE-HGF)0$$aMüller, Johannes$$b61
001018113 7001_ $$0P:(DE-HGF)0$$aNebot, Eduardo$$b62
001018113 7001_ $$0P:(DE-HGF)0$$aOda, Terri$$b63
001018113 7001_ $$0P:(DE-HGF)0$$aO'Ryan, Liam$$b64
001018113 7001_ $$0P:(DE-HGF)0$$aOster, Marco$$b65
001018113 7001_ $$0P:(DE-HGF)0$$aPakzad, Ana$$b66
001018113 7001_ $$0P:(DE-HGF)0$$aPekin, Thomas C.$$b67
001018113 7001_ $$0P:(DE-HGF)0$$aPeña, Francisco de la$$b68
001018113 7001_ $$0P:(DE-HGF)0$$aRichter, Tobias$$b69
001018113 7001_ $$0P:(DE-HGF)0$$aRitz, Robert$$b70
001018113 7001_ $$0P:(DE-HGF)0$$aSander, Kunt$$b71
001018113 7001_ $$0P:(DE-HGF)0$$aSchloz, Marcel$$b72
001018113 7001_ $$0P:(DE-HGF)0$$aSchuh, Michael$$b73
001018113 7001_ $$0P:(DE-HGF)0$$aShabih, Sherjeel$$b74
001018113 7001_ $$0P:(DE-HGF)0$$aSimson, Martin$$b75
001018113 7001_ $$0P:(DE-HGF)0$$aSparlinek, Peter$$b76
001018113 7001_ $$0P:(DE-HGF)0$$aStewart, Andy$$b77
001018113 7001_ $$0P:(DE-HGF)0$$aSukumaran, Murali$$b78
001018113 7001_ $$0P:(DE-HGF)0$$aSweeney, Eugene$$b79
001018113 7001_ $$0P:(DE-HGF)0$$aVerbeeck, Jo$$b80
001018113 7001_ $$0P:(DE-HGF)0$$aVoyles, Paul$$b81
001018113 7001_ $$0P:(DE-HGF)0$$aWatts, Benjamin$$b82
001018113 7001_ $$0P:(DE-HGF)0$$aWepf, Roger$$b83
001018113 7001_ $$0P:(DE-HGF)0$$aWilbrink, Jacob$$b84
001018113 7001_ $$0P:(DE-HGF)0$$aWilson, Lance$$b85
001018113 7001_ $$0P:(DE-Juel1)161387$$aWinkler, Florian$$b86
001018113 7001_ $$0P:(DE-HGF)0$$aWollgarten, Markus$$b87
001018113 7001_ $$0P:(DE-HGF)0$$a@theassassin$$b88
001018113 773__ $$a10.5281/ZENODO.10091431
001018113 909CO $$ooai:juser.fz-juelich.de:1018113$$pec_fundedresources$$pVDB$$popenaire
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)174151$$aForschungszentrum Jülich$$b0$$kFZJ
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)171370$$aForschungszentrum Jülich$$b1$$kFZJ
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)180323$$aForschungszentrum Jülich$$b3$$kFZJ
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)165314$$aForschungszentrum Jülich$$b21$$kFZJ
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)144121$$aForschungszentrum Jülich$$b22$$kFZJ
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)130525$$aForschungszentrum Jülich$$b23$$kFZJ
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)139077$$aForschungszentrum Jülich$$b24$$kFZJ
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)157750$$aForschungszentrum Jülich$$b26$$kFZJ
001018113 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)167381$$aForschungszentrum Jülich$$b54$$kFZJ
001018113 9131_ $$0G:(DE-HGF)POF4-535$$1G:(DE-HGF)POF4-530$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5351$$aDE-HGF$$bKey Technologies$$lMaterials Systems Engineering$$vMaterials Information Discovery$$x0
001018113 9131_ $$0G:(DE-HGF)POF4-535$$1G:(DE-HGF)POF4-530$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5353$$aDE-HGF$$bKey Technologies$$lMaterials Systems Engineering$$vMaterials Information Discovery$$x1
001018113 9141_ $$y2023
001018113 9201_ $$0I:(DE-Juel1)ER-C-1-20170209$$kER-C-1$$lPhysik Nanoskaliger Systeme$$x0
001018113 9201_ $$0I:(DE-Juel1)ER-C-2-20170209$$kER-C-2$$lMaterialwissenschaft u. Werkstofftechnik$$x1
001018113 980__ $$asware
001018113 980__ $$aVDB
001018113 980__ $$aI:(DE-Juel1)ER-C-1-20170209
001018113 980__ $$aI:(DE-Juel1)ER-C-2-20170209
001018113 980__ $$aUNRESTRICTED