Changelog

Package changes:

• Compilation now using CMake
• Added HDF5 formats for detector, output and data files. The old file formats will still work. Additionally, if HDF5 is not available, the program will revert to the old formats.
• Reconstruction directories made by dragonfly_init now include all programs and utilities

EMC code changes:

• Probabilities stored as sparse array now, enabling much larger reconstructions. One can also optionally save the probabilities after every iteration.
• Default empirically estimated signal-dependent frame-by-frame beta values used if no initial value is specified. This is the recommended behaviour now. The beta_schedule parameter acts on absolute iteration number.
• Added orientation refinement when continuing a reconstruction by increasing num_div. This is implemented by giving two numbers for num_div in the configuration file, num_div = <fine> <coarse>, where <coarse> should be the value used in the previous iteration.
• [Beta] Can now incorporate a constant background intensity with variable fluence in reconstruction (algorithm is similar to the one in J. Appl. Cryst. (2017). 50, 985-993)

GUI changes

• autoplot now connects to frameviewer in multi-mode reconstructions to show which frames went into which mode. This is especially useful in initial 2D classification.
• Added option to centro-symmetrize frame to fill in detector gaps. Of course, at high angles, patterns do not have inversion symmetry, but this makes things easier at small angles.

Hi, I followed the instructions from the quick start guide and did the following:

./dragonfly_init 
cd recon_0004
./utils/sim_setup.py
./utils/run_emc.py
./utils/autoplot.py

A GUI popped up but it didn’t seem to work.

And from the CLI, it said unable to open data.

(ana-4.0.17-py3) [psanagpu106:recon_0004]$ ./utils/autoplot.py     
Unable to open data//output/intens_001.bin
Unable to open data//output/intens_001.bin

Not sure if I've missed anything.
Any advice would be appreciated!

Changelog

• Added classifier GUI which can be used to classify patterns manually and using a few different ML methods.
• Can now support multiple detectors using the in_detector_list config option.
• Significant overhaul of python codebase. All GUIs now using PyQt rather than TkInter.
• Added Anaconda yml file to resolve additional dependencies.
• EMC C code refactored. Added some abstraction (structs and additional files) for the benefit of extensibility. Also, code reusability significantly improved since C utilities share a lot of code with main EMC code.

This line only saves the new config file to 'config.ini'. Is it a feature or a bug?

recon_emc.c:93
struct detector *orig_det = calloc(1, sizeof(struct detector)) ;
copy_detector(det, orig_det) ;

orig_det have allocated memory for one detector but orig_det[0].num_det contain actual number of detectors.

It cause memory corruption here:

recon_emc.c:137
free_detector(orig_det) ;

Changelog

Python package introduced (beta)

• Code in dragonfly/ sub-directory. Can be installed with pip install.
• Cython code with some C functions
• Provides python package to access and manipulate detector and .emc files
• pyemc console script can perform MPI+OpenMP reconstruction with same interface as conventional emc executable (mpi4py required)

EMC code changes

• Fixed compilation issue for new compilers (clarified declarations)
• Fixed multiple background file parsing
• Added weighted symmetrization
• Added n-fold axial symmetrization option (axial_sym = n in config file)
• Fixed bugs related to combining probabilities across MPI and OpenMP processes
• Changed beta_factor behavior. Instead of always defining the factor, the factor at iteration n will be beta_factor * pow(beta_jump, (n-1) // beta_period)

Miscellaneous

• Log file in reconstruction directory in logs/ sub-directory by default
• Intensity comparison script exposed (utils/compare) to align and calculate correlation coefficients between two intensity reconstructions.

Interface changes

• Added ability to open different config file without restarting Progrss Viewer GUI (autoplot)
• Added ability to change slices visualized by autoplot when looking at a 3D reconstruction
• Added analysis tools for 2D classification: CLPCA (described here) and 2D phase retrieval
• Autoplot settings saved locally for each config file upon exit
• Frame Viewer GUI can be run without config file (with -d and -e options)

Hi,

I tried to do a quick simulation by typing
./dragonfly_init
in the topmost directory, following the instruction here https://github.com/duaneloh/Dragonfly/wiki/Quick-start-with-simulations, but got an error:
(dragonfly) [psanagpu104:Dragonfly]$ ./dragonfly_init
Traceback (most recent call last):
File "./dragonfly_init", line 6, in
import init_new_recon
ImportError: No module named init_new_recon

Would you mind taking a look?

Thanks!

• Made python code 2/3 and PyQt4/5 agnostic
• Cleaned up python code using the help of pylint
• Added cython bindings in pyemc/ folder
• Added unit tests (pyemc/unit.py) and make test option to Makefile
• Styled GUIs with CSS