In this previous commit 70858bc, portions of code (routines) have been converted to functions and saved in separate files, thereby adding clarity and allowing flexibility. While these changes don't break HEEPS (no error message), their implementation seem to require additional changes. In 'metis_hci.py', multiple calls to functions (e.g. wavefront_aberrations, coronagraph, apodization, vortex, lyotstop) return an output that is not being stored, nor used. This results in unexpected results observed, where the wavefront error doesn't seem to be affected properly by the different optical elements.

As a quick fix to allow me working on #1 , I am going to update our test file 'example_coronagraph_psf.py' to run without calling 'metis_hci.py' anymore, leaving the issues in 'metis_hci.py' untouched.

This issue is not high priority, and will be addressed in the next release (HEEPS v1.1), along with more architecture updates.

A natural evolution of HEEPS package in a more Object Oriented architecture.
The main object could be a telescope instrument, created/handled/saved like this:

import heeps
from heeps.instrument import instrument

# create an HCI instrument (e.g., METIS + RAVC)
metis_ravc = instrument(scriptfile='sample_script')

# propagate wavefront errors/aberrations (path to phase screens can be passed here or, by default, in the 'sample_script')
metis_ravc.wferr.propagate(path_atm='phase_screens')

# look-up the different planes 
metis_ravc.pupil_plane.draw()
metis_ravc.focal_plane.draw()
metis_ravc.lyot_plane.draw()

# save results using pickle, e.g. into a '.heeps' file
import pickle
filename = 'metis_ravc_test.heeps'
with open(filename, 'wb') as output:
    pickle.dump(metis_ravc, output, pickle.HIGHEST_PROTOCOL)

As proposed by @ChrisDelaX we could rename HEEPS to become: "High-contrast End-to-End Performance Simulator". As a matter of fact, HEEPS will also be used to predict performance of instruments on other telescopes (e.g., the NEAR project at VLT).

Will be moved to pupil module

This change has apparently disappeared...

Originally posted by @GillesOrban in #19 (comment)

parallactic angle should vary around 0 or 180 degrees, which is currently not the case:

In [1]: import numpy as np
In [2]: length = 10
In [3]: np.linspace(-160, 160, length)
Out[3]:
array([-160.        , -124.44444444,  -88.88888889,  -53.33333333,
        -17.77777778,   17.77777778,   53.33333333,   88.88888889,
        124.44444444,  160.        ])

Instead of -160 to 160 deg, it could be defined from 160 to -160

In [4]: (np.linspace(160, 200, length) + 180) % 360 - 180
Out[4]:
array([ 160.        ,  164.44444444,  168.88888889,  173.33333333,
        177.77777778, -177.77777778, -173.33333333, -168.88888889,
       -164.44444444, -160.        ])

Using VIP, this seems to produce the same result as using 160 to 200 deg,

or even from -20 to 20 deg

In metis_modes.py:
 
if conf['mode'] == 'ELT':
        conf['LS_params'] = [1., -0.3, 0.]  # means no Lyot stop
        lyotstop(wf, conf)

If simulating the ELT mode 1st, LS parameters get updated and later running other modes get screwed up. So for ELT mode keep is empty. 

The example scripts have been moved inside heeps, I think it will be good to keep it outside, makes it easier to show how each module works. I will be also adding an end-end example.

And metis_hci has been removed, I think its good to keep it for now. Easier to do coronagraphic propagation for various modes.

The specifications can be written into a json file associated with the output of every simulation, e.g. 'metis_ravc_20181008.json'
A JSON-formatted file is a list of objects, such as variables and lists of variables.

Specifications not specified in a json file will be replaced by default values.
All specs will have default values hard-coded in constructor methods (init) of HEEPS modules.

A 'script' folder will be added, containing multiple sample scripts, e.g. 'sample_metis_ravc.json', 'sample_visir_agpm.json', 'sample_lbt_app.json' ...

a import seems to be wrong in aberrations/wavefront_aberrations.py:

Fix is probably:
from .static_ncpa import static_ncpa
by
from .ncpa_aberration import ncpa_aberration

Example of speed-up improvement from changes to np.arange and np.angle, to be implemented in future commit:

        def old():
            ramp_oversamp = 11.
            nramp = int(gridsize*ramp_oversamp) #oversamp
            # create the vortex by creating a matrix (theta) representing the ramp (created by atan 2 gradually varying matrix, x and y)
            y1 = np.ones((nramp,), dtype=np.int)
            y2 = np.arange(0, nramp, 1.) - (nramp/2) - int(ramp_oversamp)/2
            y = np.outer(y2, y1)
            x = np.transpose(y)
            return np.arctan2(y,x)
            
        def new():
            # vortex phase ramp is oversampled for a better discretization
            ramp_oversamp = 11.
            nramp = int(gridsize*ramp_oversamp)
            start = -nramp/2 - int(ramp_oversamp)/2
            end   =  nramp/2 - int(ramp_oversamp)/2
            Vp = np.arange(start, end, 1.)
            # Pancharatnam Phase = arg<Vref,Vp> (horizontal input polarization)
            Vref = np.ones(Vp.shape)
            prod = np.outer(Vref, Vp)
            return np.angle(prod + 1j*prod.T)

Time it:

In [87]: %timeit(old())
15.3 s ± 222 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [88]: %timeit(new())
7.16 s ± 45.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Those folders are currently created in the user local working directory.
Their paths should dynamically point to folders in the HEEPS repository.
Input/output folders will also be added to .gitignore to avoid storing fitsfiles on the Github.

• check the size
• do the padding
• propagate foward, backward,
• amplitudes and/or phase

Optimize:

• implement an option using pyfftw package
• it could be unabled within PROPER
• benchmark: compare results vs previous version

Hi there,

I just cloned HEEPS and wanted to run the run_heeps.py script for the APP mode. I needed to install the module proper to get it running, but it seems the package available on PyPi with the name proper is probably not the same proper that you are referring to here:

Which python package should I instead install to get HEEPS running?

Thanks in advance!

Translate Brunella's IDL routine into python, using PROPRER library.
Integrate into HEEPS.

Tibor has to ask Leo and ROy if this is already implemented in METIS simulator.
If yes, we need to cross-validate results.

I'm trying to run a fresh git clone from the repository but it crashes on "file is not a zip file" when downloading the input files. I'm assuming the website/API is just returning some other error that isn't caught. Did the input_file .zip stored on the gdrive change it's ID or did the gdrive API method change ? (I remember it was working in June 2021 and I still have those input_files but I moved them out of the way to see if there were any new files)

Downloading input files from Google Drive to
'/home/gotten/heeps_metis/input_files/'

Traceback (most recent call last):
  File "run_heeps.py", line 68, in <module>
    run_heeps()
  File "run_heeps.py", line 13, in run_heeps
    conf = heeps.config.read_config(verbose=False)
  File "/home/gotten/temp/HEEPS2/heeps/config/read_config.py", line 191, in read_config
    extract_zip(conf['gdrive_id'], conf['dir_input'])
  File "/home/gotten/temp/HEEPS2/heeps/util/download_from_gdrive.py", line 13, in extract_zip
    with zipfile.ZipFile(zipfilename,'r') as zip_ref:
  File "/home/gotten/anaconda3/envs/heeps/lib/python3.7/zipfile.py", line 1258, in __init__
    self._RealGetContents()
  File "/home/gotten/anaconda3/envs/heeps/lib/python3.7/zipfile.py", line 1325, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

The Vortex phase ramp is apparently not perfectly centered. A shift of 0.5 pixel should be added in vortex.py:

ramp_oversamp = 11.
nramp = int(gridsize*ramp_oversamp)
start = -nramp/2 - int(ramp_oversamp)/2 + 0.5
end   =  nramp/2 - int(ramp_oversamp)/2 + 0.5

With this shift, and in perfect conditions:

• the "ring of fire" (light in the Lyot plane just before masking by the Lyot stop) is perfectly symmetric in intensity
• the vortex phase ramp is numerically adjusted to be perfect for a circular aperture. Once this phase ramp is calculated and used for a ring-apodized VC (also supposed to be perfect for an annular aperture), the image in the post-corono focal plane reaches much lower flux level w/o any structure (e.g. no donut shape)

• without chromatic leakage: produce 100 off-axis PSFs spread accross the whole sequence, including all the effects present in the end-to-end simulation, and average it.
• with chromatic leakage: the whole off-axis PSF cube calculated for the chromatic leakage, is then averaged to produce the integrated off-axis PSF for ADI.

Line 199 of adi.py is probably a hack to be removed:
cc_pp = 0

I would also have a suggestion for the function adi: to add the possibility for a user defined tag name (e.g. variable tagname='') that would be appended to savename.
Might slightly improve backstracking of simulations...

There is a inconsistency in the definition of the variable 'pupil_file' in the function pupil
See
pupil_file = {
and then line 37
if pupil_file is None:

The function wavefront_aberrations allows as input zernike indices and their respective coefficients (zern_inds and zernike).
This is typically used to inject jitter (tip-tilt or Z2 and Z3) which are specified in mas and converted in this function in [nm rms] before being passed to proper.
This is ok as long as one use tip and tilt.

For higher order modes this conversion (mas -> nm rms) is not correct.

The code should be modified and should distinguish between tip-tilt and higher-order modes and use appropriate units (rad rms or nm rms). Also, the function should be documented, so the user knows which units she/he has to use

pychache folders appear in my Github Desktop as if they wanted to be sync'ed in HEEPS Github

Christian asked Brunella about her parall angles fits file, and will add the option to use them with ADI.

The ELT pupil file 'ELT_2048_37m_11m_5mas_nospiders_cut.fits' seems wrong. Some horizontal lines are missing. This bug will be addresed along with #2.

If the input_files become too big to be stored on GoogleDrive, we could move them to a secure fpt, e.g.
sftp [email protected]

HEEPS_input_files.zip current size is 26.2 MB