desihub / desimeter Goto Github PK
View Code? Open in Web Editor NEWDESI coordinates and transformations
License: BSD 3-Clause "New" or "Revised" License
DESI coordinates and transformations
License: BSD 3-Clause "New" or "Revised" License
Analyse ADC rotation data to validate the ray tracing model used in tan2fp transform.
I'm looking at FVC expnum 66575, and it has EXTNAME keys:
> listhead /tmp/fvc-00066575.fits | grep EXTNAME
EXTNAME = 'FVC '
EXTNAME = 'C0003 ' / name of this binary table extension
EXTNAME = 'F0004 '
EXTNAME = 'C0004 ' / name of this binary table extension
EXTNAME = 'F0005 '
EXTNAME = 'C0005 ' / name of this binary table extension
EXTNAME = 'F0006 '
EXTNAME = 'C0006 ' / name of this binary table extension
EXTNAME = 'F0007 '
EXTNAME = 'C0007 ' / name of this binary table extension
EXTNAME = 'F0008 '
EXTNAME = 'C0008 ' / name of this binary table extension
EXTNAME = 'F0009 '
EXTNAME = 'C0009 ' / name of this binary table extension
EXTNAME = 'F0010 '
EXTNAME = 'C0010 ' / name of this binary table extension
EXTNAME = 'F0011 '
EXTNAME = 'F0000 '
EXTNAME = 'C0000 ' / name of this binary table extension
EXTNAME = 'F0001 '
EXTNAME = 'C0001 ' / name of this binary table extension
EXTNAME = 'F0002 '
EXTNAME = 'C0002 ' / name of this binary table extension
EXTNAME = 'F0003 '
EXTNAME = 'C0037 ' / name of this binary table extension
EXTNAME = 'C0024 ' / name of this binary table extension
ie, all mixed up. It looks like desi_fvc_proc with the --sequence option, however, assumes they are in order, and writes out the first image into output file *-F0000.csv
.
OFFSET_T,P are calibration parameters defined by the relation EXT_T,P = INT_T,P + OFFSET_T,P , where INT_T,P are the "intrinsic" angles in the frame of the positioners ; and EXT_T,P represent angles in the frame of the petal. The issue is that they serve 2 different purposes in the current calibration code:
So the calibration fit should always fit for offsets, but distinguish whether to attribute this measured offset to a measurement of the actual angle of the hard stops (i.e. OFFSET_T,P), or a correction to INT_T,P.
The former case should be considered only when the calibration includes a rehoming of the positioners, AND if it is expected that the drifts of INT_T,P would be negligible (in particular for a non-linear response, the situation is more complicated).
Applying corrections to OFFSET_T,P in a calibration sequence that does not include rehoming, or when many moves have been performed since rehoming so would lead to incorrect angles for the hard stops.
basedir=/project/projectdirs/desi/spectro/data/20200107/00038674
desi_fvc_proc \
--infile $basedir/fvc-00038674.fits.fz \
--expected $basedir/coordinates-00038674.fits \
--outfile spots-00038674.csv
PETAL_LOC=0 DEVICE_LOC=533 is matched, but with PINHOLE_ID=0 for all 4 spots.
x = Table.read('spots-00038674.csv')
x['XPIX', 'YPIX', 'PETAL_LOC', 'DEVICE_LOC', 'PINHOLE_ID'][x['LOCATION'] == 533]
<Table length=4>
XPIX YPIX PETAL_LOC DEVICE_LOC PINHOLE_ID
float64 float64 int64 int64 int64
------------------ ------------------ --------- ---------- ----------
2689.077254236983 474.1507123657714 0 533 0
2688.0201166683487 479.83004331316874 0 533 0
2693.642353089485 482.22714056577547 0 533 0
2686.9455325543513 485.50391201761533 0 533 0
i.e. it seems to be successfully matching those pinholes to the correct device location (they aren't just unmatched spots) but the pinhole IDs aren't being set. Why?
From Aaron:
4 of the exposures fail in desi_fit_guide_star_coordinates with an error "ValueError: A value in
x_new is above the interpolation range.". These are expid = 44840,
44885, 44895, 44900. They're all right near zenith.
There is a residual scale factor between focal plane and sky coordinates that vary from exposure to exposure. It is likely part of this variation could be predicted as a function of the telescope pointing (HA,Dec).
GEAR_CALIB_T
and GEAR_CALIB_P
are currently 1.0 in all cases on the Mayall. There may come a date at which robots have variable values set for these. This will affect the internally-tracked POS_T
and POS_P
angles (which is fine).
However, when running fit_posparams
to deduce new values for SCALE_T
and SCALE_P
, the internally-tracked angles must be corrected for whatever GEAR_CALIB_*
values were being used at the time the robot was commanded. Otherwise, the comparison of delta(commanded) vs delta(measured) will come out wrong.
Fix:
1.fitter.py
--> Add additional arguments for vectors of GEAR_CALIB_T
and GEAR_CALIB_P
.
2. In dynamic mode, pre-adjust the dt_int
and dp_int
data back to as if GEAR_CALIB_*
== 1 at time of move.
3. Then run the best-fit.
In ptl2fp.py
, there is a utility function for getting petal alignment data:
desimeter/py/desimeter/transform/ptl2fp.py
Lines 44 to 51 in f3e2500
The file path is hard-coded for grabbing the alignment data. This will cause versioning confusion in either of the following future cases:
To avoid stateful confusion, I suggest that we have multiple configurations in petal-alignments.yaml
. Identify each one by a unique key.
And ptl2fp()
function will need an additional, required argument providing this key.
Study why desi_fvc_proc failed on 43046 or 43047 at KPNO:
INFO:findfiducials.py:279:findfiducials:matched 0 pinholes from 0 fiducials
metrology not matched.
Adjust the fiducials coordinates by stacking the measured coordinates from FVC images.
While writing unit tests for PR #7, I found that findfiducials is missing some pinholes that it previously found:
from pkg_resources import resource_filename
import numpy as np
from astropy.table import Table
from desimeter.findfiducials import findfiducials
testspotfile = resource_filename('desimeter', 'test/data/test-spots.csv')
origspots = Table.read(testspotfile)
spots = origspots.copy()
spots.remove_columns(['LOCATION', 'PINHOLE_ID', 'X_FP', 'Y_FP',
'X_FP_METRO', 'Y_FP_METRO'])
spots = findfiducials(spots)
test_pinholes = list(zip(origspots['LOCATION'], origspots['PINHOLE_ID']))
found_pinholes = list(zip(spots['LOCATION'], spots['PINHOLE_ID']))
missing = set(test_pinholes) - set(found_pinholes)
print('Missing pinholes: {}'.format(missing))
Results in
Missing pinholes: {(4321, 2), (8011, 4), (4011, 4), (1011, 4), (9011, 4), (4150, 4), (3011, 4)}
Note that DEVICE_LOC=11 is the most problematic, as is PINHOLE_ID=4 (central pinhole).
When running analyze_bright_image
, I get the following error in fvc2fp.py
:
Traceback (most recent call last):
File "/software/products/desimeter-master/bin/analyze_bright_image", line 62, in <module>
print(transfo.tojson())
File "/software/products/desimeter-master/py/desimeter/transform/fvc2fp.py", line 91, in tojson
params['meandistance'] = self.meandistance
AttributeError: 'FVC2FP' object has no attribute 'meandistance'
Command to produce this is:
analyze_bright_image --back-illuminated back1.fits --front-illuminated front1.fits -o pos1.csv
FITS files to run this example are at:
front1.fits
back1.fits
We need a thermal-mechanical model. To give a sense of scale, metrology is generally done at ~22C (I need to verify from actual reports), whereas at Mayall the focal plane is nominally operated at 11C. It is mostly aluminum (~24 ppm/C), hence metrology data will be too large by a factor 2.64e-4, or 214 um across the 812 mm diameter.
I believe this needs to occur prior to performing the fit, e.g. sometime before this line:
desimeter/py/desimeter/transform/fvc2fp/zb.py
Lines 179 to 180 in 5926626
Thermal model should be done in a separate module, not spaghettied into zb or poly2d or the like. Also, note the duplicated metrology-loading code in the fit functions. See issue #26.
One option is we move the metrology loading code directly into the base class. And immediately after loading the metrology, apply the thermal model. I'm still wrapping my head around how you guys want your code structured, and where you think interfaces should lie. But anyway, probably better not to bury metrology data prep in specific implementations of fit().
Perhaps best would to make a clean module called like "metrology.py", and it does all the loading and prep of that data, including thermal model etc.
The thermal model may be simple (single fixed temperature at 11C, lacking any real knowledge) or more complex (looking up current temperature sensors---there are many---at the focal plane, either from the online DB, from archived DB, or real-time requests to the hardware. Or even by analyzing FVC image dot separations, if we do a physical test program to correlate them with FP temperature).
Also note that the GIFs will deform somewhat differently physically from everything else, since each pair of GIFs is mounted to a titanium plate on the GFA (which in turn is retrained by an aluminum housing). It's ~10 um effect at our operating temperature. See DESI-0998.
posparams/plotter.bin_errors()
generates a dict classifying positioners by best-fit error over time. For example of how to call load up data and send it to bin_errors()
, see plot_posparams
script.
Alternately, if you've already run plot_posparams
, can just load up the .json file it outputs (which is exactly this result from bin_errors()
).
Use this binned data to output some useful reports. In particular:
For item (2), this is saying when SCALE_T
and SCALE_P
are allowed to float. This would indicate likelihood of a potentially usable "linear" type error. To do this classification, need to distinguish "big" changes in SCALE_T or SCALE_P --- a real effect --- from "small" --- just a slightly better fit.
There are a number of exposures on night 20200118 for which I obtain desimeter field center Dec coordinates that seem to be off by ~70 arcseconds. One example is EXPID = 41797. The specific command I'm using is:
desi_fit_guide_star_coordinates -i /project/projectdirs/desi/users/ameisner/GFA/reduced/v0005/20200118/00041797/gfa-00041797_catalog.fits --fits-header /project/projectdirs/desi/spectro/data/20200118/00041797/gfa-00041797.fits.fz -o gfa-00041797.json
WARNING:desi_fit_guide_star_coordinates:29::no TARGTRA in header of HDU 0, try HDU 1
INFO:fieldmodel.py:92:read_guide_stars_catalog:reading guide stars in /project/projectdirs/desi/users/ameisner/GFA/reduced/v0005/20200118/00041797/gfa-00041797_catalog.fits
INFO:fieldmodel.py:102:read_guide_stars_catalog:selection stars for which we have a good match (< 2.0 arcsec)
INFO:fieldmodel.py:131:fit_tancorr:Use 'mjd_obs' in catalog, MJD=58867.243140375576
INFO:fieldmodel.py:149:fit_tancorr:Use LST=93.87341566663235
ERROR:gfa2fp.py:31:gfa2fp:PETAL_LOC 3 GFA metrology missing
WARNING:fieldmodel.py:258:all_gfa2fp:missing metrology
ERROR:gfa2fp.py:31:gfa2fp:PETAL_LOC 7 GFA metrology missing
WARNING:fieldmodel.py:258:all_gfa2fp:missing metrology
INFO:fieldmodel.py:182:fit_tancorr:RMS coord. residual = 1.45 arcsec
INFO:fieldmodel.py:183:fit_tancorr:Rotation angle (field rot ZP) =0.044 deg
INFO:fieldmodel.py:184:fit_tancorr:Pointing correction dHA=-0.00 arcsec, dDec=-0.00 arcsec
INFO:fieldmodel.py:185:fit_tancorr:Scales sxx=1.0009 syy=1.0008 sxy=0.0000
wrote gfa-00041797.json
The desimeter output file gfa-00041797.json has (ra, dec) = (153.48500768926948, 15.662657681144372). This is using desimeter version 1d0a473.
Separately, I wrote my own code to determine the field center based on the same GFA reductions being passed to desimeter, and I get field center (RA, Dec) coordinates of (153.485567, 15.643475).
This is basically the same as what desimeter gets in RA but offset by ~70 arcseconds in Dec. The field center coordinates I derive with my own code agree very closely with the desired field center (SKYRA, SKYDEC) = (153.485423, 15.643504), which makes sense because this exposure was taken immediately after applying a pointing correction mount offset.
From this same pointing model sequence, there are 16 exposures where my field center (RA, Dec) values agree with desimeter, and these all have desimeter RMS_ARCSEC values of ~0.15-0.35:
EXPID = 41785 , RMS_ARCSEC = 0.18693942
EXPID = 41910 , RMS_ARCSEC = 0.27438225
EXPID = 41913 , RMS_ARCSEC = 0.27071138
EXPID = 41916 , RMS_ARCSEC = 0.19972174
EXPID = 41924 , RMS_ARCSEC = 0.24017903
EXPID = 41932 , RMS_ARCSEC = 0.22779664
EXPID = 41934 , RMS_ARCSEC = 0.18618517
EXPID = 41936 , RMS_ARCSEC = 0.26857016
EXPID = 41948 , RMS_ARCSEC = 0.27877184
EXPID = 41960 , RMS_ARCSEC = 0.19437467
EXPID = 41962 , RMS_ARCSEC = 0.34004326
EXPID = 41966 , RMS_ARCSEC = 0.27551664
EXPID = 41968 , RMS_ARCSEC = 0.24442977
EXPID = 41973 , RMS_ARCSEC = 0.23566265
EXPID = 41975 , RMS_ARCSEC = 0.19593430
EXPID = 41977 , RMS_ARCSEC = 0.25727198
On the other hand, there are 13 exposures from this pointing model sequence where desimeter gives field center results offset by ~70 arseconds in Dec. These cases always have intermediate values of RMS_ARCSEC in the ~1.2-1.5 range, so I'm wondering if that is providing some sort of clue as to what's happening.
EXPID = 41797 , RMS_ARCSEC = 1.4516730
EXPID = 41810 , RMS_ARCSEC = 1.2984165
EXPID = 41823 , RMS_ARCSEC = 1.4075735
EXPID = 41835 , RMS_ARCSEC = 1.3585141
EXPID = 41847 , RMS_ARCSEC = 1.4500220
EXPID = 41859 , RMS_ARCSEC = 1.3889484
EXPID = 41871 , RMS_ARCSEC = 1.3919272
EXPID = 41883 , RMS_ARCSEC = 1.4293239
EXPID = 41895 , RMS_ARCSEC = 1.4196646
EXPID = 41898 , RMS_ARCSEC = 1.4414393
EXPID = 41900 , RMS_ARCSEC = 1.2369366
EXPID = 41902 , RMS_ARCSEC = 1.4176247
EXPID = 41904 , RMS_ARCSEC = 1.3151747
My GFA reduction outputs for all of these exposures (in both of the above lists) indicate successful WCS calibrations for every single guide camera (CONTRAST > 2), so I don't think that's the problem.
Comparison of fiber to target offsets predicted by desimeter from the FVC images and field model with the measured offsets from the dither analysis.
Add transforms between sky RA,dec and focal plane x,y using GFA WCS solutions to tie down the telescope pointing, hexapod offset/rotation, and atmospheric effects.
Handle proper motions in desi_fit_guide_stars
I think there may be places in desimeter where we should add checks that the current petal-alignments.yaml
matches whatever was in the online constants DB at the time of the measurement.
For example when we are doing best fits to discover OFFSET_X
, OFFSET_Y
based on historical data, then this may be important.
I think the simplest and cleanest test is:
Get OBS_X
, OBS_Y
and PTL_X
, PTL_Y
from the online DB. Here the PTL
values were generated in the online system by applying the Constants DB transformation params to OBS
.
Use desimeter / petal-alignments.yaml
to do the analagous conversion into PTL
. (Note OBS
== what desimeter calls X_FP
, Y_FP
.)
Check that PTL
values from (1) and (2) should exactly match.
This comes out of a comment thread in #98 (comment).
When calling ptl2fp
, with an argument of zptl=None
, a dummy vector is generated for ptlz
full of zeros:
desimeter/py/desimeter/transform/ptl2fp.py
Lines 85 to 88 in f3e2500
But this does not match the curved focal surface. I would have thought we need to instead:
r_ptl = np.hypot(xptl, yptl)
.z_ptl
using a function like R2Z_lookup()
in posconstants.py
.Easy to add this r2z (and z2r) to the desimeter transforms package if desired.
I might not be understanding some subtlety of how the "ptl" coordinates are meant to be interpreted.
PR #22 implemented radec2tan
, but we'll also need tan2radec
for analysis of where fibers were on the sky, given their measured locations on the FVC and FP.
Add matching of positioner DEVICE_LOC to FVC spots based upon their expected locations. IIUC, current code only matches static fiducials, but not moving positioners.
Add support for new FVC singlet lens, using fiducial-only data in /global/cfs/cdirs/desi/cmx/fvc/newlens-20200204/ as a starting point.
At minimum desimeter/data/default-fvc2fp.json needs to be updated. Better would be to devise a date-versioned transform such that older FVC data can still be processed transparently, while also supporting a different default transform for new FVC data.
Bonus: I think the original bootstrapping procedure used a platemaker-derived reference file; it would be preferable for desimeter to have its own independent bootstrap method.
If we want to test using desimeter transforms (alternate to platemaker) on the instrument, I think the easiest thing might be to run an "xy test" with a few suggested modifications, described below. Nothing is easy, exactly, but I guess this is least pain. The system is pretty intricate --- I'm sure we'd want Duan or Kevin to do this. Anyway I took a pass through the code and think these are the changes needed. All code links below are at current revisions as of writing.
if fvc == 'desimeter':
if fvc == 'desimeter':
These are my suggestions for broad direction of getting "the analysis stuff currently in PECS" into desimeter.
Need some script(s) for grabbing move data off the onlinedb, and then caching it. See PR #71
summarize_pos_errors
xytestdata_summarize_v2.py
plot_pos_targmeas
plot_pos_arcmeas
fit_posparams_arc
plot_pos_arcmeas
(Copying this over from #192 so we don't forget)
One CI test is failing on python 3.8 but passing on 3.6 and 3.7:
FAIL: test_tan2radec (desimeter.test.test_radec2tan.TestRADEC2TAN)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/circleci/project/py/desimeter/test/test_radec2tan.py", line 139, in test_tan2radec
assert(np.abs(ddec*3600.)<0.001) # 1 milli-arcsec
Using same example as #19:
basedir=/project/projectdirs/desi/spectro/data/20200107/00038674
desi_fvc_proc \
--infile $basedir/fvc-00038674.fits.fz \
--expected $basedir/coordinates-00038674.fits \
--outfile spots-00038674.csv
there are several positioners that are matched to more than one spot, sometimes very far away, e.g.:
from astropy.table import Table
from collections import Counter
x = Table.read('spots-00038674.csv')
for (petal, loc, pinhole), n in Counter(zip(x['PETAL_LOC'], x['DEVICE_LOC'], x['PINHOLE_ID'])).items():
if n>1:
print(petal, loc, pinhole, n)
0 533 0 4
0 509 0 2
0 409 0 2
0 224 0 2
0 205 0 2
0 157 0 2
8 350 0 2
8 248 0 2
7 24 0 2
6 159 0 2
3 494 0 2
7 377 0 2
4 134 0 2
7 443 0 2
7 397 0 2
6 437 0 2
The first case (PETAL_LOC=0 DEVICE_LOC=533) is the fiducial reported in #19, but the others are positioners, e.g.
x['XPIX', 'YPIX', 'DEVICE_LOC', 'PINHOLE_ID'][x['LOCATION'] == 509]
<Table length=2>
XPIX YPIX DEVICE_LOC PINHOLE_ID
float64 float64 int64 int64
------------------ ----------------- ---------- ----------
2667.6594170561475 540.0779076216791 509 0
2695.654126737584 565.4560307807191 509 0
desimeter is already used for the focal plane calibration. The interface is in DOSlib.proxies.FVC. But the interface for the fiber positioning for sky observations has to be developed.
Currently desimeter.transform.gfa2fp
incorrectly treats the GFAs as being parallel to the xy plane, instead of being flat in 3D space that is more approximately aligned with the curved focal surface. This results in ~30 um inconsistencies in the transforms.
Fix that.
get_posmoves fails for me. See console output copied below. I believe the cause is:
[msdos@beyonce desimeter-master]$ /software/products/desimeter-master/bin/get_posmoves --host XXXX --port XXXX --password XXXX --petal-ids 1 --date-min 2020-08-06 --with-calib -o ~/jhsilber/posmovedata
select * from posmovedb.positioner_calibration_p1 where pos_id='M08275' order by time_recorded
Traceback (most recent call last):
File "/software/products/desimeter-master/bin/get_posmoves", line 99, in <module>
j=np.where(tstamp_calib<t)[0][-1]
IndexError: index -1 is out of bounds for axis 0 with size 0
[msdos@beyonce desimeter-master]$ python -m pdb /software/products/desimeter-master/bin/get_posmoves --host XXXX --port XXXX --password XXXX --petal-ids 1 --date-min 2020-08-06 --with-calib -o ~/jhsilber/posmovedata
> /software/products/desimeter-master/bin/get_posmoves(3)<module>()
-> import os,sys
(Pdb) b 99
Breakpoint 1 at /software/products/desimeter-master/bin/get_posmoves:99
(Pdb) l
1 #!/usr/bin/env python
2
3 -> import os,sys
4 import psycopg2
5 import numpy as np
6 import matplotlib.pyplot as plt
7 from astropy.table import Table
8 import datetime
9 import argparse
10
11 from desimeter.util import parse_fibers
(Pdb) c
select * from posmovedb.positioner_calibration_p1 where pos_id='M08275' order by time_recorded
> /software/products/desimeter-master/bin/get_posmoves(99)<module>()
-> j=np.where(tstamp_calib<t)[0][-1]
(Pdb) l
94
95 for k in new_keys :
96 posmoves[k]=list()
97
98 for t in tstamp_posmovedb :
99 B-> j=np.where(tstamp_calib<t)[0][-1]
100 for k1,k1 in zip(new_keys,calib.keys) :
101 posmoves[k1].append(calib[k2][j])
102
103 # save, using astropy.Table io
104
(Pdb) t
1585169115.612538
(Pdb) tstamp_calib
array([1.58516912e+09, 1.58688938e+09, 1.58689045e+09, 1.58692164e+09,
1.58692168e+09, 1.58692171e+09, 1.58697402e+09, 1.58990817e+09,
1.58990819e+09, 1.58990820e+09, 1.58990978e+09, 1.58990983e+09,
1.58990984e+09, 1.58990986e+09, 1.58991030e+09, 1.58993004e+09,
1.58993006e+09, 1.58993007e+09, 1.58993028e+09, 1.58993196e+09,
1.58993205e+09, 1.59019717e+09, 1.59019719e+09, 1.59019721e+09,
1.59051586e+09, 1.59068536e+09, 1.59068539e+09, 1.59068541e+09,
1.59072267e+09, 1.59258753e+09, 1.59258755e+09, 1.59258757e+09,
1.59355202e+09, 1.59371648e+09, 1.59413956e+09, 1.59415106e+09,
1.59415182e+09, 1.59424257e+09])
(Pdb) np.where(tstamp_calib<t)
(array([], dtype=int64),)
(Pdb) np.where(tstamp_calib>t)
(array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
35, 36, 37]),)
Add transforms between sky RA,dec and focal plane x,y based upon an optics model and instrument configuration, i.e. a blind prediction without using GFA astrometry, using hexapod settings, dateobs, RA, dec (and from which precession, airmass, and hour-angle can be derived). More sophisticated versions might use air temperature, air pressure, and focal plane temperature.
fit_posparams
currently calculates best-fits for all positioners, then saves 3 monolithic results files ("static", "dynamic", and "merged"). This has large memory overhead, and means that all processed data up to that point is lost in event of a crash or error.
Produce readme for the workflow from fvc images and posmovedb data --> positioner parameter best-fits and pass/fail histories.
C.f. comments linked below:
#63 (comment)
Currently, get_posmov_fvc_data
matches each pos move db entry to FVC coordinates of positioner, for each positioner of each petal. It does the matching via timestamps. It then outputs a CSV file per positioner, containing those results + a bunch of other columns from the posmovedb.
Coming soon:
EXPOSURE_ID
and EXPOSURE_ITER
to the database. Should simplify matching.OBS_X
, OBS_Y
, PTL_X
, PTL_Y
, PTL_Z
to the database. These are from platemaker and from postransforms at the time of measurement.To handle this, I suggest that we:
get_posmove_fvc_data
.So both get_posmov_fvc_data
and the new script---let's call it get_posmov_data
---can use (2) and (3) as well.
Does that sound like a good approach? Or would it be easier to just extend get_posmove_fvc_data
with more options?
Develop a script and routine "fit_acquisition_exposure" that
/global/cscratch1/sd/schlafly/desi/gfa_reduce_202001/guide-00038408_catalog-00000.fits
, with table columns xcentroid,ycentroid,ra_gaia,dec_gaia
)I get only the right half of the focal plane matched when running
desi_fvc_proc --zbfit -i fvc-00052009.fits.fz --extname F0002 --field-model fm-00052009-20200730.json --expected coordinates-00052009.fits -o fvc-00052009-20200817.csv
with the new desiconda install, loaded with,
module use /global/common/software/desi/${NERSC_HOST}/desiconda/20200801-1.4.0-spec/modulefiles
module load desiconda
It works fine with the old desiconda install
module use /global/common/software/desi/${NERSC_HOST}/desiconda/20190804-1.3.0-spec/modulefiles
module load desiconda
So it's either a different interpretation of the code between different python versions, or some bug with an I/O module.
Use the fitted positioner theta axis coordinates from a series of arc calibration FVC images to refit the petal offsets and rotation AND adjust the coordinates of fiducials because there is evidence that they have systematic errors of >=10 microns.
To correctly go from (x,y,z) space (e.g. FVC pixels) to positioner coordinates (e.g. Q,S), we must first align that (x,y,z) space as nearly as possible to the CS5 coordinate system. We therefore need a procedural definition of CS5 using tangible measured values. I posted a note to DESI-5435 (also attached here for convenience), which offers one suggestion.
The note also includes some background, data, and explanations describing how well we know the petals' as-mounted radial positions based on CMM data done at LBNL. That may be useful info.
test_findfidicials is now failing, starting with the matching changes in PR #40 (but alas I didn't notice before merging).
The test uses a spots file generated by desimeter on FVC data with the Canon lens, but updating the test to use py/desimeter/data/canon-lens-fvc2fp.json
as the input transform didn't work, which surprises/concerns me.
PR #41 includes updating the test to use canon-lens-fvc2fp.json and flags the test as an expected failure, so turn that off when working on this issue.
Ideally the test would confirm that we can reliably find fiducials on both canon-lens and new singlet-lens FVC data.
There is duplicate code for loading metrology data in zb.py and poly2d.py.
I suggest moving this to a higher level. Implement in one place directly in the base class.
desimeter/py/desimeter/transform/fvc2fp/zb.py
Lines 148 to 156 in 5926626
desimeter/py/desimeter/transform/fvc2fp/poly2d.py
Lines 121 to 129 in 5926626
Need to read HEXAPOD rotation angle in guide tables, not in the primary header because it can be different after the acquisition exposure.
That's the case for instance for exposure 20200206/00047450 .
New focal plane metrology (petal alignments) provided by Steve is needed for offline FP calibration analysis.
json copied to text below with updated information:
{
"type": "group",
"name": "focal_plane_metrology",
"version": 200,
"elements": [
{
"name": "0",
"version": 76298,
"constants": {
"petal_id":4,
"petal_offset_x":0.1224,
"petal_offset_y":0.0528,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":-1.885369086
}
},
{
"name": "1",
"version": 76299,
"constants": {
"petal_id":5,
"petal_offset_x":0.0105,
"petal_offset_y":0.0005,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":-1.256580211
}
},
{
"name": "2",
"version": 76300,
"constants": {
"petal_id":6,
"petal_offset_x":-0.0265,
"petal_offset_y":0.0415,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":-0.628353415
}
},
{
"name": "3",
"version": 76301,
"constants": {
"petal_id":3,
"petal_offset_x":-0.048,
"petal_offset_y":-0.0299,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":0.000150777
}
},
{
"name": "4",
"version": 76302,
"constants": {
"petal_id":8,
"petal_offset_x":-0.0196,
"petal_offset_y":0.0459,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":0.628496576
}
},
{
"name": "5",
"version": 76303,
"constants": {
"petal_id":10,
"petal_offset_x":-0.0794,
"petal_offset_y":-0.0113,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":1.256632524
}
},
{
"name": "6",
"version": 76304,
"constants": {
"petal_id":11,
"petal_offset_x":-0.0412,
"petal_offset_y":0.003,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":1.885139556
}
},
{
"name": "7",
"version": 76305,
"constants": {
"petal_id":2,
"petal_offset_x":0.0904,
"petal_offset_y":-0.102,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":2.51321797
}
},
{
"name": "8",
"version": 76306,
"constants": {
"petal_id":7,
"petal_offset_x":0.0056,
"petal_offset_y":-0.0267,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":3.14148172
}
},
{
"name": "9",
"version": 76307,
"constants": {
"petal_id":9,
"petal_offset_x":-0.014,
"petal_offset_y":0.0264,
"petal_offset_z":0.0,
"petal_rot_1":0.0,
"petal_rot_2":0.0,
"petal_rot_3":3.769961549
}
}
]
}
A configuration file with path to the various calibration files is needed.
Especially for the FVC bias file.
Should be "NotImplementedError".
And again at:
If you don't mind, I'd like to fix these myself. Just to learn the process of branching, making pull requests, etc in github.
DESI-5421 metrology is currently missing GFA data for petals 3 and 7, due to missing measurements that weren't taken in the lab. Add placeholders to desimeter/data/fp-metrology-patch.csv
using the average of the metrology from the other petals in petal-local coordinates transformed for petals 3 and 7 offsets and rotations.
Compute field model on many exposures, predict fiber position on focal plane and compare with platemaker coordinates? files.
First goal is to fit shift, rotation, dilatation and correlate differences with ADC angles , parallactic angle and zenith angle ...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.