Outlier blotting does not account for img.meta.photometry.pixelarea_steradians about jwst HOT 4 CLOSED

Comment by Ned Molter on JIRA:

I made what I believe is the requisite change to the outlier step, which was to give tblot() a scale parameter equal to sqrt(pixelarea_blotimg / pixelarea_calimg).  I confirmed in the C code that the pixel scale ratio it expects is in units of length, not area (the C code squares this factor later), and indeed the match between the median pixel values of the input cal image and the intermediate blot image is closest using the sqrt of this ratio.

The fraction of flagged outlier pixels is also a lot smaller with the code change, as follows.

Changed code: 

2024-06-11 09:46:45,689 - stpipe.outlierdetection - INFO - New pixels flagged as outliers: 27067 (10.33%)

2024-06-11 09:46:45,851 - stpipe.outlierdetection - INFO - New pixels flagged as outliers: 70571 (26.92%)

2024-06-11 09:46:45,964 - stpipe.outlierdetection - INFO - New pixels flagged as outliers: 81067 (30.92%)

2024-06-11 09:46:46,077 - stpipe.outlierdetection - INFO - New pixels flagged as outliers: 44717 (17.06%)

Old code:

2024-06-11 09:55:54,038 - stpipe.outlierdetection - INFO - New pixels flagged as outliers: 236893 (90.37%)

2024-06-11 09:55:54,206 - stpipe.outlierdetection - INFO - New pixels flagged as outliers: 203316 (77.56%)

2024-06-11 09:55:54,321 - stpipe.outlierdetection - INFO - New pixels flagged as outliers: 194587 (74.23%)

2024-06-11 09:55:54,472 - stpipe.outlierdetection - INFO - New pixels flagged as outliers: 223848 (85.39%)

However, a caveat remains.  In short, the median pixel values of the blot image remain consistently slightly offset from the median pixel values of the cal image.  For the test data above, and with the changed code, the four exposures have (median Cal / median Blot) = 0.99762994, 0.9951868, 0.9948269, 0.9964127.  These are a factor of 2 closer than before the change, but the offset is in the other direction; hard-coding the pixel scale ratio to 1 as was done before gives (median Cal / median Blot) = 1.0126708, 1.0099102, 1.0094519, 1.0112095.

Is this small offset expected for any reason? Is it a problem?

I noticed that the shape of the PSF is substantially different between the cal data and the blot data, but excising the regions around bright sources when computing the median value did not make any difference.

Is this even the correct code change to make?  Is some ratio also needed as input to resample?

Tagging David Law Melanie Clarke Howard Bushouse, not sure who else might have insight here.

edit: development branch is found here.

from jwst.

Comment by Melanie Clarke on JIRA:

I think the ratio of areas that you're computing is not quite the right one.  You have:
 
pix_ratio = np.sqrt(blot_img.meta.photometry.pixelarea_arcsecsq / median_model.meta.photometry.pixelarea_arcsecsq)
 
But instead I think you need to mirror the iscale value that's computed in the resample step, around line 265 in resample.py. Something like:
 
from jwst.resample.resample import _compute_image_pixel_area
input_pixflux_area = blot_img.meta.photometry.pixelarea_steradians
input_pixel_area = _compute_image_pixel_area(blot_img.meta.wcs)
pix_ratio = np.sqrt(input_pixflux_area / input_pixel_area)
 
Testing with this change flags outliers at about 5-6% instead of 10-30%, which seems a little more reasonable.  I do still see a little offset in the overall median value of the blotted images, though, back in the other direction again, (median cal / median blot) ~ 1.002.
 
 

from jwst.

Comment by Ned Molter on JIRA:

Yes, this indeed seems to be it, thanks Melanie!  For the test case, now the four exposures are roughly centered around unity: the ratios (median cal / median blot) = 1.0018555, 0.9990828, 0.9986338, 1.0003904.

from jwst.

Comment by Howard Bushouse on JIRA:

Fixed by #8553

from jwst.