Giter Club home page Giter Club logo

drizzle's Introduction

drizzle Documentation

Powered by Astropy Badge Drizzle's Coverage Status CI Status Documentation Status PyPI Status

The drizzle library is a Python package for combining dithered images into a single image. This library is derived from code used in DrizzlePac. Like DrizzlePac, most of the code is implemented in the C language. The biggest change from DrizzlePac is that this code passes an array that maps the input to output image into the C code, while the DrizzlePac code computes the mapping by using a Python callback. Switching to using an array allowed the code to be greatly simplified.

The DrizzlePac code is currently used in the Space Telescope processing pipelines. This library is forward looking in that it can be used with the new GWCS code.

Requirements

  • Python 3.10 or later
  • Numpy
  • Astropy

The Drizzle Algorithm

This section has been extracted from Chapter 2 of The DrizzlePac Handbook [Driz2012]

There are a family of linear reconstruction techniques that, at two opposite extremes, are represented by the interlacing and shift-and-add techniques, with the Drizzle algorithm representing a continuum between these two extremes.

If the dithers are particularly well-placed, one can simply interlace the pixels from the images onto a finer grid. In the interlacing method, pixels from the independent input images are placed in alternate pixels on the output image according to the alignment of the pixel centers in the original images. However, due to occasional small positioning errors by the telescope, and non-uniform shifts in pixel space across the detector caused by geometric distortion of the optics, true interlacing of images is generally not feasible.

Another standard simple linear technique for combining shifted images, descriptively named “shift-and-add”, has been used for many years to combine dithered infrared data onto finer grids. Each input pixel is block-replicated onto a finer subsampled grid, shifted into place, and added to the output image. Shift-and-add has the advantage of being able to easily handle arbitrary dither positions. However, it convolves the image yet again with the original pixel, thus adding to the blurring of the image and to the correlation of noise in the image. Furthermore, it is difficult to use shift-and-add in the presence of missing data (e.g., from cosmic rays) and geometric distortion.

In response to the limitations of the two techniques described above, an improved method known formally as variable-pixel linear reconstruction, and more commonly referred to as Drizzle, was developed by Andy Fruchter and Richard Hook, initially for the purposes of combining dithered images of the Hubble Deep Field North (HDF-N). This algorithm can be thought of as a continuous set of linear functions that vary smoothly between the optimum linear combination technique (interlacing) and shift-and-add. This often allows an improvement in resolution and a reduction in correlated noise, compared with images produced by only using shift-and-add.

The degree to which the algorithm departs from interlacing and moves towards shift-and-add depends upon how well the PSF is subsampled by the shifts in the input images. In practice, the behavior of the Drizzle algorithm is controlled through the use of a parameter called pixfrac, which can be set to values ranging from 0 to 1, that represents the amount by which input pixels are shrunk before being mapped onto the output image plane.

A key to understanding the use of pixfrac is to realize that a CCD image can be thought of as the true image convolved first by the optics, then by the pixel response function (ideally a square the size of a pixel), and then sampled by a delta-function at the center of each pixel. A CCD image is thus a set of point samples of a continuous two-dimensional function. Hence the natural value of pixfrac is 0, which corresponds to pure interlacing. Setting pixfrac to values greater than 0 causes additional broadening of the output PSF by convolving the original PSF with pixels of non-zero size. Thus, setting pixfrac to its maximum value of 1 is equivalent to shift-and-add, the other extreme of linear combination, in which the output image PSF has been smeared by a convolution with the full size of the original input pixels.

The Drizzle algorithm is conceptually straightforward. Pixels in the original input images are mapped into pixels in the subsampled output image, taking into account shifts and rotations between images and the optical distortion of the camera. However, in order to avoid convolving the image with the large pixel “footprint” of the camera, Drizzle allows the user to shrink the pixel before it is averaged into the output image through the pixfrac parameter.

The flux value of each input pixel is divided up into the output pixels with weights proportional to the area of overlap between the “drop” and each output pixel. If the drop size is too small, not all output pixels have data added to them from each of the input images. One should therefore choose a drop size that is small enough to avoid convolving the image with too large an input pixel footprint, yet sufficiently large to ensure that there is not too much variation in the number of input pixels contributing to each output pixel.

When images are combined using Drizzle, a weight map can be specified for each input image. The weight image contains information about bad pixels in the image (in that bad pixels result in lower weight values). When the final output science image is generated, an output weight map which combines information from all the input weight images, is also saved.

Drizzle has a number of advantages over standard linear reconstruction methods. Since the pixel area can be scaled by the Jacobian of the geometric distortion, it is preserved for surface and absolute photometry. Therefore, the flux in the drizzled image, that was corrected for geometric distortion, can be measured with an aperture size that's not dependent of its position on the image. Since the Drizzle code anticipates that a given output pixel might not receive any information from an input pixel, missing data does not cause a substantial problem as long as the observer has taken enough dither samples to fill in the missing information.

The blot methods perform the inverse operation of drizzle. That is, blotting performs the inverse mapping to transform the dithered median image back into the coordinate system of the original input image. Blotting is primarily used for identifying cosmic rays in the original image. Like the original drizzle task, blot requires the user to provide the world coordinate system (WCS) transformations as inputs.

[Driz2012]Gonzaga, S., Hack, W., Fruchter, A., Mack, J., eds. 2012, The DrizzlePac Handbook. (Baltimore, STScI)

The Drizzle Library

The Drizzle library is object-oriented and you use it by first creating an object of the Drizzle class. To create a new Drizzle output image, supply an Astropy WCS object representing the coordinate system of the output image. The other parameters are the linear pixel dimension described in the previous section, the drizzle kernel used, how each input image is scaled (by exposure time or time squared), and the pixel value set in the output image where the input images do not overlap.

After creating a Drizzle object, you add one or more images by calling the add_fits_file method. The arguments are the name of the FITS file containing the input image and optionally the name of a FITS file containing the pixel weighting. Both file names can be followed by an extension name or number in square brackets. Optionally you can pass the name of the header keywords containing the exposure time and units. Two units are understood: counts and cps (counts per second).

The following function is a demonstration of how you can create a new output image:

def drizzle_demo_one(reference, outfile, infiles):
    """
    First demonstration of drizzle

    Parameters
    ==========
    reference
        A file containing the wcs of the output image

    outfile
        The name of the output image

    infiles
        The names of the input images to be combined
    """
    # Get the WCS for the output image
    hdulist = fits.open(reference)
    reference_wcs = wcs.WCS(hdulist[1].header)

    # Initialize the output with the WCS
    driz = drizzle.drizzle.Drizzle(outwcs=reference_wcs)

    # Combine the input images into on drizzle image
    for infile in infiles:
        driz.add_fits_file(infile)

    # Write the drizzled image out
    driz.write(outfile)

Optionally you can supply the input and weight images as Numpy arrays by using the add_image method. If you use this method, you must supply the extra information that would otherwise be read from the FITS image: The WCS of the input image, the exposure time, and image units.

Here is an example of how you would call add_image:

def drizzle_demo_two(reference, outfile, infiles):
    """
    Demonstration of drizzle with add image.

    Parameters
    ==========
    reference
        A file containing the wcs of the output image.

    outfile
        The name of the output image.

    infiles
        The names of the input images to be combined.
    """
    # Get the WCS for the output image
    reflist = fits.open(reference)
    reference_wcs = wcs.WCS(reflist[1].header)

    # Initialize the output with the WCS
    driz = drizzle.drizzle.Drizzle(outwcs=reference_wcs)

    # Combine the input images into on drizzle image
    for infile in infiles:
        # Open the file and read the image and wcs
        # This is a contrived example, we would not do this
        # unless the data came from another source
        # than a FITS file
        imlist = fits.open(reference)
        image = imlist[1].data
        image_wcs = wcs.WCS(imlist[1].header)
        driz.add_image(image, image_wcs)

    # Write the drizzled image out
    driz.write(outfile)

After combining all the input images, you write the output image into a FITS file with the write method. You must pass the name of the output image and optionally the units. You can also supply a set of header cards to be added to the primary header of the output FITS file.

You can also add more images to an existing Drizzle output file by creating a new Drizzle object and passing the existing output file name as the new object is created. In that case the output WCS and all other parameters are read from the file.

Here is a demonstration of adding additional input images to a drizzled image:

def drizzle_demo_three(outfile, infiles):
    """
    Demonstration of drizzle and adding to an existing output.

    Parameters
    ==========
    outfile
        Name of output image that new files will be appended to.

    infiles
        The names of the input images to be added.
    """
    # Re-open the output file
    driz = drizzle.drizzle.Drizzle(infile=outfile)

    # Add the input images to the existing output image
    for infile in infiles:
        driz.add_fits_file(infile)

    # Write the modified drizzled image out
    driz.write(outfile)

You can use the methods blot_fits_file and blot_image to transform the drizzled output image into another WCS. Most usually this is the coordinates of one of the input images and is used to identify cosmic rays or other defects. The two methods blot_fits_file and blot_image allow you to retrieve the WCS from the FITS file header or input it directly. The optional parameter interp allows you to selct the method used to resample the pixels on the new grid, and sincscl is used to scale the sinc function if one of the sinc interpolation methods is used. This function demonstrates how both methods are called:

def drizzle_demo_four(outfile, blotfile):
    """
    Demonstration of blot methods.

    Parameters
    ==========
    outfile
        Name of output image that will be converted.

    blotfile
        Name of image containing wcs to be transformed to.
    """
    # Open drizzle using the output file
    # Transform it to another coordinate system
    driz = drizzle.drizzle.Drizzle(infile=outfile)
    driz.blot_fits_file(blotfile)
    driz.write(outfile)

    # Read the WCS and transform using it instead
    # This is a contrived example
    blotlist = fits.open(blotfile)
    blot_wcs = wcs.WCS(blotlist[1].header)
    driz = drizzle.drizzle.Drizzle(infile=outfile)
    driz.blot_image(blot_wcs)
    driz.write(outfile)

The lower level function dodrizzle is present for backwards compatibility with the existing STScI DrizzlePac code and should not be used unless you are also concerned with this compatibility.

drizzle's People

Contributors

astrofrog avatar bernie-simon avatar braingram avatar bsipocz avatar cdeil avatar crawfordsm avatar embray avatar eteq avatar hamogu avatar jamienoss avatar jdavies-st avatar jhunkeler avatar kbarbary avatar keflavich avatar kmacdonald-stsci avatar larrybradley avatar mcara avatar mdboom avatar mwcraig avatar olebole avatar pllim avatar williamjamieson avatar wkerzendorf avatar zacharyburnett avatar zanecodes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

drizzle's Issues

wt_scl is sometimes a string, sometimes a float

The interface for wt_scl is not clear. in dodrizzle is described as a float, but everywhere else it is a string. And sometimes it's an empty string. This interface needs to be cleaned up and made consistent.

Additionally, within the C code, it is wtscl, but the interface presents it as weight_scale.

Sgmentation violation when pixmap is mostly NaNs

Drizzle is throwing a segmentation violation when it interpolates a position in a pixmap that is mostly NaNs. This will happen when handling JWST spectra, as the region outside the spectra is all NaNs. The problem is that the current code blows up when asked to interpolate in a line that is entirely NaNs. The solution is to rewrite the code in shrink_segment so that it "thinks" in two dimensions at a time instead of only one. Line handling then becomes a degenerate case where the second dimension has length one. The other half of the fix is to do a better job of error handling so that when the bound for the interpolated point cannot be found, the point is skipped.

After viewing the code, I saw a few other issues. It shouldn't try so hard to find a bound by searching the entire line, it should give up after a certain range of pixels. This will speed up worst case performance, which, unfortunately, is common. Also several currently unused algorithms in the code (turbo, tophat, etc.) bypass the interpolation code, call get_pixmap directly and so will fail when the pixmap value is NaN.

I'd punch the author who made this mistake, but unfortunately I'd be punching myself in the face.

Update the help

Tyler Desjardins mentions that we should consider moving emails from help[at]stsci.edu to point to the web portal where possible and appropriate. For HST (or any non-JWST), it is https://hsthelp.stsci.edu . For JWST, it is https://jwsthelp.stsci.edu . Please update info in setup.py, setup.cfg, documentation, etc as appropriate.

Please close this issue if it is irrelevant to your repository. This is an automated issue. If this is opened in error, please let pllim know!

xref spacetelescope/hstcal#317

Allow the pixmap to be sized to the output frame

Currently, the 2D image size of the input frames determine the pixmap size, and this is always <= to the output frame size. But in the case where one would prefer to have a small output frame, i.e. to resample a section of a large input frame into a small cutout into the output, this currently fails.

More details in #30.

Flux increases as ~n^2 when rebinning in x axis by a factor of n

I am not 100% sure if this is an issue, but when I use drizzle to rebin the 2D image by a factor of n in the x axis the flux increases by n^2. When I rebin in the y axis the flux is the same.

Simple example:
My image is a 8x4 matrix of ones. If I rebin the matrix in the y axis by a factor of 2, I obtain (as expected), a 8x2 matrix of ones.
However, if I take the 8x4 matrix and I rebin in the x axis by a factor of 2, I get a 4x4 matrix of fours.

dodrizzle argument list problem

The output context image which is passed into doddrizzle is sometimes replaced by a new context image. This happens because the context image stores one bit per pixel for each input image. If there are more than 32 images, the context image needs to be expanded along its third dimension to hold the additional information. Expanding the context image, which is a numpy array, creates a new copy of the old image. But since Python calling semantics is call by value, this new image is not passed back to the code which calls dodrizzle.

[1.14.0] Tests fail on big-endian platforms

When compiling version 1.14.0 for (64-bit) big-endian platforms (s390x, ppc64), many tests error or fail in cdrizzle.tridz(), f.e.:

_______________________ test_zero_input_weight[lanczos3] _______________________

kernel = 'lanczos3'

    @pytest.mark.parametrize(
        'kernel', ['square', 'point', 'turbo', 'gaussian', 'lanczos3'],
    )
    def test_zero_input_weight(kernel):
        """
        Test do_driz square kernel with grid
        """
        # initialize input:
        insci = np.ones((200, 400), dtype=np.float32)
        inwht = np.ones((200, 400), dtype=np.float32)
        inwht[:, 150:155] = 0
    
        # initialize output:
        outsci = np.zeros((210, 410), dtype=np.float32)
        outwht = np.zeros((210, 410), dtype=np.float32)
        outctx = np.zeros((210, 410), dtype=np.int32)
    
        # define coordinate mapping:
        pixmap = np.moveaxis(np.mgrid[1:201, 1:401][::-1], 0, -1)
    
        # resample:
>       cdrizzle.tdriz(
            insci, inwht, pixmap,
            outsci, outwht, outctx,
            uniqid=1,
            xmin=0, xmax=400,
            ymin=0, ymax=200,
            pixfrac=1,
            kernel=kernel,
            in_units='cps',
            expscale=1,
            wtscale=1,
            fillstr='INDEF',
        )
E       ValueError: No or too few valid pixels in the pixel map.

drizzle/tests/test_drizzle.py:276: ValueError

This is a regression; it didn't happen with version 1.13.7.
Versions:

  • Python 3.11.4
  • Astropy 5.3.3
  • Numpy 1.24.2

Full build log on s390x

"Drizzling" an image into a smaller one does not produce the expected result

I am using Drizzle to go back and forth between an astronomical scene (the large image), and a small detector that observe in in scanning mode (building a mosaic of the large scene though an on-the-fly observing mode, and I have discovered that when the image provided to Drizzle.add_image() is larger than that that has been used to create the Drizzle instance, the result is incorrect (empty, or not filled completely, this seems to has a dependence on the image size ratio).

here is a test that I have performed with a fits file that I have created for testing purposes. The image contains a recognisable structure of stripes.

hdu = fits.open('Test_Mire.fits')
sceneWcs = WCS(hdu[0].header)
data = hdu[0].data
plt.imshow(data)

This shows correctly the stripe pattern I expect.

Now I create a smaller image by extracting the central part and updating the WCS so that its is physically centred on the same sky position

data2 = data[50:-50,50:-50].copy()
tmpWcs = sceneWcs.deepcopy()
tmpWcs.pixel_shape = [3759,3866]
tmpWcs.wcs.crpix = [1929.5-50,1983-50]

Then I use drizzle to add that small image in an drizzled map that is created with the WCS of the larger image

drizMap = Drizzle(outwcs=sceneWcs)
drizMap.add_image(data2,tmpWcs)
plt.imshow(drizMap.outsci)

This shows the expected striped pattern to the exception that there is a band of empty pixels on all 4 sides of the field, as indeed the image that I have drizzled has a smaller coverage.

Then I do the reverse, use the smaller image WCS to create the drizzled map and add the larger image into it.

drizMap = Drizzle(outwcs=tmpWcs)
drizMap.add_image(data,sceneWcs)
plt.imshow(drizMap.outsci)

this image is empty (full of 0s). And the context (outcon[0,:,:]) image is also empty as if, as far as add_image is concerned, they have no geometrical overlap.

I am not attaching my test image because it is quite large but I've discovered the problem when I was trying to make this with much smaller images (by extracting first a section of the large image, large enough still so that the detector would fit into it whatever the orientation, but still quite smaller that the image I have used for this test example).

This looks like a bug.

Marc

C test failures with version 1.13.6

Additionally to #87, I observed a few C failures:

utest_map_lookup_01 ............................................... PASS
utest_map_lookup_02 ............................................... PASS
utest_map_lookup_03 ............................................... PASS
utest_map_lookup_04 ............................................... PASS
utest_shrink_segment_01 ........................................... PASS
utest_shrink_segment_02 ........................................... PASS
utest_shrink_segment_03 ........................................... PASS
utest_map_point_01 ................................................ PASS
utest_map_point_02 ................................................ PASS
utest_map_point_03 ................................................ FAIL ***
utest_map_point_04 ................................................ FAIL ***
utest_check_line_overlap_01 ....................................... PASS
utest_check_line_overlap_02 ....................................... PASS
utest_check_line_overlap_03 ....................................... PASS
utest_check_image_overlap_01 ...................................... PASS
utest_check_image_overlap_02 ...................................... PASS
utest_check_image_overlap_03 ...................................... PASS
utest_compute_area_01 ............................................. PASS
utest_compute_area_02 ............................................. PASS
utest_compute_area_03 ............................................. PASS
utest_compute_area_04 ............................................. PASS
utest_compute_area_05 ............................................. PASS
utest_do_kernel_square_01 ......................................... PASS
utest_do_kernel_square_02 ......................................... PASS
utest_do_kernel_square_03 ......................................... PASS
utest_do_kernel_square_04 ......................................... PASS
utest_do_kernel_square_05 ......................................... PASS
utest_do_kernel_square_06 ......................................... PASS
utest_do_kernel_square_07 ......................................... PASS
utest_dobox_01 .................................................... PASS
utest_dobox_02 .................................................... PASS
utest_dobox_03 .................................................... PASS
utest_dobox_04 .................................................... PASS
utest_doblot_01 ................................................... PASS
utest_doblot_02 ................................................... PASS

----------------------------------------------------------------------------

FAILED TESTS


./src/tests/utest_cdrizzle.c(526):
    chk_eq_dbl: nan != 3.250000
./src/tests/utest_cdrizzle.c(527):
    chk_eq_dbl: nan != 5000.000000
./src/tests/utest_cdrizzle.c(547):
    chk_eq_dbl: nan != 0.250000
./src/tests/utest_cdrizzle.c(548):
    chk_eq_dbl: nan != 5000.000000



----------------------------------------------------------------------------

FAILED (33/35 tests in 0.038554s)

(The numbers of total failed tests confuses me; that's why I copied all).
Environment is the "usual" Debian unstable on x86_64:

  • Python 3.10
  • Astropy 5.1
  • Numpy 1.21.5

Missing __all__ in modules

All the Python modules should have __all__ defined. It helps with API doc generation and blind * imports.

Add ReadTheDocs check to CI

I see this package has a docs directory, and Travis CI even tests the setup.py build_docs command, but I don't see any indication that this actually renders the doc on ReadTheDocs (RTD) or anything. If RTD for this project exists, its admin should add the badge to the README of this repo and let DATB know where the RTD project link is.

1.4: test_cdrizzle _cdrizzle.c fails

With the new 1.4 version, the test_cdrizzle _cdrizzle.c tests fail:

drizzle/tests/test_cdrizzle.py::test_cdrizzle _cdrizzle.c(903):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(901):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(902):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(903):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(901):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(902):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(903):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(901):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(902):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(903):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(901):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(902):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(903):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(901):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(902):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(903):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(935):
    chk_eq_dbl: 0.000000 != 4.000000
drizzle/src/tests/utest_cdrizzle.c(935):
    chk_eq_dbl: 0.000000 != 4.000000
drizzle/src/tests/utest_cdrizzle.c(935):
    chk_eq_dbl: 0.000000 != 4.000000
drizzle/src/tests/utest_cdrizzle.c(935):
    chk_eq_dbl: 0.000000 != 4.000000
drizzle/src/tests/utest_cdrizzle.c(966):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(967):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(968):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(969):
    chk_eq_dbl: 0.000000 != 1.000000
drizzle/src/tests/utest_cdrizzle.c(970):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(971):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(972):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(973):
    chk_eq_dbl: 0.000000 != 2.000000
drizzle/src/tests/utest_cdrizzle.c(974):
    chk_eq_dbl: 0.000000 != 4.000000



----------------------------------------------------------------------------

FAILED (20/35 tests in 0.092268s)

Full build log here.

Tests require write access to data directory

Some tests write output files to DATA_DIR. If the package is installed system wide, this directory is however readonly for the user, so the tests will fail.

From looking into the code, I would assume that the output files are not needed for the tests at all, but are just there for a later analysis outside of the tests, right? Could writing be switched off by default?

Best regards

Ole

Clean up sphinx-build warnings

Currently:

$ sphinx-build docs docs/_build
...
reading sources... [100%] index                                                                                                      
/Users/jdavies/dev/drizzle/docs/../drizzle/doblot.py:docstring of drizzle.doblot.doblot:9: WARNING: Unexpected section title.

Parameters
----------
/Users/jdavies/dev/drizzle/docs/../drizzle/doblot.py:docstring of drizzle.doblot.doblot:36: WARNING: Unexpected section title.

Returns
-------
/Users/jdavies/dev/drizzle/docs/../drizzle/doblot.py:docstring of drizzle.doblot.doblot:41: WARNING: Unexpected section title.

Other Parameters
----------------
/Users/jdavies/dev/drizzle/docs/../drizzle/dodrizzle.py:docstring of drizzle.dodrizzle.dodrizzle:8: WARNING: Unexpected section title.

Parameters
----------
/Users/jdavies/dev/drizzle/docs/../drizzle/dodrizzle.py:docstring of drizzle.dodrizzle.dodrizzle:106: WARNING: Unexpected section title.

Returns
-------
/Users/jdavies/dev/drizzle/docs/../drizzle/drizzle.py:docstring of drizzle.drizzle.Drizzle.add_fits_file:4: WARNING: Unexpected section title.

Parameters
----------
/Users/jdavies/dev/drizzle/docs/../drizzle/drizzle.py:docstring of drizzle.drizzle.Drizzle.add_image:8: WARNING: Unexpected section title.

Parameters
----------
/Users/jdavies/dev/drizzle/docs/../drizzle/drizzle.py:docstring of drizzle.drizzle.Drizzle.blot_fits_file:4: WARNING: Unexpected section title.

Parameters
----------
/Users/jdavies/dev/drizzle/docs/../drizzle/drizzle.py:docstring of drizzle.drizzle.Drizzle.blot_image:4: WARNING: Unexpected section title.

Parameters
----------
/Users/jdavies/dev/drizzle/docs/../drizzle/drizzle.py:docstring of drizzle.drizzle.Drizzle.write:12: WARNING: Unexpected section title.

Parameters
----------
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index                                                                                                       
WARNING: dot command 'dot' cannot be run (needed for graphviz output), check the graphviz_dot setting
generating indices... genindex py-modindex done
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 11 warnings.

The HTML pages are in docs/_build.

Once fixed, add the -W flag to the tox job.

Remove all references to INDEF

Remove references to INDEF from this code. It is not handled properly anyway. INDEF has no meaning in C or Python anyway. It is a holdover from IRAF. And here, it is set to zero here anyway. How about we just be explicit?

$ grep -r INDEF drizzle | grep -v Binary
drizzle/drizzle.py:                 fillval="INDEF"):
drizzle/drizzle.py:            not overlap it. The default value of INDEF does not set a value.
drizzle/dodrizzle.py:              pixfrac=1.0, kernel='square', fillval="INDEF"):
drizzle/dodrizzle.py:        not overlap it. The default value of INDEF does not set a value.
drizzle/dodrizzle.py:        fillval = 'INDEF'
drizzle/src/cdrizzleapi.c:  char *fillstr = "INDEF";
drizzle/src/cdrizzleapi.c:      strncmp(fillstr, "INDEF", 6) == 0 ||

In fact, there shouldn't even be a fill string. There should be a default value of 0 or NaN. We can argue about which. 😄

MNT: Stop using ci-helpers in appveyor.yml

To whom it may concern,

If you are using https://github.com/astropy/ci-helpers in your appveyor.yml , please know that the Astropy project has dropped active development/support for Appveyor CI. If it still works, good for you, because we did not remove the relevant files (yet). But if it ever stops working, we have no plans to fix anything for Appveyor CI. Please consider using native Windows support other CI, e.g., Travis CI (see https://docs.travis-ci.com/user/reference/windows/). We apologize for any inconvenience caused.

If this issue is opened in error or irrelevant to you, feel free to close. Thank you.

xref astropy/ci-helpers#464

1.5: test_square_with_point fails on 32 bit

Hi,

sorry for bothering you again with a failures: On all 32 bit architectures (i386, but also ARM, MIPS, PowerPC etc.), test_square_with_point fails (seems to crash):

============================= test session starts ==============================
platform linux2 -- Python 2.7.13rc1, pytest-2.8.3, py-1.4.30, pluggy-0.3.1 -- /usr/bin/python2.7
cachedir: ../.cache

Running tests with Astropy version 1.2.1.
Running tests in lib.linux-i686-2.7/drizzle docs.

Date: 2016-12-08T09:33:22

Platform: Linux-3.16.0-4-amd64-i686-with-debian-stretch-sid

Executable: /usr/bin/python2.7

Full Python Version: 
2.7.13rc1 (default, Dec  4 2016, 14:12:39) 
[GCC 6.2.1 20161124]

encodings: sys: ascii, locale: ANSI_X3.4-1968, filesystem: ANSI_X3.4-1968, unicode bits: 20
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.11.2
Scipy: not available
Matplotlib: not available
h5py: not available
Pandas: not available

rootdir: /tmp/drizzle-test-dKEuQp, inifile: setup.cfg
collecting ... collected 26 items

drizzle/util.py::drizzle.util.parse_extn PASSED
drizzle/data/README.rst PASSED
drizzle/tests/test_cdrizzle.py::test_cdrizzle utest_map_lookup_01 ............................................... PASS
utest_map_lookup_02 ............................................... PASS
[...]
utest_doblot_02 ................................................... PASS

----------------------------------------------------------------------------

PASSED (35/35 tests in 0.224743s)
PASSED
drizzle/tests/test_drizzle.py::test_square_with_point debian/rules:14: recipe for target 'test-python2.7' failed
make[1]: *** [test-python2.7] Error 245

C-extension compiler warnings

When you don't use tox, you see this in the build log:

gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include -I/home/runner/work/drizzle/drizzle/src -I/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12 -c /home/runner/work/drizzle/drizzle/src/cdrizzleapi.c -o /tmp/tmp3_q9wi6l.build-temp/home/runner/work/drizzle/drizzle/src/cdrizzleapi.o
  /home/runner/work/drizzle/drizzle/src/cdrizzleapi.c: In function ‘PyInit_cdrizzle’:
  /home/runner/work/drizzle/drizzle/src/cdrizzleapi.c:610:12: warning: returning ‘PyObject *’ {aka ‘struct _object *’} from a function with incompatible return type ‘PyObject **’ {aka ‘struct _object **’} [-Wincompatible-pointer-types]
    610 |     return m;
        |            ^
  gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include -I/home/runner/work/drizzle/drizzle/src -I/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12 -c /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c -o /tmp/tmp3_q9wi6l.build-temp/home/runner/work/drizzle/drizzle/src/cdrizzleblot.o
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c: In function ‘interpolate_poly3’:
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:[120](https://github.com/spacetelescope/drizzle/actions/runs/7014084246/job/19081247879?pr=128#step:4:121):47: warning: ‘cd21[3]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    120 |       ztemp[j] = sx * (coeff[index+1] + sx2m1 * cd21[j]) +
        |                                         ~~~~~~^~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:72:18: note: ‘cd21[3]’ was declared here
     72 |   float cd20[4], cd21[4], ztemp[4];
        |                  ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:120:47: warning: ‘cd21[2]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    120 |       ztemp[j] = sx * (coeff[index+1] + sx2m1 * cd21[j]) +
        |                                         ~~~~~~^~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:72:18: note: ‘cd21[2]’ was declared here
     72 |   float cd20[4], cd21[4], ztemp[4];
        |                  ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:120:47: warning: ‘cd21[1]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    120 |       ztemp[j] = sx * (coeff[index+1] + sx2m1 * cd21[j]) +
        |                                         ~~~~~~^~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:72:18: note: ‘cd21[1]’ was declared here
     72 |   float cd20[4], cd21[4], ztemp[4];
        |                  ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:120:47: warning: ‘cd21[0]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    120 |       ztemp[j] = sx * (coeff[index+1] + sx2m1 * cd21[j]) +
        |                                         ~~~~~~^~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:72:18: note: ‘cd21[0]’ was declared here
     72 |   float cd20[4], cd21[4], ztemp[4];
        |                  ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:[121](https://github.com/spacetelescope/drizzle/actions/runs/7014084246/job/19081247879?pr=128#step:4:122):45: warning: ‘cd20[3]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    121 |                  tx * (coeff[index] + tx2m1 * cd20[j]);
        |                                       ~~~~~~^~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:72:9: note: ‘cd20[3]’ was declared here
     72 |   float cd20[4], cd21[4], ztemp[4];
        |         ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:121:45: warning: ‘cd20[2]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    121 |                  tx * (coeff[index] + tx2m1 * cd20[j]);
        |                                       ~~~~~~^~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:72:9: note: ‘cd20[2]’ was declared here
     72 |   float cd20[4], cd21[4], ztemp[4];
        |         ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:121:45: warning: ‘cd20[1]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    121 |                  tx * (coeff[index] + tx2m1 * cd20[j]);
        |                                       ~~~~~~^~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:72:9: note: ‘cd20[1]’ was declared here
     72 |   float cd20[4], cd21[4], ztemp[4];
        |         ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:121:45: warning: ‘cd20[0]’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    121 |                  tx * (coeff[index] + tx2m1 * cd20[j]);
        |                                       ~~~~~~^~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:72:9: note: ‘cd20[0]’ was declared here
     72 |   float cd20[4], cd21[4], ztemp[4];
        |         ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c: In function ‘interpolate_sinc’:
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:673:19: warning: array subscript -1 is below array bounds of ‘float[15]’ [-Warray-bounds]
    673 |         px = taper[j - 1] / ax;
        |              ~~~~~^~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:596:9: note: while referencing ‘taper’
    596 |   float taper[INTERPOLATE_SINC_NCONV];
        |         ^~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:681:19: warning: array subscript -1 is below array bounds of ‘float[15]’ [-Warray-bounds]
    681 |         py = taper[j - 1] / ay;
        |              ~~~~~^~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:596:9: note: while referencing ‘taper’
    596 |   float taper[INTERPOLATE_SINC_NCONV];
        |         ^~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:685:9: warning: array subscript -1 is below array bounds of ‘float[15]’ [-Warray-bounds]
    685 |       ac[j - 1] = px;
        |       ~~^~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:597:9: note: while referencing ‘ac’
    597 |   float ac[INTERPOLATE_SINC_NCONV], ar[INTERPOLATE_SINC_NCONV];
        |         ^~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:686:9: warning: array subscript -1 is below array bounds of ‘float[15]’ [-Warray-bounds]
    686 |       ar[j - 1] = py;
        |       ~~^~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzleblot.c:597:37: note: while referencing ‘ar’
    597 |   float ac[INTERPOLATE_SINC_NCONV], ar[INTERPOLATE_SINC_NCONV];
        |                                     ^~
  gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include -I/home/runner/work/drizzle/drizzle/src -I/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12 -c /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c -o /tmp/tmp3_q9wi6l.build-temp/home/runner/work/drizzle/drizzle/src/cdrizzlebox.o
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c: In function ‘do_kernel_point’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c:254:15: warning: unused variable ‘ybounds’ [-Wunused-variable]
    254 |     integer_t ybounds[2], osize[2];
        |               ^~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c: In function ‘do_kernel_tophat’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c:343:15: warning: unused variable ‘ybounds’ [-Wunused-variable]
    343 |     integer_t ybounds[2], osize[2];
        |               ^~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c: In function ‘do_kernel_gaussian’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c:461:15: warning: unused variable ‘ybounds’ [-Wunused-variable]
    461 |     integer_t ybounds[2], osize[2];
        |               ^~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c: In function ‘do_kernel_lanczos’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c:589:15: warning: unused variable ‘ybounds’ [-Wunused-variable]
    589 |     integer_t ybounds[2], osize[2];
        |               ^~~~~~~
  At top level:
  /home/runner/work/drizzle/drizzle/src/cdrizzlebox.c:15:13: warning: ‘buf’ defined but not used [-Wunused-variable]
     15 | static char buf[1024];
        |             ^~~
  gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include -I/home/runner/work/drizzle/drizzle/src -I/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12 -c /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c -o /tmp/tmp3_q9wi6l.build-temp/home/runner/work/drizzle/drizzle/src/cdrizzlemap.o
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c: In function ‘interpolate_point’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:108:27: warning: unused variable ‘idim’ [-Wunused-variable]
    108 |     int ipix, jpix, npix, idim;
        |                           ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:108:21: warning: unused variable ‘npix’ [-Wunused-variable]
    108 |     int ipix, jpix, npix, idim;
        |                     ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:108:15: warning: unused variable ‘jpix’ [-Wunused-variable]
    108 |     int ipix, jpix, npix, idim;
        |               ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:108:9: warning: unused variable ‘ipix’ [-Wunused-variable]
    108 |     int ipix, jpix, npix, idim;
        |         ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c: In function ‘map_point’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:201:15: warning: variable ‘status’ set but not used [-Wunused-but-set-variable]
    201 |     int i, j, status;
        |               ^~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c: In function ‘clip_polygon_to_window’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:501:16: warning: passing argument 1 of ‘orient_ccw’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
    501 |     orient_ccw(p);
        |                ^
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:438:28: note: expected ‘struct polygon *’ but argument is of type ‘const struct polygon *’
    438 | orient_ccw(struct polygon *p) {
        |            ~~~~~~~~~~~~~~~~^
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:502:16: warning: passing argument 1 of ‘orient_ccw’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
    502 |     orient_ccw(wnd);
        |                ^~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:438:28: note: expected ‘struct polygon *’ but argument is of type ‘const struct polygon *’
    438 | orient_ccw(struct polygon *p) {
        |            ~~~~~~~~~~~~~~~~^
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:495:21: warning: unused variable ‘signed_area’ [-Wunused-variable]
    495 |     double t, u, d, signed_area, app_, aww_;
        |                     ^~~~~~~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:495:15: warning: unused variable ‘u’ [-Wunused-variable]
    495 |     double t, u, d, signed_area, app_, aww_;
        |               ^
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:495:12: warning: unused variable ‘t’ [-Wunused-variable]
    495 |     double t, u, d, signed_area, app_, aww_;
        |            ^
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c: In function ‘init_image_scanner’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:911:9: warning: unused variable ‘ipoint’ [-Wunused-variable]
    911 |     int ipoint;
        |         ^~~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:910:25: warning: unused variable ‘osize’ [-Wunused-variable]
    910 |     integer_t isize[2], osize[2];
        |                         ^~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:910:15: warning: unused variable ‘isize’ [-Wunused-variable]
    910 |     integer_t isize[2], osize[2];
        |               ^~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:909:21: warning: unused variable ‘xyout’ [-Wunused-variable]
    909 |     double xyin[2], xyout[2];
        |                     ^~~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:909:12: warning: unused variable ‘xyin’ [-Wunused-variable]
    909 |     double xyin[2], xyout[2];
        |            ^~~~
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c: In function ‘map_point’:
  /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:216:1: warning: control reaches end of non-void function [-Wreturn-type]
    216 | }
        | ^
  In file included from /opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include/numpy/ndarrayobject.h:21,
                   from /opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include/numpy/arrayobject.h:5,
                   from /home/runner/work/drizzle/drizzle/src/cdrizzlemap.c:10:
  At top level:
  /opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include/numpy/__multiarray_api.h:[145](https://github.com/spacetelescope/drizzle/actions/runs/7014084246/job/19081247879?pr=128#step:4:146)8:1: warning: ‘_import_array’ defined but not used [-Wunused-function]
   1458 | _import_array(void)
        | ^~~~~~~~~~~~~
  gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include -I/home/runner/work/drizzle/drizzle/src -I/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12 -c /home/runner/work/drizzle/drizzle/src/cdrizzleutil.c -o /tmp/tmp3_q9wi6l.build-temp/home/runner/work/drizzle/drizzle/src/cdrizzleutil.o
  /home/runner/work/drizzle/drizzle/src/cdrizzleutil.c: In function ‘create_lanczos_lut’:
  /home/runner/work/drizzle/drizzle/src/cdrizzleutil.c:277:17: warning: comparison of integer expressions of different signedness: ‘integer_t’ {aka ‘int’} and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
    277 |   for (i = 1; i < npix; ++i) {
        |                 ^
  In file included from /usr/include/string.h:535,
                   from /opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12/Python.h:26,
                   from /home/runner/work/drizzle/drizzle/src/cdrizzleutil.h:5,
                   from /home/runner/work/drizzle/drizzle/src/cdrizzlemap.h:5,
                   from /home/runner/work/drizzle/drizzle/src/cdrizzleutil.c:3:
  In function ‘strncpy’,
      inlined from ‘driz_error_set_message’ at /home/runner/work/drizzle/drizzle/src/cdrizzleutil.c:39:3:
  /usr/include/x86_64-linux-gnu/bits/string_fortified.h:95:10: warning: ‘__builtin_strncpy’ specified bound 512 equals destination size [-Wstringop-truncation]
     95 |   return __builtin___strncpy_chk (__dest, __src, __len,
        |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     96 |                                   __glibc_objsize (__dest));
        |                                   ~~~~~~~~~~~~~~~~~~~~~~~~~
  gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include -I/home/runner/work/drizzle/drizzle/src -I/opt/hostedtoolcache/Python/3.12.0/x64/include/python3.12 -c /home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.c -o /tmp/tmp3_q9wi6l.build-temp/home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.o
  /home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.c: In function ‘utest_cdrizzle’:
  /home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.c:618:23: warning: unused variable ‘ybounds’ [-Wunused-variable]
    618 |             integer_t ybounds[2];       /* start of in-bounds */
        |                       ^~~~~~~
  /home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.c:642:23: warning: unused variable ‘ybounds’ [-Wunused-variable]
    642 |             integer_t ybounds[2];       /* start of in-bounds */
        |                       ^~~~~~~
  In file included from /opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include/numpy/ndarrayobject.h:21,
                   from /opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include/numpy/arrayobject.h:5,
                   from /home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.c:9:
  At top level:
  /opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/numpy/_core/include/numpy/__multiarray_api.h:1458:1: warning: ‘_import_array’ defined but not used [-Wunused-function]
   1458 | _import_array(void)
        | ^~~~~~~~~~~~~
  In file included from /usr/include/string.h:535,
                   from /home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.c:4:
  In function ‘strncpy’,
      inlined from ‘fctstr_safe_cpy’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:267:5,
      inlined from ‘fct_test_new’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:974:5:
  /usr/include/x86_64-linux-gnu/bits/string_fortified.h:95:10: warning: ‘__builtin_strncpy’ specified bound [256](https://github.com/spacetelescope/drizzle/actions/runs/7014084246/job/19081247879?pr=128#step:4:257) equals destination size [-Wstringop-truncation]
     95 |   return __builtin___strncpy_chk (__dest, __src, __len,
        |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     96 |                                   __glibc_objsize (__dest));
        |                                   ~~~~~~~~~~~~~~~~~~~~~~~~~
  In file included from /home/runner/work/drizzle/drizzle/src/tests/pandokia_fct.h:25,
                   from /home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.c:15:
  In function ‘fct_nlist__clear’,
      inlined from ‘fct_nlist__final’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:751:5,
      inlined from ‘fct_test__del’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:956:5,
      inlined from ‘fct_test__del’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:950:1,
      inlined from ‘fct_test_new’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:991:9:
  /home/runner/work/drizzle/drizzle/src/tests/fct.h:736:41: warning: ‘((fct_nlist_t *)test)[1].used_itm_num’ may be used uninitialized [-Wmaybe-uninitialized]
    736 |         for ( itm_i__=0; itm_i__ != list->used_itm_num; ++itm_i__ )
        |                                     ~~~~^~~~~~~~~~~~~~
  In file included from /usr/include/string.h:535,
                   from /home/runner/work/drizzle/drizzle/src/tests/utest_cdrizzle.c:4:
  In function ‘strncpy’,
      inlined from ‘fctstr_safe_cpy’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:[267](https://github.com/spacetelescope/drizzle/actions/runs/7014084246/job/19081247879?pr=128#step:4:268):5,
      inlined from ‘fctchk_new’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:886:5,
      inlined from ‘_fct_xchk_fn_varg’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:3581:11,
      inlined from ‘fct_xchk_fn’ at /home/runner/work/drizzle/drizzle/src/tests/fct.h:3624:9:
  /usr/include/x86_64-linux-gnu/bits/string_fortified.h:95:10: warning: ‘__builtin_strncpy’ specified bound 256 equals destination size [-Wstringop-truncation]
     95 |   return __builtin___strncpy_chk (__dest, __src, __len,
        |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     96 |                                   __glibc_objsize (__dest));
        |                                   ~~~~~~~~~~~~~~~~~~~~~~~~~

From the tox-free devdeps job in #128

Retire Python 2

Python 2 will not be maintained past Jan 1, 2020 (see https://pythonclock.org/). Please remove all Python 2 compatibility and move this package to Python 3 only.

For conda recipe (including astroconda-contrib), please include the following to prevent packaging it for Python 2 (https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html?preprocessing-selectors#skipping-builds):

build: 
  skip: true  # [py2k] 

Please close this issue if it is irrelevant to your repository. This is an automated issue. If this is opened in error, please let pllim know!

Tests cause segmentation faults

When running python -c "import drizzle; exit(drizzle.test())", I get a segmentation fault. With gdb:

drizzle/tests/test_cdrizzle.py 
Program received signal SIGSEGV, Segmentation fault.
test_cdrizzle (self=<optimized out>, args=<optimized out>)
    at drizzle/src/cdrizzleapi.c:364
364	  dat = (PyArrayObject *)PyArray_ContiguousFromAny(data, PyArray_FLOAT, 2, 2);

When I disable this test, I get another one:

drizzle/tests/test_drizzle.py 
Program received signal SIGSEGV, Segmentation fault.
tdriz (obj=<optimized out>, args=0x7fffea0fcb48, keywords=0x7fffea134910)
    at drizzle/src/cdrizzleapi.c:95
95	  img = (PyArrayObject *)PyArray_ContiguousFromAny(oimg, PyArray_FLOAT, 2, 2);

This happens for both Python 2.7 and Python 3.5.

Python test failures with version 1.13.6

While trying to package version 1.13.6 for Debian, I observed these test failures:

____________________________ test_square_with_point ____________________________

tmpdir = local('/tmp/pytest-of-pbuilder/pytest-0/test_square_with_point0')

    def test_square_with_point(tmpdir):
        """
        Test do_driz square kernel with point
        """
        output = str(tmpdir.join('output_square_point.fits'))
        output_difference = str(tmpdir.join('difference_square_point.txt'))
    
        input_file = os.path.join(DATA_DIR, 'j8bt06nyq_flt.fits')
        output_template = os.path.join(DATA_DIR, 'reference_square_point.fits')
    
        insci = read_image(input_file)
        inwcs = read_wcs(input_file)
        insci = make_point_image(insci, (500, 200), 100.0)
        inwht = np.ones(insci.shape,dtype=insci.dtype)
        output_wcs = read_wcs(output_template)
    
        driz = drizzle.Drizzle(outwcs=output_wcs, wt_scl="")
        driz.add_image(insci, inwcs, inwht=inwht)
    
        if ok:
            driz.write(output_template)
        else:
            driz.write(output)
            template_data = read_image(output_template)
    
            min_diff, med_diff, max_diff = centroid_statistics("square with point",
                                                               output_difference,
                                                               driz.outsci,
                                                               template_data, 20.0, 8)
    
>           assert med_diff < 1.0e-6
E           assert 0.05702659328654338 < 1e-06

drizzle/tests/test_drizzle.py:251: AssertionError

The same failure happens with test_square_with_grid.

Environment is the "usual" Debian unstable on x86_64:

  • Python 3.10.6
  • Astropy 5.1
  • Numpy 1.21.5

Return values from cdrizzle.tdriz make no sense

Since issue #1 was closed, I am opening a new issue here. It seems that the problems described in issue #1 have been addressed in the sense that 1) I cannot get it to crash (at least not yet); and 2) there are expected number of pixels != 0 in the output image. However, the number of pixels reported as "missing" and the number of skipped lines (second and third values in the returned tuple) make no sense:

>>> import numpy as np
>>> from drizzle import cdrizzle
>>> inmask=np.zeros((100,100),dtype=np.float32)
>>> inmask[22:25,33:49]=1
>>> inwht=np.ones_like(inmask,dtype=np.float32)
>>> outsci=np.zeros((256,256),dtype=np.float32)
>>> outwht=np.zeros((256,256),dtype=np.float32)
>>> outctx=np.zeros((256,256),dtype=np.int32)
>>> in_y, in_x = np.indices((100,100), dtype=np.float64)
>>> pixmap = np.dstack((in_y,in_x))
>>> cdrizzle.tdriz(inmask,inwht,pixmap,outsci,outwht,outctx,1,0,0,0,0,1,1,'square','cps',1,1,'-1')
('Callable C-based DRIZZLE Version 0.9 (2nd July 2015)', 55536, 156)
>>> np.any(outsci>0)
True
>>> np.sum(outsci>0)
48

First of all, I am drizzling 100x100 image onto a 256x256 output image using a mapping represented by an identity transformation (so pixel [11,15] should be mapped directly to pixel [11,15] in the output image).

Second, there are only 100 "complete lines" in my input image. How can tdriz report that it could not map 156 lines from my input image???

Third, the number of missed pixels (second value in the tuple) also does not make sense. There should not be any missed pixels at all (due to this specific mapping used in my example). To verify that tdriz indeed does not miss anything, I changed the fillval to '0' and set my input array to ones:

>>> inmask=np.ones((100,100),dtype=np.float32)
>>> outsci=np.zeros((256,256),dtype=np.float32)
>>> outwht=np.zeros((256,256),dtype=np.float32)
>>> outctx=np.zeros((256,256),dtype=np.int32)
>>> cdrizzle.tdriz(inmask,inwht,pixmap,outsci,outwht,outctx,1,0,0,0,0,1,1,'square','cps',1,1,'0')
('Callable C-based DRIZZLE Version 0.9 (2nd July 2015)', 55536, 156)
>>> np.sum(outsci)
10000.0

So, "flux" in the output image is, as expected, 10000=100x100. So, how come I have missed 55536 pixels from my input image?

TST: Clean up Travis CI matrix

I think the current Travis CI matrix was copied blindly from an old Astropy project template. It is not optimized for testing this package and needs some serious clean-up.

One example: Cron jobs are defined but it is not actually set up to run in the CI service.

cdrizzle.tdriz crashes with segmentation fault and when it "works" the drizzled images have 'fillval' only

I created a simple test of drizzling a very simple image onto a larger image. The coordinate transformation is an identity transformation so the input image should be in a quadrant of the output image. The code that crashes tdriz:

>>> import numpy as np
>>> from drizzle import cdrizzle
>>> inmask=np.zeros((100,100),dtype=np.float32)
>>> inmask[22:25,33:49]=1
>>> inwht=np.ones_like(inmask,dtype=np.float32)
>>> outsci=np.zeros((256,256),dtype=np.float32)
>>> outwht=np.zeros((256,256),dtype=np.float32)
>>> outctx=np.zeros((256,256),dtype=np.int32)
>>> in_y, in_x = np.indices((100,100), dtype=np.float)
>>> pixmap = np.dstack((in_y,in_x))
>>> cdrizzle.tdriz(inmask,inwht,pixmap,outsci,outwht,outctx,
... 1,0,0,0,0,1,1,'square','cps',1,1,'-1')
('Callable C-based DRIZZLE Version 0.9 (10th May 2014)', 65536, 156)

>>> # second try with identical parameters:
>>> import numpy as np
>>> from drizzle import cdrizzle
>>> inmask=np.zeros((100,100),dtype=np.float32)
>>> inmask[22:25,33:49]=1
>>> inwht=np.ones_like(inmask,dtype=np.float32)
>>> outsci=np.zeros((256,256),dtype=np.float32)
>>> outwht=np.zeros((256,256),dtype=np.float32)
>>> outctx=np.zeros((256,256),dtype=np.int32)
>>> in_y, in_x = np.indices((100,100), dtype=np.float64)
>>> pixmap = np.dstack((in_y,in_x))
>>> cdrizzle.tdriz(inmask,inwht,pixmap,outsci,outwht,outctx,
... 1,0,0,0,0,1,1,'square','cps',1,1,'-1')
Segmentation fault

In the first run, even though tdriz did not crash, it did "miss" all of the data points (which should not happen for an identity coordinate transformation):

>>> np.any(outsci>0)
False
>>> np.sum(outsci>0)
0

The old code from DrizzlePac works as expected:

>>> from drizzlepac import cdriz
>>> import numpy as np
>>> inmask=np.zeros((100,100),dtype=np.float32)
>>> inmask[22:25,33:49]=1
>>> inwht=np.ones_like(inmask,dtype=np.float32)
>>> outsci=np.zeros((256,256),dtype=np.float32)
>>> outwht=np.zeros((256,256),dtype=np.float32)
>>> outctx=np.zeros((256,256),dtype=np.int32)
>>> cdriz.tdriz(inmask,inwht,outsci,outwht,outctx,0,0,1,1,inmask.shape[0],
... 1.0,1.0,1.0,'center',1.0,'square','cps',1,1,'-1',0,0,1,lambda x,y:[np.asarray(x),np.asarray(y)])
('Callable C-based DRIZZLE Version 0.8 (20th May 2009)', 0, 0) # <-- nothing missed!
>>> np.any(outsci>0)
True
>>> np.sum(outsci>0)
48

Values of nan in pixmap not properly interpreted

This code relies on the input pixmap to define where the input pixels should be placed in the output frame. However, some input images have regions (at the edges) which are not defined and have values of 'nan' in the input image. This results in output positions of 'nan' in the pixmap since the input pixels are not intended to be mapped to any output pixel. Unfortunately, the pixmap values for each row are used to set the limits for many of the loops used to drizzle the input pixels to the output frame and values of 'nan' cause drizzle to completely skip all the data in those rows.

The drizzle code should have some logic integrated into it to ignore values of 'nan' as returned by the pixmap array. The code should also treat those pixels as 'skipped' in the algorithm.

The available workaround is to insure that the input pixmap does NOT contain any 'nan' values so that the current drizzle code does not get confused and skip over all the input data.

Warning during build due to usage of dash-separated keys in pyproject.toml

I noticed a few warning in the failed build log posted (the failure appears unrelated to the issue reported here):
spacetelescope/jwst#7721

      ********************************************************************************
      Usage of dash-separated 'upload-dir' will not be supported in future
      versions. Please use the underscore name 'upload_dir' instead.

      By 2023-Sep-26, you need to update your project and remove deprecated calls
      or your builds will no longer be supported.

      See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
      ********************************************************************************

I'm unable to find a migration guide but I think this should be fixed by switching the key separators from - to _. I tried to test this locally but don't see the same errors. Perhaps @zacharyburnett knows about this deprecation and which keys need to be updated before they result in build failures in September.

Segfault in drizzle 1.14.x

I'm seeing a very consistent segfault in drizzle for all 1.14.x versions, which also means it happens for all jwst 1.12.x versions as well, which require this version of drizzle. I.e. the current release. To reproduce:

from astroquery.mast import Observations
from jwst.step import ResampleStep


filename = "jw01345002001_14201_00001_nrcalong_cal.fits"
uri = f"mast:JWST/product/{filename}"
Observations.download_file(uri)

result = ResampleStep.call(filename)

and the result:

2023-09-28 16:26:11,835 - stpipe.ResampleStep - INFO - ResampleStep instance created.
2023-09-28 16:26:12,042 - stpipe.ResampleStep - INFO - Step ResampleStep running with args ('jw01345002001_14201_00001_nrcalong_cal.fits',).
2023-09-28 16:26:12,043 - stpipe.ResampleStep - INFO - Step ResampleStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': True, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '', 'pixfrac': 1.0, 'kernel': 'square', 'fillval': 'INDEF', 'weight_type': 'ivm', 'output_shape': None, 'crpix': None, 'crval': None, 'rotation': None, 'pixel_scale_ratio': 1.0, 'pixel_scale': None, 'output_wcs': '', 'single': False, 'blendheaders': True, 'allowed_memory': None, 'in_memory': True}
2023-09-28 16:26:12,309 - stpipe.ResampleStep - INFO - Using drizpars reference file: /data/beegfs/astro-storage/groups/jwst/common/crds_cache/jwst_ops/references/jwst/nircam/jwst_nircam_drizpars_0001.fits
2023-09-28 16:26:12,341 - stpipe.ResampleStep - INFO - Driz parameter kernel: square
2023-09-28 16:26:12,341 - stpipe.ResampleStep - INFO - Driz parameter pixfrac: 1.0
2023-09-28 16:26:12,341 - stpipe.ResampleStep - INFO - Driz parameter fillval: INDEF
2023-09-28 16:26:12,341 - stpipe.ResampleStep - INFO - Driz parameter weight_type: ivm
2023-09-28 16:26:12,342 - stpipe.ResampleStep - INFO - Output pixel scale ratio: 1.0
2023-09-28 16:26:12,488 - stpipe.ResampleStep - INFO - Resampling science data
2023-09-28 16:26:15,795 - stpipe.ResampleStep - INFO - Drizzling (2048, 2048) --> (2068, 2075)
Segmentation fault

It is segfaulting on CentOS Linux release 7.9.2009 (Core), and the system info of the machine:

In [1]: import platform; print(platform.platform())
   ...: import sys; print("Python", sys.version)
   ...: import astropy; print("astropy", astropy.__version__)
   ...: import numpy; print("Numpy", numpy.__version__)
   ...: import drizzle; print("drizzle", drizzle.__version__)
Linux-3.10.0-1160.76.1.el7.x86_64-x86_64-with-glibc2.17
Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0]
astropy 5.3.3
Numpy 1.25.2
drizzle 1.14.2

It runs fine on the following (no segfault) on the same machine with drizzle 1.13.7:

In [1]: import platform; print(platform.platform())
   ...: import sys; print("Python", sys.version)
   ...: import astropy; print("astropy", astropy.__version__)
   ...: import numpy; print("Numpy", numpy.__version__)
   ...: import drizzle; print("drizzle", drizzle.__version__)
Linux-3.10.0-1160.76.1.el7.x86_64-x86_64-with-glibc2.17
Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0]
astropy 5.3.3
Numpy 1.25.2
drizzle 1.13.7

No segfaults at all on my Mac laptop.

ERROR: Could not build wheels for drizzle which use PEP 517 and cannot be installed directly

I came across the following error when trying to install with pip version 21.2.4.


pip install drizzle
Collecting drizzle
  Downloading drizzle-1.13.3.tar.gz (87 kB)
     |████████████████████████████████| 87 kB 2.2 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done   
Requirement already satisfied: numpy in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from drizzle) (1.21.2)
Requirement already satisfied: astropy in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from drizzle) (4.3.1)
Requirement already satisfied: pyerfa>=1.7.3 in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from astropy->drizzle) (2.0.0)
Requirement already satisfied: importlib-metadata in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from astropy->drizzle) (4.7.1)
Requirement already satisfied: typing-extensions>=3.6.4 in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from importlib-metadata->astropy->drizzle)
 (3.10.0.0)
Requirement already satisfied: zipp>=0.5 in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from importlib-metadata->astropy->drizzle) (3.5.0)
Building wheels for collected packages: drizzle
  WARNING: Building wheel for drizzle failed: [Errno 13] Permission denied: '/home/anonymous/.cache/pip/wheels/e3/77'
Failed to build drizzle
ERROR: Could not build wheels for drizzle which use PEP 517 and cannot be installed directly

For anyone else in the same boat, downgrading pip made the error go away.

pip install pip==18.1
Collecting pip==18.1
  Downloading pip-18.1-py2.py3-none-any.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 2.0 MB/s
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.2.4
    Uninstalling pip-21.2.4:
      Successfully uninstalled pip-21.2.4
Successfully installed pip-18.1
(miragePy3p7Again) [anonymous@fenrir ~]$ pip install drizzle
Collecting drizzle
  Downloading https://files.pythonhosted.org/packages/50/55/9251a901523e6ca9b6f4644b9165536f266617cd4d926edca5a6822ee320/drizzle-1.13.3.tar.gz (87kB)
    100% |████████████████████████████████| 92kB 2.5MB/s
  Installing build dependencies ... done
Requirement already satisfied: numpy in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from drizzle) (1.21.2)
Requirement already satisfied: astropy in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from drizzle) (4.3.1)
Requirement already satisfied: importlib-metadata; python_version == "3.7" in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from astropy->drizzle) (4.7.1)
Requirement already satisfied: pyerfa>=1.7.3 in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from astropy->drizzle) (2.0.0)
Requirement already satisfied: typing-extensions>=3.6.4; python_version < "3.8" in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from importlib-metadata; python_version == "3.7"->astropy->drizzle) (3.10.0.0)
Requirement already satisfied: zipp>=0.5 in ./miniconda3/envs/miragePy3p7Again/lib/python3.7/site-packages (from importlib-metadata; python_version == "3.7"->astropy->drizzle) (3.5.0)
Building wheels for collected packages: drizzle
  Running setup.py bdist_wheel for drizzle ... done
  Stored in directory: /home/anonymous/.cache/pip/wheels/18/26/01/2cb22d6714ea3223b076166fc8a5a66ed146a008c350370e31
Successfully built drizzle
Installing collected packages: drizzle  
Successfully installed drizzle-1.13.3   

Fix long_description_content_type

$ twine check --strict dist/*
Checking dist/drizzle-1.13.2-cp39-cp39-macosx_10_9_x86_64.whl: FAILED, due to warnings
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.

How to incorporate masking when drizzling?

Hi, how can masking be incorporated when drizzling a set of images? I am using drizzle to do templates for difference imaging from ground-based images, as it simplifies a lot of steps, but I do not know how to appropriately incorporate masking of cosmic rays and detector artifacts. I was thinking on just drizzle each individual exposure and then mask and average, but probably there is a more straightforward way to do it.

Python 3.5.2 failure (ConfigParser again)

Build result:

BUILD START: drizzle-0.2.12.post1.dev1-np111py35_0
Fetching package metadata: ........
Solving package specifications: ..................................Initialized empty Git repository in /srv/iraf/bldtmp/porcelain.t5ICvF6y33/porcelain/conda-bld/git_cache/__github.com_spacetelescope_drizzle.git/
Initialized empty Git repository in /srv/iraf/bldtmp/porcelain.t5ICvF6y33/porcelain/conda-bld/work/.git/
Submodule 'astropy_helpers' (https://github.com/astropy/astropy-helpers.git) registered for path 'astropy_helpers'
Initialized empty Git repository in /srv/iraf/bldtmp/porcelain.t5ICvF6y33/porcelain/conda-bld/work/astropy_helpers/.git/
Submodule path 'astropy_helpers': checked out '5fd32d0edc34f94de9640fd20865cfe5d605e499'
+ python setup.py install
Traceback (most recent call last):
  File "setup.py", line 25, in <module>
    conf = config.ConfigParser()
AttributeError: module 'distutils.config' has no attribute 'ConfigParser'

The Python devs replaced distutils.config.ConfigParser with distutils.config.RawConfigParser in the latest release of Python 3k. Also, I'm pretty sure this bug is being handled properly in the latest release of the astropy packaging template by deprecating the use of distutils.config for configuration management altogether.

Example fixes

https://github.com/astropy/photutils/blob/master/setup.py#L24
https://github.com/spacetelescope/gwcs/blob/master/setup.py#L24

Remove test files from distributed package

Currently the test FITS files are included in the MANIFEST, which means that the pip install size is 47MB. It should be less than 1MB. Also, the test files don't need to be 11MB in size.

rey> pip install drizzle --no-cache-dir
Collecting drizzle
  Downloading https://files.pythonhosted.org/packages/63/a8/d73561940986e75b3874e35d9353d1be5734d62daefbff912d53850bd98d/drizzle-1.12.tar.gz (47.0MB)
Requirement already satisfied: astropy in /Users/jdavies/miniconda3/envs/test/lib/python3.7/site-packages (from drizzle) (3.1.2)
Requirement already satisfied: numpy>=1.13.0 in /Users/jdavies/miniconda3/envs/test/lib/python3.7/site-packages (from astropy->drizzle) (1.16.2)
Installing collected packages: drizzle
  Running setup.py install for drizzle ... done
Successfully installed drizzle-1.12

Maybe move the tests out of the distributed package too.

Missing imports in __init__.py

With the current init.py this package does not properly work for python 3.11 (in jupyter lab).
A line stating

from . import subpackage_name

is missing for each subpackage.

Blocked by

Fix internal handling of floats to be 64-bit

Currently we are seeing round-off errors that mean drizzle results can vary depending on platform and whether OpenBLAS or MKL are used on the machine. Internally, some computations are done using 32-bit FLOAT. These should be 64-bit DOUBLE.

Hopefully this will fix the round-off issues we are seeing.

Should come up with a minimally-reproducible example, though this is difficult due to platform dependency.

Simple Example Not Behaving As Expected

I created a simple example which demonstrates some unexpected behavior. This example does the following:

  1. Create a 101x101 image covered in 1s, except for the central pixel (51, 51 in 1-indexed coordinates) which is 10, with values treated as counts per second. The frame has an image scale of 1 arcsec/px and is rotated 45 deg, with (51, 51) corresponding to RA = Dec = 0.

  2. Use Drizzle to map the first image onto a new 1001x1001 frame at 0.1 arcsec/px, with north=up, and (501, 501) at RA = Dec = 0.

  3. Save the 101x101 image to in.fits, and the 1001x1001 image to out.fits.

Here's the code:

from astropy.io import fits
from astropy.wcs import WCS
from drizzle.drizzle import Drizzle
import numpy as np

img_in = np.ones([101, 101])
img_in[50, 50] = 10

w_in = WCS(naxis=2)
w_in.wcs.ctype = ['RA---TAN', 'DEC--TAN']
w_in.wcs.cdelt = [-1/3600, 1/3600]
w_in.wcs.crota = [45, 45]
w_in.wcs.crpix = [51, 51]
w_in.wcs.crval = [0, 0]
w_in.pixel_shape = [101, 101]

f_in = fits.PrimaryHDU(img_in, header=w_in.to_header())
f_in.writeto('in.fits', overwrite=True)

w_out = WCS(naxis=2)
w_out.wcs.ctype = ['RA---TAN', 'DEC--TAN']
w_out.wcs.cdelt = [-0.1/3600, 0.1/3600]
w_out.wcs.crota = [0, 0]
w_out.wcs.crpix = [501, 501]
w_out.wcs.crval = [0, 0]
w_out.pixel_shape = [1001, 1001]

driz = Drizzle(outwcs=w_out)
driz.add_image(insci=img_in, inwcs=w_in, in_units='cps')
f_out = fits.PrimaryHDU(driz.outsci, header=w_out.to_header())
f_out.writeto('out.fits', overwrite=True)

The expected output of this example (out.fits) is a 1001x1001 square with surface brightness 0.01 rotated 45 deg (with corners cut off), and a 10x10 square of surface brightness 0.1 also rotated 45 deg centered on (501, 501)---in other words, essentially the original scaled up by a factor of 10 and rotated about the center by 45 deg.

The actual output deviates in two respects:

  1. The upper right part of the big square is cut off (zero, as though it's outside the original image, which it's not). A workaround is to set w_out.pixel_shape = [2002, 2002], then crop driz.outsci to driz.outsci[:1001, :1001], which gives the full rotated square (minus the corners, as expected). (A more extreme example: if instead of 45 deg, the input image has crota of 180 deg, out.fits is completely blank)

  2. The small square is not centered at (501, 501). Instead, (501, 501) is at one edge of the square. (At input crota of 30 deg and 150 deg, it's closer to the center of the small square; at input crota of 90 deg, it's at the corner of the square)

I attached a zip file containing both the in.fits and out.fits that this example produces.

I've tested the example with drizzle 1.13.1 on two different computers (both amd64 architecture):

  1. Debian 10, Python 3.7.3, numpy 1.18.1, astropy 3.1.2
  2. RHEL 7.6, Python 3.6.5, numpy 1.18.2, astropy 3.0.3

Both appear to produce identical results, as described.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.