schwarzam / astrors Goto Github PK

• version: "0.1.7"
• Operating System: Windows 11

Description

I've been recently transitioning my current all python library into a rust + python mixed library, and have been searching for a solution to read and write FITS files. I am working with data collected from Synchrotron X-ray sources that use the FITS file format. I discovered that there are some header cards that are inaccessible using the astrors::io::Header.get_card() method.

What I Did

I attached a zip file.zip with a jupyter notebook outlining what I was playing around with and the fits file that I have been using.

There are two cards I need access to, but are somehow missing, "Beamline Current" and "Sample Theta". Using the pretty print method, I can display them

let mut testfile = File::open("~/testing/ZnPc82862-00001.fits")?;

let mut header = io::Header::new();
header.read_from_file(&mut testfile).unwrap();

header.pretty_print_advanced();

This prints out all 111 cards in the fits file and importantly shows the cards I am looking for

...
----------------------------------------
Keyword: Beam Current 
Value: 499.725799560547
Comment: Analog
----------------------------------------
...
----------------------------------------
Keyword: Sample Theta 
Value: 0
Comment: [Degrees] Motor
----------------------------------------
...

However, using either get_card() or the method under the hood of iter().find(|card| card.keyword == "Sample Theta"),

println!("━━━━━━━━━━━━━ Using `get_card()` ━━━━━━━━━━━━━");

println!("Using get_card(Beamline Current) {:?}", header.get_card("Beamline Current"));
println!("Using get_card(Sample Theta) {:?}", header.get_card("Sample Theta"));

println!("━━━━━━━━━━━━━ Using `iter().find()` ━━━━━━━━━━━━━");

println!("Using iter().find(Beamline Current) {:?}", header.iter().find(|card| card.keyword == "Beamline Current"));
println!("Using iter().find(Sample Theta) {:?}", header.iter().find(|card| card.keyword == "Sample Theta"));

I cannot find the cards I am looking for

━━━━━━━━━━━━━ Using `get_card()` ━━━━━━━━━━━━━
Using get_card(Beamline Current) None
Using get_card(Sample Theta) None
━━━━━━━━━━━━━ Using `iter().find()` ━━━━━━━━━━━━━
Using iter().find(Beamline Current) None
Using iter().find(Sample Theta) None

Not sure what is going on here or what the best fix would be. But your crate is so far wonderful aside from this annoying bug.... maybe it is not a bug, and I am showing how new I am at rust lol.

Each HDU has a correct order and mandatory keywords

There should be a better way to parse the header instead of going through blocks of 2880 bytes.

There are two problems:

• If the header does not have a END card it goes on the loop until the end of the buffer (than I think it raises an error).
• When it reaches 35 on the counter, it adds the card to the header, if the card is a CONTINUE card, this will raise unknown problems.

Snippets with suggestions:

let mut buffer= [0; 2880];
let n = file.read(&mut buffer[..])?;
let mut last_card : Option<Card> = None;

let mut counter = 0; // This is not right!
for card in buffer.chunks(80) { // This for should be replaced by a loop. 
    let card_str = String::from_utf8_lossy(card).trim_end().to_string();

else {
    last_card = Card::parse_card(card_str);
    
    if (counter) == (2880 / 80) - 1 { // Remove this if block
        self.add_card(last_card);
        break;
    }
}
counter += 1; // Remove

Images and Tables should only be loaded into memory in the background.

It seeems that the writing of string columns is wrong.
Needs to compare with a real fits.

In the function fill_columns_w_data, the buffer of the file should be read once, and not on every iteration.
This causes the code to run much slower.

The compression needs a lot of understanding.

There are concepts such as:

• (un)quantize
• types of compressions
• header keys

CFITSIO adapted files

Implement a way to insert after another card like after="SIMPLE", or at first or end of header card list.

Suggestion, modify function below:

pub fn add_card(&mut self, card: Card) {
        self.cards.push(card);
}

What is the bscale and bzero from the header?

Imagine you have a FITS file with an image, and you read the header to find BSCALE = 1.5 and BZERO = 100.0. The stored pixel values are integers, but the actual physical values might be floating-point numbers.

When you read a pixel value from this file, say 400, the actual physical value represented by this pixel would be calculated as:

Physical Value = 1.5 × 400 + 100.0 = 700.0

Reading FITS Files in Practice
When reading FITS files programmatically (e.g., using Python with libraries like astropy.io.fits), these libraries typically handle the BZERO and BSCALE conversion for you. They read the pixel data, apply the scale and zero-point, and return the data in its physical form.

If you are handling the FITS file at a low level (i.e., reading raw bytes), you need to manually apply these transformations after reading the pixel values based on the BZERO and BSCALE values found in the file's header.

Header and Card setters and getters should have a template type, because they may be one the types below and not only string.

pub enum TYPE{
    INT,
    FLOAT,
    STRING, 
    LOGICAL
}

fn check_type(s: &str) -> TYPE {
    if s.parse::<i64>().is_ok() {
        TYPE::INT
    } else if s.parse::<f64>().is_ok() {
        TYPE::FLOAT
    } else if s == "T" || s == "F" {
        TYPE::LOGICAL
    } else {
        TYPE::STRING
    }
}

This is for example wrong:

impl Card {
    // Function to set the value and take ownership of the passed variable
    pub fn set_value(&mut self, new_value: String) {
        self.value = Some(new_value);
    }

Also this on header (maybe this should be removed):

pub fn get_value(&self, keyword: &str) -> Result<&str, Error> {
        self.cards.iter().find(|card| card.keyword == keyword).map_or(
            Err(Error::new(ErrorKind::Other, format!("{} keyword not found", keyword))),
            |card| {
                match &card.value {
                    Some(value) => Ok(value),
                    None => Err(Error::new(ErrorKind::Other, format!("{} keyword has no value", keyword))),
                }
            }
        )
    }