Giter Club home page Giter Club logo

isoclassify's Introduction

Isochrone Stellar Classification Codes

DOI

Python codes to perform simple stellar classifications given a set of input observables. The code is based on the work described in Huber et al. 2017, Berger et al. 2020 and Berger et al. 2023 - please cite these papers when using the code. Additional references to be cited depending on the use-case:

Installation

  1. Download mwdust (see https://github.com/jobovy/mwdust).

  2. pip install the code:

    pip install isoclassify
  3. If the install succeeded,

    isoclassify -h

    should output

    usage: isoclassify [-h] {run,batch,multiproc,scrape-output} ...
    
    optional arguments:
      -h, --help            show this help message and exit
    
    subcommands:
      {run,batch,multiproc,scrape-output}
        run                 run isoclassify
    
  4. Optional: Set an environment variable for a path to store the MESA models downloaded in step 5. Otherwise, skip this step and use the default location (~/.isoclassify).

    export ISOCLASSIFY=/path/to/data/dir
  5. Download models into the ~/.isoclassify directory created upon installation or the directory specified in step 4.

    Bolometric correction grid:

    https://drive.google.com/file/d/1vbgKqrghAWJrDJnRBLMlv--J8VSVUOpi/view?usp=sharing

    MESA models:

    https://drive.google.com/drive/folders/1GC81YxBvMF2wu4TdL0vItvdYv-zsaLI1?usp=sharing

    Parsec models:

    https://drive.google.com/file/d/1gURKVsL5jPxWZ08ipzALXiUnX7lsjoGe/view?usp=sharing

Usage

Grid Modeling:

Derives posterior distributions for stellar parameters (teff, logg, feh, rad, mass, rho, lum, age, distance, av) through direct intregration of isochrones, given any set of observables (photometry, spectroscopy, parallax, asteroseismology) and priors. Can fit for extinction or use reddening map, depending on input parameters. The Parsec grid has been amended for M dwarfs using empirical relations by Mann et al. 2019; see Berger et al. 2023 for details.

Direct Method:

Uses bolometric corrections and extinction maps to derive stellar parameters using direct physical relations. Masses and densities are only calculated for cool stars within the empirical relations by Mann et al. 2019 (to derive masses for any star, use grid modeling mode described above). Options are:

  1. RA/DEC + Photometry + (Spectroscopy) + Parallax -> Teff, R, L, distance, Av (+ M & rho for cool stars). Uses Monte-Carlo method to implement exponentially decreasing volume density prior with a length scale of 1.35 kpc (Astraatmadja & Bailer-Jones 2016)

  2. RA/DEC + Asteroseismology + Spectroscopy -> logg, rho, rad, mass, lum, distance, Av.

Bolometric corrections are interpolated in (Teff, logg, FeH, Av) from the MIST grid (http://waps.cfa.harvard.edu/MIST/model_grids.html)

List of Input Parameters:

Parameters without uncertainties:

ra ... Right Ascension (degrees)
dec ... Declination (degrees)
band ... Photometric band to use for absolute magnitude (see nomenclature below)
dust ... Extinction map to use (none,allsky,green19)
grid ... Stellar evolution grid to use (mesa,parsec)
dmag ... Contrast for secondary star (mag). Must be in same bandpass as "band"

Parameters with uncertainties (e.g. plx_err):

plx ... Parallax (arcseconds)
teff ... Effective Temperature (K)
logg ... Surface Gravity (cgs)
feh ... Metallicity (dex)
lum ... Luminosity (solar)
bmag ... Johnson B (mag)
vmag ... Johnson V (mag)
btmag ... Tycho B (mag)
vtmag ... Tycho V (mag)
gmag ... Sloan g (mag)
rmag ... Sloan r (mag)
imag ... Sloan i (mag)
zmag ... Sloan z (mag)
gamag ... Gaia G (mag)
bpmag ... Gaia Bp (mag)
rpmag ... Gaia Rp (mag)
jmag ... 2MASS J (mag)
hmag ... 2MASS H (mag)
kmag ... 2MASS Ks (mag)
numax ... Frequency of maximum power (muHz)
dnu ... Asteroseismic frequency of maximum power (muHz)

Command line interface

The preferred mode to run isoclassify is through a command line interface (CLI), which allows single star processing as well as batch processing of many stars.

isoclassify run <mode> <star name> --csv <csv file> --outdir <output directory> --plot <plotmode> 
  1. <mode> direct or grid
  2. <star name> tells isoclassify which row of to pull the input parameters
  3. <output directory> will contain the isoclassify output files
  4. <csv file> contains as columns parameters that are passed to isoclassify
  5. <plotmode> tells isoclassify whether pop interactive plots show, save to a png save-png, or to not plot at all none.

The file examples/example.csv includes a few example stars that cover various use cases, e.g.:

mkdir -p output/sol # make sure output directory exists
isoclassify run grid sol --csv examples/example.csv --outdir output/sol --plot show

The CLI also makes parallel processing easy. To run all stars in the example.csv file:

# generate batch scripts
isoclassify batch direct examples/example.csv -o output > isoclassify.tot 

# make executable and run script
chmod +x isoclassify.tot
./isoclassify.tot

# Alternatively, run with GNU parallel to use multiple cores
parallel :::: isoclassify.tot

# Combine outputs into one CSV file
isoclassify scrape-output 'output/*/output.csv' output.csv

You can also run python-based multiprocessing through joblib and memory-mapping to reduce RAM overhead.

isoclassify multiproc <mode> <num processes> <input csv> <output csv> --baseoutdir <base output directory> --plot <plot mode>
  1. <mode> direct or grid
  2. <num processes> number of processes to run at a time. -1 uses all processors, including hyperthreading for machines that have it (num cores * 2). -1 should be used with caution depending upon the # of processors and RAM in your machine, as a single process can easily consume ~4GB of RAM by itself
  3. <input csv> location and name of csv file containing all stars you want to run
  4. <output csv> location and name of csv file, scraping all results from folders in
  5. <base output directory> location of individual stellar output folders (default = ./)
  6. <plot mode> tells isoclassify whether to save to a png save-png (default), save to a pdf save-pdf, or to not plot at all none. Does not use show as above due to issues with multiprocessing functionality.

This will run all stars within the designated csv file and produce an output csv file, so there no need to use the scrape-output function detailed above in combination with this script.

Note: isoclassify is not natively setup to run in ipython notebooks or be imported with other scripts. The example directory includes notebooks constructed by hacking pre-defined functions that allows direct interaction with posteriors. The notebooks are provided for guidance only and are not supported with the latest version of the code.

Typical Use Case

In the Gaia era, a typical use-case for isoclassify is the derivation of mass, radius, luminosity, density and age for a single star with existing constraints on Teff, log(g) metallicity from spectroscopy, a parallax from Gaia, and photometry from various surveys (Gaia, 2MASS, etc). The recommended procedure is to first use direct-mode to determine the luminosity directly through the bolometric flux and distance. Using pi Men in the example.csv file, this would look like:

# make sure output directory exists
mkdir -p output/piMendirect 
isoclassify run direct piMendirect --csv examples/example.csv --outdir output/piMendirect --plot show

Here, the bolometric flux is estimated through 2MASS K-band (less sensitive to extinction) and the bolometric correction grid.

Using the luminosity from the direct method as an input to grid-mode, together with the spectroscopic Teff and metallicity, we can then derive the remaining parameters (log(g) is not needed as input, since spectroscopic constraints are less reliable than the evolutionary state constraint from the Gaia luminosity):

# make sure output directory exists
mkdir -p output/piMengrid 
isoclassify run grid piMengrid --csv examples/example.csv --outdir output/piMengrid --plot show

Note that the uncertainties reported by isoclassify are precisions intrinsic to the grid that is used. When comparing these values to other methods, systematic errors should be considered. These can be estimated by running the same star through both grids (mesa and parsec) or following the guidance described in Tayar et al. 2022.

isoclassify's People

Contributors

alexlyttle avatar danxhuber avatar petigura avatar taberger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

isoclassify's Issues

Using Gaia Bp-Rp to derive temperature in grid mode

When using isoclassify with Gaia magnitude band inputs, I'm having an issue figuring out how to (or if it is currently possible to) derive temperature constraints using Bp-Rp while in grid mode.

I tried adding the following code into the isoclassify/grid/classify.py file, as well as several other small tweaks to include Bpmag and Rpmag inputs:

if ((input.bpmag > -99.0) & (input.rpmag > -99.0)):

ut = np.where((mod['bpmag'] - mod['rpmag'] > bprpcol - sigbprpcole) & (mod['bpmag'] - mod['rpmag'] < bprpcol + sigbprpcole))
ut = ut[0]
um = np.intersect1d(um,ut)

However, I discovered the larger issue which was that the MESA models in the mesa.h5 file don't seem to contain modeled stellar data for Bpmag or Rpmag, so there is nothing to reference when Bpmag and Rpmag inputs are give.

Is there possibly some sort of workaround or small change that can be made to allow for this color derivation from Gaia bands? It is also possible that I've severely overlooked something. Any assistance would be much appreciated. Thanks!

Mac OS - OSError: Unable to open file (unable to open file)

When running the example code from the README, I get to:
File "h5py/h5f.pyx", line 88, in h5py.h5f.open
Before it says:
OSError: Unable to open file (unable to open file)

I have tried everything I can think of to fix this error. Have you seen this before / do you have any suggestions?

add script to combine CLI output?

Hi @petigura - i've started using the CLI and it's awesome! It'd be nice to add a script that combines the csv output from individual folders into a single csv file using the parameters are column headers. Does something like this already exist?

Timeout when doing many bayestar queries with green18 dustmap

Hey, @danxhuber and @taberger.

Have you guys tried running isoclassify with the CLI in parallel? I've got 12,000 stars I'd like to run through and the following command should ship a whole bunch of commands off to the CLI.

cat isoclassify.tot | grep mkdir | parallel

When I do this I get a whole bunch of 429 errors from running the bayestar web query.

isoclassify/cks1/direct/K00174/output.log:requests.exceptions.HTTPError: 429 Client Error: TOO MANY REQUESTS for url: http://argonaut.skymaps.info/api/v2/bayestar2017/query

Have you seen this too? I could just reduce the parallelism and hope that the bayestar server doesn't return 429. Any other suggestions. It looks you guys have implemented an 'allsky' method which uses mwdust.Combined15() that's preloaded on the Kepler field. Is this an effective workaround for the Kepler field?

Parameter d51mag does not exist in mesa.h5 file

Hello,
When trying to run the grid.ipynb example script, the script errors out on line 28 of the 3rd cell:
'd51mag':np.array(file['d51mag']),\
With an error stating:
KeyError: "Unable to open object (object 'd51mag' doesn't exist)"
In fact, when I check the keys in mesa.h5, d51mag is not one of them. I have downloaded the mesa.h5 file from dropbox as indicated the readme file, and have tried re-downloading in case the file got corrupted. I also notice that 'lum' is also a field in 'model' which is not a key in the mesa.h5 file. However, it is set to the same value as 'rad' on lines 10 and 11, and so the script never tries to read a key 'lum' as it does for the key 'dmag51'.
Is there a more updated mesa.h5 file that the grid.ipynb script requires?
Thanks very much for your time.

Implement diffusion grid

The current code does not return solar M and R for solar input values since mesa.ebf does not use time-dependent [Fe/H] accounting for diffusion. @taberger is working on a solution.

Squeeze keyword preventing the full output.csv file from generating

Hello! I am running the code on the example file to test the ability to combine into a csv of output and am running into this error:

`bin/isoclassify scrape-output 'output/*/output.csv' output.csv
0
/opt/anaconda3/envs/obliquity/lib/python3.10/site-packages/isoclassify/pipeline.py:388: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.

row = pd.read_csv(f,header=None,squeeze=True, index_col=0)`

The result is that only the first star gets appended to a csv file. Would it be possible to update the code to fix this or circumvent this error from stopping the run?

Version information required for pip install

It appears pip requires version info to install. The setup.py script sets version=None which no longer works.

Problem

I'm having some issues installing isoclassify using the pip functionality. I'm getting this error and I was wondering if you could help.
Screen Shot 2023-08-03 at 2 22 33 PM

Originally posted by @SpaceyCaseyLynn in #30 (comment)

Solution

Set version in setup.py.

This is an easy fix, I'm on it!

Non-deterministic dustmap sampling

@taberger,

It would be nice to have some sort test suite that we can use to make sure that our changes to isoclassify, don't break the code. As I was working on PR #15, I ran the example stars in both grid mode and direct mode. The results were consistent within errors, but were not exactly the same. I noticed that you put in some np.random.seed() calls before some of the random sample, but I don't think the coverage is complete. I think the dustmap sampling is still non-deterministic.

bcgrid Dropbox Link Returns 404

Hi Dan & co,

I am installing isoclassify, and the second dropbox link ( https://www.dropbox.com/s/921jc0ojlz6c6ar/bcgrid.h5 ) returns a 404. I was able to successfully wget the first one, so I was wondering whether the file had just been relocated? The full text of my wget error is:

HTTP request sent, awaiting response... 301 Moved Permanently
Location: /s/raw/921jc0ojlz6c6ar/bcgrid.h5 [following]
--2020-08-27 10:05:46-- https://www.dropbox.com/s/raw/921jc0ojlz6c6ar/bcgrid.h5
Reusing existing connection to [www.dropbox.com]:443.
HTTP request sent, awaiting response... 404 Not Found
2020-08-27 10:05:46 ERROR 404: Not Found.

Thank you!
-Nicholas

Non-string id_starname

If the id_starname column in the input csv file is a combination of numbers (e.g. Gaia IDs), pandas will interpret that variable as integers (e.g. int64), causing a crash in star=df.loc[self.id_strarname] since self.id_starname = kw['id_starname'] by definition will always be a string. I suggest hard-convert this column to str by inserting df['id_starname'] = df['id_starname'].astype(str) below the following code snippet.

# Read in inputs
df = pd.read_csv(kw['csv'])

Inconsistent behavior? Sometimes no mass/density calculated.

I'm trying to derive stellar parameters for a selection of stars using 2MASS and Gaia photometry, parallax, ra/dec, and spectroscopic Teff and [Fe/H]. Although isoclassify runs successfully on all stars, it only outputs a mass for some of the stars: for the rest, the output file lists mass(msun): 0.0 + 0.0 - 0.0.

I've attached two output logs -- one of a successful mass inference, and one showing a failure (i.e., no mass). Any thoughts?

output.log
output.log

The path for bprp-solar-mist.txt is broken

When using Direct and gamag + bpmag and rpmag an error is sometimes thrown as isoclassify tries to look for bprp-solar-mist.txt in the user's .local/python path. A short version of the error appears as:

[Errno 2] No such file or directory: '/home/andrew/.local/lib/python3.8/site-packages/isoclassify/direct/bprp-solar-mist.txt'

A very easy workaround is to cd into isoclassify, then cp -r isoclassify .local/lib/python3.8/site-packages/
in the example using my error message. My workaround will get everything running again just fine as a temporary fix.

The full error message is below:

Traceback (most recent call last):
File "/home/andrew/.local/bin/isoclassify", line 8, in
sys.exit(main())
File "/home/andrew/.local/lib/python3.8/site-packages/isoclassify/isoclassify.py", line 67, in main
args.func(args)
File "/home/andrew/.local/lib/python3.8/site-packages/isoclassify/isoclassify.py", line 70, in run
isoclassify.pipeline.run(**vars(args))
File "/home/andrew/.local/lib/python3.8/site-packages/isoclassify/pipeline.py", line 36, in run
pipe.run()
File "/home/andrew/.local/lib/python3.8/site-packages/isoclassify/pipeline.py", line 277, in run
self.paras = classify_direct.stparas(
File "/home/andrew/.local/lib/python3.8/site-packages/isoclassify/direct/classify.py", line 248, in stparas
input.teff=mist_jk(jkmag)
File "/home/andrew/.local/lib/python3.8/site-packages/isoclassify/direct/classify.py", line 782, in mist_jk
mist=ascii.read(fn)
File "/home/andrew/.local/lib/python3.8/site-packages/astropy/io/ascii/ui.py", line 287, in read
dat = _guess(table, new_kwargs, format, fast_reader)
File "/home/andrew/.local/lib/python3.8/site-packages/astropy/io/ascii/ui.py", line 447, in _guess
dat = reader.read(table)
File "/home/andrew/.local/lib/python3.8/site-packages/astropy/io/ascii/core.py", line 1165, in read
self.lines = self.inputter.get_lines(table)
File "/home/andrew/.local/lib/python3.8/site-packages/astropy/io/ascii/core.py", line 304, in get_lines
with get_readable_fileobj(table,
File "/usr/lib/python3.8/contextlib.py", line 113, in enter
return next(self.gen)
File "/home/andrew/.local/lib/python3.8/site-packages/astropy/utils/data.py", line 237, in get_readable_fileobj
fileobj = io.FileIO(name_or_obj, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/home/andrew/.local/lib/python3.8/site-packages/isoclassify/direct/bprp-solar-mist.txt'

reddening code

@taberger,

We should think about the code organization in isoclassify.py My intention behind this module is to provide general driver functionality and control structure. I think it makes sense to remove all non-pipeline code to another module, e.g. extinction.py

Regarding the extinction code, there are a few long lines defining arrays or dictionaries.

    distanceSamples = np.array([0.06309573,0.07943284,0.1,0.12589255,0.15848933,0.19952627,0.25118864,0.31622776,0.3981072,0.50118726,0.6309574,0.7943282 ,1.,1.2589258,1.5848933,1.9952621,2.511887,3.1622777,3.981073,5.011873,6.3095727,7.943284,10.,12.589258,15.848933,19.952621,25.11887,31.622776,39.81073,50.11873,63.095726])*1000. # In pc, from bayestar2017 map distance samples

        return {"ab":4.1708789, "av":3.1071930, "abt":4.3358221, "avt":3.2867038, "ag":3.8281101, "ar":2.7386468, "ai":2.1109662, "az":1.4975613, "aj":0.89326176, "ah":0.56273418, "ak":0.35666104, "aga":2.4623915}

etc 

I think the code would be cleaner if these were mode into text files, e.g. in data/distance-samples.txt or data/extinction-vector-schlafly16.txt

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.