jobovy / mwdust Goto Github PK
View Code? Open in Web Editor NEWDust maps in the Milky Way
License: BSD 3-Clause "New" or "Revised" License
Dust maps in the Milky Way
License: BSD 3-Clause "New" or "Revised" License
Hi Jo,
I had two questions about the scale on which E(B-V) is output: true (S&F 2011) vs. SFD (1998). I strongly suspect that I'm misunderstanding something; thanks in advance for your patience.
(1) What do I need to do to obtain E(B-V)true from this code?
For the Marshall and Drimmel maps, my understanding is that sf10=True means that E(B-V)true is rescaled in the code from E(B-V)true back to E(B-V)SFD, by dividing E(B-V)true by 0.86, because the S&F 2011 Table 6 is supposed to be applied to the old E(B-V)SFD values in order to get the new true filter extinctions. So if what I actually want to get out of the code is E(B-V)true, I either need to set "filter=None, sf10=False", or set "filter=None, sf10=True" and multiply the results by 0.86. Meanwhile, for the Green 2015 map, the output of "filter=None" is E(B-V)SFD no matter what I set sf10 to, so I'd always need to multiply the results by 0.86 to get E(B-V)true.
Is that right? It's a little confusing to me, because the documentation says "Only use sf10=False when you have an extinction map in true E(B-V), not SFD E(B-V)." When in fact the purpose of sf10=True is to convert from E(B-V)true to E(B-V)SFD, not the other way around; the issue in Marshall and Drimmel isn't that the map is in SFD, but that S&F Table 6 requires SFD inputs.
(2) Is the Drimmel extinction being overestimated, since it was already normalized to SFD and the code is scaling it up to SFD again?
The Drimmel AV map, in its initial construction by Drimmel et al. 2003, was already normalized to SFD. Doesn't that mean that AV,Drimmel = 3.09 * E(B-V)SFD
--> E(B-V)SFD = AV,Drimmel / 3.09,
which is already what we need, because S&F 2011 Table 6 takes SFD inputs. So why apply the 1/((1-self._sf10)+self._sf10*0.86) factor in Drimmel03.py? That gives the meaningless value of (1/0.86)*E(B-V)SFD or (1/0.86)2*E(B-V)true, and the extinction will be overestimated. Or did you somewhere scale AV,Drimmel to E(B-V)true and I didn't see it?
Anyway, I'm pretty sure I'm wrong, not the code, but I'd be very grateful for any help you can offer in explaining why I'm wrong.
Thanks again,
Adrian
I'm trying to get mwdust installed on my Windows machine (so I can install isoclassify, so I can run a teammate's code) and am running into a snag on compiling the SFD C code. There are a couple of Unix-specific parts of the code; one of them is including unistd.h, which I found a workaround patch for, but Microsoft Visual C does not like 'm', and the StackOverflow advice to either remove that reference or replace it with 'msvcrt' has not worked.
Anything else I might try, or is this a sign to get my Linux box up and running again? :)
Thanks,
Jim
Dear all,
I am Rupak Roy a researcher in Astronomy, and interested to run the code MWDust. However, getting problem while installing. It will be nice if someone can help me in this regard. The detail is as given below :
I am using Python 2.7.13 :: Anaconda custom (64-bit).
Suppose my mwdust directory is at /home/rroy/Software/MWdust/mwdust-1.0
I made a directory at /home/rroy/Software/MWdust/DUST_DIR
I added this line in the .bashrc file in my home directory : export DUST_DIR="/home/rroy/Software/MWdust/DUST_DIR"
I sourced .bashrc and changed the directory to /home/rroy/Software/MWdust/mwdust-1.0
I ran the command : python setup.py install
I got the following python error :
Traceback (most recent call last):
File "setup.py", line 51, in
'maps')])
File "/home/rroy/Software/ANACONDA/anaconda2/lib/python2.7/subprocess.py", line 181, in check_call
retcode = call(*popenargs, **kwargs)
File "/home/rroy/Software/ANACONDA/anaconda2/lib/python2.7/subprocess.py", line 168, in call
return Popen(*popenargs, **kwargs).wait()
File "/home/rroy/Software/ANACONDA/anaconda2/lib/python2.7/subprocess.py", line 390, in init
errread, errwrite)
File "/home/rroy/Software/ANACONDA/anaconda2/lib/python2.7/subprocess.py", line 1024, in _execute_child
raise child_exception
TypeError: execv() arg 2 must contain only strings
I checked that before giving the error, the code generated an empty directory /home/rroy/Software/MWdust/DUST_DIR/maps
Looking forward for the suggestions..
Kind regards
Rupak Roy
Hi there,
I get the following error and traceback when I try to call in mwdust.Combined15:
---------------------------------------------------------------------------
IOError Traceback (most recent call last)
<ipython-input-85-dd23bc76c50e> in <module>()
----> 1 combine = mwdust.Combined15()
/pkg/linux/anaconda2/lib/python2.7/site-packages/mwdust-1.0-py2.7-linux-x86_64.egg/mwdust/Combined15.pyc in __init__(self, filter, sf10, load_samples, interpk)
32 HierarchicalHealpixMap.__init__(self,filter=filter,sf10=sf10)
33 #Read the map
---> 34 with h5py.File(os.path.join(_combineddir,'dust-map-3d.h5'),'r') \
35 as combineddata:
36 self._pix_info= combineddata['/pixel_info'][:]
/pkg/linux/anaconda2/lib/python2.7/site-packages/h5py/_hl/files.pyc in __init__(self, name, mode, driver, libver, userblock_size, swmr, **kwds)
310 with phil:
311 fapl = make_fapl(driver, libver, **kwds)
--> 312 fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
313
314 if swmr_support:
/pkg/linux/anaconda2/lib/python2.7/site-packages/h5py/_hl/files.pyc in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
140 if swmr and swmr_support:
141 flags |= h5f.ACC_SWMR_READ
--> 142 fid = h5f.open(name, flags, fapl=fapl)
143 elif mode == 'r+':
144 fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/h5f.pyx in h5py.h5f.open()
IOError: Unable to open file (truncated file: eof = 229863588, sblock->base_addr = 0, stored_eof = 585045564)
Would this be an issue with the data file itself, or an H5Py issue?
Hi - I'm using Python 3 installed with miniconda on a mac (osx 10.13.6). I am unable to install mwdust as I get the following error:
> sudo -E python3 setup.py install
...
/private/var/folders/tl/_5z6xmqd5_x4y0ljl2txp3740000gn/T/easy_install-cqf3s9qs/healpy-1.12.4/healpixsubmodule/src/cxx/autotools/cxxsupport/datatypes.h:74:10: fatal error:
'cstdint' file not found
#include
...
Google indicates that the problem may be some Clang/GCC issue - my Python installation uses GCC4.2.1, which it seems doesn't include cstdint. For example if I open Python3 it says:
Python 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 12:04:33)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
My Xcode is up to date. Attached is the full installation report.
Cheers,
Martin
mwdust_install.txt
Hi all,
I would like to use this package inside one of my code for calculating the extinction of a number of stars distributed in all sky.
My preference is to use the maps from mwdust.Green19, however they do not include some part of the sky.
I would like to ask what is the footprint of mwdust.Combined19, does it cover all the sky ? My feeling is that it will have the same footprint of mwdust.Green19.
If this is the case, there is some alternative map that cover all sky using as primary source mwdust.Green19, and that fill the missing parts with other maps ?
Thanks in advance
I've included the bash history of my attempt to install via instructions in the README, however I'm running into a problem: the environment variable SUDO_USER
does not exist on my machine, but USER
does. Simply replacing all instances of SUDO_USER
with USER
in setup.py
fixes the problem for me. It may be useful to have setup.py
check whether or not SUDO_USER
is a valid environment variable and either throw an exception specific to that problem.
$ python --version
Python 2.7.12 :: Anaconda 4.2.0 (x86_64)
$ git clone https://github.com/jobovy/mwdust.git
Cloning into 'mwdust'...
remote: Counting objects: 432, done.
remote: Total 432 (delta 0), reused 0 (delta 0), pack-reused 432
Receiving objects: 100% (432/432), 97.67 KiB | 0 bytes/s, done.
Resolving deltas: 100% (252/252), done.
$ cd mwdust
$ mkdir DUST_DIR
$ export pathname="${PWD}/DUST_DIR"
$ echo $pathname
/Users/jah5/Documents/installs/mwdust/DUST_DIR
$ echo "export DUST_DIR=\"${pathname}\"" >> ~/.bashrc
$ . ~/.bashrc
$ echo $DUST_DIR
/Users/jah5/Documents/installs/mwdust/DUST_DIR
$ mkdir ../mwdust_install
$ python setup.py install --prefix=../mwdust_install
Traceback (most recent call last):
File "setup.py", line 51, in <module>
'maps')])
File "/Users/jah5/anaconda2/lib/python2.7/subprocess.py", line 536, in check_call
retcode = call(*popenargs, **kwargs)
File "/Users/jah5/anaconda2/lib/python2.7/subprocess.py", line 523, in call
return Popen(*popenargs, **kwargs).wait()
File "/Users/jah5/anaconda2/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/Users/jah5/anaconda2/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
TypeError: execv() arg 2 must contain only strings
In addition, I found that by manually setting the installation directory a la python setup.py install --prefix=...
causes Python to complain that:
running install
Checking .pth file support in ../mwdust_install/lib/python2.7/site-packages/
/Users/jah5/anaconda2/bin/python -E -c pass
TEST FAILED: ../mwdust_install/lib/python2.7/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
../mwdust_install/lib/python2.7/site-packages/
and your PYTHONPATH environment variable currently contains:
''
Here are some of your options for correcting the problem:
* You can choose a different installation directory, i.e., one that is
on PYTHONPATH or supports .pth files
* You can add the installation directory to the PYTHONPATH environment
variable. (It must then also be on PYTHONPATH whenever you run
Python and want to use the package(s) you are installing.)
* You can set up the installation directory to support ".pth" files by
using one of the approaches described here:
https://setuptools.readthedocs.io/en/latest/easy_install.html#custom-installation-locations
Please make the appropriate changes for your system and try again.
I may be misusing the --prefix
option, but I found that simply installing as usual (python setup.py install
) worked without a problem, and on systems where permissions are an issue, I believe that specifying the --user
option (python setup.py install --user
) should avoid the problems I ran into when using the --prefix
option.
I am using Python 3.6.3 :: Anaconda custom (64-bit).I have installed mwdust using __" sudo python3 setup.py install --no-downloads"__
The dust data is also downloaded in the DUST_DIR.On importing mwdust I am getting the following error:
Traceback (most recent call last):
File "", line 1, in
File "/home/akriti/dust/mwdust-master/mwdust/init.py", line 1, in
from mwdust.SFD import SFD
File "/home/akriti/dust/mwdust-master/mwdust/SFD.py", line 7, in
from mwdust.util.read_SFD import read_SFD_EBV
File "/home/akriti/dust/mwdust-master/mwdust/util/read_SFD.py", line 27, in
raise IOError('SFD/C module not found')
OSError: SFD/C module not found
Also, if I close the terminal and open a new one i need to run setup.py again otherswise it says:
ModuleNotFoundError: No module named 'mwdust'
Hello, I am running into issues with installation of mwdust. Running the setup.py script for mwdust gives:
running bdist_egg
running egg_info
writing mwdust.egg-info/PKG-INFO
writing dependency_links to mwdust.egg-info/dependency_links.txt
writing requirements to mwdust.egg-info/requires.txt
writing top-level names to mwdust.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
adding license file 'AUTHORS.txt'
writing manifest file 'mwdust.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.9-x86_64/egg
running install_lib
running build_py
running build_ext
building 'sfd_c' extension
After the last step listed, I got a long error attached to this message. I am running it on an M1 mac, but I also tested it on an intel chip mac to troubleshoot. Do you have any insight into this issue?
Hi,
I'd like to propose a new dust map to be added to the list of mwdust
maps. Skowron+ 21 published a dust map in the direction of Magellanic Clouds based on OGLE red clump stars. The map is interactively accessable via their website. The reason why I'd really appreciate to implement this dust map is that the existing maps overestimate the extinction in the direction of MCs. This can be seen if one compares the Skowron's map to SFD on the linked website. Our group just recently published a paper (Molnár+ 21), where we also have addressed this issue (see Sect 3.2.1. and Fig. 4.).
For a sufficiently long DUST_DIR path length, the setup.py script fails. I don't have the terminal error. The path length which failed was
/Users/homefoldername/Documents/GitHub/Research/externaldata/DUST_DIR
changing the path to
/Users/homefoldername/Documents/DUST_DIR did install correctly.
I was trying to install mwdust
with pip, but it failed with an error that says it can't find a C header file. After cloning the repo and using pip install .
it worked. There might be a problem with the path of included directories.
If DUST_DIR and git-cloned "mwdust" are on different partitions, installation by
cd mwdust
python setup.py install
fails reporting:
[...]
Downloading Sale et al. (2014) dust maps ...
[...]
2018-09-15 11:09:54 (9.21 MB/s) - ‘/home/[...]/MWDUST/sale14/Amap.tar.gz’ saved [111829141/111829141]Amap.dat
ReadMe
Traceback (most recent call last):
File "setup.py", line 265, in
'Amap.dat'))
OSError: [Errno 18] Invalid cross-device link
Python 2.7.15rc1 on Ubuntu 18.04 machine.
regards, Michal
Electronic version here: http://iopscience.iop.org/0004-637X/737/2/103/suppdata/apj398709t6_ascii.txt
Requires 2MASS to be added ...
Hi,
During the installation of this module, the Drimmel data which is given as an example for the usage is not accessible right now (ftp://ftp.oato.inaf.it/astrometria/extinction/data-for.tar.gz).
I was able to skip this download in the setup, but maybe you can add this file to the repository as you say it does not occupy much space??
I am a second-year undergraduate currently working on a project that involves the installation of the mwdust package. The installation commands are intended for Unix, yet I am trying to run them on a Windows machine. When I run setup.py, I get the following error message: "FileNotFoundError: [WinError 2] The system cannot find the file specified". Is there another way to install mwdust?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.