j-faria / kima Goto Github PK

Write some examples in the wiki on how to start using kima.

What's missing until we can submit

• Implement the test suite with make test
• Clearly list all dependencies (especially Python packages)
• (if possible) install dependencies automatically
• what did I forget?

Put titles at the very least!

Also, maybe these shouldn't show up by default.
For an average user, this will be the first plots they see and they will be hard to interpret.

As of now, in the analysis of results, the first two lines of the datafile are always skipped.
This should be a model setting read from kima_model_setup.txt.

This is causing the BL2009 example to appear broken when it's really not (for example)

This is a point I hinted at in my paper with Courtney Donovan - there are phase changes in the shape of the likelihood function which make annealing unreliable. Therefore Nested Sampling is required to evaluate marginal likelihoods.

I can write a sentence or two if you like.

The installation of pykima in Travis is broken...

$ python setup.py install
running install
running bdist_egg
running egg_info
creating pykima.egg-info
writing pykima.egg-info/PKG-INFO
writing top-level names to pykima.egg-info/top_level.txt
writing dependency_links to pykima.egg-info/dependency_links.txt
writing entry points to pykima.egg-info/entry_points.txt
writing manifest file 'pykima.egg-info/SOURCES.txt'
reading manifest file 'pykima.egg-info/SOURCES.txt'
writing manifest file 'pykima.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
warning: install_lib: 'build/lib' does not exist -- no Python modules to install
creating build
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying pykima.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pykima.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pykima.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pykima.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying pykima.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/pykima-0.1-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing pykima-0.1-py2.7.egg
Copying pykima-0.1-py2.7.egg to /home/travis/miniconda/lib/python2.7/site-packages
Adding pykima 0.1 to easy-install.pth file
Installing kima-showresults script to /home/travis/miniconda/bin
Installed /home/travis/miniconda/lib/python2.7/site-packages/pykima-0.1-py2.7.egg
Processing dependencies for pykima==0.1
Finished processing dependencies for pykima==0.1

The command "python setup.py install" exited with 0.

$ python -c "import pykima"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named pykima

The command "python -c "import pykima"" exited with 1.

$ kima-showresults -h
Traceback (most recent call last):
  File "/home/travis/miniconda/bin/kima-showresults", line 9, in <module>
    load_entry_point('pykima==0.1', 'console_scripts', 'kima-showresults')()
  File "/home/travis/miniconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/__init__.py", line 549, in load_entry_point
  File "/home/travis/miniconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/__init__.py", line 2542, in load_entry_point
  File "/home/travis/miniconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/__init__.py", line 2202, in load
  File "/home/travis/miniconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg/pkg_resources/__init__.py", line 2208, in resolve
ImportError: No module named pykima.showresults
The command "kima-showresults -h" exited with 1.

All three examples (BL2009, CoRoT7, many_planets) yield an error message during compilation (appened below). It looks like the error is caused by the recent renaming of RVmodel():objects into RVmodel():planets (cf. commit ffbc792).

This issue will be easy to fix, but it demonstrates that this software will benefit from having the examples tested automatically upon each change (e.g. using Travis). I will be opening a separate issue about this.

~/dev/kima/examples/CoRoT7 $ make
g++ -c -I../../src -I../../DNest4/code -I../../eigen -I../../celerite/cpp/include -o kima_setup.o kima_setup.cpp -pthread -std=c++11 -O3 -w
kima_setup.cpp: In constructor ‘RVmodel::RVmodel()’:
kima_setup.cpp:22:6: error: class ‘RVmodel’ does not have any field named ‘objects’
     :objects(5, 5, false, RVConditionalPrior())
      ^
kima_setup.cpp:24:58: error: no matching function for call to ‘DNest4::RJObject<RVConditionalPrior>::RJObject()’
     ,C(Data::get_instance().N(), Data::get_instance().N())
                                                          ^
In file included from ../../DNest4/code/RJObject/RJObject.h:129:0,
                 from ../../DNest4/code/DNest4.h:18,
                 from kima_setup.cpp:2:
../../DNest4/code/RJObject/RJObjectImpl.h:2:1: note: candidate: DNest4::RJObject<ConditionalPrior>::RJObject(int, int, bool, const ConditionalPrior&, DNest4::PriorType) [with ConditionalPrior = RVConditionalPrior]
 RJObject<ConditionalPrior>::RJObject(int num_dimensions, int max_num_components, bool fixed,
 ^
../../DNest4/code/RJObject/RJObjectImpl.h:2:1: note:   candidate expects 5 arguments, 0 provided
In file included from ../../DNest4/code/DNest4.h:18:0,
                 from kima_setup.cpp:2:
../../DNest4/code/RJObject/RJObject.h:27:7: note: candidate: DNest4::RJObject<RVConditionalPrior>::RJObject(const DNest4::RJObject<RVConditionalPrior>&)
 class RJObject
       ^
../../DNest4/code/RJObject/RJObject.h:27:7: note:   candidate expects 1 argument, 0 provided
../../DNest4/code/RJObject/RJObject.h:27:7: note: candidate: DNest4::RJObject<RVConditionalPrior>::RJObject(DNest4::RJObject<RVConditionalPrior>&&)
../../DNest4/code/RJObject/RJObject.h:27:7: note:   candidate expects 1 argument, 0 provided
Makefile:23: recipe for target 'kima_setup.o' failed
make: *** [kima_setup.o] Error 1

(This issue is part of openjournals/joss-reviews#487)

I've run python setup.py install and pykima seems to have been put in my site-packages correctly. However, I get a ModuleNotFoundError when I try to import it. Any idea why this would happen?

A key component of kima is the analysis code located under scripts, which contains a significant number of Python functions and classes used to interpret and analyze results. Because this code is re-used in all the examples, and because you encourage users to re-use it as well, I strongly encourage you to turn this directory into a Python package.

Creating a Python package is surprisingly easy! It essentially takes three steps:

1. Rename scripts into kima (or however you want to name the Python package).
2. Add an empty __init__.py file inside kima (this is what turns a directory into a Python package).
3. Add a setup.py file to the repository to contain the metadata and make it installable (i.e. the docs would then instruct the user to type python setup.py install).

Doing this would make it much easier to re-use the Python code, e.g. users will be able to use import kima.dnest4 or import kima.display from any location.

Right now, it looks like users are requested to copy code from scripts to a custom work directory. Because the showresults.py script requires modules from scripts, I see that the code has been modified in the examples directory to include a sys.path.append() statement. This hard-coded method of ensuring that the required modules can be imported is error-prone, and the docs do not explain it in sufficient detail. Instead I recommend creating a package, it is the standard way to enable re-use of Python functions and classes.

Two extra tips:

• An extra advantage of a package is that you can configure the setup.py file to add command-line tools to the user's path upon installation. For example, if you add the option entry_points={'console_scripts': ['kima-showresults = kima.showresults:showresults_main']}, then installing kima will add a command-line tool called kima-showresults which executes the function kima.showresults.showresults_main when a user runs it on the command line.
• Once you have created the package, you can install it using python setup.py develop. This will install the package using symlinks to the development directory so that you don't have to type python setup.py install after each change. It makes developing a package more fun.

If this sounds like a good idea but you get stuck while creating the setup.py file, then do shout for help!

(This issue is part of openjournals/joss-reviews#487)

Hi,
kima favors a 2 planets model but the posterior distribution of the orbital periods shows that one is the 1-day alias of the other. How can I prevent this?
Thank you!

Currently, we have dependencies on

• numpy
• scipy
• matplotlib
• pandas
• (maybe I forgot some?)

Should the makefile test this and suggest their installation?

Response to the JOSS referee (issue here)

We need to check for any license issue with celerite, eigen and DNest4

We need to verify that we get the same results when running with the same seed but in different systems.
I'm not sure this is the case now and it would be important to create a complete test suite.

Include an example that takes 1-2 min to finish so that users can use kima right away

I strongly encourage the authors to add automated tests to enable users to verify whether their installation of kima works correctly, and to help prevent future code changes from introducing bugs (e.g. #18 could have been avoided this way).

It is great to see that the repository already contains a .travis.yml file which verifies that the C++ code can be compiled. This is a great and important first step, but the package could be improved by adding tests which also verify the correct functionality of the code.

In Python this is really easy thanks to the py.test package. For example, you can create a file called tests/test_kima.py which contains one or more functions with the prefix test_ that execute assert statements. Then, when you run py.test on the command line, it will automatically discover those tests and tell you whether any of the assert statements evaluate to False.

An alternative to automated tests could be to include additional input/output in the docs to allow users to manually verify the correctness. Some output is already shown in the docs (at https://github.com/j-faria/kima/wiki/5.-Analysis-of-results), but it is not clear what the exact input was.

(This issue is part of openjournals/joss-reviews#487.)

Otherwise it's not clear you have to close one to get the next one

• A small readme explaining the example and how to run it

In both the JOSS paper and the README/docs, please acknowledge and cite other software packages that enable similar RV analyses. In particular, it is important to clarify exactly what the unique contribution of kima is compared to those other packages.

For example: is kima the first tool to use nested sampling? Is it the first to use GPs? Is it the first to allow the number of planets to vary? Is it faster? Is it more accurate? Are any of these features important? etc, etc...

I'm sure you can think of some key advantages offered by kima. It is important to state clearly what they are. I have opened this issues because the guidelines from the Journal of Open Source Software include the following two requirements:

[The submission] should be a significant contribution to the available open source software that either enables some new research challenges to be addressed or makes addressing research challenges significantly better (e.g., faster, easier, simpler)

Submissions that implement solutions already solved in other software packages are accepted into JOSS provided that they meet the criteria listed above and cite prior similar work.

FYI: Here are the first five alternative packages which I find via Google:
https://github.com/California-Planet-Search/radvel
https://github.com/nespinoza/exonailer
https://github.com/jdeast/EXOFASTv2
https://github.com/oscaribv/pyaneti
https://github.com/mrtommyb/ktransit

(This is part of openjournals/joss-reviews#487.)

Weird thing is happening

I have two versions of kima, one before you made the more recent updates and another up to date.

The older version doesn't give me problems when I make ./run in the terminal. The new one gives me:

#Loaded 175 data points from file /home/joaocamacho/corot.txt
#Using 0 threads.
#Target compression factor between levels = 
Segmentation fault (core dumped)

Both use the same OPTIONS parameters.

Plus add a legend for the timespan line

Everytime I set hyperpriors = true this happens:

# Loaded 100 data points from file spots20andplanets3.txt
# Using 4 threads.
# Target compression factor between levels = 2.7182818284590451
# Seeding random number generators. First seed = 1512056330.
# Generating 20 particles from the prior...done.
*** Error in `./main': malloc(): smallbin double linked list corrupted: 0x00007ffac8000b80 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7ffae043f7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x82651)[0x7ffae044a651]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7ffae044c184]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_Znwm+0x18)[0x7ffae0f5be78]
./main[0x414f47]
./main[0x415de5]
./main[0x4180b4]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80)[0x7ffae0f86c80]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7ffae07996ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7ffae04cf3dd]
======= Memory map: ========
00400000-00423000 r-xp 00000000 08:01 18486454                           /home/joaocamacho/Dropbox/kima/main
00622000-00623000 r--p 00022000 08:01 18486454                           /home/joaocamacho/Dropbox/kima/main
00623000-00624000 rw-p 00023000 08:01 18486454                           /home/joaocamacho/Dropbox/kima/main
01dc9000-01f9b000 rw-p 00000000 00:00 0                                  [heap]
7ffac8000000-7ffac8052000 rw-p 00000000 00:00 0 
7ffac8052000-7ffacc000000 ---p 00000000 00:00 0 
7ffacc000000-7ffacc021000 rw-p 00000000 00:00 0 
7ffacc021000-7ffad0000000 ---p 00000000 00:00 0 
7ffad0000000-7ffad0054000 rw-p 00000000 00:00 0 
7ffad0054000-7ffad4000000 ---p 00000000 00:00 0 
7ffad4000000-7ffad4050000 rw-p 00000000 00:00 0 
7ffad4050000-7ffad8000000 ---p 00000000 00:00 0 
7ffad8000000-7ffad8052000 rw-p 00000000 00:00 0 
7ffad8052000-7ffadc000000 ---p 00000000 00:00 0 
7ffade3c4000-7ffade3c5000 ---p 00000000 00:00 0 
7ffade3c5000-7ffadebc5000 rw-p 00000000 00:00 0 
7ffadebc5000-7ffadebc6000 ---p 00000000 00:00 0 
7ffadebc6000-7ffadf3c6000 rw-p 00000000 00:00 0 
7ffadf3c6000-7ffadf3c7000 ---p 00000000 00:00 0 
7ffadf3c7000-7ffadfbc7000 rw-p 00000000 00:00 0 
7ffadfbc7000-7ffadfbc8000 ---p 00000000 00:00 0 
7ffadfbc8000-7ffae03c8000 rw-p 00000000 00:00 0 
7ffae03c8000-7ffae0588000 r-xp 00000000 08:01 58462571                   /lib/x86_64-linux-gnu/libc-2.23.so
7ffae0588000-7ffae0788000 ---p 001c0000 08:01 58462571                   /lib/x86_64-linux-gnu/libc-2.23.so
7ffae0788000-7ffae078c000 r--p 001c0000 08:01 58462571                   /lib/x86_64-linux-gnu/libc-2.23.so
7ffae078c000-7ffae078e000 rw-p 001c4000 08:01 58462571                   /lib/x86_64-linux-gnu/libc-2.23.so
7ffae078e000-7ffae0792000 rw-p 00000000 00:00 0 
7ffae0792000-7ffae07aa000 r-xp 00000000 08:01 58458205                   /lib/x86_64-linux-gnu/libpthread-2.23.so
7ffae07aa000-7ffae09a9000 ---p 00018000 08:01 58458205                   /lib/x86_64-linux-gnu/libpthread-2.23.so
7ffae09a9000-7ffae09aa000 r--p 00017000 08:01 58458205                   /lib/x86_64-linux-gnu/libpthread-2.23.so
7ffae09aa000-7ffae09ab000 rw-p 00018000 08:01 58458205                   /lib/x86_64-linux-gnu/libpthread-2.23.so
7ffae09ab000-7ffae09af000 rw-p 00000000 00:00 0 
7ffae09af000-7ffae09c5000 r-xp 00000000 08:01 58462545                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffae09c5000-7ffae0bc4000 ---p 00016000 08:01 58462545                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffae0bc4000-7ffae0bc5000 rw-p 00015000 08:01 58462545                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffae0bc5000-7ffae0ccd000 r-xp 00000000 08:01 58462529                   /lib/x86_64-linux-gnu/libm-2.23.so
7ffae0ccd000-7ffae0ecc000 ---p 00108000 08:01 58462529                   /lib/x86_64-linux-gnu/libm-2.23.so
7ffae0ecc000-7ffae0ecd000 r--p 00107000 08:01 58462529                   /lib/x86_64-linux-gnu/libm-2.23.so
7ffae0ecd000-7ffae0ece000 rw-p 00108000 08:01 58462529                   /lib/x86_64-linux-gnu/libm-2.23.so
7ffae0ece000-7ffae1040000 r-xp 00000000 08:01 28575312                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7ffae1040000-7ffae1240000 ---p 00172000 08:01 28575312                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7ffae1240000-7ffae124a000 r--p 00172000 08:01 28575312                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7ffae124a000-7ffae124c000 rw-p 0017c000 08:01 28575312                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7ffae124c000-7ffae1250000 rw-p 00000000 00:00 0 
7ffae1250000-7ffae1276000 r-xp 00000000 08:01 58458200                   /lib/x86_64-linux-gnu/ld-2.23.so
7ffae12c0000-7ffae1451000 rw-p 00000000 00:00 0 
7ffae1472000-7ffae1475000 rw-p 00000000 00:00 0 
7ffae1475000-7ffae1476000 r--p 00025000 08:01 58458200                   /lib/x86_64-linux-gnu/ld-2.23.so
7ffae1476000-7ffae1477000 rw-p 00026000 08:01 58458200                   /lib/x86_64-linux-gnu/ld-2.23.so
7ffae1477000-7ffae1478000 rw-p 00000000 00:00 0 
7ffd7b384000-7ffd7b3a5000 rw-p 00000000 00:00 0                          [stack]
7ffd7b3be000-7ffd7b3c0000 r--p 00000000 00:00 0                          [vvar]
7ffd7b3c0000-7ffd7b3c2000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

Took .250521446 seconds

Not sure if its a problem of my computer or of kima, the rest works fine.

It would be helpful to have correct, non-summarised headers in sample.txt, so the user knows exactly what is in each column. This will involve writing the description functions well, and in such a way that handles all the different combinations of options the user might have.

The kima_model_setup.txt file is not always saved in the right directory.
It is always created in the current directory, but it should instead be created in the same directory as sample.options.sample_file.

@eggplantbren, any idea how to access that directory from within the RVmodel methods?

Running standard example in Python 2 gives ConfigParser instance has no attribute __getitem__

dnest4.py:175: RuntimeWarning: invalid value encountered in multiply
  H_estimates[z] = -logz_estimates[z] + np.sum(P_samples[:,z]*logl)
log(Z) = -106.72234101 +- 0.0
Information = nan +- nan nats.

This is happening in the BL2009 example

I upgraded by Ubuntu installation and now have gcc 7.2.0. With this new version, I get an error message about bind not being a member of std. This has been fixed in upstream DNest4 (which we should be using anyway). Alternatively if you want to fix it in your copy of DNest4 you should just add #include to the top of Sampler.h.

I highly recommend the following default, or something very similar.

2
10000
10000
100
0
10
100
20000

Is there a reason for the choices in the repository?

My original goal for how to set the priors was

• there's a default set of priors, which we can use with `#include "default_priors.h"
• after doing that, the user could change a few priors individually, either inside or outside of the RVmodel constructor, depending on whether they depend on the data

But that has created a few complications and was the cause of #13 and #15 at least.
So I was thinking of refactoring things to be able to do

#include "default_extra_priors.h"  // sets Cprior, Jprior, maybe slope_prior
#include "default_GP_priors.h"  // sets the eta priors
#include "default_planet_priors.h"  // P, K, e, phi and w priors
#include "default_hyper_priors.h"  // sets log_muP_prior, wP_prior, log_muK_prior

Maybe the last include should also set P and K priors, but it will get confusing fast...
Let's use this issue to discuss

Change loadnew() to load()

Just a suggestion: have you considered using the Python bindings for dnest4, or otherwise creating a Python wrapper for your own C++ code?

This could simplify this package by bypassing the need for compilation instructions, and bypassing the need for documenting both the C++ and the Python code. It would also make it easier to treat dnest4 as a standard Python dependency (i.e. it could automatically be installed and compiled via pip or setup.py just like e.g. astropy).

(This suggestion is part of openjournals/joss-reviews#487.)

Use the gprn branch and open issues at will ;)

Running kima from the main directory with the maximum number of levels set to 1, followed by

kima-checkpriors 0 --code "samples=np.random.uniform(-47.6, 74, size=nsamples)"

gives something like

shouldn't they be more similar?

That's probably not the right word, but we might want to assign specific priors for a given planet (for the orbital period if detected with transits for example).
I started working on this adding a integer id argument to the RVConditionalPrior::from_uniform and to_uniform but:

• not sure if there is a better way to go at it
• log_pdf also needs access to id
• this might make setting the priors very complicated...

I think it'd be a good idea to have all examples be able to run. So the corot7.txt data file is needed.

I was running kima on a dataset of mine and it used up all my RAM and started using virtual memory. Obviously this is a problem that needs to be fixed.

This doesn't seem to affect the functionality, and I have run kima through a valgrind check and it passes. So it is only an issue of C++ style. There are some strange uses of new to create variables on the heap and it's not clear where they get deleted. There is almost certainly a better way to manage these things.

On a Mac, kima-showresults fails because of matplotlib, which in turn fails because it's run with a non-application python.
It seems that in OS X the anaconda installation of python has both a python and a pythonw executable. I think the scripts should use pythonw.

To Reproduce

Fresh kima install, python setup.py install and call kima-showresults
This seems to happen on Python 2.7, not tested on 3.x
Importing and using pykima from within IPython seems to work fine...

Possible solution

replace console_scripts with gui_scripts in the setup.py file?

Given the uncertainties on the RVs, RVerr, the uncertainties in the indicators are

• for the FWHM, 2.35 * RVerr
• for the BIS, 2 * RVerr

The many_planets example does not compile because three header files appear to be missing from the repository:

(This issue is part of openjournals/joss-reviews#487)

I just cloned the repo and everything compiled with no problems. But when I try to run it, it says it can't find data/data_to_test_priors.txt. It'd be good to have that file in the repository.

No idea how to test for this...

And infer RV offsets between instruments as needed.

In paper.md you still have ...

tags:

• example
• tags
• for the paper

😄

The order of the columns in the data file should always be

time   RV   RVerr   FWHM   BIS   R'hk   R'hkerr

right, @jdavidrcamacho ?

In terms of units, FWHM and BIS are (usually) in the same units as RV and RVerr. I think it's safe to assume that. Once this is implemented, we need to

• write documentation