linkedearth / pyleoclim_util Goto Github PK
View Code? Open in Web Editor NEWPython Package for the Analysis of Paleoclimate Data. Documentation at
Home Page: https://pyleoclim-util.readthedocs.io
License: GNU General Public License v3.0
Python Package for the Analysis of Paleoclimate Data. Documentation at
Home Page: https://pyleoclim-util.readthedocs.io
License: GNU General Public License v3.0
It would be nice to add a lowpass filtering capability that does not distort phase.
Splines are pretty awesome for that.
I'd start with:
http://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.spline_filter1d.html#scipy.ndimage.spline_filter1d
or
http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.UnivariateSpline.html#scipy.interpolate.UnivariateSpline
On no account should a running mean ever be used!!!
http://mres.uni-potsdam.de/index.php/2017/02/10/better-avoid-running-means/
Installation should be through pip but if want to use RBchron, needs a little tweaking.
R needs to be present. Add to documentation.
And integrate to Timeseries module
Integrate querying and downloading from the wiki
Mean spacing: mean of the median for a realization
start_time: max(min)
end_time: min(max)
Proposed change:
Workflows would either allow for a LiPD object (dictionary) or for a user dictionary containing appropriate information as specified by function (e.g., time, value for Series).
Can be used to create personal workflows. See Jupyter Notebook as example. These methods will be re-used in workflow.py
Classes to consider:
Change the import to fortran code to be within the function so the package doesn't fail on import. Put a warning that the method is slow and we strongly suggest installing said compiler.
Feng, Julien wanted to integrate your notebook into Pyleoclim. He said using statsmodel. More details would be nice.
It's important for our plots to have a distinct look and feel.
To me that involves two things:
Currently our style is a bit of a mish-mash, so we need to get coherent. What style do you prefer? I'm pretty happy with the new matplotlib default, with ggplot a close second.
Another consideration is customizability: it's essential to have sensible defaults, but it should be easy for users to override that if they have a preferred style. I think this can be done easily by explaining in the documentation that the package sets its own Matplotlib style, but explain how users can override that in one line of code.
Switch Lists to NumPy arrays.
Currently, Pyleoclim does not interpret the variable "distance_from_top" as "depth". Thus for example, in the summaryTs function, age model plot can not be generated.
Several functions, like corrsig and rednoise, look like they are not used and should therefore be canned to avoid confusion.
Some of the functions have while loop asking for the right term. raise error instead.
Hello! Just a heads up:
Users will get an error for your basemap
requirement when following the install guide.
Can recreate this on a clean conda environment on OSX or Linux:
conda create --name spacemonkey
source activate spacemonkey
conda install python=3.6 pip
pip install
The last pip
command gives:
Could not find a version that satisfies the requirement basemap>=1.0.7 (from pyleoclim) (from versions: )
No matching distribution found for basemap>=1.0.7 (from pyleoclim)
See here: https://www.statsmodels.org/stable/generated/statsmodels.tsa.stattools.grangercausalitytests.html
*ready for api.py
I remember we investigated this at some point. What's the status on this? If not possible, comment on why and close the issue
Propose to replace it with nitime
.
Installation issues are real (particularly for Windows users, for whom Anaconda is hit or miss, and Basemap is still a major issue). One idea to circumvent them is to run our notebooks on
JetStream (https://jetstream-cloud.org). First, we need to check that it is feasible. Can someone give it a try?
Use this: https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp2d.html instead of the bumpy version
If ensembles are available, show the envelope on the plot
Add this feature for evenly-spaced timeseries
Using clustering
Class in in timeseries.py.
PCA algorithm here -> https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html
--finished
Needs to show only the dates (circles), together with their uncertainties (error bars). No line connecting them unless there is a ChronData table or EnsembleTable in the lipd file.
Should we create an upper workflow with all the spectral methods in? In this case, we need to make sure that inputs/ouputs are dictionaries to allow for the variability in outputs for each functions
Add examples to selected functions.
Pass the code to insert ensembles into LiPD files in lipdutils
enabled SSA as a method for Series in ui.py
enable MSSA as a method for MultipleSeries in ui.py
Hello again!
If I work around Issue #21 by installing basemap from conda, I can open a python shell or ipython or what have you and run:
import pyleoclim as pyleo
At this point I'm super excited for the wave of paleo awesomeness that is about to hit me. But instead I get:
~/miniconda/envs/spacemonkey/lib/python3.6/site-packages/pyleoclim/__init__.py in <module>()
29 from pyleoclim import SummaryPlots
30 from pyleoclim import Plot
---> 31 from pyleoclim import Spectral
32 from pyleoclim import Stats
33 from pyleoclim import Timeseries
~/miniconda/envs/spacemonkey/lib/python3.6/site-packages/pyleoclim/Spectral.py in <module>()
34
35 if sys.platform.startswith('darwin'):
---> 36 from . import f2py_wwz as f2py
37
38 '''
ImportError: dlopen(/Users/notarealuser/miniconda/envs/spacemonkey/lib/python3.6/site-packages/pyleoclim/f2py_wwz.cpython-36m-darwin.so, 2): Library not loaded: @rpath/libiomp5.dylib
Referenced from: /Users/notarealuser/miniconda/envs/spacemonkey/lib/python3.6/site-packages/pyleoclim/f2py_wwz.cpython-36m-darwin.so
Reason: image not found
And the library is not imported. Looks like it might be an issue with f2py needing an Intel lib.... I thought it could be something with MKL or GDAL...
Installing MKL from conda and then reloading the shell and importing pyleoclim gives me:
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-2828cd9a80d8> in <module>()
----> 1 import pyleoclim as pyleo
~/miniconda/envs/spacemonkey/lib/python3.6/site-packages/pyleoclim/__init__.py in <module>()
32 from pyleoclim import Stats
33 from pyleoclim import Timeseries
---> 34 from pyleoclim import RBchron
35
36
~/miniconda/envs/spacemonkey/lib/python3.6/site-packages/pyleoclim/RBchron.py in <module>()
13 import pandas as pd
14 import datetime
---> 15 import rpy2.robjects as robjects
16 from itertools import chain
17 from rpy2.robjects.packages import importr
~/miniconda/envs/spacemonkey/lib/python3.6/site-packages/rpy2/robjects/__init__.py in <module>()
14 import itertools
15 from datetime import datetime
---> 16 import rpy2.rinterface as rinterface
17 import rpy2.rlike.container as rlc
18
~/miniconda/envs/spacemonkey/lib/python3.6/site-packages/rpy2/rinterface/__init__.py in <module>()
49 del(os)
50
---> 51 from rpy2.rinterface._rinterface import (baseenv,
52 emptyenv,
53 endr,
ImportError: dlopen(/Users/notarealuser/miniconda/envs/spacemonkey/lib/python3.6/site-packages/rpy2/rinterface/_rinterface.cpython-36m-darwin.so, 2): Library not loaded: /usr/local/opt/xz/lib/liblzma.5.dylib
Referenced from: /Users/notarealuser/miniconda/envs/spacemonkey/lib/python3.6/site-packages/rpy2/rinterface/_rinterface.cpython-36m-darwin.so
Reason: Incompatible library version: _rinterface.cpython-36m-darwin.so requires version 8.0.0 or later, but liblzma.5.dylib provides version 6.0.0
I wasn't able to get around this by updating with packages on the conda-forge repos, either. So, there is still a dependency that not being taken care of?
Is there a plan to implement unit tests or continuous integration checks (e.g. TravisCI)? This will definitely ease development as the project grows.
I may be able to help with this if wanted.
Should we also group them so a method is chosen.
We need to check the package for issues that may pop up with the numpy upgrade, specifically the fact that it doesn't accept floats for indices.
We need a function like this one http://www.clidyn.ethz.ch/imputation/standardize.m
Notice that it returns 3 things: the z-score, the original mean and original standard deviation.
Among other things, it can be used to scale a record to a match a given variance.
PCA available for MultipleSeries in ui.pu
@fzhu2e and @CommonClimate Feng proposed to do this for the functions that are workflows but don't use a LiPD file for the data.
I agree with him that it will make things cleaner for a user, who won't have to mess with modules if they don't want LiPD.
My only suggestion would be to not name this api.py
Since the update to v0.11, statsmodels.tsa.AR
has been deprecated in favor of statsmodels.tsa.AutoReg
and statsmodels.tsa.SARIMAX
.
the tsmodel.py module, and the various functions herein (e.g. ar1_sim), need to be updated accordingly.
asked by @CommonClimate Good time to be doing this.
Warning: all notebooks not using the main api for Pyleoclim would have to be updated.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.