Giter Club home page Giter Club logo

pysolid's Introduction

Language CircleCI Conda Download Version License Citation

PySolid

The Python based solid Earth tides (PySolid) is a thin Python wrapper of the solid.for program (by Dennis Milbert based on dehanttideinelMJD.f from V. Dehant, S. Mathews, J. Gipson and C. Bruyninx) to calculate solid Earth tides in east, north and up directions (section 7.1.1 in the 2010 IERS Conventions). Solid Earth tides introduce large offsets in SAR observations and long spatial wavelength ramps in InSAR observations, as shown in the Sentinel-1 data with regular acquisitions and large swaths (Yunjun et al., 2022).

This is research code provided to you "as is" with NO WARRANTIES OF CORRECTNESS. Use at your own risk.

1. Install

PySolid is available on the conda-forge channel and the main archive of the Debian GNU/Linux OS. The released version can be installed via conda as:

# run "conda update pysolid" to update the installed version
conda install -c conda-forge pysolid

or via apt (or other package managers) for Debian-derivative OS users, including Ubuntu, as:

apt install python3-pysolid

Or build from source:

PySolid relies on a few Python modules as described in requirements.txt and NumPy's f2py to build the Fortran source code. You could use conda to install all the dependencies, including the Fortran compiler, or use your own installed Fortran compiler and pip to install the rest.

a. Download source code
# run "cd PySolid; git pull" to update to the latest development version
git clone https://github.com/insarlab/PySolid.git
b. Install dependencies
# option 1: use conda to install dependencies into an existing, activated environment
conda install -c conda-forge fortran-compiler --file PySolid/requirements.txt --file PySolid/tests/requirements.txt

# option 2: use conda to install dependencies into a new environment, e.g. named "pysolid"
conda create --name pysolid fortran-compiler --file PySolid/requirements.txt --file PySolid/tests/requirements.txt
conda activate pysolid

# option 3: have a Fortran compiler already installed and use pip to install the rest dependencies
python -m pip install -r PySolid/requirements.txt -r PySolid/tests/requirements.txt
c. Install PySolid
# option 1: use pip to install pysolid into the current environment
python -m pip install PySolid

# option 2: use pip to install pysolid in develop mode (editable) into the current environment
python -m pip install -e PySolid

# option 3: manually compile the Fortran code and setup environment variable
cd PySolid/src/pysolid
f2py -c -m solid solid.for
export PYTHONPATH=${PYTHONPATH}:~/tools/PySolid
d. Test the installation

To test the installation, run the following:

python -c "import pysolid; print(pysolid.__version__)"
python PySolid/tests/grid.py
python PySolid/tests/point.py

2. Usage

PySolid could compute solid Earth tides in two modes: point and grid. Both modes produce displacement in east, north and up directions.

  • Point mode: compute 1D tides time-series at a specific point for a given time period
  • Grid mode: compute 2D tides grid at a specific time for a given spatial grid

2.1 Point Mode [notebook]

import datetime as dt
import pysolid

# prepare inputs
lat, lon = 34.0, -118.0                 # point of interest in degree, Los Angles, CA
step_sec = 60 * 5                       # sample spacing in time domain in seconds
dt0 = dt.datetime(2020, 1, 1, 4, 0, 0)  # start date and time
dt1 = dt.datetime(2021, 1, 1, 2, 0, 0)  # end   date and time

# compute SET via pysolid
dt_out, tide_e, tide_n, tide_u = pysolid.calc_solid_earth_tides_point(
    lat, lon, dt0, dt1,
    step_sec=step_sec,
    display=False,
    verbose=False,
)

# plot the power spectral density of SET up component
pysolid.plot_power_spectral_density4tides(tide_u, sample_spacing=step_sec)

2.2 Grid Mode [notebook]

import datetime as dt
import numpy as np
import pysolid

# prepare inputs
dt_obj = dt.datetime(2020, 12, 25, 14, 7, 44)
meta = {
    'LENGTH' : 500,                # number of rows
    'WIDTH'  : 450,                # number of columns
    'X_FIRST': -126,               # min longitude in degree (upper left corner of the upper left pixel)
    'Y_FIRST': 43,                 # max laitude   in degree (upper left corner of the upper left pixel)
    'X_STEP' :  0.000925926 * 30,  # output resolution in degree
    'Y_STEP' : -0.000925926 * 30,  # output resolution in degree
}

# compute SET via pysolid
tide_e, tide_n, tide_u = pysolid.calc_solid_earth_tides_grid(
    dt_obj, meta,
    display=False,
    verbose=True,
)

# project SET from ENU to satellite line-of-sight (LOS) direction with positive for motion towards the satellite
# inc_angle : incidence angle of the LOS vector (from ground to radar platform) measured from vertical.
# az_angle  : azimuth   angle of the LOS vector (from ground to radar platform) measured from the north, with anti-clockwirse as positive.
inc_angle = np.deg2rad(34)   # radian, typical value for Sentinel-1
az_angle = np.deg2rad(-102)  # radian, typical value for Sentinel-1 descending track
tide_los = (  tide_e * np.sin(inc_angle) * np.sin(az_angle) * -1
            + tide_n * np.sin(inc_angle) * np.cos(az_angle)
            + tide_u * np.cos(inc_angle))

3. Citing this work

  • Yunjun, Z., Fattahi, H., Pi, X., Rosen, P., Simons, M., Agram, P., & Aoki, Y. (2022). Range Geolocation Accuracy of C-/L-band SAR and its Implications for Operational Stack Coregistration. IEEE Trans. Geosci. Remote Sens., 60, 5227219. [ doi | arxiv | data | notebook ]
  • Milbert, D. (2018), "solid: Solid Earth Tide", [Online]. Available: http://geodesyworld.github.io/SOFTS/solid.htm. Accessd on: 2020-09-06.

pysolid's People

Contributors

avalentino avatar dependabot[bot] avatar jhkennedy avatar pbrotoisworo avatar piyushrpt avatar rtburns-jpl avatar scottstanie avatar yunjunz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pysolid's Issues

The pysolid package is not available on conda-forge for "osx-arm64"

I was running a MintPy installation with conda (conda install -c conda-forge --file ~/tools/MintPy/requirements.txt), and it failed to find the pysolid package in conda-forge for my M1 Mac:

PackagesNotFoundError: The following packages are not available from current channels:

  - pysolid
 
Current channels:

  - https://conda.anaconda.org/conda-forge/osx-arm64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/osx-arm64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/osx-arm64
  - https://repo.anaconda.com/pkgs/r/noarch

bug: same `solid.txt` filename conflicts with multi-threads run

A bug found with a solution suggested by @fastice:

I ran into a problem with pysolid.calc_solid_earth_tides_grid().

Basically I am running multiple threads at the same time from the same directory, which means that the individual threads are overwriting or deleting the solid.txt file.

As simple fix so that files from multiple threads would not conflict would be to make the file names unique with something like.

import uuid

txt_file = os.path.abspath(fsolid.{uuid.uuid1().hex}.txt')
if os.path.isfile(txt_file):
  os.remove(txt_file)

No need to change soon on my account, I found a workaround but others might run into a similar problem in a production environment.

This unique TXT filename should be passed to the Fortran code directly as a function argument.

`f2py` appears to skip code in `solid.for` that is not supported for Mac M2s

forwarded from Paul Rosen (on trying to install it on an M2 machine):

F2py appears to be skipping code in solid.fort that is not supported, in my case an implied DO in a DATA statement that is common in old fortran. A possible solution is to modify the setup to use –std=legacy in gfortran, but I don’t know how robust that is for all compilers and all machines.

More details

Pip failed until I had Michael Aivazis guide me through what is actually happening with pip and we got the code to rebuild the source with the right pip command line switch (--no-binary).
I ran f2py by hand in verbose mode and it told me that the compiler did not understand a particular implied do array definition so it was skipping the code! So I have no confidence that the code is actually correctly compiled. According to Aivazis, you must use –std=legacy to get old code like this to compile properly, but I have no idea how to do that with the pip infrastructure.

it might be solved now that you can Conda install it even for osx-arm... but it's probably a bad sign for future stability that a new macbook causes the standard compilation to fail.

Support Python 3.12+

PySolid currently does not support Python 3.12+ because it relies on numpy.distutils to build the compiled Fortran extension. numpy.distutils has been removed for Python 3.12+ builds of numpy:
https://numpy.org/devdocs/reference/distutils_status_migration.html#distutils-status-migration

Reccomendations there are to upgrade to:

I (@jhkennedy) lean towards scikit-build-core out of familiarity, but I haven't used mesons enough to have a well-formed opinion.

grid: empty values in solid.txt file on Windows OS

When running test_SET_grid.py the test fails after running the solid_grid function because the output solid.txt file contains empty values (#12).

Sample of file with empty values:

# program solid -- UTC version -- 2018jun01
# year, month, day = 2020 12 25
# hour, minute, second = 14  7 44
# S, N, W, E =   33.479166795   33.800000000 -118.200000000 -117.796666828
# step_lat, num_lat =   -0.009166663    35
# step_lon, num_lon =    0.009166663    44
,  ,  ,  ,  
,  ,  ,  ,  
,  ,  ,  ,  
,  ,  ,  , 
etc...

Licensing

The setup.py file has been recently updated to specify the license as license='GPL-3.0-or-later'.

Anyway in the classifiers section it is is specified:

License :: OSI Approved :: GNU General Public License v3 (GPLv3)

which corresponds to GPL-3.0-only, while GPL-3.0-or-later corresponds to

License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)

Please clarify the licensing.

DLL load failed while importing solid for manual installation on Windows

(python 3.11.3 windows)

when running python -c "import pysolid; print(pysolid.__version__)"
i get 0.3.2 which means pysolid was installed succesfully ?
when i run python PySolid/tests/grid.py i get the following error :


D:\test_gnss\PySolid\tests\grid.py
Traceback (most recent call last):
File "D:\test_gnss\gnss\Lib\site-packages\pysolid\grid.py", line 45, in calc_solid_earth_tides_grid
from pysolid.solid import solid_grid
ImportError: DLL load failed while importing solid: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\test_gnss\PySolid\tests\grid.py", line 33, in
tide_u) = pysolid.calc_solid_earth_tides_grid(dt_obj, atr, verbose=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\test_gnss\gnss\Lib\site-packages\pysolid\grid.py", line 50, in calc_solid_earth_tides_grid
raise ImportError(msg)
ImportError: Cannot import name 'solid' from 'pysolid'!
Maybe solid.for is NOT compiled yet.
Check instruction at: https://github.com/insarlab/PySolid.

`swath` mode

I agree with you, that's the timing difference along the orbit! If it's not accounted for, this timing variation will introduce a small ramp along the azimuth direction for a long track (I described this in section III.E.2 in my 2022 TGRS paper, after Dr. Dennis Milber brought it up in our email communications); and for the multiple frames stitching, as in your case here, it looks like a jump in our current implementation!

I intended to write a swath mode, in addition to the current grid and point mode, for this scenario. The implementation will be much easier if we have a true API-like talking between the python and fortran code (#2), instead of the current talking via text file. Therefore, I have not done it yet.

Originally posted by @yunjunz in #54 (comment)

Since solid.for could return numpy array since #56, we could implement this feature.

Inconsistency between overlapping frames

When using calc_solid_earth_tides_grid through MintPy for two separate runs involving overlapping frames, I notice unexpected behavior.

Specifically, I would expect the overlapping region between the two to be consistent after referencing epochs in the stack to the same reference date and point, but I instead find a non-zero, sub-mm level mean offset (~0.1 mm, and a range of ~0.5mm) and stdev (~0.33 mm):
image

Here is the residual within the overlapping region:
image

Please let me know how I can help to debug, if I should clarify anything, and if also perhaps it would be best to share the input files I used on the MintPy end. Many thanks

f2py: pass array/scalar directly instead of writing/reading text file

It seems straightforward to pass array and scalar variables from Fortran subroutine to Python function, as shown here: https://notmatthancock.github.io/2017/02/10/calling-fortran-from-python.html. Thus, we should eliminate the writing/reading of solid.txt file and pass solid.for results directly to point/grid.py.

Relevant link: https://numpy.org/doc/stable/f2py/f2py.getting-started.html#the-quick-and-smart-way.

Contributions are welcomed!

Support for ARM64-based MacOS via pip or conda

In addition to the x86_64 builds currently supported by pysolid, it would be great if you could support an arm64 build for MacOS. This would enable newer Mac users to use pysolid. Also, since pysolid is a dependency of MintPy, MintPy is not currently installable on arm64 Macs.

A pip-installable version of pysolid would fix my immediate issue, but long-term it would be good to have conda support as well.

I have tried to install pysolid via pip, but ran into low-level errors when trying to build the pysolid wheels. I'm happy to provide the error messages here if they would be useful.

How to install in centos

I want to install this package in centos, and this machine haven't anaconda.
So I have to build and install locally.
But I encountered this error when I followed the steps of readme.

ERROR: Command errored out with exit status 1: /usr/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/pyserver_dev/PySolid/setup.py'"'"'; __file__='"'"'/home/pyserver_dev/PySolid/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_iefdv6v/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/python3/include/python3.6m/pysolid Check the logs for full command output.

grid: reshape error with numpy=1.22

With the latest release of numpy 1.22, pysolid now has a reshape error for the pysolid.grid module as below:

python ${HOME}/tools/PySolid/tests/test_SET_grid.py
Test solid Earth tides calculation for a spatial grid at a specific time
PYSOLID: ----------------------------------------
PYSOLID: datetime: 2020-12-25T14:07:44
PYSOLID: SNWE: (33.4666668, 33.8, -118.2, -117.7833335)
SOLID  : calculate solid Earth tides in east/north/up direction
SOLID  : shape: (36, 45), step size: -0.0092 by 0.0092 deg
SOLID  : calculating / writing data to txt file: /root/tools/PySolid/solid.txt
PYSOLID: read data from text file: /root/tools/PySolid/solid.txt
Traceback (most recent call last):
  File "/root/tools/PySolid/tests/test_SET_grid.py", line 34, in <module>
    tide_u) = pysolid.calc_solid_earth_tides_grid(dt_obj, atr, verbose=True)
  File "/root/tools/miniconda3/lib/python3.9/site-packages/pysolid/grid.py", line 101, in calc_solid_earth_tides_grid
    tide_e = fc[:, 0].reshape(length, width)
ValueError: cannot reshape array of size 1614 into shape (36,45)

If I pin the numpy version to <1.22, there is no issue. Thus, this seems to be an f2py module behavior change introduced in numpy 1.22 release.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.