Giter Club home page Giter Club logo

phaseworm's Introduction

PhaseWorm

This python script wraps-up PhaseNet for use as a picker within an Earthworm installation. For manual validation of the results, we recommend to use SeisComP or SeisComP3 software to build a catalog of the events with magnitudes.

Retailleau, L., Saurel, J.-M., Zhu, W., Satriano, C., Beroza, G. C, Issartel, S., Boissier, P., OVPF Team and OVSM Team (2022). A wrapper to use a machine-learning-based algorithm for earthquake monitoring, Seismological Research Letter, XX, 1-9, 10.1785/0220210279.

flow-chart _ PhaseWorm can access data from 4 differents data sources, using ObsPy clients:

  • SeedLink
  • FDSN webservice
  • Earthworm WaveServerV
  • SDS disk file archive

Picks can be written as Earthworm TYPE_PICK_SCNL messages files or simply written to the standard output.

PhaseWorm can run in an infinite loop in an almost real-time manner and can also process (replay) old data.

PhaseNet identifies P and S arrivals and PhaseWorm binds them to, respectively, vertical and horizontal channels.


Installation and configuration

The PhaseWorm code has been developed with Python 3.8, ObsPy 1.2.2 and Earthworm v7.10. The PhaseNet included version uses TensorFlow 2.

Note that TensorFlow 2 is currently not compatible with Python 3.9.

PhaseWorm

Download PhaseWorm by cloning the repository into your local installation directory.

git clone https://github.com/jmsaurel/phaseworm.git

Preparing your environment

A python environment with proper libraries and modules is necessary to run PhaseWorm and the underlying PhaseNet.

Using Anaconda (recommended)
conda create --name phaseworm python=3.8
conda activate phaseworm
conda install tensorflow=2.3
conda install matplotlib scipy numpy
conda install obspy -c conda-forge
Using virtualenv
pip install virtualenv
virtualenv .phaseworm
source .phaseworm/bin/activate

Installing PhaseWorm

You can install PhaseWorm using pip from the main directory:

pip install .

Or, in "developer mode":

pip install -e .

Earthworm

PhaseWorm is designed to output TYPE_PICK_SCNL Earthworm messages files that will be read by the file2ew module and injected into a RingBuffer.

Because PhaseNet is designed to use overlapping time sequences, it is recommended to use a pkfilter module to remove duplicated picks.

file2ew

PhaseWorm pick file should be looked for regularly by the file2ew module to be injected in the correct RingBuffer. We recommend a scan every 0.5 second and no logging extensive logging. Pick files are written by PhaseWorm with the file extension .pick.

Because pick files are written, and then a hard symlink made to the file2ew deposit directory, it's recommended to not activate the SaveDataFiles option.

The SuffixType option should be configure with the InsitutionID used in the Earthworm setup.

CheckPeriod     0.5                # sleep this many seconds between looks
OpenTries       2                  # How many times we'll try to open a file
OpenWait        100                # Milliseconds to wait between open tries
SaveDataFiles   0                  # 0 = remove files after processing
LogOutgoingMsg  0
SuffixType  .pick     TYPE_PICK_SCNL       INST_xxx

pkfilter

PhaseNet picks predictions are equally accurate when working on overlapping data windows. We recommend a value of 0.05 seconds for PickTolerance.

When working with 30s timewindows, a pick could be detected only on the next overlapping time window. We then allow 60s OlderPickLimit to be checked.

PickTolerance  0.05
OlderPickLimit 60

binder_ew

Binder_ew module should be setup as usual, according to the local area settings. However, to take full advantage of the P and S phase identifications from PhaseNet, two parameters should be activated:

  • no_S_on_Z (only PhaseNet P readings are sent on vertical channels)
  • no_P_on_Horiz (only PhaseNet S readings are sent on horizontal channels)

Since S phases are sent to horizontal channel, binder_ew shouldn't be allowed to initiate the stack on horizontal channels.

Currently, PhaseNet doesn't output any amplitude pick measurements. It's then recommended not to use the S to P amplitude ratio setting.

# stack_horizontals
# s_to_p_amp_ratio 2.0
no_S_on_Z
no_P_on_Horiz

eqassemble

Because PhaseNet picks S readings, it's important to activate them in eqassemble module.

ReportS 1

Usage

By default, PhaseWorm will use the configuration from the config.cfg file.

phaseworm

Alternative configuration file can be passed on the command line.

phaseworm --config-file /my/path/to/my_config.cfg

The configuration file is divided into 3 different sections.

Earthworm section

This section defines variables linked to the Earthworm setup that will be used to process the pick messages (module_id, inst_id, message_id). This section also defines the directory in which TYPE_PICK_SCNL messages will be written to.

PhaseNet section

This section defines PhaseNet related variables. Currently there is only one available variable: the neural network training file to use. The recommended default is to use the file shipped with PhaseNet and included in this bundle.

If you have trained PhaseNet on your own data, you might want to use your training file.

General section

The general section contains various variables to set-up PhaseWorm:

  • data source
  • time window lenght to process
  • network.stations to look for
  • channels priority list to look for each station
  • replay or real-time mode
  • whether to activate or not the TYPE_PICK_SCNL message writing.

Examples

Two configuration examples are shipped: config_replay.cfg and config_rt.cfg.

Replay mode

The file config_replay.cfg contains a configuration for replay use.

PhaseWorm reads data from FDSN webservice, starting from xxx until xxx.

Real-time mode

The file config_rt.cfg contains a configuration for near real-time use.

PhaseWorm reads data from an Earthworm WaveServerV (recommended) or SeedLink.

PhaseWorm write picks in the TYPE_PICK_SCNL directory.


Acknowledgements

PhaseNet code was simplified on purpose by Weiqiang Zhu.

Data preprocessing was designed and implemented by Lise Retailleau.

Overall wrapping was done by Jean-Marie Saurel.

Claudio Satriano provided extensive code review.

Testing was performed on Mayotte seismo-volcano crisis for the Réseau de surveillance volcanologique et sismologique de Mayotte: REVOSIMA.

phaseworm's People

Contributors

claudiodsf avatar jmsaurel avatar lisesismo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

phaseworm's Issues

Conda environment instructions do not work anymore

The instructions in README.md for creating a conda environment do not work anymore on my Mac (at least).

The following creates a working environment for me, with tensorflow 2.4 (which should work, according to #9):

   conda create -n phaseworm -c defaults -c conda-forge python=3.8 tensorflow=2.4 obspy

phaseworm launches (all the dependencies are installed), but I couldn't test the code yet with an example.

TensorFlow 2.6.x not compatible

Works and tested on TensorFlow versions 2.3.x and 2.4.x.
TensorFlow version 2.6.x doesn't work.
Add precisions in the doc about that.

Use more standard package structure, protect vendored phasenet package

Hi Jean-Marie,

I just saw your recent PR #12. I think it introduces two problems:

  1. The directory name phaseworm_sources does not respect the general recommendation, which is to name this directory like the package (i.e. phaseworm).

In its current form, when one does pip install ., she will get the following directories in the site-packages directory:

➜  ~ conda activate phaseworm
(phaseworm) ➜  ~ ls -lrt $CONDA_PREFIX/lib/python*/site-packages
...
drwxr-xr-x  13 claudio staff    416 Apr 12 09:06 phasenet
drwxr-xr-x   9 claudio staff    288 Apr 12 09:06 phaseworm_sources
drwxr-xr-x  11 claudio staff    352 Apr 12 09:06 phaseworm-1.1.dist-info

It would be better, in my opinion (and for coherency with other installed packages in site-packages) to drop the suffix _sources.

  1. The above directory listing also illustrates the second problem: phaseworm distributes its own specific version of phasenet (this is technically called "vendoring"). Currently, phasenet is installed as a top-level package, meaning that a pip install phasenet will overwrite the version shipped with phaseworm, with the possibility of generating weird bugs.
    To protect this specific version, it should go inside the phaseworm directory.

So, my proposed directory structure is:

phaseworm
├── CITATION.cff
├── CONTRIBUTORS.txt
├── LICENSE.txt
├── MANIFEST.in
├── README.md
├── doc
│   └── flow-chart.png
├── examples
│   ├── config.cfg
│   ├── config_replay.cfg
│   └── config_rt.cfg
├── phaseworm
│   ├── __init__.py
│   ├── get_data.py
│   ├── hinv_station_rw.py
│   ├── phasenet
│   │   ├── LICENSE
│   │   ├── README.md
│   │   ├── app.py
│   │   ├── data_reader.py
│   │   ├── detect_peaks.py
│   │   ├── model
│   │   │   └── 190703-214543
│   │   │       ├── checkpoint
│   │   │       ├── config.log
│   │   │       ├── loss.log
│   │   │       ├── model_95.ckpt.data-00000-of-00001
│   │   │       ├── model_95.ckpt.index
│   │   │       └── model_95.ckpt.meta
│   │   ├── model.py
│   │   ├── requirements.txt
│   │   ├── run.py
│   │   ├── test_app.py
│   │   └── util.py
│   ├── phaseworm.py
│   ├── read_config.py
│   └── stationxml2hinv.py
└── setup.py

tmp directory not checked

PhaseWorm should check the existence of the tmp directory inside the configured pick_dir and it should be created if not existing

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.