Giter Club home page Giter Club logo

t21-hack-footprint's Introduction

t21-hack-footprint

Repo for the TRANSFORM 2021 Hackathon project on seismic acquisition footprint noise filtering (discussion here)

Interactive app name

F.R.I.D.A.: Footprint Removal Interactive Design App

Demo

Slack channel

t21-hack-footprint

People

Conda setup

To create the conda environment (called "footprint") for this repo, use

     $ conda env create -f environment.yml

To activate the environment, use

     $ conda activate footprint

To deactivate the environment, use

     $ conda deactivate

Requirements

matplotlib==3.2.2
param==1.10.1
scipy==1.5.0
scikit_image==0.16.2
holoviews==1.14.3
torch==1.8.1
dask==2.19.0
panel==0.11.3
hvplot==0.7.1
xarray==0.17.0
numpy==1.20.2

How to Install

  1. Clone the Repository.
  2. Install the requirements :
pip install -r requirements.txt
  1. Run :
python -m panel serve ./t21-footprint/smallfoot/app.py --dev
  1. Go to your browser and access http://localhost:5006/app

Description and background

  • Pre-hackathon meetup PowerPoint presentation here
  • Introduction to post-hack presentation here
  • For a definition of acquisition footprint read the SEG Wiki
  • For some examples, read this blog post, and also go to the Resources and reading material below
  • Project background: Elwyn and I have done some work putting together a tool to remove acquisition footprint from seismic data; this has been my longest-lived side project.
  • If you are curious about it, please read Chapter 39 of the upcoming 52 Things You Should Know About Geocomputing and then head over to the Tutorial notebook; give it a spin

Goals

The long term goal (AKA, the "dream") would be to create an open-source tool, ideally part of Awesome Open Geoscience, that can be used by geophysicists and geologists to remove footprint from seismic data, WHEN 5D interpolation or other costly post-stack processing works are not an options. The objectives of the hack are:

  • The primary objective of this projects would be to find and eliminate performance bottlenecks in the existing code. We already improved computations by switching from Astropy convolution to Scipy (see here and here)
  • Secondary objective, test with noisier examples (we will need to find open data with footprint, F3 has some)
  • Tertiary objective would be to create documentation for the tool, either by expanding/completing the Tutorial notebook or a separate document
  • Final objective, time permitting, would be to put together a VERY minimal Panel app (for example load data from numpy file >> display a time slice and its 2D spectrum >> derive filter >> save filter to numpy file) and deploy
  • I am including this for information purposes only: down the road the tool will need a way to automatically recognize and segment time slices of irregular shape into components of polygonal shape (by whatever means)

Resources and reading material

Skills needed

Elwyn and I will be working on this no-matter-what. But we welcome participants that are interested in this project. Useful skills would be:

  • Programming / performance optimization /profiling (Numpy broadcasting / Dask, etc.)
  • Geophysics / signal analysis
  • Testing
  • Documentation
  • Ideas - any ideas or suggestion is welcome

t21-hack-footprint's People

Contributors

leocd91 avatar markogauk avatar mycarta avatar scibbatical avatar wesleythegeolien avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

t21-hack-footprint's Issues

Access to full Penobscot 3D seismic volume (0-6000 milliseconds)

It would be good to have a full-sized 3D for testing various things, beginning with how the 2D and 3D implementations of FFT scale up.

  • Download the full 6000 milliseconds Penobscot project and access 3D volume
  • Use OpendTect to export the full milliseconds
  • Use segyio to load it and save it a .npy file (example code in this notebook and also the array with samples (values of times in milliseconds for each time slice)
  • Store on Dropbox
  • Use pooch (here) for retrieval, as suggested by Matt Hall on a discussion on SWUNG, if we want to do everything programmatically

Rewrite readme

Once enhancement 2 from #6 (adding dropdown menu for seismic volume choice) is completed:

  1. record a demo of the app
  2. add as a gif to the readme
  3. adjust all the text to reflect we are now in post-hackathon, with a working prototype to move forward

Dask

The goal is to be able to apply this workflow to larger cubes. One idea is to use Dask to do this as it is possible to apply python functions.

Dask works by chunking the dataset into smaller cubes and carrying out the analysis on each chunk before recombining.

We would have to check the results to make sure the FFT are being carried out correctly

Comment app

Assigned everyone to this collaborative effort.
Let's go back to the app and heavily comment it. I'd like to see it all the way to ELI5 level (explain it like I am 5). Egoistically for my benefit, but also anyone who wants to dig deeper into the code.

Update requirements for minimal working environment

We are using later features of some of the packages, notably templates in panel which are only available in panel > 10.X it would be good to update the environment files so that it is easier to install

Create a Dockerfile

In the line of making the tool easier #12 to install a docker file would probably make this more portable! IT would also facilitate deployment.

Why not also do a GitHub CICD to build the image automatically so people can run docker run -p 5006:5006 Frida to get the whole tool to work!

3D vectored Code

Currently the code works on 2D time slices and when applying to a cube we iterate over each time slice one by one.

The objective is to run this code in native 3D

  • Pad
  • Run FFTs
  • Unpad

The padding and un-adding can be extended from the current code.

For the FFT bumpy proposes an fftN version of their fft2. From what I understand this generalizes the FFTs to a N dimensional space.

However fft2 can be applied to a data cube (or N dimensions for that matter) but ONLY works along 2 axes.

Question: Do we need/want an N dimensional implementation or does a 2D ftt along each time slice work fine?

Interactive App

Using tools like Panel it may be possible to create an interactive app to create a visualise the results of at the filter before applying it to all levels

Python Package

There are starting to have multiple bits of code being reused in different notebooks.

So we can make sure that we are always using the latest version of the code (unless reason not to) I propose to write the common functions inside a small package that can be called from the notebooks to start consolidating some of the code.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.