gwpy / vet Goto Github PK

The Gravitational-wave Veto Evaluation and Testing suite

License: GNU General Public License v3.0

Python 100.00%

vet's Introduction

GWpy is a collaboration-driven Python package providing tools for studying data from ground-based gravitational-wave detectors.

GWpy provides a user-friendly, intuitive interface to the common time-domain and frequency-domain data produced by the LIGO and Virgo observatories and their analyses, with easy-to-follow tutorials at each step.

https://gwpy.github.io

Release status

Development status

Installation

To install, you can do:

conda install -c conda-forge gwpy

python -m pip install gwpy

You can test your installation, and its version by

python -c "import gwpy; print(gwpy.__version__)"

License

GWpy is released under the GNU General Public License v3.0 or later, see here for a description of this license, or see the LICENSE file for the full text.

vet's People

Contributors

Stargazers

Watchers

Forkers

duncanmmacleod sforsyth6 lauranuttall erikacowan1 dethodav sandychuang alurban andrew-lundgren tjma12 iaraota

vet's Issues

Glitchgrams (Omicron plots) plot low SNR points on top of high SNR points

New runs of VET have glitchgrams (Omicron plots) in which high SNR points are plotted first, followed by low SNR points, so the low SNR points obscure any high SNR points under them. This is a known issue which is discussed here: "Change in formatting of Omicron plots in summary pages" - https://git.ligo.org/detchar/ligo-summary-pages/-/issues/171 . Investigation has not turned up the exact cause of the change in plotting order (which is detailed on the above LIGO Summary Page issue), but a GWpy fix has been submitted in a pull request ("Handle changes in kwarg defaults in Axes.scatter (take 2)" - gwpy/gwsumm#333 ), which is awaiting approval. Some work is also being done to determine exactly what software change (probably in a Conda environment) caused the change in plotting behavior.

VET failing again

VET is consistently crashing at both LHO and LLO with the following error, which Duncan notes may be related to the function actually living gwsumm.channels

 File "/home/detchar/.conda/envs/ligo-summary-3.7/lib/python3.7/site-packages/gwvet/tabs.py", line 36, in <module>
    from gwsumm.data import get_channel
ImportError: cannot import name 'get_channel' from 'gwsumm.data' (/home/detchar/.conda/envs/ligo-summary-3.7/lib/python3.7/site-packages/gwsumm/data/__init__.py)

VET Failing

When executing a VET script that previous completed without error, I am getting the error, "KeyError: 'peak'". The following full error trace was run on LLO with a VET using Omicron glitches and the ligo-summary-3.10 env.

amber.stuver@ldas-pcdev5 Omicron-milestone2]$ ./run.sh
gw_summary 2024-05-24 00:19:21 CDT     INFO:  -- GW interferometer summary information system -- 
gw_summary 2024-05-24 00:19:21 CDT    DEBUG: This is process 3220090
gw_summary 2024-05-24 00:19:21 CDT    DEBUG: You have selected gps mode
gw_summary 2024-05-24 00:19:21 CDT    DEBUG: Start time: 2023-08-01 00:00:00.000 (1374883218.0)
gw_summary 2024-05-24 00:19:21 CDT    DEBUG: End time: 2023-10-01 00:00:00.000 (1380153618.0)
gw_summary 2024-05-24 00:19:21 CDT    DEBUG: Output directory: /home/amber.stuver/public_html/O4/electrical_glitches/Omicron-milestone2/gps/1374883218-1380153618
gw_summary 2024-05-24 00:19:21 CDT     INFO: Loading configuration
gw_summary 2024-05-24 00:19:21 CDT    DEBUG:  -- Loaded 1 plugins:
gw_summary 2024-05-24 00:19:21 CDT    DEBUG:         <module 'gwvet.tabs' from '/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwvet/tabs.py'>
gw_summary 2024-05-24 00:19:21 CDT    DEBUG:     Loaded 0 units
gw_summary 2024-05-24 00:19:21 CDT    DEBUG:     Loaded 0 channels
gw_summary 2024-05-24 00:19:21 CDT    DEBUG:     Loaded 3 states
gw_summary 2024-05-24 00:19:21 CDT    DEBUG:     Loaded 0 rcParams
gw_summary 2024-05-24 00:19:22 CDT     INFO:     Loaded 10 tabs [1 parents overall]
gw_summary 2024-05-24 00:19:27 CDT    DEBUG: Processing L1:PEM-EY_VMON_ETMY_ESDPOWER24_DQ/SNR90
    Downloaded 139 segments for L1:DMT-ANALYSIS_READY:1 (99.97% coverage).
States finalised [1 total]
    Observing: 139 segments | 4075821.0 seconds [DEFAULT]
Processing 'Observing' state:
    Read 5153 segments for L1:DCH-EY_ESD24_SNR90:1 (100.00% coverage).
    Grabbing Omicron triggers for L1:GDS-CALIB_STRAINTraceback (most recent call last):
  File "/home/detchar/.conda/envs/ligo-summary-3.10/bin/gw_summary", line 10, in <module>
    sys.exit(main())
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwsumm/__main__.py", line 845, in main
    tab.process(config=config, nds=args.nds,
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwsumm/tabs/data.py", line 377, in process
    self.process_state(state, config=config, nproc=nproc,
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwvet/tabs.py", line 347, in process_state
    before = get_triggers(str(self.channel), self.etg, state,
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwsumm/triggers.py", line 244, in get_triggers
    trigs = read_cache(segcache, SegmentList([segment]), etg,
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwsumm/triggers.py", line 465, in read_cache
    cache = [this_trigger_file for this_trigger_file in cache if
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwsumm/triggers.py", line 466, in <listcomp>
    len(EventTable.read(this_trigger_file, **kwargs)) != 0]
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwpy/table/table.py", line 235, in read
    return io_read_multi(vstack, cls, source, *args, **kwargs)
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwpy/io/mp.py", line 90, in read_multi
    output = mp_utils.multiprocess_with_queues(
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwpy/utils/mp.py", line 123, in multiprocess_with_queues
    return list(map(_inner, inputs))
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwpy/utils/mp.py", line 118, in _inner
    return func(x)
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwpy/io/mp.py", line 119, in _read_single_file
    return fobj, io_read(cls, fobj, *args, **kwargs)
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/astropy/io/registry/compat.py", line 52, in wrapper
    return getattr(registry, method_name)(*args, **kwargs)
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/astropy/io/registry/core.py", line 219, in read
    data = reader(*args, **kwargs)
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/gwpy/table/io/utils.py", line 52, in wrapper
    return tab[columns]
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/astropy/table/table.py", line 2084, in __getitem__
    [self[x] for x in item], copy_indices=self._copy_indices
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/astropy/table/table.py", line 2084, in <listcomp>
    [self[x] for x in item], copy_indices=self._copy_indices
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/astropy/table/table.py", line 2075, in __getitem__
    return self.columns[item]
  File "/home/detchar/.conda/envs/ligo-summary-3.10/lib/python3.10/site-packages/astropy/table/table.py", line 263, in __getitem__
    return OrderedDict.__getitem__(self, item)
KeyError: 'peak'

The previous execution results that ran can be found here:
https://ldas-jobs.ligo-la.caltech.edu/~amber.stuver/O4/electrical_glitches/Omicron-milestone2/gps/1374883218-1380153618/l1_pem_ex_vmon_etmx_esdpower24_dq/snr40/

This issue was brought to my attention by Brennan who is having the same issue (he asked me to confirm).

Update glue dependence to 1.60+

The glue.segments module is also being deprecated in favor of ligo.segments.

Expand rho range when plotting cWB results

The current plots for cWB have the rho range limited to >6. This agreed with the way cWB was implemented prior to O4 but rho no longer is limited, i.e. results for any rho are reported. Also, there aren't as many events with high rho to plot.

Please remove the plotting range limit on rho.

Request: Create web pages for cWB statistics only

This is a request:

Running VET over cWB triggers has been broken for a long time due to plotting errors that occur after the basic statistics are calculated. Brennan and I have been copy & pasting the statistics that VET reports to the terminal. However, it would be nice if we can temporarily disable the plotting so that the VET pages are created with the statistics so that there is something more official than our copy/paste to share with each other, and the collaboration.

tl;dr - until VET is fixed for cWB, please make cWB VET pages without plots so that we can have the statistics documentation.

VET error

VET no longer runs (including analyses that ran fine before). I am seeing this error:

 .  /home/duncan.macleod/opt/gwpysoft-2.7/bin/activate
(gwpysoft-2.7) [ray.dean@ldas-pcdev2 BlipL]$ gw_summary gps 1168452598 1168798198 -f defaults.ini -f Blipl.ini --verbose
/home/duncan.macleod/opt/gwpysoft-2.7/bin/gw_summary:58: UserWarning: 
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.

The backend was *originally* set to 'TkAgg' by the following code:
  File "/home/duncan.macleod/opt/gwpysoft-2.7/bin/gw_summary", line 36, in <module>
    from gwpy import time
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwpy/__init__.py", line 35, in <module>
    from . import (
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwpy/plot/__init__.py", line 35, in <module>
    from .plot import Plot

  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwpy/plot/plot.py", line 30, in <module>
    from matplotlib import (backends, figure, get_backend, _pylab_helpers)
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 16, in <module>
    line for line in traceback.format_stack()

  use('Agg')
Traceback (most recent call last):
  File "/home/duncan.macleod/opt/gwpysoft-2.7/bin/gw_summary", line 78, in <module>
    from gwsumm.tabs import (TabList, get_tab)
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwsumm/tabs/__init__.py", line 25, in <module>
    from .builtin import *
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwsumm/tabs/builtin.py", line 36, in <module>
    from ..plot import get_plot
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwsumm/plot/__init__.py", line 56, in <module>
    from .sei import *
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwsumm/plot/sei.py", line 26, in <module>
    from matplotlib.pyplot import subplots
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/pyplot.py", line 115, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 62, in pylab_setup
    [backend_name], 0)
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/backends/backend_tkagg.py", line 4, in <module>
    from . import tkagg  # Paint image to Tk photo blitter extension.
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/backends/tkagg.py", line 5, in <module>
    from six.moves import tkinter as Tk
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/six.py", line 203, in load_module
    mod = mod._resolve()
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/six.py", line 115, in _resolve
    return _import_module(self.mod)
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/six.py", line 82, in _import_module
    __import__(name)

ImportError: No module named Tkinter

Any assistance is welcome!

Change all paths in output files to relative paths

Currently, VET output files have their run paths hard-coded into the HTML files, which makes it very difficult to move or rename the runs. E.g., a run located at /home/detchar/public_html/vet/o4_review/h1_omicron_test1/ could be copied to /home/detchar/public_html/vet/o4_review/h1_omicron_test1_renamed/, but all of the links on the different pages will still point to files and directories in /home/detchar/public_html/vet/o4_review/h1_omicron_test1/, including links to images. If the original run needed to be archived, a restored version would only function properly if restored to the same path as it was originally run from.

A sed or similar text-modification command could probably be used to change the links, but that hasn't been developed or tested.

It might be possible to modify the code that generates the output files to use relative paths in all output HTML files, so that the run dir can be easily moved, and the paths will be agnostic and indifferent to their location, as long as the relative paths (mainly to images) are preserved. (Note that I don't know how big of a change this would require within GWPy, GWSumm, and/or other packages or how difficult that would be.)

Default url for get_segments is out-of-date

The default URL for gwvet.segments.get_segments points at the pre-O1 testing URL, and should be updated to https://segments.ligo.org.

Add VET to standard IGWN Conda environment

Adding VET to the standard IGWN Conda environment will simplify running it by end users and make results more reproducible.

VET failing

I tried running VET and it's failing due to the following:

Loading configuration...
Traceback (most recent call last):
  File "/home/detchar/opt/gwpysoft-2.7/bin/gw_summary", line 470, in <module>
    plugins = config.load_plugins()
  File "/home/detchar/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwsumm/config.py", line 235, in load_plugins
    mods.append(import_module(plugin))
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/detchar/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwvet/__init__.py", line 31, in <module>
    from .metric import *
  File "/home/detchar/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwvet/metric/__init__.py", line 328, in <module>
    from metrics import *
  File "/home/detchar/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwvet/metric/metrics.py", line 34, in <module>
    from gwpy.table.utils import get_table_column
ImportError: No module named utils

No way to access vetoed triggers list

The number one thing I want that isn't already on these awesome VET pages is a list of triggers (omicron, cWB, ahope) that is vetoed by each flag. It would be great to have this accessible as a link from the page. Additionally, it could be good to have a list of un-vetoed triggers, if that file were not way too large.

Bug in trying to plot an empty flag

in LLO, on my directory: /home/karelle.siellez/public_html/O1/test/veto_definer_file_VET/test2
Then I run

gwvet-vdf https://code.pycbc.phy.syr.edu/detchar/veto-definitions/download/master/burst/O1/H1L1-HOFT_C01_O1_BURST.xml 1126051217 1127271617

It produces the .sh file, and when I execute it by

./gwpy-vet.sh ```
It gives me a warning that seems to be linked with an attempt to try to analyse/plot a flag with no segments in it. Could you have a look into it please ? 

```/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/patches.py:107: UserWarning: Setting the 'color' property will overridethe edgecolor or facecolor properties. 
  warnings.warn("Setting the 'color' property will override"
/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/patches.py:107: UserWarning: Setting the 'color' property will overridethe edgecolor or facecolor properties. 
  warnings.warn("Setting the 'color' property will override"
/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/__init__.py:892: UserWarning: axes.color_cycle is deprecated and replaced with axes.prop_cycle; please use the latter.
  warnings.warn(self.msg_depr % (key, alt_key))
Traceback (most recent call last):
  File "/home/duncan.macleod/opt/gwpysoft-2.7/bin/gw_summary", line 682, in <module>
    datafind_error=opts.on_datafind_error, **cache)
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwsumm/tabs/data.py", line 370, in process
    **stateargs)
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwvet/tabs.py", line 310, in process_state
    minduration=self.minseglength, vetotag=str(state))[0]
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwvet/core.py", line 88, in evaluate_flag
    out[metric] = _metric(flag, triggers, after=after)
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwvet/metric/__init__.py", line 203, in __call__
    return Quantity(self.method(*args, **kwargs), unit=self.unit)
  File "<string>", line 2, in loudest_event
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwvet/metric/metrics.py", line 53, in _use_dqflag
    return f(segments, *args, **kwargs)
  File "/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/gwvet/metric/metrics.py", line 227, in loudest_event
    arank = get_table_column(after, column).max()
  File "/usr/lib64/python2.7/site-packages/numpy/core/_methods.py", line 10, in _amax
    out=out, keepdims=keepdims)
ValueError: zero-size array to reduction operation maximum which has no identity
/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/patches.py:107: UserWarning: Setting the 'color' property will overridethe edgecolor or facecolor properties. 
  warnings.warn("Setting the 'color' property will override"
/home/duncan.macleod/opt/gwpysoft-2.7/lib/python2.7/site-packages/matplotlib/__init__.py:892: UserWarning: axes.color_cycle is deprecated and replaced with axes.prop_cycle; please use the latter.
  warnings.warn(self.msg_depr % (key, alt_key))

"Export to CSV" button does not work

The "Export to CSV" button does not seem to do anything at all. This is actually a larger LIGO Summary Pages issue, as discussed here: ""Export to CSV" button does not do anything after ~Nov. 2019" - https://git.ligo.org/detchar/ligo-summary-pages/-/issues/176

Frequency/SNR cuts on input triggers would be useful

Currently VET doesn't have any way to downselect the triggers read in from an ETG, it just reads everything it can. It would be nice to be able to place cuts on frequency/SNR (for example).

Text in new plots (titles, axis labels, etc.) is different from older plots, and sometimes difficult to read

The text in newer VET plots does not match the size and formatting of older VET plots, which in some cases causes difficulty in reading some (new) text, as detailed in this LIGO Summary Page issue: "Change in formatting of Omicron plots in summary pages". This is probably due to a different Conda environment being used to run the software that generates the plots (GWpy), and thus a different version or configuration of some software generating the plots, though as the issue page above notes, the differences in output text have been seen in a pair of Summary Pages that self-document to have been created using the same versions of software, so the differences might be due to settings in the environment or due to changes in software whose version is not tracked. Investigation is being done to determine if running in different Conda environments can duplicate both versions, which could then be used to track down which software package is responsible for the differences.

Update documentation for VET

VET documentation is currently limited to a few options:

The "official" documentation, at https://ldas-jobs.ligo.caltech.edu/~duncan.macleod/vet/latest/index.html , which is from 2016
A (very nice) tutorial that Amber Stuver put together, at https://wiki.ligo.org/DetChar/VetoEvaluationTool
Some information in the Inside LIGO wiki page (search for "VET =")
The source code ( https://github.com/gwpy/vet )

It would be good to update the "official" documentation. Maybe create an official landing page for VET, so users have a starting point, like https://wiki.ligo.org/DetChar/VET? Or a home page on the Computing Guide? Or a section in a DetChar Tools page, like Virgo has?