Giter Club home page Giter Club logo

tigger-lsm's Introduction

Tigger-LSM: LSM Libs/utils

Installing Tigger-LSM

Ubuntu package

Enable the KERN suite and install the tigger-lsm package.

from pypi or from source

requirements:

  • Assorted python packages: astropy, numpy, scipy, astLib, python-casacore, future.

With the exception of astLib, these are already present in most Linux distros. astLib may be downloaded here: http://astlib.sourceforge.net/

  • Purr/Kittens. Available from pip as astro-kittens. Else, install the purr package from a MeqTrees binary

distribution (see http://www.astron.nl/meqwiki/Downloading). Alternatively, check it out from svn (see below), and make sure the parent of the Kittens directory is in your PYTHONPATH.

To obtain on ubuntu you can run:

$ sudo apt-get install python-kittens python-pyfits python-astlib python-scipy python-numpy

now from pip:

$ pip install astro-tigger-lsm

or from source:

$ git clone https://github.com/ska-sa/tigger-lsm
$ cd tigger-lsm
$ python setup.py install

Using Tigger-LSM

In python:

$ import Tigger $ model = Tigger.load("foo.lsm.html")

In the shell

$ tigger-convert foo.txt foo.lsm.html

Questions or problems

Open an issue on github

https://github.com/ska-sa/tigger-lsm

Travis

https://travis-ci.org/ska-sa/tigger-lsm.svg?branch=master

tigger-lsm's People

Contributors

o-smirnov avatar gijzelaerr avatar sphemakh avatar bennahugo avatar razman786 avatar sjperkins avatar sebokolodi avatar athanaseus avatar bmerry avatar mulan-94 avatar

Stargazers

 avatar

Watchers

 avatar Griffin Foster avatar James Cloos avatar Wesley New avatar Jason Manley avatar  avatar Ruby van Rooyen avatar Iniyan Natarajan avatar  avatar Modhurita Mitra avatar Roger Deane avatar  avatar Tony Willis avatar  avatar  avatar  avatar  avatar Liju Philip avatar  avatar Antonio Bradley Peters avatar  avatar  avatar  avatar Francois Joubert avatar Nunhokee Chuneeta avatar  avatar  avatar Gyula Jozsa (Josh) avatar Kshitij Thorat avatar  avatar Julien N Girard avatar  avatar Landman Bester avatar ratt-priv-ci avatar  avatar

tigger-lsm's Issues

FITS file loading error CUNIT4

When trying to load particular FITS files the following error is shown:

File ".local/lib/python3.8/site-packages/astro_tigger_lsm-1.7.1-py3.8.egg/Tigger/Coordinates.py", line 359, in __init__
    self.wcs = WCS(header)
  File ".local/lib/python3.8/site-packages/astropy/wcs/wcs.py", line 537, in __init__
    self.wcs.set()
astropy.wcs._wcs.InvalidTransformError: ERROR 6 in wcs_units() at line 3195 of file cextern/wcslib/C/wcs.c:
In CUNIT4 : Mismatched units type 'charge': have 'm/S', want 'm/s'.

Error loading FITS image ../CD-joint/wsrt.fits: ERROR 6 in wcs_units() at line 3195 of file cextern/wcslib/C/wcs.c:
In CUNIT4 : Mismatched units type 'charge': have 'm/S', want 'm/s'.

Issue getting PA from MS file

Hi when running the following command:

tigger-convert --force --type Gaul --append-type auto --int-to-app --primary-beam         
/home/mika/ladumaredux/pipeline/input/dummy_$(xy)_$(reim).fits --fits-l-axis px --fits-m-axis py --beam-clip 0.001 --pa-from-ms /home/mika/ladumaredux/pipeline/msdir1/1602964865_sdp_l0.8k-J03323_28075-corr.ms --linear-pol
/home/mika/ladumaredux/pipeline/input/vlamodel.gaul vlamodel.lsm.html

I get the following error:

# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line 389)
Requested data table IERSeop97 cannot be found in the searched directories:                           
# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line     
389)+ /usr/share/casacore/data/ephemerides/                                                           
# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line     
389)+ /usr/share/casacore/data/geodetic/                                                              
# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line 389)
Requested data table IERSpredict cannot be found in the searched directories:                         
# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line     
389)+ /usr/share/casacore/data/ephemerides/                                                           
# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line     
389)+ /usr/share/casacore/data/geodetic/                                                              
# 2024-06-05 15:06:06   INFO    MeasIERS::fillMeas(MeasIERS::Files, Double)     Requested JD 59139.8  
is outside the range of the IERS (Earth axis data) table.                                             
# 2024-06-05 15:06:06   INFO    MeasIERS::fillMeas(MeasIERS::Files, Double) +   Calculations will     
proceed with less precision                                                                           
# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line 389)
Requested data table TAI_UTC cannot be found in the searched directories:                             
# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line     
389)+ /usr/share/casacore/data/ephemerides/                                                           
# 2024-06-05 15:06:06   WARN    MeasIERS::findTab (file /code/measures/Measures/MeasIERS.cc, line     
389)+ /usr/share/casacore/data/geodetic/                                                              
# 2024-06-05 15:06:06   SEVERE  MeasTable::dUTC(Double) (file /code/measures/Measures/MeasTable.cc,   
line 4283)  Cannot read leap second table TAI_UTC                                                     
# Reading /home/mika/ladumaredux/pipeline/input/vlamodel.gaul (PyBDSM .gaul/.srl file)                
# Model contains 9 sources                                                                            
# Converting intrinsic fluxes to apparent                                                             
# importing InterpolatedBeams in standalone mode, ignoring MeqTrees-related stuff                     
# Loading FITS Beams /home/mika/ladumaredux/pipeline/input/dummy_xx_re.fits                           
/home/mika/ladumaredux/pipeline/input/dummy_xx_im.fits                                                
#   omitting xy beam due to --beam-diag                                                               
#   omitting yx beam due to --beam-diag                                                               
# Loading FITS Beams /home/mika/ladumaredux/pipeline/input/dummy_yy_re.fits                           
/home/mika/ladumaredux/pipeline/input/dummy_yy_im.fits                                                
# Using FITS beams with reference frequency 1519.500000 MHz                                           
# Getting PA range from MS                                                                            
/home/mika/ladumaredux/pipeline/msdir1/1602964865_sdp_l0.8k-J03323_28075-corr.ms, field 0             
# Successful readonly open of default-locked table                                                    
/home/mika/ladumaredux/pipeline/msdir1/1602964865_sdp_l0.8k-J03323_28075-corr.ms: 28 columns, 5114179 
rows                                                                                                  
# Successful readonly open of default-locked table                                                    
/home/mika/ladumaredux/pipeline/msdir1/1602964865_sdp_l0.8k-J03323_28075-corr.ms/ANTENNA: 8 columns,  
59 rows                                                                                               
# Successful readonly open of default-locked table                                                    
/home/mika/ladumaredux/pipeline/msdir1/1602964865_sdp_l0.8k-J03323_28075-corr.ms/FIELD: 9 columns, 1  
rows                                                                                                  
# Traceback (most recent call last):                                                                  
#   File "/home/mika/.quarticalenv/bin/tigger-convert", line 780, in <module>                         
#     pa1 = [(dm.do_frame(dm.epoch("UTC", dq.quantity(t, "s"))) and dm.posangle(fld,                  
#   File "/home/mika/.quarticalenv/bin/tigger-convert", line 780, in <listcomp>                       
#     pa1 = [(dm.do_frame(dm.epoch("UTC", dq.quantity(t, "s"))) and dm.posangle(fld,                  
#   File "/home/mika/.quarticalenv/lib/python3.10/site-packages/casacore/measures/__init__.py", line  
945, in posangle                                                                                      
#     return _measures.posangle(self, m0, m1)                                                         
# RuntimeError: 2024-06-05 15:06:06     SEVERE  MeasTable::dUTC(Double) (file                         
/code/measures/Measures/MeasTable.cc, line 4283)  Cannot read leap second table TAI_UTC 

Upstream breakage in latest astropy

�[0m�[91m2020-11-03 18:16:59	INFO	imager::createFTMachine() 	Performing interferometric gridding...
�[0m�[91m2020-11-03 18:17:01	INFO	ImageFitsConverter::ImageToFITS 	Copying '/src/pyxis/Pyxis/recipies/meqtrees-batch-test/WSRT.MS.MODEL_DATA.channel.1ch.img' to file Copying plane by plane (1048576 pixels).
�[0mlwimager normally ended
Image /src/pyxis/Pyxis/recipies/meqtrees-batch-test/WSRT.MS.MODEL_DATA.channel.1ch.fits ready. No image viewer selected (see TDL Exec menu).
�[91mTraceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/Timba/TDL/Compile.py", line 139, in import_tdl_module
    _tdlmod = imp.load_source(modname,filename,infile);
  File "/usr/lib/python3.6/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/src/pyxis/Pyxis/recipies/meqtrees-batch-test/sim.py", line 70, in <module>
    meqmaker.add_sky_models([gridded_sky,transient_sky,fitsimage_sky,lsm,TiggerSkyModel(verbose=2)]);
  File "/usr/local/lib/python3.6/dist-packages/Cattery/Siamese/OMS/tigger_lsm.py", line 45, in TiggerSkyModel
    classobj = importTigger(verbose=verbose);
  File "/usr/local/lib/python3.6/dist-packages/Cattery/Siamese/OMS/tigger_lsm.py", line 18, in importTigger
    import Tigger.SiameseInterface
  File "/usr/local/lib/python3.6/dist-packages/Tigger/SiameseInterface.py", line 46, in <module>
    from Tigger.Models.SkyModel import SkyModel
  File "/usr/local/lib/python3.6/dist-packages/Tigger/Models/SkyModel.py", line 31, in <module>
    from Tigger.Coordinates import angular_dist_pos_angle, DEG
  File "/usr/local/lib/python3.6/dist-packages/Tigger/Coordinates.py", line 62, in <module>
    from astLib.astWCS import WCS
  File "/usr/local/lib/python3.6/dist-packages/astLib/astWCS.py", line 29, in <module>
    import astropy.wcs as apywcs
  File "/usr/local/lib/python3.6/dist-packages/astropy/wcs/__init__.py", line 26, in <module>
    from .wcs import *
  File "/usr/local/lib/python3.6/dist-packages/astropy/wcs/wcs.py", line 57, in <module>
    from .wcsapi.fitswcs import FITSWCSAPIMixin, SlicedFITSWCS
  File "/usr/local/lib/python3.6/dist-packages/astropy/wcs/wcsapi/fitswcs.py", line 11, in <module>
    from astropy.coordinates import SpectralCoord, Galactic, ICRS
  File "/usr/local/lib/python3.6/dist-packages/astropy/coordinates/__init__.py", line 16, in <module>
    from .builtin_frames import *
  File "/usr/local/lib/python3.6/dist-packages/astropy/coordinates/builtin_frames/__init__.py", line 46, in <module>
    from . import supergalactic_transforms
  File "/usr/local/lib/python3.6/dist-packages/astropy/coordinates/builtin_frames/supergalactic_transforms.py", line 14, in <module>
    @frame_transform_graph.transform(StaticMatrixTransform, Galactic, Supergalactic)
  File "/usr/local/lib/python3.6/dist-packages/astropy/coordinates/transformations.py", line 674, in deco
    register_graph=self, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/astropy/coordinates/transformations.py", line 1251, in __init__
    matrix = matrix()
  File "/usr/local/lib/python3.6/dist-packages/astropy/coordinates/builtin_frames/supergalactic_transforms.py", line 17, in gal_to_supergal
    mat2 = rotation_matrix(90 - Supergalactic._nsgp_gal.b.degree, 'y')
  File "/usr/local/lib/python3.6/dist-packages/astropy/coordinates/baseframe.py", line 1638, in __getattr__
    in_frame_units=True)
  File "/usr/local/lib/python3.6/dist-packages/astropy/coordinates/baseframe.py", line 1110, in represent_as
    datakwargs[comp] = datakwargs[comp].to(new_attr_unit)
  File "/usr/local/lib/python3.6/dist-packages/astropy/units/quantity.py", line 689, in to
    return self._new_view(self._to_value(unit, equivalencies), unit)
  File "/usr/local/lib/python3.6/dist-packages/astropy/units/quantity.py", line 661, in _to_value
    equivalencies=equivalencies)
  File "/usr/local/lib/python3.6/dist-packages/astropy/units/core.py", line 987, in to
    return self._get_converter(other, equivalencies=equivalencies)(value)
  File "/usr/local/lib/python3.6/dist-packages/astropy/units/core.py", line 918, in _get_converter
    raise exc
  File "/usr/local/lib/python3.6/dist-packages/astropy/units/core.py", line 904, in _get_converter
    self, other, self._normalize_equivalencies(equivalencies))
  File "/usr/local/lib/python3.6/dist-packages/astropy/units/core.py", line 888, in _apply_equivalencies
    unit_str, other_str))
astropy.units.core.UnitConversionError: 'deg' (angle) and 'deg' (angle) are not convertible
E
======================================================================
ERROR: batch_test.testMeqtreesBatchJob
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/src/pyxis/Pyxis/recipies/meqtrees-batch-test/batch_test.py", line 170, in testMeqtreesBatchJob
    mod,ns,msg = Compile.compile_file(mqs,script,config=None);
  File "/usr/local/lib/python3.6/dist-packages/Timba/TDL/Compile.py", line 305, in compile_file
    (tdlmod,text) = import_tdl_module(filename,text=text,config=config);
  File "/usr/local/lib/python3.6/dist-packages/Timba/TDL/Compile.py", line 167, in import_tdl_module
    raise TDL.CumulativeError(*ns.GetErrors());
Timba.TDL.TDLimpl.CumulativeError: 
 [  1] 'deg' (angle) and 'deg' (angle) are not convertible

check tigger-lsm packaging

@gijzelaerr, as discussed, I've split out the Python libraries and standalone scripts into this repo. This should be packaged as tigger-lsm (contents of bin/) and python-tigger-lsm. I've done basic tests, and this looks to be functional -- could you please do whatever is necessary to get this out the door? If the packaging works, then we can already ship them.

Next in line -- converting the GUI.

.gaul list is not parsed by Tigger

I was trying to load .gaul list im3-MFS-image.gaul.txt and I got this error:

### running match_catalogs
# invoking callable catalog_utils.match_catalogs() using external interpreter
# Loading im3-MFS-image.srl: PyBDSM .gaul/.srl file
# loaded 8306 sources from 'main' im3-MFS-image.srl
# Loading im3-MFS-image.gaul: PyBDSM .gaul/.srl file
# Traceback (most recent call last):
#   File "<string>", line 17, in <module>
#   File "/home/gcilitshana/Project/New/madroon-1/catalog_utils.py", line 38, in match_catalogs
#     model = Tigger.load(catalog)
#   File "/home/gcilitshana/venv4/lib/python3.8/site-packages/Tigger/Models/Formats/__init__.py", line 126, in
load
#     return import_func(filename)
#   File "/home/gcilitshana/venv4/lib/python3.8/site-packages/Tigger/Models/Formats/PyBDSMGaul.py", line 88, in
load
#     raise ValueError("this .gaul file does not appear to contain a format string")
# ValueError: this .gaul file does not appear to contain a format string

Model parsing discussion

As promised in the meeting, I am opening this to discuss all things model parsing related @o-smirnov @JSKenyon @sjperkins @bennahugo. It seems that there is a need to parse complex models, possibly consisting of a mixture of component models (for both morphology and frequency spectrum) as well as pixelated images/facets. The former should be evaluated as accurately as possible using a parametrised DFT while the latter get's processed using a degridder. This functionality already exists (to some extent) in CubiCal but the model specification has become rather convoluted. The dream is to have a single parser that can concatenate different combinations of models and keep track of how each one should be predicted. I think the consensus was that tigger-lsm should be the parser with the API possibly living in codex-africanus. I see we were already thinking along these lines in 2019 ratt-ru/codex-africanus#85. But it may also be preferable to have a separate application to handle the predict. Ideas?

ImportError: No module named html.parser

Getting this error. As near as I can tell, this is because in Python 3 the module has been moved to html.parser. But somehow when I run tigger (using the modernize branch), it gets executed by Python 2. @gijzelaerr help, how do I force it to run in Python 3? How is it even possible that it runs in 2?

defer import of astropy & co until necessary

Problem exposed by ratt-ru/meqtrees-cattery#86

Astropy (at least some versions) uses pkg_resources, which breaks under MeqTrees for reasons unknown.

Since this import is not strictly needed for TiggerSiameseInterface to function, move it from the top level of Coordinates.py to the function where it's actually invoked.

Error converting from BBS to Tigger

When I try to convert from BBS to Tigger I get an error.

tigger-convert -t BBS -o Tigger single-source.skymodel single-source.lsm

Reading single-source.skymodel (BBS source catalog)
Error loading model: name 'file' is not defined

The contents of single-source.skymodel is:

# (Name, Type, Ra, Dec, I) = format
A, POINT, 23:58:15.43335658, +35:14:14.96586417, 1.0

Test suite fails

0.2Gb lsmformats(ASCII.py:366:load): /code/test/3C147-HI6.refmodel.lsm:679: could not convert string to float: mdlval="1000"><A>apply:1000</A>, skipping
0.2Gb lsmformats(ASCII.py:366:load): /code/test/3C147-HI6.refmodel.lsm:680: could not convert string to float: </TABLE>, skipping
0.2Gb lsmformats(ASCII.py:366:load): /code/test/3C147-HI6.refmodel.lsm:682: could not convert string to float: properties</H1>, skipping
0.2Gb lsmformats(ASCII.py:366:load): /code/test/3C147-HI6.refmodel.lsm:683: could not convert string to float: </A>, skipping
0.2Gb lsmformats(ASCII.py:366:load): /code/test/3C147-HI6.refmodel.lsm:684: list index out of range, skipping
Error loading support for format 'ModelHTML', see above. Format will not be available.
Loading /code/test/3C147-HI6.refmodel.lsm: ASCII table
Loaded model /code/test/3C147-HI6.refmodel.lsm
Selection leaves 0 source(s), of which 0 are point source(s)
There's nothing to convert into a brick.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.