setiathcro / ata-utils Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 6.0 51.82 MB

Makefile 0.03% C 0.72% Ruby 0.87% Shell 0.87% Python 11.59% Jupyter Notebook 85.87% HTML 0.05% OpenEdge ABL 0.01%

ata-utils's People

Contributors

Stargazers

Watchers

Forkers

ngizani realtimeradio mydonsolutions pranavp25 gsingh-0-0-1 tusay

ata-utils's Issues

make_and_track_ephems doesn

The function make_and_track_ephems:

https://github.com/SETIatHCRO/ATA-Utils/blob/ata-rest/pythonLibs/ATATools/ata_control.py#L294

should wait until antennas have fully acquired position before returning to replicate the behaviour of the previous function (atatrackephem -w ...). Currently, observing scripts can start calibrating/observing while the antennas are slewing

requests needs to be added to pythonLibs requirements

With the addition of the REST backend, requests is now used in pythonLibs, so it should be added as a requirement in pythonLibs/setup.py. Otherwise import ATATools.ata_control will give an ImportError saying it can't find requests.

Develop `ata*`-style scripts that can be run on `obs-node1`

Python scripts that we need include:

Inaccurate `get_source_ra_dec` results

ata_control.get_source_ra_dec (https://github.com/SETIatHCRO/ATA-Utils/blob/master/pythonLibs/ATATools/ata_control.py#L162-L189) is returning values that are apparently only accurate to 3 decimal parts.
Assuming a rounding is happening, a maximum error of 0.0005 would result in RA and Dec. RA is expressed in hours, and so the effect would be ~27 arcseconds, whereas dec would be 1.8 arcseconds.
The digitizer board delay engine uses this utility function to phase center the array, thus these values have to be accurate to sub-arcsecond level.

Adding default hashpipe targets

To make it easier on observers, why not have hashpipe target templates already available for the seti-nodes? We know for a fact that LOb and LOc are independently passed to seti-nodes1-4 and 5-8 respectively.

ATA-Utils/pythonLibs/SNAPobs/snap_hpguppi/record_in.py

Line 104 in dc53d7a

hashpipe_targets=None,

Something like:

from defaults import LOb_targets # or something similar
record_in(10, 300, LOb_targets)

Porting `atalistsats` to `ATATools`

Request:
A function that returns a similar output to atalistsats (assuming is interfaceable via REST) with the optional flags that the command-line tool provides, but from within ATATools.

Method to produce string version of PointingModel

Add a method to produce a string of the antenna model coefficients, similar to what is produced by atapointer.java

https://github.com/SETIatHCRO/ATA-Utils/blob/master/pythonLibs/ATATools/ata_pointing.py

atacheck REST equivalent

An enhancement to the REST API is to add support to call an atacheck.
This will help with implementing logic inside the python observing scripts

make set_atten() function fall back to REST API

set_atten() function:
https://github.com/SETIatHCRO/ATA-Utils/blob/master/pythonLibs/ATATools/ata_control.py#L970
currently uses an ssh call to an if-switch that doesn't exist anymore. It should fall back to the REST server as per:
http://restgw.hcro.org:12345/antenna/help/pams

Also, to follow the naming scheme, I think it's probably better to name this function set_pams() (as get_pams() already exists).
We can potentially keep set_atten() for backward compatibility.

ata_control.create_ephems2() issue with negative offset

After running ata_control.create_ephems2(source, az_offset, el_offset) with az_offset < 0, ata_control.point_ants2(source, "off", ant_list) doesn't actually make the antennas point to the off-source and it returns instantaneously without an error message.

ata_control.py: focus_data iterator fix

diff --git a/pythonLibs/ATATools/ata_control.py b/pythonLibs/ATATools/ata_control.py
index 673767c..da71a2b 100755
--- a/pythonLibs/ATATools/ata_control.py
+++ b/pythonLibs/ATATools/ata_control.py
@@ -593,7 +593,7 @@ def get_freq_focus(ant_list):
         raise
         
     retdict = {}
-    for ant, focus in focus_data:
+    for ant, focus in focus_data.items():
         # Return only short names like '1a'
         ant = ant.replace('ant', '')
         retdict[ant] = focus

ata_control.make_and_track_tle() does not work

I'm testing the REST version of ATATools and it seems that TLE tracking isn't working. With a TLE I have in my home dir, I get

>>> ac.make_and_track_tle('44453.tle', ['1a'])
/ephemeris got error: Command failed - ephemeris generation failed - The source, /home/obs/ephemeris/44453, was not found or no tracks generated.
Traceback (most recent call last):
  File "/home/destevez/ATA-Utils/pythonLibs/ATATools/ata_rest.py", line 71, in _do_op
    raise ATARestException(json[cls.RETURN_MSG_KEY])
ATATools.ata_rest.ATARestException: Command failed - ephemeris generation failed - The source, /home/obs/ephemeris/44453, was not found or no tracks generated.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/destevez/ATA-Utils/pythonLibs/ATATools/ata_control.py", line 329, in make_and_track_tle
    retval = ATARest.post(endpoint, json={'tle': data})
  File "/home/destevez/ATA-Utils/pythonLibs/ATATools/ata_rest.py", line 124, in post
    return cls._do_op(cls._OP_POST, endpoint, **kwargs)
  File "/home/destevez/ATA-Utils/pythonLibs/ATATools/ata_rest.py", line 78, in _do_op
    raise ATARestException(str(e))
ATATools.ata_rest.ATARestException: Command failed - ephemeris generation failed - The source, /home/obs/ephemeris/44453, was not found or no tracks generated.

TLE tracking isn't working either with the pre-REST version, but the error is different:

RuntimeError                              Traceback (most recent call last)
<ipython-input-22-9020ec345402> in <module>
----> 1 ac.make_and_track_tle('/home/destevez/44453.tle', ['1a'])

/usr/local/lib/python3.8/dist-packages/ATATools/ata_control.py in make_and_track_tle(tle, antstr)
    318     tle_basename = os.path.basename(tle)
    319 
--> 320     result,errormsg = ata_remote.callObs(['ataephem',
    321         '--tle', '${HOME}/tle/%s' %tle_basename])
    322     logger.info(result)

/usr/local/lib/python3.8/dist-packages/ATATools/ata_remote.py in callObs(myargs)
    161         str_out,str_err = callProg(myargs);
    162     else:
--> 163         str_out,str_err = callProg( ['ssh', OBS_USER +'@' + OBS_HOST ] + myargs )
    164 
    165     return str_out,str_err

/usr/local/lib/python3.8/dist-packages/ATATools/ata_remote.py in callProg(myargs)
     84         logger = logger_defaults.getModuleLogger(__name__)
     85         logger.warning("process '%s' returned error\n %s" % (myargs,str_err))
---> 86         raise RuntimeError("process %s failed" % myargs)
     87 
     88     return str_out,str_err

RuntimeError: process ['ssh', 'obs@control', 'ataephem', '--tle', '${HOME}/tle/44453.tle'] failed

Running directly ataephem on control gives

obs@control:~ 01:40:04 > ataephem --tle tle/44453.tle
some unhandled exception - PredictServer not ready/initialized
Problem with object look up (e.g. solar sys, satellite).
PredictServer not ready/initialized
The source, tle/44453, was not found or no tracks generated.

So perhaps the problem doesn't have anything to do with ATATools but with the PredictServer at control?

Add capability to track user-defined ephem file

The ata_control module does not provide a function that allows users to track a user-defined ephemeris file.
What spawned this request is the ability to perform a raster scan in az/el, around a geostationary satellite.

@tkoumrian what's your take on this? Assuming an experienced user creates ATA-compliant ephem files, with the correct TAI_nano/az/el/IR, is this doable?

REST call to retrieve pointing model

Request: a REST GET method to retrieve pointing model (TPOINT coefficients + other parameters as seen in the PointingModel_antXX.dat files) for antennas in json format. They are needed to apply the pointing corrections and needed when observations are being taken.
I can then write python to wrap around them

Place to check log files

I was running the GPS satellite tracking code yesterday, it ran for a few hours, with no issues until this:

2021-03-20 15:31:15 ERROR ATATools.ata_control: /ephemeris got error: Command failed - ephemeris generation failed - The source, GPS-BIIR-8---PRN-16-, was not found or no tracks generated.
Traceback (most recent call last):
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_rest.py", line 71, in _do_op
    raise ATARestException(json[cls.RETURN_MSG_KEY])
ATATools.ata_rest.ATARestException: Command failed - ephemeris generation failed - The source, GPS-BIIR-8---PRN-16-, was not found or no tracks generated.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "atapointer_multi.py", line 289, in <module>
    main()
  File "atapointer_multi.py", line 193, in main
    ata_control.create_ephem(name,
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_control.py", line 1061, in create_ephem
    retval = ATARest.post(endpoint, json=ephem_kwargs)
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_rest.py", line 124, in post
    return cls._do_op(cls._OP_POST, endpoint, **kwargs)
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_rest.py", line 78, in _do_op
    raise ATARestException(str(e))
ATATools.ata_rest.ATARestException: Command failed - ephemeris generation failed - The source, GPS-BIIR-8---PRN-16-, was not found or no tracks generated.
2021-03-20 15:31:15 INFO ATATools.ata_control: Reserving "3c,2b" from bfa to none
2021-03-20 15:31:15 INFO ATATools.ata_control: Querying group bfa
2021-03-20 15:31:15 INFO ATATools.ata_control: Querying group none

@tkoumrian Is there a log file where the exceptions can be read?

Exception raised by ATArest focus get/set

@tkoumrian I think this happens when the focus of certain antennas is set to a large frequency, say 10.5GHz, and then I try to set it back down to 1.5GHz.

Exception:

2021-06-17 16:42:13 ERROR ATATools.ata_control: /antennas/1a,1f,1k,5c,1h,4j,3d,4g,2h,2a,1c,2b/focus got error: Command failed - Command returned: ''
Traceback (most recent call last):
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_rest.py", line 71, in _do_op
    raise ATARestException(json[cls.RETURN_MSG_KEY])
ATATools.ata_rest.ATARestException: Command failed - Command returned: ''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "observe_j0332+5434.py", line 50, in <module>
    main()
  File "observe_j0332+5434.py", line 33, in main
    ata_control.set_freq(freqs, ant_list)
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_control.py", line 833, in set_freq
    set_freq_focus(freq, antlist)
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_control.py", line 777, in set_freq_focus
    ATARest.put(endpoint, data={'value': freq})
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_rest.py", line 109, in put
    return cls._do_op(cls._OP_PUT, endpoint, **kwargs)
  File "/home/obsuser/miniconda3/envs/ATAobs/lib/python3.8/site-packages/ATATools/ata_rest.py", line 78, in _do_op
    raise ATARestException(str(e))
ATATools.ata_rest.ATARestException: Command failed - Command returned: ''

Should be easy to test on any of the other antennas

ATAcontrol.get_ant_pos() doesn't work

It seems that

ATA-Utils/pythonLibs/ATATools/ata_control.py

Line 135 in 3c81bcd

def get_ant_pos(ant_list):

doesn't work:

>>> ac.get_ant_pos(['1h','4g'])
process '['ssh', 'obs@control', 'fxconf.rb', 'antpos']' returned error
 b"usage: fxconf [-h] [--host HOST] [-p PORT] {help,sals,sagive} ...\nfxconf: error: argument operation: invalid choice: 'antpos' (choose from 'help', 'sals', 'sagive')\n"

The problem is that the fxconf.rb at control doesn't accept the antpos argument that get_ant_pos() uses to call it.

Create ephemeris + track_and_offset functions

@tkoumrian I added this in my local repository to do the ephemeris creation + track offsets.
Maybe something like this can be added to our python library.

_create_ephem_offset_source = None
def create_ephem(source, **ephem_kwargs):
    logger = logger_defaults.getModuleLogger(__name__)
    endpoint = '/ephemeris'

    global _create_ephem_offset_source
    _create_ephem_offset_source = source

    try:
        ephem_kwargs['source'] = source
        set_ephemeris_defaults(ephem_kwargs)
        retval = ATARest.post(endpoint, json=ephem_kwargs)
        ephem_id = retval['id']
    except Exception as e:
        logger.error('{:s} got error: {:s}'.format(endpoint, str(e)))
        raise


def track_and_offset(source, antstr, **ephem_kwargs):
    logger = logger_defaults.getModuleLogger(__name__)
    assert source == _create_ephem_offset_source
    try:
        antstr = snap_array_helpers.input_to_string(antstr)
        logger.info("Tracking source {:s} with {:s}".format(source, antstr))
        ephem_id = source
        json = {'id': ephem_id, 'wait': True}
        json.update(ephem_kwargs)
        endpoint = '/antennas/{:s}/track'.format(antstr)
        ATARest.put(endpoint, json=json)
    except Exception as e:
        logger.error('{:s} got error: {:s}'.format(endpoint, str(e)))
        raise

An observing script will essentially look something like this:

from ATATools import ata_control
ants = ["1a", "1f"]

ata_control.reserve_antennas(ant_list)

# create ephemeris file
source = "GPS-BIIRM-3--PRN-12-"
ata_control.create_ephem(source)

# now start cycling through pointings:
offset = [10., 0.]
ata_control.track_and_offset(source, ant_list, offset=offset)
record_data()

offset = [10., 10.]
ata_control.track_and_offset(source, ant_list, offset=offset)
record_data()
...