Giter Club home page Giter Club logo

pocs's Introduction

Welcome to POCS documentation!

PAN001


GHA Status codecov Documentation Status PyPI version

Project PANOPTES

PANOPTES is an open source citizen science project designed to find transiting exoplanets with digital cameras. The goal of PANOPTES is to establish a global network of of robotic cameras run by amateur astronomers and schools (or anyone!) in order to monitor, as continuously as possible, a very large number of stars. For more general information about the project, including the science case and resources for interested individuals, see the project overview.

POCS

POCS (PANOPTES Observatory Control System) is the main software driver for a PANOPTES unit, responsible for high-level control of the unit.

For more information, see the full documentation at: https://pocs.readthedocs.io.

Install

POCS Environment

If you are running a PANOPTES unit then you will most likely want an entire PANOPTES environment, which includes the necessary tools for operation of a complete unit.

There is a bash shell script that will install an entire working POCS system on your computer. Some folks even report that it works on a Mac.

To install POCS via the script, open a terminal and enter (you may be prompted for your sudo password):

curl -fsSL https://install.projectpanoptes.org > install-pocs.sh
bash install-pocs.sh

Or using wget:

wget -qO- https://install.projectpanoptes.org > install-pocs.sh
bash install-pocs.sh

The install script will ask a few questions at the beginning of the process. If you are unsure of the answer the default is probably okay.

In addition to installing POCS, the install script will create the Config Server and Power Monitor services, which will automatically be restarted upon reboot of the computer.

POCS Module

If you want just the POCS module, for instance if you want to override it in your own OCS (see Huntsman-POCS for an example), then install via pip:

pip install panoptes-pocs

If you want the extra features, such as Google Cloud Platform connectivity, then use the extras options:

pip install "panoptes-pocs[google,focuser,testing]"

Running POCS

POCS requires a few things to properly run:

  1. A panoptes-utils config-server running to provide dynamic configuration.
  2. An Observatory instance that has details about the location of a POCS unit (real or simulated), which hardware is available, etc.

A minimal working example with a simulated Observatory would be:

import os
from panoptes.utils.config.server import config_server
from panoptes.pocs.core import POCS

os.environ['PANDIR'] = '/var/panoptes'
conf_server = config_server('conf_files/pocs.yaml')
I 01-20 01:01:10.886 Starting panoptes-config-server with  config_file='conf_files/pocs.yaml'
S 01-20 01:01:10.926 Config server Loaded 17 top-level items
I 01-20 01:01:10.928 Config items saved to flask config-server
I 01-20 01:01:10.934 Starting panoptes config server with localhost:6563

pocs = POCS.from_config(simulators=['all'])
I 01-20 01:01:20.408 Initializing PANOPTES unit - Generic PANOPTES Unit - Mauna Loa Observatory
I 01-20 01:01:20.419 Making a POCS state machine from panoptes
I 01-20 01:01:20.420 Loading state table: panoptes
S 01-20 01:01:20.485 Unit says: Hi there!
W 01-20 01:01:20.494 Scheduler not present
W 01-20 01:01:20.495 Cameras not present
W 01-20 01:01:20.496 Mount not present
I 01-20 01:01:20.497 Scheduler not present, cannot get current observation.

pocs.initialize()
W 01-20 01:01:28.386 Scheduler not present
W 01-20 01:01:28.388 Cameras not present
W 01-20 01:01:28.389 Mount not present
S 01-20 01:01:28.390 Unit says: Looks like we're missing some required hardware.
Out[10]: False

For a more realistic usage, see the full documentation at: https://pocs.readthedocs.io.

For actually deploying a PANOPTES unit, refer to the Operating Guider.

Developing POCS

See Coding in PANOPTES

To test the software, you can use the standard pytest tool from the root of the directory.

By default all tests will be run. If you want to run one specific test, give the specific filename as an argument to pytest:

pytest tests/test_mount.py

Links

pocs's People

Contributors

anthonyhorton avatar blackflip14 avatar bsipocz avatar capsulecorplab avatar danjampro avatar fergusl avatar gdrosos avatar huntsmanscope avatar immortalrabbit avatar jaalvarado-montes avatar jamessynge avatar jeremylan avatar jermainegug avatar jlibermann avatar joshwalawender avatar kmeagle1515 avatar kstoreyf avatar lspitler avatar lucasholucasho avatar mikebutterfield avatar mimming avatar sme12435 avatar wtgee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pocs's Issues

Set up Wiki page for metadata to be stored with data

We need to compile a master list of meta data to be stored in each raw image and in each "postage stamp cube" generated from an image sequence. A wiki page where people can contribute to the list seems like the right place for now.

See also #42 .

Pretty Pictures

Determine best way to do pretty pictures. This could be manually, automatic for each image, only for first image, etc. Also figure out best techniques for generating pretty pictures automatically. Possible develop web-based interface for PAWS that would allow unit owners to dynamically alter images.

Mount Simulator

Develop the mount simulator so that POCS can be used without an actual mount.

See also #29

Accelerometer Positioning

We want to be able to recover the pointing position of the mount based on the accelerometer readings located inside the camera box. Various ways to do this exist. See e.g. (in no particular order):

http://perso-etis.ensea.fr/~pierandr/cours/M1_SIC/AN3397.pdf
http://arxiv.org/abs/1407.0035
http://www.it.bton.ac.uk/staff/gw4/papers/Displacement%20from%20Accelerometer.pdf

Values are available directly off the accelerometer (stored in mongodb locally) or off BigQuery. Values take the form:

                "camera_box" : {
                        "accelerometer" : {
                                "o" : 7,
                                "x" : -0.96,
                                "z" : -0.16,
                                "y" : -0.15
                        }
                }

where o represents the orientation (which can determine which side of the pier the accelerometer currently is)

See also #41 as we want to switch how we are reading the data as well.

libgphoto2 client (gphoto2 replacement)

gphoto2 is being used to control the cameras but has a number of problems when dealing with multiple cameras. Even when the model and port are specified the client will issue a USB reset on the hub, effectively killing any other cameras that are currently in operation and resetting the USB device IDs. This is also complicated by potential power cable issues on PAN001 which makes it hard to set on the current setup.

A new client can be written directly from the libgphoto2 library that is specific to PANOPTES. This client could handle the details of controlling both cameras (or not). This client would replace gphoto2's main.c: https://github.com/gphoto/gphoto2/blob/master/gphoto2/main.c

Prototype Mount Module

Build proof of concept mount module to control iOptron mounts.

See wiki for some specifications:

When the methods of the mount class communicate (via serial) with the mount hardware, we strongly encourage the author of the code for that particular mount type (i.e. Orion, Celestron, Meade, etc.) to communicate with the mount using physical mount coordinates: Hour Angle (HA) and Declination. There are several advantages to this:

  • Keeping all calculations relating to the conversion between J2000 (RA and Dec) to physical coordinates (HA and Dec) means that those calculations will always be consistent because we will know which corrections have been handled (i.e. precession to Jnow, refraction correction, etc.),
  • Issuing slew commands in physical coordinates relieves the author of the code for the individual mount type of having to insert checks for the internal mount time and location values. This avoids potential confusion of the mount time and location are not matched to the time and location POCS.
  • Using physical cordinates means that the same serial slew command to the mount can also be used for parking.

Assemble Camera Control Electronics

Assemble prototype arduino control system for the cameras. Should include:

  1. Control of camera shutter via TTL signal
  2. Relay to connect/disconnect camera from USB
  3. Temperature and humidity sensing
  4. Orientation sending via accelerometer or IMU

Stop / Restart with Bad Weather

Seems to be an issue with the unit starting back up automatically after bad weather has been triggered. Related to movement through state machine, not from bad weather readings. Happening intermittently.

Handling of Observatory Conditions

Hi all,

I have a question on the philosophy of the observatory conditions. There are things like weather.safe and mount.slewing and camera.exposing which typically have two possible values and which are critical for decision making in the state machine. In each while_(state) function, I query those conditions and check to see if they match the expected situation for that state.

Should I query them at the beginning of each function and populate a variable/property, then use that variable/property in the logic or should I query the condition multiple times. For example, consider a while function for something like the slewing state:

currentCondition = "slewing" weather.query_condition() # populates the weather.safe property with True or False mount.query_slew_state() # populates mount.slewing if weather.safe and mount.slewing: ## do stuff for getting ready state else: ## this means conditions don't match, so we need ## to figure out what to do based on how they don't match if not mount.slewing and weather.safe: ## we're done slewing so move on to imaging or some other state if not weather.safe: ## park because weather is not safe

Compare this with an alternative in which each test of weather.safe (or any other condition) queries it:

currentCondition = "slewing" if weather.safe() and mount.slewing(): ## do stuff for getting ready state else: ## this means conditions don't match, so we need ## to figure out what to do based on how they don't match if not mount.slewing() and weather.safe(): ## we're done slewing so move on to imaging or some other state if not weather.safe(): ## park because weather is not safe

This second example will make 3 queries to weather and 2 queries to mount compared to a single query to each in the first example.

I'm inclined to use something like the first situation so we query each property only once during the execution of the function. The reason I prefer this is that even though each function should take a very short time to execute, it might happen that a condition changes halfway though the execution of the while_(state) function. If that happens, the logic may get totally screwed up as a conditional statement evaluates differently at different positions in the function code.

Anyway, I wanted to see if there were any thoughts on this before I refactor the while_(state) functions I've written to use that methodology.

-Josh

Camera Simulator

Camera simulator needs to be able to produce images, including pointing images (ideally with some offset). Need a folder of test images to work with this.

Split `images` utils file.

panoptes.utils.images is now way too large. Should be split into smaller logical sub-util modules, e.g. panoptes.utils.images.fits, etc.

Target / Observation / Exposure Refactor

The implementations of the Target and Observation classes are currently clunky and need some major cleaning up. This should be done with respect to what we want the eventual Scheduler to look like (see #44 ). cameras should not needlessly be passed into objects.

Data Pipeline

The data pipeline handles everything from the raw image off the camera to the processing that will happen for images across the PANOPTES network. It is composed of a number of distinct parts.

Camera Stop/Shutdown

Have a better method for interrupting camera to guarantee we don't get in a locked out state with exposure. Should be able to build into the subprocess/async.subprocess easily.

Create CONTRIBUTING file

We need a CONTRIBUTING.md file for each of our repos. This would include the standard Getting Started, Making Changes, Code Formatting, etc.

Integrate weather station in to computer enclosure

The weather station components must be integrated in to the computer enclosure. This requires some to be outside and some to be inside, so careful consideration of weatherproofing and access will need to be done.

Scheduler Constraints

Developer scheduler Constraints. This should be done with consideration to the ongoing development in astroplan. Minimum first Constraints to be developed should maximize observable time so that targets about to go below horizon are not selected.

Coordinate Conversion Utility Function

From wiki:

Because we will need to convert between J2000 coordinates and physical mount coordinates for all mounts, we should have a utility which does that conversion which all mounts use (rather than reproduce it for each mount). We need a method, probably as part of the observatory class, which takes in J2000 coordinates and returns HA and Dec.

To begin with I think this should:

  • Convert from J2000 to Jnow
  • Convert Jnow to HA and Dec

We can later add things like atmospheric refraction correction, geocentric vs. topocentric coordinates, etc. as we see fit.

Webcam Development

We currently have two webcams attached to each unit. These are currently used simply for checking on the unit in the day. These timelapse photos are then archived as a simple movie each day.

Potential other uses for the webcams (brightness levels? cloud cover?) should be explored.

Preprocessing Step (after image sequence complete)

This step can be handled during Housekeeping or immediately after full sequence is complete.

  • Image files converted from .cr2 to .fits: CAMID_YYYYMMDD_HHMMSS.fits
  • Postage stamp cubes created around each catalog entry
  • Populate metadata and add data quality metrics for postage stamp cubes
  • First image of sequence plate solved and compared to existing catalog of stars of interest
  • Move postage stamp cubes to Cloud
  • Co-add image sequence
  • Upload metadata for sequence

Part of #50

Assemble Camera Enclosure Hardware

Assemble camera enclosure with mounting scheme for cameras and components. Electronics must be coordinated with the Camera Control Electronics project.

Supervisord

Review supervisord scripts and make sure we are starting all items appropriately.

  • Weather scripts (#25)
  • Sensors scripts (#25)
  • PAWS site
  • Unit script
  • Other? (e.g. maintenance scripts, data processing, updating, etc.)

Web Cams for Panoptes

We need hardware and software for web cams for the Panoptes baseline. The cameras Oli is using in the prototype work fine in daytime, but I'm wondering if we can get better sensitivity. I've ordered one of the Raspberry Pi camera boards with the IR filter removed (i.e. http://www.adafruit.com/products/1567) for a personal project. I'm going to try to set it up to do integrations and see how sensitive it can get. If this works, we might use this sort of system as a DIY web cam. If not, I suspect we can easily reproduce what Oli did in the prototype.

Assemble Weather Station

Assemble a prototype weather station module using Arduino board and components. This is a prototype in that assembly will use a breadboard instead of soldering components in to a mounting.

Better Sensor Reading Scripts

Currently handled by scripts/read_serial.py. Could be more robust. Also could integrate with panoptes/read_weather.py. Could also integrate with supervisord.

Incorporate trim rates in RA and Dec based on pointing/tracking model

Based on feedback, set or update a "trim rate" on the RA and Dec tracking rates. The goal is to maintain the position of the stars on the pixel grid to sub-pixel accuracy over the duration of an observation (many images).

One way to generate the feedback is to only look at pointing errors, then make a physical model of the mount. This is the traditional way of doing this. In the earlier version of the prototype @oguyon did this differently. He measured the offset between subsequent images (via cross correlation) and estimated a tracking rate error for that position in mount coordinates (HA and Dec). With a sufficient number of these points, he fit a polynomial to the data to create the model. Then each time a target was slewed to, he evaluated this model and applied the appropriate trim rates. I tend to prefer this method as easier to understand than the physical model which was often modified by arbitrary polynomials anyway, so it wan't very physical.

There are also a few ways we might input these trim rates. First, we could use mount commands (either direct serial commands or via an INDI layer). Second, we could use the arduino in the electronics package to make "guiding" corrections to the mount. The rate correction would basically be a PWM signal output by the arduino to the mount guide inputs.

Weather Params Tuning

The current weather settings are adequate but not ideal. Tuning of rain, cloud, and humidity values should be based on observed conditions at site. See also #24

Arduino / Firmata Interaction

We have custom arduino code both in the computer and camera boxes. With the updates to the actual electronics interface we also want to start placing Firmata directly on the boards so that the control can be handled over to Python (or some other client code). This needs to be implemented and the clients need to be written to digest and interact with the arduinos

Daily Housekeeping

Determine tasks that need to be accomplished during the Housekeeping state, which is run after the unit is parked. This should either be run directly after parking or at some set time each morning.

  • Cleanup observations directory
  • Upload images to cloud storage
  • Upload metadata to (TBD) cloud.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.