Comments (13)
Hi @cdeil, thanks!
A TeV-catalog sounds like it would be a fantastic addition! And I think our framework could be easily adapted for your purposes. We've started a wiki page describing how to create a new catalog --- a process which I'm working on myself as-well. It's only a rough outline at the moment, but will continue to be expanded.
The current procedure is to clone the astrocats source code, in addition to an existing catalog (e.g. supernovae
) which is then copied and modified to create a new one. Currently it would be much easier to both contain the repositories under github/com/astrocatalogs
and the backend on @guillochon 's server space --- we're happy to host it for you.
from astrocats.
@cdeil So as you guys already have your own domain, it would definitely be possible to have the catalog on your domain and hosted by us (or even hosted by you on your own server). Right now I see what you guys have is a nice map as your home page, but it might be nice to have the catalog and the map be separate pages that are both available through the gamma-sky.net domain.
The first step is what @lzkelley suggested: To get the import script working such that you can use the scripts that generate the web-based catalog. We are happy to host the final product in whichever form you prefer, but it's probably worth discussing what's best before we make a decision. I'll also gladly link to it from the astrocats.space domain once it is at least in a "beta" state.
Feel free to join us in our slack channel: https://astrochats.slack.com/messages/general/
from astrocats.
@lzkelley and @guillochon - Thanks for the quick and positive reply.
My preference would be if the TeV catalog is maintained under your Github repo and domain, to have this more clearly be a community thing that's separate from the Github org I've started https://github.com/gammapy/ and the webpage @vorugantia and I have started http://gamma-sky.net/ .
For years there has been http://tevcat.uchicago.edu/ as a separate catalog and website, and as I mentioned in the other issue, they don't want to make their catalog fullly open. My hope would be that their TeVCat and this new TeV source catalog I want to start here will unite into a community-maintained catalog like what you've set up for other domains of astronomy. For now and in any case, we'd just follow your community open catalog model and do it for TeV sources.
I'd like to get started on this tomorrow morning and see if I can ingest some initial TeV catalog and get it working. Thanks for the link to your wiki and writing up the steps.
Should I start with repos under the gammapy org for now or are does it seem better to you to make repos for this under your Github org now?
from astrocats.
It should just be "Open TeV catalog" and repo name e.g. open-tevcat
(to distinguish it from the existing http://tevcat.uchicago.edu/
from astrocats.
Looks like https://astrochats.slack.com/messages/general/ is password-protected. Is anyone with a Slack account able to join or do you have to invite me before I can join? My email is listed on my Github profile page.
from astrocats.
Hi @cdeil, I just invited you.
from astrocats.
@cdeil As for where the repos should live, you should just fork them for now and when you think it's close to being ready we'll fork it into astrocatalogs. I can tell you more over Slack.
from astrocats.
FYI: I'm starting this now here: https://github.com/gammapy/gamma-cat
For the next few days I'll just roll my own Python scripts. I think that's easier for me to get something done, and once it's there you'll see what TeV catalog we have and can better advice if / how using astrocats
makes sense.
I did try to follow
https://github.com/astrocatalogs/astrocats/wiki/Catalog-Creation-Guide
but then
git clone [email protected]:astrocatalogs/supernovae.git astrocats/supernovae
and
python -m astrocats supernovae import --tasks internal
did take a lot of space (6 GB I think) and time, and it wasn't even clear to me what the result / output would be.
So if you have the time, I think it would be much better to have the "getting started" guide use an example catalog with very few data (1 MB and few files) that can be processed within seconds, so that people have something useful running quickly and see what astrocats
does and try to understand how it works.
from astrocats.
@cdeil, this isn't in that guide I don't think but did you try this:
python -m astrocats catalog import
That runs the example import, which is a much more lightweight set of tasks.
Sorry things aren't working out immediately! We'll try better to make the install process more straightforward.
from astrocats.
gamma-cat is continuing outside of the astrocats framework.
from astrocats.
Yes, for gamma-cat, I've now decided to write my own Python scripts, that generate a static webpage via Sphinx (coming soon).
I could never quite figure out if the astrocats framework would work for our application, and what benefits and drawbacks it would bring, compared to just rolling our own.
But @lzkelley and @guillochon - please know that even if we're not using astrocats, I'm very grateful for your feedback and help and to show the way how something like this can be set up (text files, git, webpage, community effort, paper). Thank you!
from astrocats.
No hard feelings whatsoever @cdeil! Please keep us posted when gamma-cat becomes available, would be interested to see if any of the choices you made could be inform our future design decisions.
from astrocats.
One choice I've made is that input files are YAML for hierarchical data, and ECSV for tables (see input
folder in our repo. For input validation I'm using JSON schema and Python scripts, with the JSON schema again read from YAML files. Basically because I don't like editing JSON files, YAML files are more pleasant.
For the output, the plan (to be implemented in the next days) is to expose the data via index JSON files that list and link to all available data. And for the individual data files I'm not sure yet, I was thinking JSON for hierarchical data, and ECSV for tables. For the summary catalog, I'm planning to offer a FITS and ECSV version. Basically I don't know yet what format users prefer for the output, because we only have very few users so far. We're also developing a Python package to access and work with the data with domain-specific classes (e.g. spectral model, spectral points, morphology model, lightcurve) here and we're using gamma-cat as one of the source catalogs on http://gamma-sky.net/ (not live yet, a preliminary version should go live in the next days) to make browsing the catalog and comparing the MWL data easy. For gamma-sky.net we have a proceeding published, for gamma-cat nothing yet.
So it's different in detail from your choices, but the overall concept is very similar.
I think in the end, we had a lot of domain-specific things, so that using your framework would not have been the right choice. But I might be wrong and should just have taken some time to learn what you have and see if / how our stuff fits in.
from astrocats.
Related Issues (20)
- Can't use --depth command anymore? HOT 2
- Photometry: 'u_time': MJD is being added as default even to 'host' entries without time value HOT 2
- Move git-related `supernovae` scripts to `catalog` methods.
- Separate `PATHS` class from `Catalog`, and combine with `git-` methods. HOT 1
- Get 'update' mode working again HOT 2
- The path to v1.0
- Improve logging levels and logging file
- Logger output conflicts with tqdm
- iframe doesn't resize when entry is opened in background tab
- Replace `future` with `six` HOT 2
- Combine `Entry._init_cat_dict()` and `Entry._add_cat_dict()`
- Entry.sanitize() fails (at times) with photometry that dont have `TIME` values.
- Add setting to determine warning/logging/error level when adding data fails
- Cleanup and improve PHOTOMETRY.TIME parameter HOT 3
- Consider using astroquery for image cutouts
- Importing the example catalogue [supernovae] results in error RuntimeError: Return value should have been None! HOT 2
- astrocats cannot import catalogs when installed with Anaconda HOT 3
- Failed OSC update HOT 2
- api.astrocats.space certificate expired HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from astrocats.