Specify LSS columns about ssim_dc1 HOT 16 CLOSED

@cwwalter

Chris, first I need to understand a bit better what is going on here: what is the reason we don't just get the full DM output? I.e. all columns look potentially interesting and while we can make our compressed value added catalog, having the full table and ability to test-drive DB queries seem like a valuable experience.

from ssim_dc1.

The scripts for filling every entry in every table in the baseline schema from the Stack output do not exist as far as I know. So the reason for asking is so that we can allocate the resources from our limited manpower to implement what you think you really need.

from ssim_dc1.

Ok, I see. So, what do we get by default? If we come with some minimal list, can we go back to the stack output directly and look at it ourselves? Presumably all this lives at NERSC?

from ssim_dc1.

The Stack produces catalogs from the analysis of the coadds with these columns, so we could provide db table access to any of those. Nevertheless, the FITS catalogs containing those data will be available at NERSC, so you can extract what you need from them directly.

from ssim_dc1.

From what I've seen in other issues it looks like we will have very low seeing variations (clear sky in PhoSim configuration) if any. Magnitudes from CatSim will be extinction corrected as well, right? Airmass will change in this simulation or will every target be simulated with airmass 1.0?

I think that we want, in addition to the DM columns, the extinction map consistent with the input (if any), an average airmass per exposure/target, and a map with seeing variations if there are any. And if depth variations are included at this point, we would need them as well (due to dithering or any other effect).

from ssim_dc1.

The airmass is in the OpSim info that drives PhoSim. Is there currently a way for us to pass that info into the final table in the work flow?

from ssim_dc1.

Right now the scripts that fill the db tables are run outside of our version of the Level 2 pipeline. In principle, we have control over the schemas for the tables, so we can add whatever columns we want.

from ssim_dc1.

Yeah, an extra column would work. At the end of the day we would produce some sort of map (healpix or other) with the average airmass (or any other systematic) per pixel across the footprint, and then cross-correlate it with our signal. If you are using MAF airmass/seeing/extinction, etc. maps as an input, we could use those in our analysis without adding any extra columns. With that being said: I would prefer to have the extra columns and the maps (if you use them), the more information, the better.

from ssim_dc1.

Yes, it's great to see javier on top of this. I know it is DC1 and so really testing the testing pipeline, but at some point we need a full enumeration of all simplifications that were done wrt to everything phosim is capable of doing.

from ssim_dc1.

Right now the scripts that fill the db tables are run outside of our version of the Level 2 pipeline. In principle, we have control over the schemas for the tables, so we can add whatever columns we want.

Do we have the ability to connect the info in the opsim schema with the output tables in the current pipeline?

from ssim_dc1.

Some of the info in the opsim db file does appear in the Visit table in the baseline schema. I'm not sure that the Level 2 pipeline is the most sensible place for that information to be "connected". Filling the Visit table would be a fairly trivial task that wouldn't require any special Level 2 code.

from ssim_dc1.

@jchiang87 Do we need another issue to make sure that the airmass gets added to the visit table?

Also, I am a bit unclear whether we have decided in this issue if (other than the airmass issue) we are happy with the default values Jim posted from the schema.

from ssim_dc1.

@jchiang87 Do we need another issue to make sure that the airmass gets added to the visit table?

I don't think so. When we fill the Visit table, we'll just fill it with the relevant info from the opsim db file, and that includes airmass.

from ssim_dc1.

The db default columns are good for us and as long as we have the visit table, and this table includes airmass, extinction, sky-brightness, seeing, and we know the dithering pattern. I guess that if we need some additional information, we will be able to get it even if it's not a straightforward process but, right now I can't think of anything else that we need.

from ssim_dc1.

@jchiang87 and @fjaviersanchez Are we good to close this issue now?

I think the conclusion is that we will take the default db database and add the relevant opsim info to the visit table.

from ssim_dc1.

yes, I think we're good.

from ssim_dc1.