Poor NIRSpec Performance about eureka HOT 12 CLOSED

The difference in MAD values has to due with having different wavelength ranges. A lot of the extra noise in this case is coming in from >5 microns. This is why the S4 calculation of MAD is limited to the specified wavelength range. It allows for an apples-to-apples comparison.

from eureka.

Hey Taylor, I think @kevin218 got pretty much the same result as this at some point in the past. I'm in the process right now of finalising a new NIRSpec dataset / tiny version and don't want to spend too much (if any) time looking at the old dataset as there are a few changes since then. Once I have that done (I'm hoping today / Monday next week), I'll do a quick run through Eureka to see whether the same bug crops up again and let you know.

One thing I am confident of is that the data isn't this bad, and it's some pipeline step that's going wrong / not interacting with the data in the way we expect it to.

from eureka.

Very much understand that - thanks for looking into this when you get the chance! One thing to note is that I started from the Stage 1 outputs, and given your comment on the ERS Slack I suspect these observations also do not have a transit injected into them at this stage? Good to keep in mind, but the data should indeed look far cleaner than this

from eureka.

Oh right, these plots look almost the exact same as in Issue #35

from eureka.

Yeah if you started from the uncal.fits file, there won't be any transit injected.

from eureka.

Ahh, I started from the _rateints.fits file, so there is actually a transit injected. Kevin's guess on Issue #35 was right that our threshold constraints were too tight - increasing p3thresh, p5thresh, and p7thresh to 10, 20, 20 instead of 5, 10, 10 significantly improves the outputs of Stage 3 (see attached image where a transit is actually visible). I'm not sure what is driving the rest of the awfulness, but it's possible we need even looser thresholds

from eureka.

Alright, I was able to get at least fairly reasonable outputs with threshold sigmas of 5, 10, 40 now. It is the p7thresh value that was really the issue. I tried using smooth and poly profiles as well, and both gave what appeared to be all NaN values.

The outputs of Stage 4 look horrible because the occasional drops in flux down to zero do not get clipped at any point. We'll probably want to add at least a mild sigma clipping in Stage 4 to remove the extreme outliers.

from eureka.

Not surprised about poly, but smooth should have performed well enough. One more issue to look into, I suppose.

Glad the looser threshold values are working out, but that's also an indication of a poorly constructed weighting profile. Not sure if this is due to the fact we're using CV3 data or if there's something wrong with our NIRSpec reduction.

from eureka.

Are people happy to say we're no longer in the "poor" performance regime for the NIRSpec data reduction? I've attached the MAD plot I made just before the Baltimore data challenge and it's a nice improvement over the predecessors. @cpiaulet @erinmmay @evamariaa, did you do better than mine / would you agree that we can close this?

from eureka.

Mine looks similar (see plot below) so I agree that this issue can be closed.

from eureka.

Sure! I think there is still a bunch to investigate though (will open another issue on this) as to the sensitivity of the quality of the extracted light curves to the S3 ecf parameters.

from eureka.

I think this issue can be close now as well. The one thing I'm surprised about is how different the MAD values are between Caroline and Aarynn's plots as I would have said Aarynn's reduction looks far better than Caroline's but the MAD values disagree. That's not really the same issue though

from eureka.