mustaric / lambda-tess-search Goto Github PK

Is GPL acceptable for DSII?

We're already on the second version of make straws, and straws created by different versions of the code are incompatible, mostly because of metadata changes. We should store a version number in the metadata so we can check we're reading in the right kind of straws in LoadCube()

• Move codes that work in final prototypes out of sandbox.
• Write it all up to the level of "oh that's what they did" in repo README.

As required by DSII grant.

• Introduction: Present the problem we are trying to solve (big data, serverless, ...)
• Describe the lambda-calling-lambdas method.
• Describe the straws method.
• Comparison table between the two.
• Lessons learn.
• Conclusion: Summarize lessons learned and mention possible future work.

Then, send it off for printing. Do we want a fabric poster?

Expected each FFI to produce a CSV, but out of 1000-1300, only get back 800-900. @jbcurtin sees the same behavior with his own testing and will investigate.

@mustaric , do you know the typical size of the sub-cube you would need to extract from TESS cube off S3? In profiling the memory portion of it in EC2, I find that it takes incredibly long to grab a 1000 x 1000 slice off the cube, but 10 x 10 was almost instantaneous, and there is already some wait time for 100 x 100.