gwallison / openff-build Goto Github PK

• Number of unique "reckey" is the same as the total number of records
• All records should have a bgCAS (no NaN)
• all records with calcMass meet criteria
• all flagged duplicates are removed in std_filtered
• APINumbers are appropriately formatted

Since reckey is supposed to be a unique identifier for every record in the full data frame, why doesn't the number of unique == the number of records?

This feature was experimental. Sort of interesting but seems like there are better ways finding and labeling location related wells.

These nominally boolean flags were not included in the SkyTruth data and so currently we mark them with NaN. There were many disclosures from that era without those fields included on the PDF. However, all directly downloaded data from FracFocus have either a True or False value. Because it is set up as a boolean in FracFocus, some default value is added when the value is not reported. It seems possible that some of those T or F should actually be "unknown" or "unreported".

Need to investigate further...

Change the name to "chemrecs"

With the change to auto-detected sets, this flag is no longer used

Currently, there are over 500,000 records for which the calculated mass is zero. (A little more than half of those are non-data records; lines that Open-FF names ambiguous or sysApp.) That doesn't seem right. If a chemical is reported as being used in a fracking job, it MUST also have weight. Therefore, when a company reports a ZERO mass for chemical, it is an obvious non-disclosure. But by giving it a value for calcMass, it makes it appear that more masses are disclosed than is the reality.

This happens when companies set their PercentHFJob values to 0.0.
Thus, it makes sense to change mass calculations to filter 0 values out.

This change will appear in v12 (and v11-beta after Dec. 19, 2021).

These have not been used and are cluttering the variable space