Check negative net generation methods and outputs about open-grid-emissions HOT 3 CLOSED

With regards to cleaning negative numbers: it appears that the only place this is currently happening in our pipeline is with the raw EIA-930 data. We should probably fix this as part of #113

from open-grid-emissions.

There are several areas where there is a risk of multiplying by a negative number, primarily in src.impute_hourly_profiles:

1. In calculate_scaled_residual() the scaled residual is calculated as the product of the cems net generation and a scaling factor, which is min(eia930 net gen / cems net gen). If the cems profile is negative at any porint, it could lead to a negative scaling factor. Also, if the cems profile itself is negative, it could cause issues. We should investigate this further and implement fixes.
2. In shape_monthly_eia_data_as_hourly() we multiply each of the data columns by the profile as a percentage. We have already implemented a check so that in any months where the net generation is negative, we assingn it a flat profile. However, we have not checked whether the profile itself is negative.
3. When scaling the partial cems data, we have checks implemented to catch negative scaling values.

When we convert our profile to a percent, we divide each hourly value by the monthly total value. However, if the profile has any negative values in it, this will affect the monthly total and potentially create larger swings in the data. We should probably check whether this monthly total is ever negative

from open-grid-emissions.

Another issue that I'm discovering with the CEMS net generation data is that our current adjustment methodology sometimes results in hours or days where the negative net generation is in the hundreds of MW - it seems unlikely that a non-energy-storage generator would ever be consuming that much electricity as a house load. We should use data from EIA-923 or the disposition data to get an idea of maximum negative draw and filter out methods that would cause large negative values.

Based on the EIA-923 data, the largest negative generation for a single generator in a month is about -19,000 MWh, which works out to about -25MW on average in each hour. Thus, we probably should not see negative net generation values much larger than that created as a result of our gross to net conversion.

We can set a threshold such that the gross to net shift factor does not result in a negative net generation value less than, say -50 MW in any single hour. If it would, we can drop the shift factor and use the scaling factor instead, which would result in no negative net generation in any hour, but still result in a monthly total net generation that matches the EIA-reported value.

from open-grid-emissions.