read_many_patterns breaks aggregation when missing values for distance_from_home about safegraphr HOT 6 CLOSED

Shoot! I will take a look at this, thanks for letting me know.

from safegraphr.

Figured it out - this is due to now the na.rm option works. It only removes missing values for variables specified in expand_int or expand_cat. It doesn't drop missings for distance_from_home, then, so sum() returns an NA if there are any missing values.

I will try to get a fix for this. In the meantime, you can get around this by specifying an aggregation function that handles NAs. So fun = function(x) sum(x, na.rm = TRUE) in your read_patterns() call should handle it.

from safegraphr.

Thanks Nick! Appreciate any help you can offer with this, will give that a try.

from safegraphr.

I gave that a try and received this error:

Attempted to find start_date from filename. I think it's YYYY-MM-DD 2020-03-02
Error in `[.data.table`(patternsb, , lapply(.SD, fun), by = by) : 
  Column 4 of result for group 2 is type 'double' but expecting type 'integer'. Column types must be consistent for each group.

Any idea on what the problem could be? Thanks!

from safegraphr.

Hmm, two possibilities come to mind - the first is that you've left some columns in there that sum() doesn't accept (try the select() option to get rid of them). Another is that, within states, sum() produces a double in one group but not another for some reason.

Two approaches to fixing:

1. Try `fun = function(x) sum(as.double(x), na.rm = TRUE)
2. The benefits of read_patterns really show up when you're using one of the expand options. Since you're not, you might want to just use fread to read the file directly, use fips_from_cbg() to extract the state and county FIPs values (see the help file for examples), and then aggregate it yourself, which will give you more control.

from safegraphr.

Should be fixed by 2bc66ad

from safegraphr.