Currently sometimes state either long (Victoria) or short name (VIC)

• standardise state values so they are the same, to assist with joining.
• standardise columns to be just "lga" instead of "lga_name" - no need for "lga_code" as not all data have it?
• use "population" as the value column where appropriate

Add ability to include known household age distributions as offsets in the 'home' setting model, in place of the whole population distribution (pop_age_to)

We were running into issues getting education data cleaned up - here https://github.com/njtierney/conmat/blob/master/data-raw/clean-education.R#L120

• so we have lower and upper age bounds
• make it easier for people to add their own data source to clean it up (so column names aren't hard coded)

Documentation was generated by sinew - which was really useful to template out, but it needs filling in.

The modelled results have a more pronounced 'ridge' of contacts between people of the same or similar ages for adults in 'school' and 'other' settings than is evident in the data.

Needs a tweak of the model structure to capture that ridge in the other settings but not this one.

this function fails:
abs_age_lga("Fairfield (C)") Error: Problem with mutate()columnage_group. i age_group = readr::parse_number(age_group). x is.character(x) is not TRUE

Issue seems to be that age_group is a factor.

Can be solved by wrapping it in as.character in function

abs_pop_age_lga_2020 %>% dplyr::filter(lga == lga_name) %>% dplyr::select(lga, age_group, year, population) %>% dplyr::mutate(age_group = readr::parse_number(as.character(age_group))) %>% dplyr::rename(lower.age.limit = age_group)

Separately - it might be useful to return both lower age limit and age_class in original factor form?

I think it makes sense to include in this repo functions for applying age-specific per-contact transmission probabilities to contact matrices in order to produce next generation matrices. There is various code for doing this in the contact_matrices_australia private repo that could be brought in here. None of the relevant functions or data are sensitive.

The interface might be something like:

ngm_5y_fairfield <- make_ngm(
  contact_matrices = synthetic_settings_5y_fairfield,
  age_contributions = davies_covid19(),
  R = 6
)

which involves the find_m() to do calibration, and apply_age_contribution() to apply the Davies estimates (though those will need interpolating from 10y age bins in order to be used more generally)

e.g.,

Using POLYMOD social contact data. To cite this in a publication, use the 'cite' function

• predict_contacts.R

This will break downstream code, but we might want to ensure we are consistent with how Prem and Mossong structure their contact matrices.

E.g.,

For example, #41 . The idea is then that one function gets the household size distributions for a given place, and function another to convert that into a per-capita household size.

Each of the setting-specific models include an offset term representing the age distribution of the contactable population. The model therefore fits to the difference between this and the observed number of contacts - estimating a propensity to contact each contactable person. That way, the contact matrix can be extrapolated to a new population based on the age distribution of that population - applying the model with an updated offset. This is the same as the population ratio reweighting in Prem, but in a model-based way.

Currently the contactable population for every setting is taken to be the whole population. So the rate of contacts is proportional to the fraction of the population in that age group - extrapolating to a younger population means more contacts with young people and fewer with old. That makes sense for most settings (population distribution predicts household age structure, age of workforce, age of casual contacts), but not school contacts.

Schools contacts are heavily cohorted - students are in classes with people either their calendar age, or the age 1 above or 1 below. Regardless of the population age distribution of the population, the age and number (class size) for students will remain the same. The only things that will change are the average number of contacts from adults to kids in school (fewer kids = fewer classes, and adults are less likely to be teachers), and the average number of contacts from kids to adults in school (teachers are likely to be younger).

As a consequence, predictions currently massively over/under estimate school contacts when extrapolated to younger/older populations. Redefining the offset for schools to capture class-based mixing (independent of population age distribution) for contacts between students, but also population distribution and for contacts between adults in schools and students. The prediction of the number of contacts within the same/adjacent calendar ages therefore wouldn't change when extrapolating, but contacts with adults would.

😱😱😱😱

Only when specified in the formula, by the looks of it. This makes extrapolation behave in some very weird ways, e.g. places with more kids have far fewer school contacts.

Will investigate and can always pass in with the offset argument instead

As mentioned in #33 by @goldingn

... population is paired with the contact data (represents the population when and where the survey took place, to capture the opportunity for contacts) so in a future implementation we might want to wrap them up in a single data object. That will make it easier when we start fitting to different/multiple surveys.

We should think about a way to wrap up the contact data and survey population into a single object/dataframe/list to pair them together.

Write up a vignette that walks through the entire process and motivation for how conmat creates the contact matrices.

This should be written up in a way that is aimed at people who might not be as familiar with epidemiology. So, it will go into detail on what each of the steps in the model fitting does, what the polymod data is, and answer questions like:

• "What is a contact matrix"
• "How do I adjust contact matrices for my age population"
• "How do I estimate the impact of vaccination on spread"

This will involve fixing and simplifying the README, which is #35.

From get_polymod_contact_data.R

age_from = part_age,
age_to = cnt_age,

imma fix it

paper: https://www.medrxiv.org/content/10.1101/2020.07.22.20159772v2
code: https://github.com/kieshaprem/synthetic-contact-matrices

e.g., fit the model to data from Melbourne and polymod simultaneously.

Currently it takes a long time to fit - would be good to resolve this using smaller data and examples

There's a lot of output in the README that could be cut down, and also the images and outputs keep changing on each render, when they should be fixed, so might need to fix the set.seed value?

There is code in the contact_matrices_australia private repo to adjust synthetic household contact matrices to account for household size distributions for that population . We should bring this over and incorporate it as an optional argument to predict_setting_contacts(). Probably a bit like this:

synthetic_settings_5y_fairfield <- predict_setting_contacts(
  population = fairfield_age_pop,
  contact_model = setting_models,
  age_breaks = c(seq(0, 85, by = 5), Inf),
  household_size_distribution = fairfield_household_sizes
)

The number of contacts is not proportional to the number of people in the area, only the relative number of contacts per age group relative to the relative population in each age group. So use normalised populations as offsets for fitting and prediction.

See https://github.com/njtierney/conmat/blob/master/_pkgdown.yml#L10-L57

We need to check that all the functions in the package appear in the lines above

This code is run a lot

fit_setting_contacts(
contact_data_list = get_polymod_setting_data(),
population = get_polymod_population()
)

We should export this as a data object to be used in the package

Depends on #38, which will introduce the ability to turn sets of setting-specific contact matrices into next generation matrices.

Code to digitise per-contact transmission probability matrices is in the contact_matrices_australia private repo. Note that these will be setting-specific (one each for home, work/school or other) so should be combined with the corresponding contact matrix before summing to get the NGM

Ie. for developing countries that don't have abs-equivalent data. Prem et al used DHS survey data. You need to apply to access that, but we have access at MAP that we might be able to use for this purpose?

Population count data

• Number of people in a given 5 year age bracket (0-5, 6-10, etc.)

Workforce data

• Age group breakdown of people in different type of workforces

Border control
Healthcare workers

depends on #38

Once we have functions to generate next generation matrices, add functions to apply vaccination to them in the same way as in the contact_matrices_australia private repo. This could look something like this:

ngm_5y_fairfield_vacc_0.5 <- apply_vaccination(
	ngm = ngm_5y_fairfield,
	coverage_by_age = age_coverage,
	efficacy_susceptibility = efficacy_susceptibility,
	efficacy_infectiousness = efficacy_infectiousness
)

https://njtierney.github.io/conmat/reference/predict_contacts.html

There's a few times where lapply is used where it might be beneficial to parallelise the code

i.e. when aggregating integer age bins with breaks c(0, 5, 10), 0 is being discarded, 5 is being assigned to the first bin and not the second.

This would be fine if we were explicit and consistent about it, and always indexed from 0, but polymod data indexes from 0, and socialmixr's wpp_age(), which we use, defines breaks based on the lowest age, and indexes from 0. So we just need to flip the definition in calls to cut(), so that in the example above, 0s get assigned to the first bin, 5s get assigned to the second, etc.

Hi @njtierney , Started a new issue on this one so that I work on everything related to this on another branch.

It seems lgas that look like the below ones are present in other datasets in conmat. If I change it in abs_lga_lookup , I should change it in other datasets like abs_household_lga and so on so that the existing functions (eg. get_per_capita_household_size()) don't break.

library(tidyverse)
conmat::abs_household_lga %>%
  filter(str_detect(lga,patterns))%>%
  distinct(state,lga)

#> # A tibble: 20 x 2
#>    state lga                                   
#>    <chr> <chr>                                 
#>  1 NSW   Campbelltown (C) (NSW)                
#>  2 NSW   Central Coast (C) (NSW)               
#>  3 NSW   Migratory - Offshore - Shipping (NSW) 
#>  4 VIC   Kingston (C) (Vic.)                   
#>  5 VIC   Latrobe (C) (Vic.)                    
#>  6 VIC   Migratory - Offshore - Shipping (Vic.)
#>  7 QLD   Central Highlands (R) (Qld)           
#>  8 QLD   Flinders (S) (Qld)                    
#>  9 QLD   Migratory - Offshore - Shipping (Qld) 
#> 10 SA    Campbelltown (C) (SA)                 
#> 11 SA    Kingston (DC) (SA)                    
#> 12 SA    Migratory - Offshore - Shipping (SA)  
#> 13 WA    Migratory - Offshore - Shipping (WA)  
#> 14 TAS   Central Coast (M) (Tas.)              
#> 15 TAS   Central Highlands (M) (Tas.)          
#> 16 TAS   Flinders (M) (Tas.)                   
#> 17 TAS   Latrobe (M) (Tas.)                    
#> 18 TAS   Migratory - Offshore - Shipping (Tas.)
#> 19 NT    Migratory - Offshore - Shipping (NT)  
#> 20 ACT   Migratory - Offshore - Shipping (ACT)

Created on 2022-05-26 by the reprex package (v2.0.1)

using select in mgcv

library(conmat)
library(readr)
library(dplyr)

oz_pop <- abs_pop_age_lga_2020 %>%
group_by(age_group) %>%
summarise(
population = sum(population)
) %>%
mutate(
lower.age.limit = parse_number(as.character(age_group))
) %>%
mutate(
country = "Australia"
)
oz_pop

fit_setting_contacts(
contact_data_list = get_polymod_setting_data(),
population = oz_pop
)
#> Error in if (sum(uconv.ind) == 0) {: missing value where TRUE/FALSE needed

Thanks to @goldingn for working out when this error typically occurs:

Because the interpolated population function was predicting 0 population for some ages, which then led to a non-finite offset

The goal here it to either try and capture that error exactly, and give a more informative error, replacing it with suggestions.

Alternatively, identify this ahead of time somehow, before model fitting, and describe how to fix that issue.

- [x] called setting_weights - for each of the settings (home, school , work, other)
- [ ] Provide greta code to reproduce these weights in data-raw
- [ ] Provide a function to appropriately map the weights to the transmission probability settings

E.g. as function of household sizes, population distributions, and socio economic/cultural indicators.

Could use Prem et al methods or something more simple.

abs_household_lga uses data with an older set of LGAs than abs_pop_age_lga_2020, so the code to adjust household sizes doesn't work for all LGAs.

We'll need to use a correspondence to line them up. I'll see what I can hack together for this one off code I'm writing and paste here.

e.g., https://github.com/njtierney/conmat/blob/master/R/add_school_work_participation.R

We want to have functionality where people can refit models, but it doesn't need to be refit each time

https://github.com/njtierney/conmat/blob/master/R/get_age_population_function.R#L3-L5

Currently this requires columns from socialmixr::wpp_age(), which must contain lower.age.limit and population - make this more general so you can name the columns.

Alternatively/in addition, could have a function that is specific to socialmixr::wpp_age().

When outputting matrices, 'age_group_from' should correspond to columns, and "age_group_to" to rows. That way, after conversion to a next generation matrix, a matrix multiply will give the number of infections in each age group at the next step.

Currently the output matrix is being output the other way: https://github.com/njtierney/conmat/blob/master/R/predictions_to_matrix.R#L17-L18 ('from' on the rows)

This also applies to the setting transmission matrices.

This doesn't appear in the plots, which are fine since it's the other way around in matrix_to_predictions() too, so it cancels out.

plot_setting_matrices() requires the patchwork package, but that isn't imported, so it errors if it isn't loaded separately

in predict_contacts(), the contacts are predicted at integer resolution using the population age interpolator function to define population sizes, and then integrated to get the aggregate matrices. For some population age distributions, this predicts 0 population within the age bounds of the integration, which leads to non-finite prediction of contact rates. The integration range needs to be restricted to prevent this from happening during prediction.

A number of lgas listed in abs_lga_lookup aren't listed in abs_household_lga:

> anti_join(abs_lga_lookup, abs_household_lga, by = "lga")
# A tibble: 13 × 3
   state lga_code lga                              
   <chr>    <dbl> <chr>                            
 1 NSW      10500 Bayside (A)                      
 2 NSW      12160 Cootamundra-Gundagai Regional (A)
 3 NSW      12390 Dubbo Regional (A)               
 4 NSW      15700 Nambucca Valley (A)              
 5 SA       40150 Adelaide Plains (DC)             
 6 SA       45400 Orroroo-Carrieton (DC)           
 7 WA       54200 Kalamunda (C)                    
 8 WA       54280 Kalgoorlie-Boulder (C)           
 9 TAS      60210 Break O`Day (M)                  
10 TAS      62410 Glamorgan-Spring Bay (M)         
11 TAS      65410 Waratah-Wynyard (M)              
12 NA       99399 Unincorp. Other Territories      
13 NA          NA NA