egap / methods-guides Goto Github PK

Assigned @taraslough

@malisi The mechanisms guide is invisible (https://egap.org/methods-guides/10-things-mechanisms). The github source looks reasonable. Worth fixing ASAP.

@jwbowers I noticed that this guide's html file wasn't updated by your command last week. @amwilk and I have been attempting to knit it to html on its own, and it looks like there's an issue with a code chunk that's preventing it. Since you're the original guide author, I wanted to check with you to see if you can advise and/or knit it.

The guide should be edited. Small errors in the code should be fixed.

Hi @graemeblair and @nahomi. Where would you like comments and pull requests on the guides. It looks like they exist in two repositories now.

I found a typo in the meta-analysis guide. Happy to make a pull request but wasn't sure which repository was being archived https://docs.github.com/en/repositories/archiving-a-github-repository/archiving-repositories and which was being maintained.

FYI on the meta-analysis guide:

This next should have $\frac{1}{\hat{\sigma}_2^2}$ in the numerator of the second term rather than $\frac{1}{\hat{\sigma}_1^2}$

$$ \hat{ATE_{pooled}} = \frac{\frac{1}{\hat{\sigma}_1^2}}{\frac{1}{\hat{\sigma}_1^2} + \frac{1}{\hat{\sigma}_2^2}}\hat{ATE_1} + \frac{\frac{1}{\hat{\sigma}_1^2}}{\frac{1}{\hat{\sigma}_1^2} + \frac{1}{\hat{\sigma}_2^2}}\hat{ATE_2} $$

Tagging @malisi and @maartenvoors. Maarten reports slow loading of them. Maybe worth a test around the world? And a discussion with our web hosting folks? (Configure Cloudfront, AWS, etc.. appropriately)

From edits here 53d05f7

-[] Spanish
-[] French

This is just a place to record ideas for new methods guides:

• Attrition (Missing Outcomes)
• All the ways to analyze block randomized experiments https://www.tandfonline.com/doi/abs/10.1080/19345747.2020.1831115

see https://egap.org/resource/10-things-to-know-about-multiple-comparisons/

We want people to easily contribute to and improve on the guides. So, we should update the README.md that people see when arriving at the repo with some basic instructions about forking and making pull requests (only people with access to the repo itself can use the branching model that I've been using for my own edits). Assigning this to @malisi to make a draft of this README.

... to reflect the changes in this pull request: 4749e87

• Spanish
• French

Assigned to @cdgrady21 and @lulachen

@acoppock We've been recompiling guides and ran into a problem here. We had to remove the plot to get it to run through --- and the summary shows NAs. We'd like to have the nice plot again. Any ideas about what we should do? (I changed condition_names to conditions to update the call to declare_ra FYI.) The chunk starts on line 196 and is in the section headed # 7. Randomization inference for multi-arm trials

three_arm_dat <- read.csv("three_arm_dat.csv")
three_arm_dec <- declare_ra(N = 200, 
                            conditions = c("Control", "Treatment 1", "Treatment 2"))

ri_out <-
    conduct_ri(
      formula = Y ~ Z,
      declaration = three_arm_dec,
      data = three_arm_dat,
      sims = sims
    )


summary(ri_out)
## plot(ri_out)

@malisi @graemeblair I don't see the multisite guide on the main EGAP website. Are there edits that I've forgotten to do? It would be nice to have it online.

@jwbowers I remember the hypothesis testing guide in English was having trouble knitting to html after I added the linking script. can you either 1) knit this one as well or 2) show me how you got it to work last time?

The methods guides page https://egap.org/list-methods-guides is looking a bit disorganized because of the lack of images. I think even blank images would be better than what we have now if we do not have funds to pay folks to create images. Assigning this to you @malisi

Another possible guide.

@malisi I just found a few typos in the measurement guide. Worth a read through

The last word in this paragraph is not linked as it should be:

Randomization Protocol. What kind of randomization will you be employing? Simple randomization gives all subjects an equal probability of being in the treatment group, and then performs a (possibly weighted) coin flip for each. Complete randomization is similar, but it ensures that exactly a certain number will be assigned to treatment. Block randomization is even more powerful — it ensures that a certain number within a subgroup will be assigned to treatment. A restricted random assignment rejects some random assignments based on some set of criteria — lack of balance perhaps. These various types of random assignment can dramatically increase the power of an experiment at no extra cost. Read up on randomization protocols here.

In https://egap.org/resource/10-things-to-know-about-randomization/ this link doesn't work:
https://egap.org/methods-guides/10-strategies-figuring-out-if-x-caused-y

@lilymedina I think that "salida potencial" sounds weird. A couple of other folks who study causal inference and are native Spanish speakers agree. We suggest "resultado potencial" or "respuesta potencial". However, if the rest of the non-Chilean world thinks that "salida" is best, I'd defer.

Tagging @nahomi for her knowledge.

Reflecting these edits d0bf026

• Spanish
• French

Assigned to @malisi

See the changes here: aff3525

• French
• Spanish

I edited power.Rmd and submitted a pull request. However, I did not submit a pull request for a new power.html because I did not see code to recreate the figure. Is it possible to create the figure within the power.Rmd file itself? Or is there something about word press + Rmarkdown which suggests that it would be better to have a different file to create, in this case, the file simulatedpvals.png via some other route?

See the new paragraph here: 3f94d33

• French Version
• Spanish Version

@malisi The links on the tables of contents in the methods guides do not work on any of my browsers. Do they work on yours? Is there a fix for this? Some alternative solution that some searching on stackoverflow might suggest?

Assigned to @jwbowers

People commonly use F-tests or Likelihood Ratio tests (Even with cluster randomized studies). Hansen and Bowers 2008 showed that these approaches do not control the false positive rate in small samples. Explain and compare the different approaches.

Assigned to Brandon de la Cuesta

Use the Google Doc from Gareth and Winston currently in the Google Drive as a guide for editing power.Rmd.

Tagging @nahomi @maartenvoors and @malisi on this.

Meanwhile, I think that this piece would be of use for the initial "what is a good research question" part of the piece Fuck Nuance. I've assigned it my research design class week on "Theory" (alone with Theory-based medicine and the role of evidence: why the emperor needs new clothes, again and a couple of articles from the Stanford Encyclopedia of Philosophy) and I'm enjoying it.