Refutation tests of ATE and heterogeneous effect about econml HOT 2 OPEN

It would be great to have a bit more information - like what are the units of the quantities that you're measuring, what are you using as an instrument, etc. Also, you mention the point estimates, but it would also be important to look at the confidence intervals - it's possible that 243.18 is in the 95% confidence interval for your ATE even if the point estimate is 7.15, which could just mean that there's a ton of uncertainty in your estimate, not that it's "wrong" (and likewise, the p-value of .27 means that even when the point estimate was 243.18, you couldn't reject the null hypothesis even at a75% confidence level). If your estimates are very uncertain, it might be worth trying to develop better first-stage models for predicting the outcome and treatment, but it could also just be the case that there is not enough useful variation to make a good estimate.

To directly answer your question, though, if your ATE estimate is way off, then I would not trust the CATE estimates, and estimating the CATE is a harder statistical problem than estimating the ATE and so you should generally expect your estimates of the CATE to be less precise than those of your ATE.

from econml.

Hi @kbattocchi,

Thanks a lot for your answer. Here is a bit more information about my model: the outcome (incidence of malaria) is expressed in cases per 10,000 inhabitants, and the treatment (human footprint index) corresponds to a scale (0 to 50) according to this paper https://www.nature.com/articles/s41597-022-01284-8.
I used the concentration of four aerosols pollutants (PM2.5, black carbon, SO2, and SO4) to develop the instrument variable, implementing an XGBoost algorithm, which means I got the instrument variable using the aerosols as predictors and the values of the human footprint index as the response variable (Y). Note that the aerosols are not associated with the incidence of malaria, except through treatment (human footprint index). The predictions of Y corresponded to the instrument variable, and the R2 of the XGBoost predictions of Y was 0.64.
Following your suggestion, I tested other algorithms for IV, and now I obtain an ATE=2.55 with 95% CI=-2.31 - 7.41. The refutation tests give me the next values: Add a random common cause=-2.56, replace a random subset=1.33 p value=0.14, and add a placebo treatment=0.96 p-value=0.24.

Here is the plot of the heterogeneous effect of human footprint index (HFP index) on the incidence of malaria:

from econml.