Comments (3)
For your second bullet, imagine, for example, that ice cream demand is high when temperatures are above 50 degrees Fahrenheit and low when temperatures are below 50 degrees. Then if summer temperatures are always above 50 and winter temperatures are always below 50 then any attempt to extrapolate from one season to the other will fail. So you either need to ensure that there is overlap (the same situations occur in both seasons, but maybe with different frequencies), or that the functional relationship varies smoothly in a way that you can safely extrapolate from one season to the other.
For your last question, I don't think it's possible, both because of the extrapolation issue (your Y(treatment, weather) model can have zero error but still not generalize correctly out-of-domain), as well as the fact that if the Y model does not fit perfectly, there's no way to tell if this is due to harmless random noise or due to a missing confounder that would actually lead you to compute a different treatment effect if accounted for.
from econml.
I'm afraid I'm not fully sure I understand what you're asking. It sounds like since you have (different) randomized treatments in each period, you can compute the treatment effect in each period, but that you worry that the treatment effect from one season can't be assumed to apply to the other. This seems like a very reasonable concern.
If you are willing to assume that "season" acts on the outcome via some other measured covariates (say, temperature and rainfall), then if you include those covariates in W or X (depending on whether you think the season affects only the outcome or possibly also the strength of the effect) and fit a model on the combined data then the model could take those factors into account. However, I would be concerned that there might not be enough in-season variation in these factors to extrapolate from one season to another, and also that "seasonal" factors might include a whole host of other things that you haven't measured (e.g. maybe there are more tourists in town at one time than another), so I wouldn't take any such estimates very seriously unless you're confident that you understand the mechanism by which the season affects the outcomes, that you have measured all necessary factors, and that those factors affect the outcome in a way which an ML model will be able to generalize from one season to the other.
from econml.
@kbattocchi Thank you and yes, you understood this question correctly. To make sure I understood your reply, let me repeat it back:
- Theoretically, the DRPolicyForest should control for confounder (i.e. seasonal difference between those variants).
- The concern is that there may not be sufficient in-season variation to extrapolate from one season to another.
Can you give an example what you mean? Why would there be not enough variation?
- The other concern is that there may be more confounder that are unobserved and may bias treatment effect.
Can you confirm/clarify above?
Furthermore, the issue with the unobserved confounders (3rd bullet) is "we dont know what we dont know", right? Is there any test that you recommend to check how "good" the adjusted treatment effect is (from the DR model) given that I have ground truth for both experiments (since those were randomized trials)? In other words, do you have any idea how we could leverage ground truth (for each variant) to check how good the model controls for confounder? Is this possible? I want to get a temperature if we can trust the model in cases where we train the policy on randomized data that came from difference periods/seasons.
from econml.
Related Issues (20)
- IV Models: CATE or LATE? HOT 1
- Background Variable for Causal Analysis Object HOT 2
- Scaling Othrolearners using Ray HOT 6
- Only one CATE value is recorded from EconML (when called from DoWhy) HOT 1
- W and X in documentation, what are the difference? HOT 2
- Calculate ATE when T is continuous in double machine learning HOT 4
- PEP 517 support HOT 3
- Policy using Treatment Classes with DynamicDML HOT 1
- Wrong ATE estimation result, expected positive ATE got negative ATE HOT 2
- CausalForestDML need X to successfully run HOT 2
- how can i save model? HOT 2
- Bug with SHAP library v.0.41 HOT 1
- NumbaDeprecationWarning via shap HOT 1
- Changing covariance logic in DRIV HOT 1
- do you have files with real data and code this data ? HOT 1
- SparseLinearDML and DML with Debaised Lasso output only NaNs
- Effect Interval for DR Methods Not Possible When T0 is non-zero HOT 1
- Changing categories and refit_final HOT 1
- Parameter tuning for Policy Learning
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from econml.