Comments (4)
Hi @NickCH-K, thanks so much, this is exactly the type of feedback I had hoped for!
from pyfixest.
Hi @NickCH-K , I have finally gotten around to take a look at this, and I am not sure if R "gets it right". Depending on the respective datetime, lm
and feols
do all sorts of transformations, and I am not sure if it's particularly safe or user friendly what pops out.
But I very much agree that an error message is quite annoying, so I am thinking about internally transforming date variables to categoricals and to add a clear warning message that if users want a continuous time variable, they need to turn their datetime variable into a numeric "by hand". On the other hand, I this might backfire if the date variable is high dimensional. What do you think?
Below are R example of different transformations of POSIXct
and Date
variables:
library(fixest)
library(causaldata)
data(restaurant_inspections)
restaurant_inspections$Time = lubridate::ymd_hms(paste0(restaurant_inspections$Year, '-01-01 00:00:00'))
class(restaurant_inspections$Time)
# [1] "POSIXct" "POSIXt"
fit = feols(Year~Time, data = restaurant_inspections)
fixest:::model.matrix.fixest(fit, type = "rhs") |> head()
# (Intercept) Time
# 1 1 1483228800
# 2 1 1420070400
# 3 1 1451606400
# 4 1 1041379200
# 5 1 1483228800
# 6 1 1199145600
restaurant_inspections$Time2 <- as.Date(restaurant_inspections$Time)
class(restaurant_inspections$Time2)
# Date
fit = feols(Year~Time2, data = restaurant_inspections)
fixest:::model.matrix.fixest(fit, type = "rhs") |> head()
# (Intercept) Time2
# 1 1 17167
# 2 1 16436
# 3 1 16801
# 4 1 12053
# 5 1 17167
# 6 1 13879
fit = feols(Year~as.factor(Time2), data = restaurant_inspections)
fixest:::model.matrix.fixest(fit, type = "rhs") |> head()
from pyfixest.
The R transformations are such that you can estimate a linear time trend with them though, right? Even if the scale of the variable isn't easily guessable. It even handles stuff like leap years properly, fitting the time trend on the actual amount of time passed. That seems to me a reasonable expectation when you put a date variable in the non-FE part of the model.
as.numeric(as.Date('2020-01-01'))-as.numeric(as.Date('2019-01-01'))
# 365
as.numeric(as.Date('2021-01-01'))-as.numeric(as.Date('2020-01-01'))
# 366 (leap year!)
as.numeric(lubridate::ymd_hms('2020-01-01 00:00:00'))-as.numeric(lubridate::ymd_hms('2019-01-01 00:00:00'))
# 31536000 = 60*60*24*365
from pyfixest.
Ah ok, I get it, this indeed makes sense! Thanks!
So one simple solution is to convert all dates to int
/ POSIX
:
int(pd.Timestamp('2020-01-01').timestamp()) - int(pd.Timestamp('2019-01-01').timestamp())
# 31536000
I'll try to implement that tomorrow =)
from pyfixest.
Related Issues (20)
- Add "lean" argument to `feols()` and `fepois()` HOT 4
- Publish `pyfixest` on conda
- Add `solver` argument to `feols()`, `fepois()` APIs HOT 5
- Support for weights as an optional parameter for did2s? HOT 4
- Add benchmarks against `linearmodels` and `fastreg` HOT 8
- Add default IV Diagnostics to `pf.summary()` and `pf.etable()` HOT 1
- installing via pip broken by stargazer HOT 4
- Adding Standard Error of Prediction HOT 6
- BUG: coefplot() and iplot() show only point estimates, no confidence intervals HOT 1
- Implementing outcome plot in panelview()
- Scalable IV regression using sufficient statistics
- Small bugs in `pf.etable()` regarding fixed effects information HOT 4
- Unexpected behavior with resids and predict post feols HOT 5
- `pre-commit` CI failing for every PR
- CI: Run Tests only on earliest and latest supported Python Versions & Parallelize CI Tests
- CI: Add workflow to automatically render all Example Notebooks
- Refine `resid()` output
- IV: Fix bug in `first_stage` method with special `formulaic()` syntax
- Bring back extended tests HOT 3
- Request to Bring Back DataFrame ouput for 'etable()' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyfixest.