Comments (6)
matthieugomez
commented on May 14, 2024
Hi Johannes,
Thanks for the bug report. Could you downlad the current version and try
with this?
It should give you an explicit error rather than a segmentation fault.
If so, could you give me the error message?
…On Sat, Jan 14, 2017 at 9:35 AM, jmboehm ***@***.***> wrote:
Hi,
I'm getting a segmentation fault when trying to use clustered standard
errors. The dataset is fairly large (46m obs); it could be that I'm just
running out of memory. What do you think?
The regression command I'm using is
reg( C ~ interaction |> outputinput , df, VcovCluster([:outputinput]))
and it yields
signal (11): Segmentation fault: 11
while loading no file, in expression starting on line 0
macro expansion at /Users/johannes.boehm/.julia/
v0.5/FixedEffectModels/src/vcov/vcovcluster.jl:74 [inlined]
macro expansion at ./simdloop.jl:73 [inlined]
helper_cluster at /Users/johannes.boehm/.julia/v0.5/FixedEffectModels/src/
vcov/vcovcluster.jl:73
unknown function (ip: 0x316729720)
jl_call_method_internal at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/gf.c:1942
shat! at /Users/johannes.boehm/.julia/v0.5/FixedEffectModels/src/
vcov/vcovcluster.jl:55
unknown function (ip: 0x316728f66)
jl_call_method_internal at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/gf.c:1942
vcov! at /Users/johannes.boehm/.julia/v0.5/FixedEffectModels/src/vcov/vcovcluster.jl:31
[inlined]
#reg#48 at /Users/johannes.boehm/.julia/v0.5/FixedEffectModels/src/
reg.jl:332
unknown function (ip: 0x3166e0f30)
runRegressions at /Users/johannes.boehm/Documents/IndiaCourts/
verticalDistance/verticalDistanceModule.jl:256
unknown function (ip: 0x3166baf26)
jl_call_method_internal at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/gf.c:1942
do_call at /Users/osx/buildbot/slave/package_osx10_9-x64/build/src/
interpreter.c:66
eval at /Users/osx/buildbot/slave/package_osx10_9-x64/build/src/
interpreter.c:190
jl_toplevel_eval_flex at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/toplevel.c:558
jl_toplevel_eval_in_warn at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/builtins.c:590
eval at ./boot.jl:234
jlcall_eval_19752 at /Applications/Julia-0.5.app/
Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_call_method_internal at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/gf.c:1942
eval_user_input at ./REPL.jl:64
unknown function (ip: 0x31667b8e6)
jl_call_method_internal at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/gf.c:1942
macro expansion at ./REPL.jl:95 [inlined]
#3 <https://github.com/matthieugomez/FixedEffectModels.jl/issues/3> at
./event.jl:68
unknown function (ip: 0x3166780df)
jl_call_method_internal at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/osx/buildbot/slave/
package_osx10_9-x64/build/src/gf.c:1942
jl_apply at /Users/osx/buildbot/slave/package_osx10_9-x64/build/src/./julia.h:1392
[inlined]
start_task at /Users/osx/buildbot/slave/package_osx10_9-x64/build/src/
task.c:253
Allocations: 1077545854 (Pool: 1077544121; Big: 1733); GC: 866
Segmentation fault: 11
Running it without VcovCluster() works fine. Any help/thoughts are
appreciated.
Best, Johannes
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<https://github.com/matthieugomez/FixedEffectModels.jl/issues/11>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AF733Q0gtuTuCctzXZKhluwS6JKQuHjZks5rSN0XgaJpZM4Ljo_v>
.
from fixedeffectmodels.jl.
jmboehm
commented on May 14, 2024
Thanks for the quick response Matthieu. I went to the master branch using Pkg.checkout("FixedEffectModels")
but I'm still getting the same segfault. I should perhaps add that memory use before the segfault is not particularly high...
from fixedeffectmodels.jl.
jmboehm
commented on May 14, 2024
Seemed to have indeed been a memory issue.
from fixedeffectmodels.jl.
matthieugomez
commented on May 14, 2024
Ok. Out of curiosity could you tell me the size of your memory compared to the size of the dataset?
from fixedeffectmodels.jl.
jmboehm
commented on May 14, 2024
The dataframe was about 20gb (according to Base.summarysize() ); total memory on the machine was 64gb. Not sure it is/was really related to FixedEffectModels, actually. These days I tend to trim down my dataframes before running regressions, and that works ok for me.
from fixedeffectmodels.jl.
matthieugomez
commented on May 14, 2024
Ok thanks!
from fixedeffectmodels.jl.
Related Issues (20)
- MethodError: no method matching fe(::CategoricalValue{String15, UInt32}) HOT 1
- Implement StatsAPI.fit() HOT 1
- Generate confidence intervals for `predict` HOT 1
- Ignore rows with `Inf`s? HOT 3
- error in using gpu? HOT 3
- Feature request: GPU support in MacOS HOT 2
- Drop regressors that are collinear with the fixed effects (depending on tolerance for partialling-out)
- Get " run `reg` with the option save = :residuals" despite doing exactly that HOT 3
- How can I get the dof? HOT 2
- Degrees of freedom always 1 HOT 1
- Demeaning HOT 3
- Print name of dependent variable in FixedEffectModel results display? HOT 1
- tolerance in `invsym!` HOT 1
- Next release please! For Stata front-end HOT 4
- `predict` for fixed effects HOT 4
- Feature request: Saving an object of type `FixedEffectModel` HOT 1
- Using r2 as a regression name conflicts with r2 (r-squared?) HOT 1
- Wrong results in a large data set with one set of FE HOT 51
- `predict` doesn't work for FE-only models
- Poor multithreading performance with very small FE groups HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fixedeffectmodels.jl.