Comments (6)
andrey-khropov
commented on June 2, 2024
malloc/free calls AFTER catboost is loaded cause segfaults like this in the JVM:
Do you mean that the issue is that some memory that had been allocated by one allocator (that had been used by JVM process) is then tried to be freed by mimalloc implementation that replaced free after loading of catboost-prediction library's dynamic library?
Also, can you provide macOS version and CPU architecture (x86_64 or arm64 (Apple Silicon)) ?
We'll look into it, meanwhile you can try to build catboost-prediction JVM applier without mimalloc allocator yourself, remove lines from CMakeLists that are relevant to you CPU architecture here or here and then use these instructions.
from catboost.
andrey-khropov
commented on June 2, 2024
And also what JRE do you use?
from catboost.
mikekap
commented on June 2, 2024
Updated the first post with the version details (x86_64 & temurin 17).
I'm not sure if it's related to allocations before/after - my most consistent reproduction (the one above) loads two JNI/JVMTI libraries, but I load catboost first.
Let me try to compile the build myself without mimalloc.
from catboost.
mikekap
commented on June 2, 2024
I can confirm removing mimalloc fixes the reproduction I have. Here's a reproduction you can try at home (you'll need to install Clojure & the clj tool - see here):
$ clj -J-Djdk.attach.allowAttachSelf=true -Sdeps '{:deps {ai.catboost/catboost-prediction {:mvn/version "1.2.3"} com.clojure-goes-fast/clj-async-profiler {:mvn/version "1.0.5"}}}}'
Clojure 1.11.1
user=> ai.catboost.CatBoostModel ;; Load catboost JNI
ai.catboost.CatBoostModel
user=> (require '[clj-async-profiler.core :as prof])
nil
user=> (prof/profile (dotimes [i 10000] (reduce + (range i))))
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x000000013f9da245, pid=90163, tid=27915
...
from catboost.
aakashb-kayzen
commented on June 2, 2024
I am facing the same issue.
Catboost Version : 1.2.2
Host: "MacBookPro17,1" arm64, 8 cores, 8G, Darwin 21.6.0, macOS 12.5.1 (21G83)
JVM :
java --version
openjdk 11.0.15 2022-04-19 LTS
OpenJDK Runtime Environment Zulu11.56+19-CA (build 11.0.15+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.56+19-CA (build 11.0.15+10-LTS, mixed mode)
Stack trace :
Stack: [0x000000016f6fc000,0x000000016f8ff000], sp=0x000000016f8fdf30, free space=2055k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libcatboost4j-prediction6819137113863618542.dylib+0x175610] _mi_free_block_mt+0x78
C [jnilib-18296746703685709260.tmp+0x109b2a8] arrow::Schema::Impl::~Impl()+0x4c
C [jnilib-18296746703685709260.tmp+0x106d56c] arrow::Schema::~Schema()+0x2c
C [jnilib-18296746703685709260.tmp+0xc5e68] arrow::dataset::DatasetFactory::Inspect(arrow::dataset::InspectOptions)+0xe0
C [jnilib-18296746703685709260.tmp+0x59b0] Java_org_apache_arrow_dataset_jni_JniWrapper_inspectSchema+0x60
j org.apache.arrow.dataset.jni.JniWrapper.inspectSchema(J)[B+0
j org.apache.arrow.dataset.jni.NativeDatasetFactory.inspect()Lorg/apache/arrow/vector/types/pojo/Schema;+26
j org.apache.arrow.dataset.jni.NativeDatasetFactory.finish()Lorg/apache/arrow/dataset/jni/NativeDataset;+2
j org.apache.arrow.dataset.jni.NativeDatasetFactory.finish()Lorg/apache/arrow/dataset/source/Dataset;+1
from catboost.
Related Issues (20)
- Show default values on the Documentation HOT 1
- Restore handle on RDS saved model HOT 1
- catboost.cv with YetiRank fails with "CatBoostError: /src/catboost/catboost/libs/train_lib/options_helper.cpp:88: Targets are required for YetiRank loss function." HOT 1
- Here is a negation overflow in lua HOT 4
- Potential OOB vulnerability may happen HOT 4
- Error "Attempt to call single feature writer on packed feature writer" with embedding features on GPU HOT 2
- CatBoostRegressor.staged_predict(eval_period=0) goes in an indefinite loop
- used_ram_limit not working for some cases
- Integrate ZLUDA for AMD CUDA HOT 1
- Param alpha is mandatory for expectile loss
- Fitting with text features uses too much memory (~35GB for 5M examples)
- Incorrect Documentation in algorithm-main-stages_bootstrap-options.md HOT 1
- Illegal instruction (core dumped) HOT 2
- Upgrade scikit-learn 1.4.0 to 1.4.1 HOT 2
- RMSEWithUncertainty not supported on GPU with posterior_sampling=True
- Feature Request: BrierScore for multiclass problems (auto adding :type=Mu to ordinary BrierScore, by averaging across classes). HOT 1
- Catboost website regularly offline? HOT 8
- Usage example of `libcatboostmodel_static` doesn't work HOT 1
- getting CatBoostError: Invalid cat_features[6] = 22 value: index must be < 22 but len(x_train.columns) is 23 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from catboost.