Giter Club home page Giter Club logo

Comments (3)

qingqing01 avatar qingqing01 commented on May 3, 2024

This problem is similar to #63, please try that solutions. You can set pool_size as follows in dataprovider.py to reduce data cache memory.

@provider(init_hook=hook, pool_size=1024)

from paddle.

hdulbj avatar hdulbj commented on May 3, 2024

I did what you advised ,but it still has problems
lbj@ubuntu:~/work/Paddle-master/demo/sentiment$ ./train.sh
I0912 07:26:11.295408 4219 Util.cpp:144] commandline: /usr/local/bin/../opt/paddle/bin/paddle_trainer --config=trainer_config.py --save_dir=./model_output --job=train --use_gpu=false --trainer_count=4 --num_passes=10 --log_period=10 --dot_period=20 --show_parameter_stats_period=100 --test_all_data_in_one_period=1
I0912 07:26:11.296186 4219 Util.cpp:113] Calling runInitFunctions
I0912 07:26:11.296751 4219 Util.cpp:126] Call runInitFunctions done.
[INFO 2016-09-12 07:26:12,275 networks.py:1122] The input order is [word, label]
[INFO 2016-09-12 07:26:12,275 networks.py:1129] The output order is [cost_0]
I0912 07:26:12.353705 4219 Trainer.cpp:169] trainer mode: Normal
I0912 07:26:13.966311 4219 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-12 07:26:14,002 dataprovider.py:22] dict len : 101745
I0912 07:26:14.200013 4219 PyDataProvider2.cpp:219] loading dataprovider dataprovider::process
[INFO 2016-09-12 07:26:14,218 dataprovider.py:22] dict len : 101745
I0912 07:26:14.218703 4219 GradientMachine.cpp:134] Initing parameters..
I0912 07:26:15.289685 4219 GradientMachine.cpp:141] Init parameters done.
I0912 07:26:19.343603 4220 ThreadLocal.cpp:39] thread use undeterministic rand seed:4221
I0912 07:26:19.514636 4221 ThreadLocal.cpp:39] thread use undeterministic rand seed:4222
I0912 07:26:19.536335 4222 ThreadLocal.cpp:39] thread use undeterministic rand seed:4223
I0912 07:26:19.704776 4223 ThreadLocal.cpp:39] thread use undeterministic rand seed:4224

I0912 07:29:23.252763 4219 TrainerInternal.cpp:162] Batch=10 samples=1280 AvgCost=0.692175 CurrentCost=0.692175 Eval: classification_error_evaluator=0.485156 CurrentEval: classification_error_evaluator=0.485156

I0912 07:32:37.542174 4219 TrainerInternal.cpp:162] Batch=20 samples=2560 AvgCost=0.666661 CurrentCost=0.641147 Eval: classification_error_evaluator=0.401172 CurrentEval: classification_error_evaluator=0.317188

I0912 07:35:50.129344 4219 TrainerInternal.cpp:162] Batch=30 samples=3840 AvgCost=0.626617 CurrentCost=0.54653 Eval: classification_error_evaluator=0.342448 CurrentEval: classification_error_evaluator=0.225

I0912 07:38:51.197893 4219 TrainerInternal.cpp:162] Batch=40 samples=5120 AvgCost=0.585538 CurrentCost=0.462299 Eval: classification_error_evaluator=0.307422 CurrentEval: classification_error_evaluator=0.202344

I0912 07:41:56.019923 4219 TrainerInternal.cpp:162] Batch=50 samples=6400 AvgCost=0.551691 CurrentCost=0.416303 Eval: classification_error_evaluator=0.282031 CurrentEval: classification_error_evaluator=0.180469

I0912 07:44:51.390555 4219 TrainerInternal.cpp:162] Batch=60 samples=7680 AvgCost=0.525094 CurrentCost=0.392109 Eval: classification_error_evaluator=0.264844 CurrentEval: classification_error_evaluator=0.178906

I0912 07:47:44.667140 4219 TrainerInternal.cpp:162] Batch=70 samples=8960 AvgCost=0.502319 CurrentCost=0.365668 Eval: classification_error_evaluator=0.250112 CurrentEval: classification_error_evaluator=0.161719
/usr/local/bin/paddle: line 81: 4219 Killed ${DEBUGGER} $MYDIR/../opt/paddle/bin/paddle_trainer ${@:2}

from paddle.

qingqing01 avatar qingqing01 commented on May 3, 2024

@hdulbj please check your available memory size. If it is not enough, you can try to reduce dictionary size or batch size.

from paddle.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.