Comments (11)
gnannicini
commented on June 28, 2024
You should use alg.optimize() rather than alg.optimize_parallel(). Since you set num_cpus=4, it will run in parallel.
As you have already observed firsthand, the function .optimize_parallel() is an advanced function of the "use at your own risk" kind.
from rbfopt.
mawi363
commented on June 28, 2024
Thanks for the Quick response.
Sadly setting: num_cpus=4 and using alg.optimize() shows the same behaviour.
any other Ideas ?
Is there perhaps a working example for an optimization thats running in parallel available? so i can test the rest of my setup ?
from rbfopt.
gnannicini
commented on June 28, 2024
The parallel optimization is based on the multiprocessing module, which imports all your modules for each thread. If you have includes that don't resolve properly, or if importing one module causes the optimization to start over, it may not work.
Where is the objective function defined? Make sure the corresponding module is properly wrapped with:
if name == 'main':
otherwise multiprocessing will keep loading your 'main' function indefinitely.
from rbfopt.
gnannicini
commented on June 28, 2024
oops, bad syntax. Imean:
if __name__ == '__main__'
from rbfopt.
mawi363
commented on June 28, 2024
I tried it using the minimal example but i still run into the same problem:
a brief spike in cpu usage than it drops down and starts to idle
import rbfopt
import numpy as np
def obj_funct(x):
return x[0]*x[1] - x[2]
if __name__ == '__main__':
bb = rbfopt.RbfoptUserBlackBox(3, np.array([0] * 3), np.array([10] * 3),
np.array(['R', 'I', 'R']), obj_funct)
settings = rbfopt.RbfoptSettings(max_evaluations=500, num_cpus=4)
alg = rbfopt.RbfoptAlgorithm(settings, bb)
val, x, itercount, evalcount, fast_evalcount = alg.optimize()
any other ideas ? maybe a mistake during setup ? I am fairly new at Python so it could be something obvious...
from rbfopt.
gnannicini
commented on June 28, 2024
The example that you wrote above should work, and in fact it does work: I just tried it on a fresh installation. It works from console as well as when executed as a script.
Are you not seeing any output at all? I suspect it may be a problem in your installation but I am not sure what.
from rbfopt.
mawi363
commented on June 28, 2024
I am seeing a little output. It builds the chart for the output data but that's it. The program seem to get stuck during the first initialization step. I tried different numbers of cores but everything above 1 runs into the same problem.
Iter Cycle Action Objective value Time Gap
---- ----- ------ --------------- ---- ---
I tried leaving it running for a while but even after 5min runtime nothing changes (using a single core it barely takes a second to get past the initialization steps)
from rbfopt.
gnannicini
commented on June 28, 2024
Can you try to figure out what function it is spending time in?
On Linux just Ctrl + C while the process is running will tell you what received the interrupt. Otherwise you could try a profiler or something like that.
As I said the code snippet you posted works, so I'm really not sure what's the problem.
from rbfopt.
mawi363
commented on June 28, 2024
sorry for the long wait!
I think you are right and its an error in my installation. So i made a fresh set up on another PC (Windows 10) here is what i used:
- Winpython 3.8 64-bit (from: http://winpython.github.io/#releases )
- Bonmin 64bit (from: https://ampl.com/products/solvers/open-source/)
- Ipopt 64bit (from: https://ampl.com/products/solvers/open-source/)
Then i used:
pip install pyomo
pip install rbfopt
pip install numpydoc
then using JupyterLab i ran the following code:
import rbfopt
import numpy as np
import rbfopt
Path1= r'C:\Users\PC2\Downloads\bonmin-win64\bonmin'
Path2= r'C:\Users\PC2\Downloads\ipopt-win64\ipopt'
def obj_funct(x):
return x[0]*x[1] - x[2]
if __name__ == '__main__':
bb = rbfopt.RbfoptUserBlackBox(3, np.array([0] * 3), np.array([10] * 3),
np.array(['R', 'I', 'R']), obj_funct)
settings = rbfopt.RbfoptSettings(max_evaluations=500, num_cpus=4,minlp_solver_path=Path1, nlp_solver_path=Path2)
alg = rbfopt.RbfoptAlgorithm(settings, bb)
val, x, itercount, evalcount, fast_evalcount = alg.optimize()
Result is the same problem as before: the code works with num_cpus=1 but frezzes in num_cpus=4:
This is the output when it freezes:
Iter Cycle Action Objective value Time Gap
---- ----- ------ --------------- ---- ---
now in difference to anaconda winpython also leaves the comand prompt (i think) open so i can finally see a real error message.
This is the readout:
[I 05:07:25.073 LabApp] Build is up to date
[I 05:07:26.120 LabApp] Kernel started: XXXXX I deleted this because i didn't know what is was and it kinda looked like my mac address XXXXX
Process SpawnPoolWorker-2:
Traceback (most recent call last):
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\queues.py", line 358, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'obj_funct' on <module '__main__' (built-in)>
Process SpawnPoolWorker-3:
Traceback (most recent call last):
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
Process SpawnPoolWorker-4:
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\queues.py", line 358, in get
return _ForkingPickler.loads(res)
Traceback (most recent call last):
AttributeError: Can't get attribute 'obj_funct' on <module '__main__' (built-in)>
Process SpawnPoolWorker-1:
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\pool.py", line 114, in worker
task = get()
Traceback (most recent call last):
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\queues.py", line 358, in get
return _ForkingPickler.loads(res)
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
AttributeError: Can't get attribute 'obj_funct' on <module '__main__' (built-in)>
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\queues.py", line 358, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'obj_funct' on <module '__main__' (built-in)>
Process SpawnPoolWorker-8:
Traceback (most recent call last):
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "C:\Users\PC2\Downloads\WPy64-3830\python-3.8.3.amd64\lib\multiprocessing\queues.py", line 358, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'obj_funct' on <module '__main__' (built-in)>
[I 05:09:26.067 LabApp] Saving file at /Untitled.ipynb
Is this useful ? If not could you give me a rundown how you set up your installation ?
from rbfopt.
gnannicini
commented on June 28, 2024
Yes, this is very useful. It is definitely an error of the multiprocessing module in Windows.
You should be able to fix it following the suggestion reported here:
https://stackoverflow.com/questions/41385708/multiprocessing-example-giving-attributeerror/42383397
basically you need to define obj_funct in a separate, importable module.
from rbfopt.
mawi363
commented on June 28, 2024
Separating the obj_funct into its own module and than importing it worked !
Thank you for helping me !!
from rbfopt.
Related Issues (20)
- Cannot import after pip install HOT 1
- NameError: global name 'sys' is not defined HOT 3
- Error: "retrieving immutable Param value..." when calling .optimize() HOT 1
- sample_size not being casted to integer HOT 1
- No attribute RbfoptBlackBox HOT 1
- Refactor RbfoptAlgorithm the way to make it possible to a library user drive the get_point-evaluate_point-update_model loop himself HOT 2
- Input initial guess of the parameters HOT 1
- import rbfopt causes Excel to close HOT 3
- Problem with restarting pool while running optimize_parallel HOT 1
- Is there a way to skip an output value? HOT 4
- Is there a way to exstract the rbf surrogate model? HOT 1
- Constraint handling HOT 2
- Addition of kwargs to RbfoptUserBlackBox
- Add optional arguments to UserBlackBox function HOT 2
- TypeError at end of iterations HOT 2
- UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 81: character maps to <undefined> HOT 5
- Unexpected error: "IndexError: index 160 is out of bounds for axis 0 with size 158" HOT 7
- Tests never finish, stuck on "Solving hartman6 with random seed 512319876" HOT 13
- The complexity of the rbfopt algorithm HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rbfopt.