Comments (8)
Nice to hear, that this could solve your issue. In the upcoming release the logging structure will anyways be adjusted since currently for every instantiation of DEHB the logs will be written to some specific file. In the upcoming release it will just be written to dehb.log
and thus will be overwritten by new instatiations of DEHB (assuming that you do not adjust the output path).
from dehb.
Hey, I'm assuming you are using DEHB.run
. If so, have you tried using the verbosity flag, i.e. DEHB.run(..., verbose=False)
. Then you should only be left with outputs regarding state saving/logging.
from dehb.
Anyways, thanks for you issue! Since the state saving logs could also fill stdout with a lot of logs, we should maybe think about adding a verbosity level for the whole DEHB class and not just the run
function.
from dehb.
Hey, I'm assuming you are using
DEHB.run
. If so, have you tried using the verbosity flag, i.e.DEHB.run(..., verbose=False)
. Then you should only be left with outputs regarding state saving/logging.
Yes that's correct, verbose is set to False, but I'm getting lots of these messages to stdout.
2024-05-29 12:04:56.361 | INFO | dehb.optimizers.dehb:reset:119 -
RESET at 05/29/24 12:04:56 PDT
2024-05-29 12:04:56.417 | INFO | dehb.optimizers.dehb:reset:119 -
RESET at 05/29/24 12:04:56 PDT
2024-05-29 12:04:56.468 | INFO | dehb.optimizers.dehb:reset:119 -
RESET at 05/29/24 12:04:56 PDT
2024-05-29 12:04:56.519 | INFO | dehb.optimizers.dehb:reset:119 -
RESET at 05/29/24 12:04:56 PDT
Somewhat related is that this package appears to be accumulating lots of logs that does not get cleaned up after a run is complete. For me, dehb has accumulated almost to 20GB of log data since I've installed 4 weeks ago.
![Capture d’écran 2024-05-29 à 12 18 30](https://private-user-images.githubusercontent.com/71951740/334935341-1d2b1108-e5b4-4202-a61f-1b16ed3eebde.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjAwMTcwNTQsIm5iZiI6MTcyMDAxNjc1NCwicGF0aCI6Ii83MTk1MTc0MC8zMzQ5MzUzNDEtMWQyYjExMDgtZTViNC00MjAyLWE2MWYtMWIxNmVkM2VlYmRlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MDMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzAzVDE0MjU1NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU3OThlMmYyYzJhNDA4YmY3Nzk0NTI5MmQyYzRjNzFlN2Y0ZDY4MzY2YTkwYzk2NmU3ZmRhYTk1NDIxNDFkYjQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.emJAQjPEe-mFbi5FHFFVnreYjoatVP42SNi_I_KbK7s)
^ After clearing it 2 weeks ago.
from dehb.
lots of these messages to stdout.
They appear to be coming from dehb.py:119, removing that line solved the issue for me
def reset(self):
self.inc_score = np.inf
self.inc_config = None
self.population = None
self.fitness = None
self.traj = []
self.runtime = []
self.history = []
self.logger.info("\n\nRESET at {}\n\n".format(time.strftime("%x %X %Z")))
from dehb.
Good to know, thanks for looking into the issue and looking forward to the fix.
In the upcoming release it will just be written to dehb.log and thus will be overwritten by new instatiations of DEHB
Will this affect the ability to run multiple DEHB instances running simultaneously? In my use case, the host program optimizes many different models in parallel.
from dehb.
Will this affect the ability to run multiple DEHB instances running simultaneously?
This should not, assuming each DEHB instance is an independent optimization run since then the output_path
should vary.
@Bronzila could you please correct me or add anything if required.
from dehb.
Sorry, I must have overlooked your comment @u3Izx9ql7vW4
As @Neeratyoy pointed out, as long as each DEHB instance has a unique output_path
no results will be overwritten.
from dehb.
Related Issues (20)
- Continuously develop unit tests
- Keep active brackets in a dictionary HOT 1
- Implement ask-and-tell interface for DEHB HOT 1
- Use of deprecated np.int in dimension type checking HOT 3
- [Bug] `isinstance(self, Client)` will never be `True` HOT 6
- Configuration and budgets in each brackets HOT 6
- Using Dask ```Client``` as context manager
- No support for Constant type in vector_to_configspace() and configspace_to_vector() HOT 3
- Update CHANGELOG with the historic releases
- Updating and populating the documentation
- Implement versioning for documentation HOT 1
- Introduce IDs for evaluated configurations HOT 2
- Continuing configuration evolution when run budget is exhausted. HOT 2
- Implement parallel DE
- [Bug] Seeding is not applied to configspace HOT 3
- ERROR - Failed to communicate with scheduler during heartbeat. followed by TimeoutError: No valid workers found HOT 2
- Test backward compatibility with NumPy2.0 HOT 4
- Ask & Tell for DE
- Update numpy dependency as soon as ConfigSpace works with numpy 2.0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dehb.