ooanalyzer timeouts about pharos HOT 8 CLOSED

Ok seems to be a default config file. Maybe the defaults could be documented in ooanalyzer --help.

from pharos.

The timeouts are all a bit hacky. :-( There are whole program and per function timeouts and memory limits. The goal was to allow the tools to finish in a reasonable amount of time even when presented with a few unusual functions that have bad performance (in time or memory). The whole program limits were set for batch runs of multiple files, and should perhaps be higher by default.

You will find the documentation for the timeouts in the man pages for the tools. Hopefully that will make all the various settings a little bit clearer. And yes, if you want to change the defaults without having to specify the command line options every time, you can do that in your config file.

from pharos.

I see.

from pharos.

It's not clear though if increasing the relative time limit actually helps. It definitely heavily increases the total runtime but there are still lots of functions that fail to be analyzed.

from pharos.

The relative timeout is basically a per-function timeout (hopefully the man pages made that clear -- the name is old and little poor). If the functions that failed to be analyzed all have nearby addresses, that probably means that the whole program limit was reached. As with the other recently opened issue, the CPU and RAM limits are significantly affecting what OOAnalyzer can accomplish on large programs, and that's an area that we're working on. More resources need to be freed here:

https://github.com/cmu-sei/pharos/blob/master/libpharos/ooanalyzer.cpp#L195

Including the output state referenced in the no_output_state message. Unfortunately, that state is currently needed by other later parts of the analysis, so it's not as simple as just freeing the state at that point, so some code refactoring will need to occur. Specifically we need get whatever is needed (the this-pointer symbolic value?) out of the output state and save it before freeing the output state.

from pharos.

Indeed, it's a memory issue after all. It only hit the time limit cause it ran oom. Large swap files do work though as most of the data is not needed until much later.

Memory usage is also lower after restarting it using the serialized CFG. Maybe there are leaks.

And it doesn't seem to release the analyzer memory after the facts have been added: https://github.com/cmu-sei/pharos/blob/master/libpharos/oosolver.cpp#L188

from pharos.

After the recent changes, concerns about urgency of memory usage and CPU performance should be somewhat reduced, but there's still lots of opportunity for improvement. I think that reloading the serialized partitioner results takes less RAM because not everything is restored exactly. Just the things that the ROSE developers knew were important?

I agree that we should work on retracting the facts from the fact base as OOAnalyzer nears completion. What's less clear to me is whether we should do that in C++, Prolog or both. I'll leave the issue open as a reminder to both free more facts and to revisit whether the default timeouts should be larger. I'm inclined to believe that they should because OOAnalyzer is working more reliably on larger programs now, so we need to give it more time...

from pharos.

Default timeouts have been largely removed

from pharos.