resume after analysis about pharos HOT 14 CLOSED

Yes kind of, but the mechanism is a little cryptic, and it's intended for debugging instead of general use. The general idea is that once the fact file has been dumped, you can run the Prolog step without the C++ components at all. This best demonstrated in the ctests that begin ooprolog_xxx.

By running, "ctest -V -R ooprolog" then reading the --help output for the script that ctest runs, and specifying -c to show the commands run by the script, you can get all the way down to how XSB Prolog is invoked from the command line.

Additionally, this script and the others in that directory will show you how to run the Prolog step with various options such as debugging spew enabled. There's a fair bit of assumed knowledge about how to run these scripts, but with the clues provided from ctest you should be able to figure it out.

from pharos.

I though so. Something like /usr/local/xsb-3.8.0/bin/xsb --noprompt -e "asserta(library_directory('pharos-master/share/prolog/oorules')),([rulerun],assert(debuggingEnabled),assert(rTTIEnabled),(run('facts.txt');true)),halt." does work but still won't give you the final json output file.
Btw not all files are installed in share/prolog/oorules/CMakeLists.txt.

Would be nice if the facts and results options behaved like the serialize option to resume ooanalyzer if they already exist.

from pharos.

Doh! Good point. :-( Being developers more than actual users, we haven't usually been too concerned with generating the JSON from the Prolog results. We're usually testing one component at a time, and we've never really needed to do that. This is a good argument in favor of generating JSON within Prolog rather than within C++, and would make more of the tool more easily accessible in Ghidra, but we haven't done that yet.

The ooXXX scripts weren't "installed" because we don't consider them part of the user interface (they're basically for development and debugging). But it's obvious that you've dug into the guts of the system enough to want to know the what's possible rather than what's "supported".

The serialization idea is a good one in concept, although I doubt that it would be handled with boost::serialize like the partitioner serialization is. It wouldn't be a huge change to the C++ code though to add some additional options that would skip fact generation (or even result generation) and instead just read the Prolog assertions from a file, then inject them into Prolog so that the latter steps could complete normally in C++. That's a useful suggestion, but one I doubt we'll get to unless you or someone else work on it (someone else made a similar suggestion at some point).

from pharos.

The ooXXX scripts weren't "installed" because we don't consider them part of the user interface (they're basically for development and debugging).

No I meant the xwam files. It installs 2 of them but not the rest.

The serialization idea is a good one in concept, although I doubt that it would be handled with boost::serialize like the partitioner serialization is. It wouldn't be a huge change to the C++ code though to add some additional options that would skip fact generation (or even result generation) and instead just read the Prolog assertions from a file

Yes the latter, not boost::serialize.
So you won't lose everything if something happens during the final phases.

from pharos.

This is a good argument in favor of generating JSON within Prolog rather than within C++

Seems reasonable. And probably way easier than in C++ 😁
Already wondered why you coupled Prolog that much, even embedding it. Wouldn't it be better as a separate executable? The current analysis memory issues wouldn't be a problem, other frontends could be supported (e.g. facts from dynamic analysis), ...

from pharos.

I think that the other xwams are not installed because they're all compiled into setup.xwam.

The tight integration of Prolog was driven by the desire to produce an end-user tool that you just pointed at an executable and ran (instead of a multi-step process involving different tools). We may not have reached a "positive end-user experience" yet, but we're trying.

JSON generation from Prolog should be a change that would be easily independent of the rest of the system, and we'd happily accept a pull request. :-)

Yes, Others have suggested alternative fact generation front ends such as dynamic fact generators, and I think it's an interesting avenue of research.

from pharos.

The tight integration of Prolog was driven by the desire to produce an end-user tool that you just pointed at an executable and ran (instead of a multi-step process involving different tools).

Hmm couldn't the driver just kick off the prolog command after analysis and exit?

from pharos.

Yes, but only if we didn't need to return to the Pharos C++ framework for any kind of additional analysis after that. The intention was that someday you would invoke the OOAnalysis pass to obtain better data about virtual call targets and so forth before conducting some subsequent analysis into broader program behavior. The presumption was that the bigger analysis pass would be back in C++. For example the data flow of the OO results is actually Prolog -> C++ data structures -> JSON. That middle step was considered important, at least at the time, but isn't actually used right now.

from pharos.

I've recently given some thought to having Ctrl-C report whatever Prolog conclusions have been reached so far. Then we could have a mode in which the Prolog phase could be restarted with the already reached conclusions, by simply loading them into the fact base at startup. Of course this would disable proper backtracking through incorrectly reached conclusions. I have no idea how to incorporate this gracefully into the C++ portion of the until someone writes a JSON export in Prlog however. I'll leave the issue open as a reminder to continue thinking about how to improve the case where we want to "resume after (an aborted) analysis".

from pharos.

the data flow of the OO results is actually Prolog -> C++ data structures -> JSON.

Is it possible to manually re-run this last step with the results file?
For example there was a json export fix recently but you don't really want to run all the analysis again.

from pharos.

Sadly, there is no easy way to do this. :-( That's probably another argument in favor of implementing JSON exporting in Prolog. If it took many hours to run the analysis, you could probably modify solve() in setup.P to just load the results file (presuming that you saved it) instead of actually doing the analysis. Then when you "ran" OOAnalyzer, it would "analyze" very quickly, and then do the JSON exporting. But that whole plan is wildly untested, and is fairly tricky to setup.

Just writing a JSON exporter in Prolog might make more sense in that case. In summary, the only real choice is to just re-run the entire process.

from pharos.

Looks like this got fixed?
https://github.com/cmu-sei/pharos/blob/master/share/prolog/oorules/README.md#step-three-prolog-analysis

from pharos.

It would appear that you are right! :)

from pharos.

Though I guess it could be incorporated into the ooanalyzer exe by calling the appropriate sub commands when resuming.

from pharos.