Not serializable about yauaa HOT 10 CLOSED

So far I have solved this issue by simply instantiating the analyzer on the node that does the actual execution. So this is what you see in the Pig UDF.

I'll have a look if I can make it serializable. No promises at this moment.

from yauaa.

Thanks for answering, since we're using Spark SQL and UDFs that would make it "once by execution" which is pretty bad in terms of executions. How can I help ?

from yauaa.

btw thx your work is the best we've been trying so far

from yauaa.

we may be blocked by antlr/antlr4#233

from yauaa.

The Parse Tree does not need to be serializable. I'll have a look on how I can solve this.

from yauaa.

I did some experimentation and getting this to work right would be 'really hard'.
Background:
Parsing the config files may yield parse errors (so you want to do that early) and the resulting data structures need the parser system to validate it all. Then 'only' serializing what NEEDS to be serialized becomes tricky.

from yauaa.

I found a way to do this that seems to work.
I test it by running a significant part of the tests with a UserAgentAnalyzer that was serialized and then deserialized: https://github.com/nielsbasjes/yauaa/blob/master/analyzer/src/test/java/nl/basjes/parse/useragent/TestSerialization.java

Can you please verify if this also works in your context?

from yauaa.

Question: You say "best so far". What makes this library for you the best?

from yauaa.

@ogirardot Have you been able to check if it works now?

from yauaa.

These changes are in the 1.4 release.
So for now I'm closing this as 'fixed'.
Please reopen if you find any issues.

from yauaa.