Unable to distribute simulation batches about alchemist HOT 11 OPEN

Apparently an object is not serializable: how does Ignite serialize?

from alchemist.

As far I underdand, it uses it's Binary Marshaller

from alchemist.

Can you deepen the reason why it does refuse to serialize? It looks like it's failing at serializing Kotlin's lambda expressions - maybe we can program it?

from alchemist.

Ok, I'll try to debug

from alchemist.

Most of the serialization exceptions seem to origin from classes whose fields in some ways end up dealing with org.danilopianini.jirf.FunctionEdge. I guess the problem may be that it does not implement Serializable.

Other classes with similar issue are:

• org.danilopianini.jirf.FactoryImpl
• it.unibo.alchemist.loader.m2m.Context
• org.jgrapht.graph.DefaultDirectedGraph (although this one extends a class who does implement Serializable)
• it.unibo.alchemist.loader.m2m.SimulationModel$fromMap$3

And then there's another kind of exception regarding which I could not find so many information.

java.lang.UnsupportedOperationException: can't get field offset on a hidden class: private final kotlin.jvm.functions.Function1 it.unibo.alchemist.loader.m2m.ObjectFactory$$Lambda$145/0x0000000801176228.arg$1

from alchemist.

Okay, we must change the way we serialize, then, it's too much stuff to make Serializable. Also, it may be the case that Ignite internally does not use the Java serialization (and would be right in doing so).
We should send away only the environment, and the environment should not bring a Java Implicit Reflective Factory within, nor the YAML-to-Alchemist model-to-model stuff.

I think that the "strange thing" you see is a Kotlin default argument value.

from alchemist.

I tried to avoid sending away the Loader by making the field in which it is stored transient and store in each RemoteSimulation an InitializedEnvironment. Most of the previous exceptions are gone, but now there's a problem with the serialization of classes with companion objects. Here's the error log.

from alchemist.

Can we replace the serializer of ignite with something else?

from alchemist.

Yes, there's a method to set a (i believe custom) Marshaller but, it is deprecated:

Deprecated. Since 2.1. Some Ignite features will not work if non-null marshaller is set (IgniteCache.withKeepBinary(), .NET, CPP, ODBC)
Sets marshaller to use within grid.

Javadoc

from alchemist.

Okay, we must change the way we serialize, then, it's too much stuff to make Serializable. Also, it may be the case that Ignite internally does not use the Java serialization (and would be right in doing so). We should send away only the environment, and the environment should not bring a Java Implicit Reflective Factory within, nor the YAML-to-Alchemist model-to-model stuff.

@DanySK I'm working on the distribution of simulations, still having issues with serialization. I'm avoiding using the default Java serialization by using protocol buffers. The doubt that I now have is: to be able to serialize any Environment and/or Exporter (contained in an InitializedEnvironment) using protobuf, or any other framework, I should define a custom serializer for every specific implementation, right? (It seems to me like it would take a lot of effort, I want to be sure I'm not taking the wrong path).

from alchemist.

I think that you should have your custom exporter and throw errors if any other is being used

from alchemist.