caroljmcdonald / mapr-sparkml-churn Goto Github PK

Hi Carol,

Thanks for this example, but I found an error when running it on my cluster. The error is:

org.apache.spark.sql.AnalysisException: Unable to generate an encoder for inner class

The error line is: case class Account(state: String, len: Integer, acode: String, .....

This means that you are adding a new outer scope to this context that can be used when instantiating an inner class during deserialization.

Inner classes are created when a case class is defined in the Spark REPL and registering the outer scope that this class was defined in allows us to create new instances on the spark executors.

In normal use (your case), you shouldn't need to call this function.

EDIT: You'll also need to move your case classes outside of the Churn object.

This answer was taken from this page ANSWER

Kr,

Rshad :)