Task not serializable error with HashingTF about ispark HOT 7 CLOSED

I think that's not the fault of ISpark.
I think if you compile the code and submit you will get the same message.

from ispark.

The problem does not exist on Zeppelin, though.

from ispark.

I probably encounter it somewhere else before, could you try:

@transient val tf = new HashingTF(numFeatures = 10000)

and let me know the result
?

from ispark.

I tried with the @transient, but I'm getting the same error.

Would you need extra info about my configuration? I can post my ipython_config.py if you want.

from ispark.

I've tested again on the latest build and didn't see this problem, it may be a bug on 1.3.0 but was resolved 0.0.1 later, I'll upload the latest jar tomorrow.

from ispark.

New jar and source code uploaded. Please use the following code to test:

import org.apache.spark.mllib.feature.HashingTF
val text ="Apache Spark is an open-source cluster computing framework originally developed in the AMPLab at UC Berkeley. In contrast to Hadoop's two-stage disk-based MapReduce paradigm, Spark's in-memory primitives provide performance up to 100 times faster for certain applications.[1] By allowing user programs to load data into a cluster's memory and query it repeatedly, Spark is well suited to machine learning algorithms.[2]"
val spamText = sc.parallelize(text.split("[.,]"))
val tf = new HashingTF(numFeatures = 10000)
val spamFeatures = spamText.map(email => tf.transform(email.split(" ")))
spamFeatures.count()

case closed

from ispark.

Thank you for the quick fix.

from ispark.