jleetutorial / python-spark-streaming Goto Github PK

@matthew-mcateer Hi, I am running the code in local instead of VM, the code was running fine in VM and able to see the twitter data but in local i am facing issue like Py4JJavaError: An error occurred while calling o25.sql.
: org.apache.spark.sql.AnalysisException: Table or view not found: tweets; line 1 pos 23

Any help ?

While debugging, when i compare VM and local machine, after the streaming started,
My local :

VM:

So the tweets table is not creating in my local. I have not figured the solution yet.

Hello Matthew,

I see in the join demo that you pass a range object to SparkContext.parallelize

https://github.com/jleetutorial/python-spark-streaming/blob/master/3_advanced/1_Stream-Stream%20Join%20Demo.ipynb

In the Spark documentation it says that SparkContext.parallelize takes an iterable or a collection.

https://spark.apache.org/docs/2.1.1/programming-guide.html#parallelized-collections

I am not quite sure that a range object is one of those (iterable orcollection) so I got surprised the example works. Actually I does not work for me and the resulting rdd seems empty.

Any comment on this would be appreciated

Thanks

Jorge