learningjournal / spark-programming-in-python Goto Github PK

Sir for The view SP20 Creating Spark Session for the playlist Spark Programming for beginner using Pyspark3
I have a doubt...what does organization name refers to 5:22 ..how can I find out that

will add meaningful comments to understand the code.

Priority Support

• @Chad-Udit is using Mintycode to fund this issue.
• If you would like to accept bounty for solving this issue join Mintycode.
• Thank you in advance for helping.

Hi Sir, @LearningJournal
I have the clone of the project and want to execute the .py files using spark-submit command , but facing an error. Can you guide me through on how can I execute these scripts in spark env.

command:./bin/spark-submit /home/ubuntu/Spark-Programming-In-Python-master/01-HelloSpark/HelloSpark.py

the error logs are pasted below for reference:

21/07/16 14:01:56 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/lib/python3.8/configparser.py", line 846, in items
d.update(self._sections[section])
KeyError: 'SPARK_APP_CONFIGS'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ubuntu/Spark-Programming-In-Python-master/01-HelloSpark/HelloSpark.py", line 7, in
conf = get_spark_app_config()
File "/home/ubuntu/Spark-Programming-In-Python-master/01-HelloSpark/lib/utils.py", line 25, in get_spark_app_config
for (key, val) in config.items("SPARK_APP_CONFIGS"):
File "/home/ubuntu/anaconda3/lib/python3.8/configparser.py", line 849, in items
raise NoSectionError(section)
configparser.NoSectionError: No section: 'SPARK_APP_CONFIGS'