Current:

python mr-mr_s3_log_parser.py -r emr s3://bucket-source/ --output-dir=s3://bucket-dest/"

Should be:

python mr_s3_log_parser.py -r emr s3://bucket-source/ --output-dir=s3://bucket-dest/"

flake8 testing of https://github.com/donnemartin/data-science-ipython-notebooks on Python 3.7.0

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./scipy/thinkplot.py:475:14: F821 undefined name 'xp'
        xs = xp.delete(xs, 0)
             ^
./data/titanic/myfirstforest.py:84:19: E999 SyntaxError: invalid syntax
print 'Training...'
                  ^
./data/titanic/gendermodel.py:47:46: E999 SyntaxError: invalid syntax
print 'Proportion of women who survived is %s' % proportion_women_survived
                                             ^
./data/titanic/genderclassmodel.py:42:10: F821 undefined name 'xrange'
for i in xrange(number_of_classes):
         ^
./data/titanic/genderclassmodel.py:43:14: F821 undefined name 'xrange'
    for j in xrange(number_of_price_brackets):
             ^
./data/titanic/genderclassmodel.py:84:14: F821 undefined name 'xrange'
    for j in xrange(number_of_price_brackets):
             ^
./deep-learning/keras-tutorial/deep_learning_models/vgg19.py:163:9: F821 undefined name 'preprocess_input'
    x = preprocess_input(x)
        ^
./deep-learning/keras-tutorial/deep_learning_models/vgg19.py:167:25: F821 undefined name 'decode_predictions'
    print('Predicted:', decode_predictions(preds))
                        ^
./deep-learning/keras-tutorial/deep_learning_models/resnet50.py:242:9: F821 undefined name 'preprocess_input'
    x = preprocess_input(x)
        ^
./deep-learning/keras-tutorial/deep_learning_models/resnet50.py:246:25: F821 undefined name 'decode_predictions'
    print('Predicted:', decode_predictions(preds))
                        ^
./deep-learning/keras-tutorial/deep_learning_models/vgg16.py:161:9: F821 undefined name 'preprocess_input'
    x = preprocess_input(x)
        ^
./deep-learning/keras-tutorial/deep_learning_models/vgg16.py:165:25: F821 undefined name 'decode_predictions'
    print('Predicted:', decode_predictions(preds))
                        ^
./deep-learning/keras-tutorial/solutions/sol_112.py:2:1: E999 SyntaxError: invalid syntax
%timeit -n 1 -r 1 ann.train(zip(X,y), iterations=100)
^
./deep-learning/keras-tutorial/solutions/sol_111.py:2:1: E999 SyntaxError: invalid syntax
%timeit -n 1 -r 1 ann.train(zip(X,y), iterations=2)
^
./deep-learning/theano-tutorial/rnn_tutorial/lstm_text.py:255:22: E999 SyntaxError: invalid syntax
        print 'epoch:', epoch
                     ^
./deep-learning/tensor-flow-examples/multigpu_basics.py:84:37: E999 SyntaxError: invalid syntax
print "Single GPU computation time: " + str(t2_1-t1_1)
                                    ^
./deep-learning/tensor-flow-examples/input_data.py:95:34: F821 undefined name 'xrange'
      fake_image = [1.0 for _ in xrange(784)]
                                 ^
./deep-learning/tensor-flow-examples/input_data.py:97:35: F821 undefined name 'xrange'
      return [fake_image for _ in xrange(batch_size)], [
                                  ^
./deep-learning/tensor-flow-examples/input_data.py:98:31: F821 undefined name 'xrange'
          fake_label for _ in xrange(batch_size)]
                              ^
6     E999 SyntaxError: invalid syntax
13    F821 undefined name 'xrange'
19

Machine learning models with solved datasets and solutions

I'm trying to clean my data and do some preprocessing but i don't have much understanding of the Columns if the zero in them are normal or missing values i'm using the dataset cic-collection on kaggle if any expert would help i'd be much thankful

Alexnet implementation in tensorflow has incomplete architecture where 2 convolution neural layers are missing. This issue is in reference to the python notebook mentioned below.

https://github.com/donnemartin/data-science-ipython-notebooks/blob/master/deep-learning/tensor-flow-examples/notebooks/3_neural_networks/alexnet.ipynb

Hi Everyone,

I'll be temporarily unavailable starting April 17 and won't be able to respond to issues or pull requests. I'm hoping I'll be available in a couple of weeks, but I might not be responsive for several weeks.

Sorry for the inconvenience.

-Donne

Mac users can benefit from a much simplified installation method thanks to Homebrew.

In the code cell 3:
data = ['peter', 'Paul', None, 'MARY', 'gUIDO']
data.map(lambda s: s.capitalize())
was changed to
[s.capitalize() for s in data if s!= None ]
and now the code woks fine.

Hello,

When I try to open the hyperlinks which should direct me to the correct ipython notebook, it returns me "Error 503 No healthy backends"
"No healthy backends
Guru Mediation:
Details: cache-fra1236-FRA 1462794681 3780339426
Varnish cache server"

Thanks
Jiahong Wang

https://github.com/facebookresearch/dynalab/blob/main/.gitignore#L1-L27

Title says it all.

Is it possible to include a section for Sage related notebooks?

fifofoof

"Bokeh is a Python interactive visualization library that targets modern web browsers for presentation. Its goal is to provide elegant, concise construction of novel graphics in the style of D3.js, but also deliver this capability with high-performance interactivity over very large or streaming datasets. Bokeh can help anyone who would like to quickly and easily create interactive plots, dashboards, and data applications."

Bokeh seems like a good candidate to feed data from Spark streaming and sharing results to stakeholders who don't use visualization tools like Tableau.

Bokeh at Pycon

Why don't menction also Disco Project (http://discoproject.org/)?
It's an easy Map Reduce framework in Python (jobs) + Erlang (scheduling) by Nokia

I want to update this notebook using the latest TensorFlow version 2.4.1.

Please assign this issue to me.

data science learning is difficult.

data-science-ipython-notebooks

Reported by core_dumpd on Reddit /r/DataScience.

Solution seems to be discussed in Stack Overflow here.

core_dumpd reports the following works, need to confirm and update repo:

I end up running this:
PYSPARK_DRIVER_PYTHON_OPTS="notebook --profile=pyspark" /usr/local/spark/bin/pyspark

With:
PYSPARK_PYTHON=/opt/anaconda/bin/ipython PYSPARK_DRIVER_PYTHON=/opt/anaconda/bin/ipython

I'm running on docker based on sequenceiq/hadoop-docker:latest with Spark/MiniConda added on top. The only real config options in the profile are for the ip = '*' and open_browser = False.

Hey, I see that there are no tutorial notebooks for implementing machine learning algorithms and neural networks in PyTorch in this repo yet. PyTorch is gaining a lot of traction lately and is really going to be one of the most popular frameworks due to its dynamic computational graph & eager execution.

I would like to add such tutorial notebooks in PyTorch.

Allen Downey's workshop on statistical inference with computation methods in Python was a great tutorial during Pycon 2015.

https://github.com/AllenDowney/CompStats

Hey,

I see that there are no tutorial notebooks for Trax implementations in this repository yet. Trax is an end-to-end library for deep learning that focuses on clear code and speed. It is actively used and maintained in the Google Brain team.

I would like to add such tutorial notebooks in Trax

Spark 1.3 introduced DataFrames:

A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a data frame in R/Python, but with richer optimizations under the hood. DataFrames can be constructed from a wide array of sources such as: structured data files, tables in Hive, external databases, or existing RDDs.

https://spark.apache.org/docs/1.3.0/sql-programming-guide.html#dataframes

Spark 1.6 introduces DataSets:

A new Dataset API. This API is an extension of Spark’s DataFrame API that supports static typing and user functions that run directly on existing JVM types (such as user classes, Avro schemas, etc). Dataset’s use the same runtime optimizer as DataFrames, but support compile-time type checking and offer better performance. More details on Datasets can be found on the SPARK-9999 JIRA.

https://databricks.com/blog/2015/11/20/announcing-spark-1-6-preview-in-databricks.html

Note: PySpark will included DataSets in the upcoming Spark 2.0.

I tried rnn-lstm in Keras, the link seems to be expired.

There are other many links too showing 404 error. Please fix them.

The split of this notebook from matplotlib.ipynb left out key imports, which is now causing errors when running the notebook.

sir ,plz send me code along with churn dataset