rasbt / python_reference Goto Github PK

View Code? Open in Web Editor NEW

3.7K 235.0 939.0 9 MB

Useful functions, tutorials, and other Python-related things

CSS 0.12% Python 1.29% HTML 3.69% Shell 0.02% Jupyter Notebook 94.88%

python-tips learning-python

python_reference's Introduction

A collection of useful scripts, tutorials, and other Python-related things

// Python tips and tutorials
// Python and the web
// Algorithms
// Plotting and Visualization
// Benchmarks
// Python and "Data Science"
// Useful scripts and snippets
// Other
// Links

Python tips and tutorials [back to top]

A collection of not so obvious Python stuff you should know! [IPython nb]
Python's scope resolution for variable names and the LEGB rule [IPython nb]
Key differences between Python 2.x and Python 3.x [IPython nb]
A thorough guide to SQLite database operations in Python [Markdown]
Unit testing in Python - Why we want to make it a habit [Markdown]
Installing Scientific Packages for Python3 on MacOS 10.9 Mavericks [Markdown]
Sorting CSV files using the Python csv module [IPython nb]
Using Cython with and without IPython magic [IPython nb]
Parallel processing via the multiprocessing module [IPython nb]
Entry point: Data - using sci-packages to prepare data for Machine Learning tasks and other data analyses [IPython nb]
Awesome things that you can do in IPython Notebooks (in progress) [IPython nb]
A collection of useful regular expressions [IPython nb]
Quick guide for dealing with missing numbers in NumPy [IPython nb]
A random collection of useful Python snippets [IPython nb]
Things in pandas I wish I'd had known earlier [IPython nb]

Python and the web [back to top]

Creating internal links in IPython Notebooks and Markdown docs [IPython nb]
Converting Markdown to HTML and adding Python syntax highlighting [Markdown]

Algorithms and Data Structures [back to top]

This category has been moved to a separate GitHub repository rasbt/algorithms_in_ipython_notebooks

Sorting Algorithms [Collection of IPython Notebooks
Linear regression via the least squares fit method [IPython nb]
Dixon's Q test to identify outliers for small sample sizes [IPython nb]
Counting points inside a hypercube [IPython nb]
Singly Linked List [ IPython nbviewer ]

Plotting and Visualization [back to top]

The matplotlib-gallery in IPython notebooks has been moved to a separate GitHub repository matplotlib-gallery

Featured articles:

Preparing Plots for Publication [IPython nb]

Benchmarks [back to top]

Simple tricks to speed up the sum calculation in pandas [IPython nb]

*More benchmarks can be found in the separate GitHub repository [One-Python-benchmark-per-day](https://github.com/rasbt/One-Python-benchmark-per-day)*

Featured articles:

(C)Python compilers - Cython vs. Numba vs. Parakeet [IPython nb]
Just-in-time compilers for NumPy array expressions [IPython nb]
Cython - Bridging the gap between Python and Fortran [IPython nb]
Parallel processing via the multiprocessing module [IPython nb]
Vectorizing a classic for-loop in NumPy [IPython nb]

Python and "Data Science" [back to top]

The "data science"-related posts have been moved to a separate GitHub repository pattern_classification

Featured articles:

Entry Point: Data - Using Python's sci-packages to prepare data for Machine Learning tasks and other data analyses [IPython nb]
About Feature Scaling: Standardization and Min-Max-Scaling (Normalization) [IPython nb]
Principal Component Analysis (PCA) [IPython nb]
Linear Discriminant Analysis (LDA) [IPython nb]
Kernel density estimation via the Parzen-window technique [IPython nb]

Useful scripts and snippets [back to top]

watermark - An IPython magic extension for printing date and time stamps, version numbers, and hardware information.
Shell script For prepending Python-shebangs to .py files.
A random string generator function.
Converting large CSV files to SQLite databases using pandas.
Sparsifying a matrix by zeroing out all elements but the top k elements in a row using NumPy.

Other [back to top]

Links [back to top]

PyPI - the Python Package Index - The official repository for all open source Python modules and packages.
PEP 8 - The official style guide for Python code.
PEP 257 - Python's official docstring conventions; pep257 - Python style guide checker

// News

Python subreddit - My favorite resource to catch up with Python news and great Python-related articles.
Python community on Google+ - A nice and friendly community to share and discuss everything about Python.
Python Weekly - A free weekly newsletter featuring curated news, articles, new releases, jobs etc. related to Python.

// Resources for learning Python

Dive Into Python / Dive Into Python 3 - A free Python book for experienced programmers.
The Hitchhiker’s Guide to Python - A free best-practice handbook for both novices and experts.
Think Python - How to Think Like a Computer Scientist - An introduction for beginners starting with basic concepts of programming.
A Byte of Python - a free book on programming using the Python language.
Python Patterns - A directory of proven, reusable solutions to common programming problems.
Intro to Computer Science - Build a Search Engine & a Social Network - A great, free course for learning Python if you haven't programmed before.

// My favorite Python projects and packages

The IPython Notebook - An interactive computational environment for combining code execution, documentation (with Markdown and LateX support), inline plots, and rich media all in one document.
matplotlib - Python's favorite plotting library.
NumPy - A library for multi-dimensional arrays and matrices, along with a large library of high-level mathematical functions to operate on these arrays.
SciPy - A library that provides various useful functions for numerical computing, such as modules for optimization, linear algebra, integration, interpolation, ...
pandas - High-performance, easy-to-use data structures and data analysis tools build on top of NumPy.
Cython - C-extensions for Python, an optimizing static compiler to combine Python and C code.
Numba - A just-in-time specializing compiler which compiles annotated Python and NumPy code to LLVM (through decorators)
scikit-learn - A powerful machine learning library for Python and tools for efficient data mining and analysis.

python_reference's People

Contributors

Stargazers

Watchers

Forkers

alep jeffreywugz icsaas wavelets nacyot prabampm renyuanl aitatanit mscs-usp tsnguyen momenehforoutan bigbv upbeast jarvizx maksimovica d13sl0w nrmjba krishnakumarraghu psjoshi nfredrik goryszewskig majuma li-ch datahacking xuanhan863 wujerry utahdave fnaval cnh jcity ilovejs faisalp4p jaquesgrobler jamesxuesj rohit01 satejk vchollati yingzheng vranjan gnuwolf afthill ukanukolanu justraghu codetasks lafengnan sunwaylive keithlyon22 jongoodnow shayanb kingsky23 manithnuon rachitmehrotra donatp jorgejavieri haijieg sampathweb ckj634167 mittald uroborus dhetting cdsj ahmb narayana1208 astitwlathe rbhim shankarmsy simplyali241 suranands henrilin28 wibowo87 ajiokoewj gdgupta11 jakobularius ljdawn xiaozan-pku jxlijunhao adrianhust nonva hihihippp hoangddt lseyesl giserh caohy1988 gepcel ccxuy lreelenn srve4 springcoil honglongwu wubr2000 odconfront markdays cauyrd prathapnagaraj beernlinux alexliberzonlab mengzhaoji anasrana rjpearsoniv mindis

python_reference's Issues

python-reference

Broken link in SQLite3_howto README

In the following README:
https://github.com/rasbt/python_reference/tree/master/tutorials/sqlite3_howto
the link in the line "Download the script: create_new_db.py" appears to be broken, it refers to https://raw.githubusercontent.com/rasbt/python_reference/master/tutorials/code/create_new_db.py which no longer exists.

The link URL should probably be replaced by
https://github.com/rasbt/python_reference/blob/master/tutorials/sqlite3_howto/code/create_new_db.py

Slight lambda-in-closures issue

This is great stuff!

In the not-so-obvious python tutorial, one thing caught me eye - you say that the lambda-closure problem doesn't apply to generators, but it actually does. Your example appears to work because each function returned by the generator is going to return the current value of n, which indicates how far along in generator you've gotten. The list example fails because the whole list is evaluated; the generator example "succeeds" because when you call the ith function, n is still only equal to i. To see this in action:

>>> my_gen = (lambda:n for n in range(5))
>>> a = next(my_gen)
>>> a() # a() returns 0 because that's what n is right now
0
>>> a()
0
>>> # Advancing the generator increases n by one
>>> b = next(my_gen)
>>> b() # b returns 1, as expected
1
>>> a() # but now a returns 1, too
1
>>> # calling list(my_gen) will advance the generator to the end
>>> _ = list(my_gen)
>>> a(), b() # now n is 4
(4, 4)

Sentence in Example 3.1 "Here is Why" ends prematurely

Thanks for this very helpful intro. Hope this isn't just an annoying nitpick. The sentence:

"As we go up in Python's L -> E -> G -> B hierarchy, the function a_func() finds len() already in the global scope first before it attempts"

is missing an ending such as "to search the Built-in namespace."
I was referred to your tutorial by MITx 6.00.1x Intro to CS & Programming Using Python

Sorting Algorithms IPython nb link is bad.

In the readme file, there is a hyperlink for the IPython notebook for Sorting Algorithms, but it's pointing to "404: Not Found".

Thanks.

More difference between range@py3 and xrange@py2

One thing I keeps telling people about range/xrange difference is that, the range in python3 implement the __contains__ method, which makes it very easy to test if a value is in range.

For example, in Python2, the following code is super slow:

x = 10000000 # a very large number
if x/2 in xrange(x):
    pass

But in Python3, the following code returns immediately:

x = 10000000 # a very large number
if x/2 in range(x):
    pass

To archive the same performance in Python2, one have to write the code like this:

x = 10000000 # a very large number
if 0 <= x/2 <= x-1:
    pass

%watermark addition.

FYI, IPython.sys_info() can be of help and give you commit hash or IPython and other stuff.

In [1]: import IPython
In [3]: print(IPython.sys_info())
{'commit_hash': b'4aea4a2',
 'commit_source': 'repository',
 'default_encoding': 'UTF-8',
 'ipython_path': '/Users/bussonniermatthias/ipython/IPython',
 'ipython_version': '3.0.0-dev',
 'os_name': 'posix',
 'platform': 'Darwin-11.4.2-x86_64-i386-64bit',
 'sys_executable': '/usr/local/opt/python3/bin/python3.4',
 'sys_platform': 'darwin',
 'sys_version': '3.4.0 (default, Apr  9 2014, 11:57:22) \n'
                '[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)]'}

Workaround for HTML Conversion in Table of Contents

I found a workaround for an issue you describe in tutorials/table_of_contents_ipython.ipynb when converting internal links to an HTML format. Instead of being forced to put the id-anchor tag in a separate cell like you did in your notebook:

You can simply place a line break between the id-anchor and whatever text you'd like to include:

Which will render properly without actually including any line break visual:

Bad generated html

On http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/not_so_obvious_python_stuff.ipynb an anchor is closed in wrong place:

Chapter "Only the first clause of generators is evaluated immediately" is highlighted with the code altogether on mouse over.

See screenshot:
http://h.ctf.su/img/tmp.79rYIPXEi7Ua.png

Also chapter "Assigning types to variables as values" is the only chapter title highlighted on mouse over.

Checked on Chrome 34.0.1847.132 and firefox (linux).

PS: Wanted to patch it myself, but .ipynb syntax is mad. Also it seems HTML generated isn't good at all:

...
<a name="variable_types"></p>
</div>
</div>
</div>
...

Modifying a list while looping through it

A nice addition to your "not so obvious" notebook would be the modifying a list while looping through it pitfall. For instance, you might be tempted into believing that the following will remove all even values from the list a

>>> a = [1, 2, 3, 4, 5]
>>> for i in a:
...     if not i % 2:
...         a.remove(i)
...
>>> a
[1, 3, 5]

But if you try a different example:

>>> a = [2, 4, 5, 6]
>>> for i in a:
...     if not i % 2:
...         a.remove(i)
...
>>> a
[4, 5]

I'll leave it as an exercise for you to figure out exactly what is going on here.

Dictionaries will protect you from this by raising an exception if they change during iteration, but for lists, you should use a copy (for i in a[:]), or convert the for loop into a while loop.

Great notebook by the way!

Could you please address this?

rasbt / python_reference Goto Github PK

python_reference's Introduction

A collection of useful scripts, tutorials, and other Python-related things

python_reference's People

Contributors

Stargazers

Watchers

Forkers

python_reference's Issues

Recommend Projects

Recommend Topics

Recommend Org