pysal / notebooks Goto Github PK

jupyter/ipython notebooks demonstrating PySAL functionality

Jupyter Notebook 8.93% Python 88.45% Dockerfile 2.62%

notebooks's Introduction

PySAL Notebooks Project

This project uses jupyter-book to build a book with all the notebooks of the packages in the PySAL federation. The result is available at:

http://pysal.org/notebooks

Dependencies

If you want to access the built book, simply head over to the URL above. If you want to build the book locally, you will need the following:

A recent version of jupyter-book
Unix OS: the builder relies on a few shell commands (rm, mkdir, cp, mv, cd)
pandoc (for .rst to .md conversion)
git
Python libraries listed in build-requirements.txt

A recommended route is to run this repository inside a gds_env (use the gds_dev flavour) container, which is the platform used to build it. If you have it installed, you can launch it from the notebooks folder by running:

> docker run --rm -ti -p 8888:8888 -p 4000:4000 -v ${PWD}:/home/jovyan/work darribas/gds_dev:4.0

Build process

The current build process involves two main steps:

Pull repositories to extract the files used to build the book. This can be done by running the following command from the root folder:

> python lib/build.py --pull

This will generate a notebooks folder that contains all the files required to build the book.
Build the book from the downloaded notebooks. This can be done by running the following command from the root folder:

> python lib/build.py --build

This will create/update the docs folder in the root folder so it contains everything needed for the hosting server to serve the book. Commit the changes and the updated book will be available online shortly.

The built website is available on the ./docs folder and is ready to be pushed to the repository for serving on Github Pages. If you want to serve it locally, you will need to:

Navigate into the ./docs directory:

> cd docs
Serve the site:

> jekyll serve

If you are serving it inside a container, fix the host so it can be accessed from the host machine:

> jekyll serve --host="0.0.0.0"

Then point your browser to localhost:4000/notebooks/

Additionally, the command line tool lib/build.py incorporates experimental support for testing the notebooks. Again, two options are available:

Test all the .ipynb files are well-formated and can be converted:

> python lib/build.py --test_no_run
Execute all the .ipynb files:

> python lib/build.py --test_run

notebooks's People

Stargazers

Watchers

notebooks's Issues

Master's `intro_scicomp_python.ipynb` has broken data

In this section, we read data from http://ubuntuone.com/4oIpVJDCpdREhzNdSRwMx8, which no longer exists.

I'm not sure what the dataset was supposed to be, but it looks like it was a set of tweets?

Add continuous integration for workshop branches

This should:

grab each branch of this repo we want to put into test coverage (I'd say all of them!)
use that workshop's environment.yml to set up an environment using conda env create -f environment.yml
run all notebooks, maybe with nbval
report on change.

Draw from these for sphinx galleries of upstream pysal

Use these notebooks for source material on subpackages & use cases for our initiative on designing our better documentation

Rasterio sourcing

Rasterio was consistently difficult to handle installs on windows and linux.

This may resolve with rasterio 1.0, but for now we need to identify the best way to install this on multiple platforms if possible. Again, like #26, I suspect conda forge will be our friend here in the future.

Import Error : OD_weights

OD Weights

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-4217c6ea02bd> in <module>
      1 import pysal as ps
----> 2 from pysal import weights as w
      3 import numpy as np
      4 import scipy.sparse as sp

ImportError: cannot import name 'weights' from 'pysal' (/srv/conda/lib/python3.7/site-packages/pysal/__init__.py)

Customize look & feel of book

This is lower priority but important nevertheless. It involves replacing the Jupyter book logo for a PySAL one and other changes we'd like to make to the appearance of our book.

[ENH] Do some identification/search on the correct number of regions

this is just pure sklearn, but we can't assume people have too much sklearn knowledge.

[ENH] Ensure linkage between notebooks/gallery and pysal site

We have a lot of great content so our challenge is having it all discoverable and centralized. @weikang9009 has done some great work on the landing site for the meta package so we should leverage the gallery and notebooks to be discoverable from there.

[GENERAL] Write book builder

Development for this is currently taking place at the nbs2.0 branch of @darribas fork. The roadmap is sketched under the nbs2.0 tag. Opening this as an umbrella issue for more general discussions about what we want to do. To mark as completed when the project goes live.

Resolve issues with conda/conda-forge/pypi

This may go away in the future, but we had some difficulty at the SciPy workshop with mixing conda and conda-forge.

We should probably move straight to the forge for most of the geospatial stack, since conda maintainers have been more active in applying patches to make the ecosystem work.

Add belab to notebooks site

Turn on option for the belab.

Re-order outline so it fits intro

As suggested by @knaaptime on #76:

is it possible to reorder the chapters so they follow the same outline described here (lib -> explore -> viz -> model) ?

bug: imports in notebooks have not been updated for meta package

They will fail when run on binder:

Source

[ENH] really make clear what the data borrowing is before you fit a model

eventually, replace this with the gds book spatial regression chapter, which is excellent on this front

ModuleNotFoundError-Binder Interactive Notebooks - library dependencies

Rank Based Methods Jupyter Notebook

Directional Analysis of Dynamic LISAs

Full Rank Markov and Geographic Rank Markov

Exploratory Analysis of Spatial Data: Spatial Autocorrelation

Distance Based Statistical Method for Planar Point Patterns

Minimum Bounding Circle

Quadrat Based Statistical Method for Planar Point Patterns

Point Pattern Windows

Facility Location

Snapping point to segments

splot.libpysal: assessing neigbors & spatial weights

Exploratory Analysis of Spatial Data: Visualizing Spatial Autocorrelation with splot and esda

Space-time visualisations giddy

Mapping with splot and PySAL

Spatially Varying Coefficients

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-3-5fcf88860e5d> in <module>
      1 import pandas as pd
----> 2 import libpysal
      3 import geopandas as gpd
      4 import numpy as np
      5 

ModuleNotFoundError: No module named 'libpysal'

Measures of Income Mobility

Spatially Explicit Markov Methods

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-d3b35564319e> in <module>
----> 1 from giddy import markov,mobility
      2 get_ipython().run_line_magic('pinfo', 'mobility.markov_mobility')

ModuleNotFoundError: No module named 'giddy'

Centrography of Point Patterns

Point Processes

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-2d32b704dc0b> in <module>
      1 import numpy as np
----> 2 from pointpats import PointPattern
      3 get_ipython().run_line_magic('matplotlib', 'inline')
      4 import matplotlib.pyplot as plt
      5 points = [[66.22, 32.54], [22.52, 22.39], [31.01, 81.21],

ModuleNotFoundError: No module named 'pointpats'

Spatial Graphs, Networks, Topology, & Inference

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-3-7d57733dc214> in <module>
----> 1 import spaghetti as spgh
      2 from libpysal import examples
      3 import geopandas as gpd
      4 import matplotlib.pyplot as plt
      5 import matplotlib.lines as mlines

ModuleNotFoundError: No module named 'spaghetti'

Using the Sampler

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-5f54e2b05722> in <module>
----> 1 import spvcm as spvcm #package API
      2 spvcm.both_levels.Generic # abstract customizable class, ignores rho/lambda, equivalent to MVCM
      3 spvcm.both_levels.MVCM # no spatial effect
      4 spvcm.both_levels.SESE #  both spatial error (SE)
      5 spvcm.both_levels.SESMA # response-level SE, region-level spatial moving average

ModuleNotFoundError: No module named 'spvcm'

Sparse vs. Dense Grav

Test Grav

Sparse_Grav

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-48f4e22a45ce> in <module>
      1 import numpy as np
      2 import pandas as pd
----> 3 from gravity import Gravity, Production, Attraction, Doubly, BaseGravity
      4 from entropy import Unconstrained, ProductionConstrained, AttractionConstrained, DoublyConstrained
      5 import statsmodels.formula.api as smf

ModuleNotFoundError: No module named 'gravity'

Example_NYCBikes_AllFeatures

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-3bf3e2d02e84> in <module>
----> 1 from pysal.contrib.spint.gravity import  BaseGravity, Gravity, Production, Attraction, Doubly
      2 from pysal.contrib.spint.dispersion import phi_disp
      3 from pysal.contrib.spint.vec_SA import VecMoran
      4 import pysal as ps
      5 import pandas as pd

ModuleNotFoundError: No module named 'pysal.contrib'

New Distance Band

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-50084c00507b> in <module>
      2 from scipy.spatial import distance
      3 import scipy.spatial as spatial
----> 4 from pysal.weights import W
      5 from pysal.weights.util import isKDTree
      6 from pysal.weights import Distance as Distance

ModuleNotFoundError: No module named 'pysal.weights'

GLM Speed

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-3-5e087b8bf1ca> in <module>
      1 import numpy as np
      2 import pandas as pd
----> 3 from spint.gravity import Gravity, Production, Attraction, Doubly, BaseGravity
      4 #from entropy import Unconstrained, ProductionConstrained, AttractionConstrained, DoublyConstrained
      5 import statsmodels.formula.api as smf

ModuleNotFoundError: No module named 'spint'

Sparse Categorical

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-d9a4f3428b8b> in <module>
      1 import numpy as np
      2 from scipy import sparse as sp
----> 3 from statsmodels.tools.tools import categorical
      4 from datetime import datetime as dt

ModuleNotFoundError: No module named 'statsmodels'

ODW Example

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-3d6a9795512b> in <module>
      1 import sys
      2 sys.path.append('/Users/toshan/dev/pysal/pysal/weights')
----> 3 from spintW import ODW
      4 import pysal as ps

ModuleNotFoundError: No module named 'spintW'

Autograd Test

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-18bc241f5db4> in <module>
      1 import pandas as pd
      2 import scipy.optimize as sc
----> 3 import autograd.numpy as np
      4 import autograd
      5 from autograd.convenience_wrappers import multigrad

ModuleNotFoundError: No module named 'autograd'

Poisson GLM

Gaussian_GLM

Binomial_GLM

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-1c4fb77a18fe> in <module>
----> 1 from spglm.glm import GLM
      2 from spglm.family import Poisson
      3 import libpysal.api as ps
      4 import numpy as np

ModuleNotFoundError: No module named 'spglm'

[ENH] Do silhouettes for agglomerative clustering

this will show that cluster "certainty" in spatially-constrained clustering often reveals some latent within-zone heterogeneity that the unconstrained clustering picks up. It helps to build intuition, I think.

Also, this could be made better by mapping silhouettes, rather than just scatterplotting them between methods.

add spatial optimization notebook to jupyter book

http://nbviewer.jupyter.org/gist/jGaboardi/a671d31a796e06d9df669d018d5b66ba @jGaboardi

SyntaxError: Multiple notebooks

sparse_scipy_optim

def poiss_loglike(x, data, a):
    print x
    return -np.sum(data*np.log(x)-x)*a


params = sc.fmin(poiss_loglike, 0, args=(data ,1))
print params

File "<ipython-input-11-adb8f934fbcb>", line 2
    print x
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(x)?

Sparse Categorical Bottleneck

File "<ipython-input-1-09b63899e645>", line 23
    print index                                                                    #the most intense part of the algorithm
              ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(index                                                                    #the most intense part of the algorithm)?

Sparse Categorical Speed

File "<ipython-input-6-a5621e230dc1>", line 3
    print np.allclose(spcategorical1(o).toarray(), spcategorical2(o).toarray())
           ^
SyntaxError: invalid syntax

NYC_Bike_Example

os.chdir('/Users/toshan/dev/pysal/pysal/contrib/spint')
from gravity import Gravity, Production, Attraction, Doubly

model = Gravity(flows, o_vars, d_vars, cost, 'exp')
print model.params
print model.deviance

  File "<ipython-input-2-72a7a70c129d>", line 5
    print model.params
              ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(model.params)

Change sourcing of notebooks from federated packages to meta repository

Moving forward, we want to grab the latest notebooks not from each separate repo (as currently do).

Front-End Feature Suggestions for New Users/Citizen Scientists

Journey Mapping – On home page (http://pysal.org/) for new users, create a separate user experience (new user/citizen scientist) versus Developer

(https://www.optimizely.com/optimization-glossary/user-journey-map/)

Add infrastructure to test notebooks locally before pushing to website

NameError: Sparse Categorical Speed, 4d_distance

Sparse Categorical Speed

spcat1a = []
for n in np.arange(25,250,25):
    o = np.tile(np.arange(n),n)
    s = dt.now()
    b = spcategorical1a(np.array(o))
    e = dt.now()
    spcat1a.append((e-s).total_seconds())

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-8-23d91b0c87a5> in <module>
      3     o = np.tile(np.arange(n),n)
      4     s = dt.now()
----> 5     b = spcategorical1a(np.array(o))
      6     e = dt.now()
      7     spcat1a.append((e-s).total_seconds())

NameError: name 'spcategorical1a' is not defined

4d_distance

%timeit nplinalg()
%timeit distpython()
%timeit scpkdtree()

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-3-75f6979764bc> in <module>
      1 get_ipython().run_line_magic('timeit', 'nplinalg()')
----> 2 get_ipython().run_line_magic('timeit', 'distpython()')
      3 get_ipython().run_line_magic('timeit', 'scpkdtree()')

/srv/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
   2305                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2306             with self.builtin_trap:
-> 2307                 result = fn(*args, **kwargs)
   2308             return result
   2309 

</srv/conda/lib/python3.7/site-packages/decorator.py:decorator-gen-60> in timeit(self, line, cell, local_ns)

/srv/conda/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188 
    189         if callable(arg):

/srv/conda/lib/python3.7/site-packages/IPython/core/magics/execution.py in timeit(self, line, cell, local_ns)
   1145             for index in range(0, 10):
   1146                 number = 10 ** index
-> 1147                 time_number = timer.timeit(number)
   1148                 if time_number >= 0.2:
   1149                     break

/srv/conda/lib/python3.7/site-packages/IPython/core/magics/execution.py in timeit(self, number)
    159         gc.disable()
    160         try:
--> 161             timing = self.inner(it, self.timer)
    162         finally:
    163             if gcold:

<magic-timeit> in inner(_it, _timer)

<ipython-input-2-8bde5fb2158b> in distpython()
     34 
     35 def distpython():
---> 36     dist(np.array((67, 46, 92, 67)),np.array((44, 97, 25, 50)))
     37     dist(np.array((67, 46, 92, 67)),np.array((84, 37, 66, 53)))
     38     dist(np.array((67, 46, 92, 67)),np.array((30, 46, 23, 80)))

NameError: name 'dist' is not defined

Binder redirects to `darribas`

Binder links are built to create container off the darribas account instead of the pysal org.

404 Error: Read the Docs

https://pysal.readthedocs.io/en/latest/developers/index.html

PySAL_esda failling on epd 7.3-2 (32-bit)

In [14]:

srv = pd.Series(mr.sim)
f = figure()
ax = f.add_subplot(111)
srv.plot(kind='kde', ax=ax)
axvline(mr.I, color='r')
xlabel('I')
ylabel('f(I)')

title('Empirical distribution and value of $Ir$')

UnboundLocalError Traceback (most recent call last)
in ()
2 f = figure()
3 ax = f.add_subplot(111)
----> 4 srv.plot(kind='kde', ax=ax)
5 axvline(mr.I, color='r')
6 xlabel('I')

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pandas/tools/plotting.py in plot_series(series, label, kind, use_index, rot, xticks, yticks, xlim, ylim, ax, style, grid, logy, **kwds)
591 label = series.name
592
--> 593 plot_obj = klass(series, kind=kind, rot=rot, logy=logy,
594 ax=ax, use_index=use_index, style=style,
595 xticks=xticks, yticks=yticks, xlim=xlim, ylim=ylim,

UnboundLocalError: local variable 'klass' referenced before assignment

How to handle local files?

Although we could use pandas to read distributed files, many of the notebooks that we've used in short courses rely on local files. We need to come up with a scheme to support this for the notebooks in this repos.

Add instructions for adding classes and presentations to the wiki

File Error Not Found - Multiple notebooks

Dispersion Test

import os
os.chdir('/Users/toshan/dev/pysal/pysal/contrib/spint')
from count_model import CountModel
import numpy as np
import pandas as pd
from gravity import Gravity, Production, Attraction, Doubly, BaseGravity
import statsmodels.formula.api as smf
from statsmodels.api import families
os.chdir('/Users/toshan/dev/pysal/pysal/contrib/glm')
from glm import GLM
from family import Poisson, QuasiPoisson

import pysal
import os
os.chdir('/Users/toshan/dev/pysal/pysal/contrib/spint')
from dispersion import alpha_disp, phi_disp

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-1-b8d025f70293> in <module>
      1 import os
----> 2 os.chdir('/Users/toshan/dev/pysal/pysal/contrib/spint')
      3 from count_model import CountModel
      4 import numpy as np
      5 import pandas as pd

FileNotFoundError: [Errno 2] No such file or directory: '/Users/toshan/dev/pysal/pysal/contrib/spint'

NetW

import numpy as np
import os
os.chdir('/Users/toshan/dev/pysal/pysal/weights')
from spintW import netW, mat2L

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-1-a4e23daf5780> in <module>
      1 import numpy as np
      2 import os
----> 3 os.chdir('/Users/toshan/dev/pysal/pysal/weights')
      4 from spintW import netW, mat2L

FileNotFoundError: [Errno 2] No such file or directory: '/Users/toshan/dev/pysal/pysal/weights'

remove aliasing in the imports?

some polite ribbing in the lightning talk suggested we stop importing using aliases.

PySAL devs also might want to revisit the libpysal.api recommendations, if this is a prevailing sentiment.

[ENH] add inline exercises

This is done partially in geopandas/scipy2018-geospatial-data#15, but we need to be more proactive about including this in our material.

Two examples of how this is done are provided in the scipy18 branch, the case-conflict-mapping.ipynb and case-gini-in-a-bottle.ipynb files.

Web hosting of book

This involves figuring out:

Automatic build of book
Automatic push to Github Pages
Automatic re-build of the website after any commit to the main repo

adding kernels vs. starting the notebook from within an environment.

Users were occasionally confused about which kernels were active and how they were active.

We were working with users who were likely to have more than one environment installed, since they were attending more than one workshop.

We should be consistent in our installation directions to either install nb_conda_kernelsinto our environment from conda forge, which provides the user with a better kernel browser and adds the install environment to the browser, or be clear that we expect the user to start jupyter notebook from within the environment.

I prefer the kernel browser in UI rather than in the terminal, so I would prefer using nb_conda_kernels.

BUG: build encountering JSON error in a notebook

(pysal-notebooks) src/notebooks - [master●] » python lib/build.py --build
python scripts/generate_book.py
Convert and copy notebook/md files...
 69%|██████████████████████████████████████████████████████▋                        | 36/52 [00:25<00:11,  1.38it/s]Traceback (most recent call last):
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/nbformat/reader.py", line 14, in parse_json
    nb_dict = json.loads(s, **kwargs)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 244 column 1 (char 31035)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "scripts/generate_book.py", line 242, in <module>
    cleaner = NotebookCleaner(tmp_notebook)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/nbclean/clean.py", line 19, in __init__
    self.ntbk = _check_nb_file(ntbk)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/nbclean/utils.py", line 8, in _check_nb_file
    ntbk = nbf.read(ntbk, nbf.NO_CONVERT)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/nbformat/__init__.py", line 139, in read
    return read(f, as_version, **kwargs)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/nbformat/__init__.py", line 141, in read
    return reads(fp.read(), as_version, **kwargs)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/nbformat/__init__.py", line 74, in reads
    nb = reader.reads(s, **kwargs)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/nbformat/reader.py", line 58, in reads
    nb_dict = parse_json(s, **kwargs)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/nbformat/reader.py", line 17, in parse_json
    raise NotJSONError(("Notebook does not appear to be JSON: %r" % s)[:77] + "...")
nbformat.reader.NotJSONError: Notebook does not appear to be JSON: '{\n "cells": [\n  {\n   "cell_type": "c...
Exception ignored in: <function tqdm.__del__ at 0x10be78598>
Traceback (most recent call last):
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/tqdm/_tqdm.py", line 931, in __del__
    self.close()
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/tqdm/_tqdm.py", line 1133, in close
    self._decr_instances(self)
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/tqdm/_tqdm.py", line 496, in _decr_instances
    cls.monitor.exit()
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/site-packages/tqdm/_monitor.py", line 52, in exit
    self.join()
  File "/anaconda3/envs/pysal-notebooks/lib/python3.7/threading.py", line 1029, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread
make: *** [book] Error 1

Use diverging colormaps when plotting diverging values

In ESDA, some maps use 'Rd', which is a single color ramp. But, in some cases (such as the box-and-whisker map), we need to use a divergent scheme with a zero point at the median. This might require some additional code, such as the centered choropleth functionality in pysal/splot.

input cells in giddy/Mobility_measures not rendering properly

when converting the notebooks from ipynb to html directly with nbconvert, everything works fine, but the cells in the book are rendered like this

[ENH] show silhouette box plots for identified zones

for a clustering (maybe all or more than one) plot the distributions of silhouette scores to show how they work within groups, and their relationship to the map-average.

This helps solidify what they mean as measures of cluster fit.

errors in last cells of splot/moran_bv_test

there is an error in the last cell of the notebook that looks like maybe the example wasn't finished before the last commit?

Remove static `jupyter-book` template and move it to a dependency

We currently host a static template of jupyter-book. Given how vibrant of a package it is, we should be requiring it as a dependency and running it to create a template everytime we rebuild.

[ENH] visualize one of the synthetic maps next to the observed map

This could be done using covariate[np.random.permutation(graph.n)].

It would be helpful, since it would visually illustrate how a random map looks, and how it relates to the observed pattern.

Further, we should also seriously using @darribas's spatial autocorrelation widget, but in the berlin districts.

Set up copy of notebooks into `content` folder for book

This requires to take all the notebooks pull from disparate git repos into one and then moved into book/content.

[ENH] Clear Notebooks

Clear the notebooks so that the user has to run the code in order to see the result on their machine.

This increases the likelihood the user will actually run the code and interact with the work.

If we are concerned, we can also publish all notebooks in html formats to our readthedocs page.

berlin-districts.geojson is invalid because it is not unprojected WGS84

berlin-districts.geojson (sometimes called berlin-neighborhoods.geojson in the geopython2018 branch, is not in WGS84 (unprojected) coordinates.

For it to be standards-compliant geojson, it should be unprojected.

images in package descriptions not imported/ rendering

I noticed that most packages have a static image in their README.md which is not rendered in the notebook book. I suppose due to the images not being imported in this repo/ and the image links being broken since the hard-coded image path changed.

It seems like GitHub markdown does not really support absolute image paths, so maybe it is possible to pull the images in this repo as well?

consider jupyter-bookifying recent workshops

many of our recent workshops link directly to a github repository where the source for the workshop is stored. However, @thedatalass correctly notes that this is very alienating for many users.

One thing I've looked into is making jupyter-books for workshop content. This has a few advantages.

Jupyter book has a content-focused UI, meaning that the intent of the UI is to show you the content of the workshop, not the filestructure of the workshop. This means:
- inline/rich images in jupyter notebooks
- navigation hints, like section headings and "next/previous" buttons.
- appropriate detail/linking back to the source project
Jupyter book code can be executed in the browser if the environment is specified, using thebelab (if we figure out the execution environment within which the code executes)

The process involved in making jupyter-books out of our workshops will not be complicated. I propose that we do this at least for the first 5/10 workshops.

Build `data/toc.yml`

This needs to be on the fly with a structure that represents the notebooks pull into content (see #46).

In addition, the data/toc.yml file also can have a base structure that sets up parts of the book that are static and do not depend on content from other repos.

[ENH] Do some residual analysis to ground the spatial model search

This will be done in the spatial regression book chapter, and should be ported to the next time these notebooks are used.

look at the residual map of aspatial regression
look at a regional model of regression residuals (e.g. make boxplots of residuals by region)

Again, if we synthesize this from other in-development material, this will be provided.

Value Error: Sparse Categorical Speed

Sparse Categorical Speed

x = np.arange(25, 250, 25)
plt.plot(x, spcat1, x, spcat1a, x, spcat1b, x, spcat2)
plt.legend(('spcat1', 'spcat1a', 'spcat1b', 'spcat2'))
plt.title('Speed of Sparse Dummy Functions')
plt.xlabel('Sample Size')
plt.ylabel('Seconds')

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-15-afa502da7217> in <module>
      1 x = np.arange(25, 250, 25)
----> 2 plt.plot(x, spcat1, x, spcat1a, x, spcat1b, x, spcat2)
      3 plt.legend(('spcat1', 'spcat1a', 'spcat1b', 'spcat2'))
      4 plt.title('Speed of Sparse Dummy Functions')
      5 plt.xlabel('Sample Size')

/srv/conda/lib/python3.7/site-packages/matplotlib/pyplot.py in plot(scalex, scaley, data, *args, **kwargs)
   2809     return gca().plot(
   2810         *args, scalex=scalex, scaley=scaley, **({"data": data} if data
-> 2811         is not None else {}), **kwargs)
   2812 
   2813 

/srv/conda/lib/python3.7/site-packages/matplotlib/__init__.py in inner(ax, data, *args, **kwargs)
   1808                         "the Matplotlib list!)" % (label_namer, func.__name__),
   1809                         RuntimeWarning, stacklevel=2)
-> 1810             return func(ax, *args, **kwargs)
   1811 
   1812         inner.__doc__ = _add_data_doc(inner.__doc__,

/srv/conda/lib/python3.7/site-packages/matplotlib/axes/_axes.py in plot(self, scalex, scaley, *args, **kwargs)
   1609         kwargs = cbook.normalize_kwargs(kwargs, mlines.Line2D._alias_map)
   1610 
-> 1611         for line in self._get_lines(*args, **kwargs):
   1612             self.add_line(line)
   1613             lines.append(line)

/srv/conda/lib/python3.7/site-packages/matplotlib/axes/_base.py in _grab_next_args(self, *args, **kwargs)
    391                 this += args[0],
    392                 args = args[1:]
--> 393             yield from self._plot_args(this, kwargs)
    394 
    395 

/srv/conda/lib/python3.7/site-packages/matplotlib/axes/_base.py in _plot_args(self, tup, kwargs)
    368             x, y = index_of(tup[-1])
    369 
--> 370         x, y = self._xy_from_xy(x, y)
    371 
    372         if self.command == 'plot':

/srv/conda/lib/python3.7/site-packages/matplotlib/axes/_base.py in _xy_from_xy(self, x, y)
    229         if x.shape[0] != y.shape[0]:
    230             raise ValueError("x and y must have same first dimension, but "
--> 231                              "have shapes {} and {}".format(x.shape, y.shape))
    232         if x.ndim > 2 or y.ndim > 2:
    233             raise ValueError("x and y can be no greater than 2-D, but have "

ValueError: x and y must have same first dimension, but have shapes (9,) and (0,)

Missing Static Notebook Titles

Missing Notebook title for each of the following notebooks:
GLM Speed
Sparse vs Dense
Validate Gravity
Local SI
Sparse Categorical
Test_Grav
Sparse_Grav
Sparse Categorical Bottleneck
OD Weights
Sparse Categorical Speed
ODW Example
Autograd Test
NYC_Bike_Example
4d distance
Poisson_GLM
Gaussian_GLM
Binomial GLM

Make master readme automatically, listing all the workshops stored in the repo

This could be done with a Makefile that

fetches the upstream
parses the branches
inserts the correctly-formatted hyperlinks into a unique workshops.md file
concatenates the readme together cat a.md workshops.md b.md > README.md

Set up `interact` to work on deployed website

Currently, the setup does not work because:

a) It is still linked to the original cholgraff repo
b) The book repository will require to have all the notebooks and requirements set for Binder, which is currently not the case

Binder details can be set on the "Interact link settings" of the _config.yml file under lib/jupyter-book-master folder.

pysal / notebooks Goto Github PK

notebooks's Introduction

PySAL Notebooks Project

Dependencies

Build process

notebooks's People

Stargazers

Watchers

Forkers

notebooks's Issues

title('Empirical distribution and value of $Ir$')

Recommend Projects

Recommend Topics

Recommend Org