quantecon / lecture-python-programming.myst Goto Github PK

Hi @mmcky ,
Import errors occur when it imports quantecon as qe
For example, please see below in lecture.

Once we are happy with the final output of this repository. We will need to:

Hosting and Deployment:

• final migration of lecture-python-programming and freeze old repository.
• merge in manual adjustments tagged as PRs
• setup github pages for hosting this website
• update our AWS https settings to point to github pages rather than an s3 bucket. (to go live)
• work through CI workflows for simple updates to the live gh-pages branch.

Cleanup:

• rename lecture-python-programming -> lecture-python-programming.rst (as a reference)
• rename lecture-python-programming.myst -> lecture-python-programming (new source repo)
• reconfigure gh-pages for new repo location

Hi @mmcky ,

The link in footnotes does not work in the end of Pandas lecture.
[Website]

[netlify.app]

In rst,

.. [#mung] Wikipedia defines munging as cleaning data from one raw form into a structured, purged one.

In md, link disappeared

Wikipedia defines munging as cleaning data from one raw form into a structured, purged one.

There has been a request to add an introductory lecture on sympy

Hi @mmcky ,

In lectures, it often refers to other lectures and codes written in own lecture. However, this does not work in all lectures in myst.
(referring to Wikipedia or other website works ok)

For example:
in An introductory Example, it refers to Numpy. So, Numpy should be written blue like below in current website

however, in netlify.app,

In rst file:

The first line imports :doc:`NumPy <numpy>`, a favorite Python package for tasks like

But in Myst: <numpy> is dropped.

The first line imports NumPy, a favorite Python package for tasks like

Morning @mmcky and @AakashGfude ,

I noticed that there are contents for each lecture in the IPYNB files, which, however, are missing in the website/HTML files. For example, for lecture about_py, please see

• IPYNB file, for which there are simply contents,
• HTML file, which shows no contents for this lecture.

You can also see it from the following screenshot, where I labelled the contents with red lines:

Do you think it is a good idea to add contents, consistent with those in the IPYNB files, to the website/HTML files?

I find there are detailed contents for the new lecture series continuous-time Markov Chain, e.g., please see one of their lectures, memoryless, its contents are on the top right-hand side and will not disappear as we scroll down, which looks really cool!

Please look at this:

Do you think it is a good idea to make the contents specific to each lecture like what we have in lectures continuous-time Markov Chain?

There is some extra > markup in list objects

generated md:

We will break this program into two parts:

1. A user-defined function that generates a list of random variables.
2. The main part of the program that
    1. > calls this function to get dat
    2. > plots the dat

source rst:

We will break this program into two parts:

#. A user-defined function that generates a list of random variables.

#. The main part of the program that

    #. calls this function to get data

    #. plots the data

Hi @mmcky ,

Referring to equations by allocated number does not work.
For example:
In website, Equation (1) is referred by the sentence " A steady state of the model is a k that solves (1) when kt+1=kt=k."

However, in netlify.app, the sentence become " A steady state of the model is a 𝑘 that solves solow_lom when 𝑘𝑡+1=𝑘𝑡=𝑘."
It does not refer to equation by (1).

In rst,

A **steady state** of the model is a :math:`k` that solves :eq:`solow_lom` when :math:`k_{t+1} = k_t = k`.

In myst, :eq: is dropped.

A **steady state** of the model is a $k$ that solves `solow_lom` when $k_{t+1} = k_t = k$.

https://5f727dfc5850e4220dd680a0--epic-agnesi-957267.netlify.app/index.html

• missing numfocus badge?

Need to add hide-output for the initial code block as not needed and produces long output using the jupyter-book suite.

Hi @mmcky ,
In this lecture,
numba jitclass warning occurs. I think this issue happened in main web site lecture before...so I'm not sure this occurs because of this converting process.

Currently if there is no content in the right hand pane it displays: back to top

should this be empty when no content?

Morning @mmcky and @AakashGfude ,

I find another issue that the numbering for some lists in normal lecture contexts can be problematic:

• in source files, we just number some lists with 1. ...., 1. ..., 1. ..., etc.
• but when they are converted into IPYNB and HTML files, they look differently: IPYNB (still 1. ...., 1. ..., 1. ..., etc.) vs HTML (1. ...., 2. ..., 3. ..., etc.)

For example, please see the lecture getting_started here:

I also notice this issue in some other lectures in basic and advanced lectures.

Do you think it is a better idea to modify the source files directly or do something in the converting job?

Should always use: -n -W —keep-going as SPHINXOPT

Update the front page to show authors. Can you come from _config.yml?

Currently there is no obvious way to get back to the main entry page (index)

Once you leave the index page (#47) but also the top level TOC isn't available on the Left Hand Menu?

@AakashGfude is this something that might be different when we build with jupyter-book?

Hi @mmcky ,
Here, the sentence "Let’s look at an example that reads data from the CSV file pandas/data/test_pwt.csv that can be downloaded here." is shown twice in netlify.app.
Please see the last sentence in each screenshot below.
[Website]

[Netlify.app]

This issue occurs because only:: is ignored in netlify.app as follows.
in rst,

.. only:: html

    Let's look at an example that reads data from the CSV file ``pandas/data/test_pwt.csv`` that can be downloaded
    :download:`here <_static/lecture_specific/pandas/data/test_pwt.csv>`.

.. only:: latex

    Let's look at an example that reads data from the CSV file ``pandas/data/test_pwt.csv`` and can be downloaded
    `here <https://lectures.quantecon.org/_downloads/pandas/data/test_pwt.csv>`__.

In md,

Let's look at an example that reads data from the CSV file `pandas/data/test_pwt.csv` that can be downloaded
<a href=_static/lecture_specific/pandas/data/test_pwt.csv download>here</a>.

Let's look at an example that reads data from the CSV file `pandas/data/test_pwt.csv` and can be downloaded
[here](https://lectures.quantecon.org/_downloads/pandas/data/test_pwt.csv).

When we deploy we will need to configure gh-pages to use a custom domain name.

https://docs.github.com/en/free-pro-team@latest/github/working-with-github-pages/configuring-a-custom-domain-for-your-github-pages-site

(cc: @DrDrij)

Hi @mmcky ,
In this lecture, this code block does not work well as follows.

The code block shows additional > after git clone https://github.com/QuantEcon/QuantEcon.py.

@AakashGfude we have these raw html in our source markdown files. I am looking to get rid of them in favour of a pure md/myst solution due to thebe config getting added by jupyter-book automatically making sphinx-tojupyter acceptance of raw html nodes problematic.

```{raw} html
<div id="qe-notebook-header" align="right" style="text-align:right;">
        <a href="https://quantecon.org/" title="quantecon.org">
                <img style="width:250px;display:inline;" width="250px" src="https://assets.quantecon.org/img/qe-menubar-logo.svg" alt="QuantEcon">
        </a>
</div>
```

Does this have any affect on our html. It doesn't seem to -- it looks like it is just some html that is used in the download notebooks as a quantecon logo header? Is that right?

@AakashGfude would you mind merging #32 and #38 into a new single PR?

Title: Manual Adjustments for Migration

with the description updated to include:

• migrate to quantecon-book-theme
• simplify project ... etc.

That way we will get a better view of the migration end product.

The home button doesn't take you back to the index page:

https://5f727dfc5850e4220dd680a0--epic-agnesi-957267.netlify.app/index_learning_python.html#

We need to migrate this REPO to tag based publishing workflows

Hi @mmcky,
The link that refers to particular sections of lecture doesn't work.

Good examples are as follows:
In scipy lecture, 'recursive function calls' and 'described above' should refer to other particular sections in lectures like current website.

[Website]

[netlify app]

We have index elements that generate an index page called genindex.html by sphinx.

We could integrate this index page into the theme somewhere?

We now have download notebooks available so we can enable Google Colab.

https://5fc1d2df2e54967ee4401de0--hungry-lovelace-7d0512.netlify.app

However the notebooks are assumed to run in anaconda and google colab doesn't use anaconda. This has become an issue as we have swapped to using conda to install quantecon.

The home for current gh-pages setup doesn't work where it is hosted

https://quantecon.github.io/lecture-python-programming.myst/intro.html

The home button takes you to https://quantecon.github.io

@AakashGfude has mentioned that a custom domain may help fix this issue as it won't be nested in a folder

https://python-programming.quantecon.org

This is a list of fixes needed manually:

One executablebooks/sphinx-exercise#16 it merged need to update CI

Hi @mmcky and @AakashGfude , good morning!

After comparing IPYNB and HTML files for all programming lectures, I noticed an important issue in the tags or numbering in math expressions of jupyter notebooks:

• the numbering or tags for math expressions or equations in IPYNB files is different from those in HTML files,
• when we cited those equations in both versions, the citation formats are the same.

For example, let's consider the lecture python_oop and look at the same equation, equation (1) in the ipynb (you need to view it in Anaconda or VS code)and equation (7.1) in the HTML:

• In the following contexts of that equation, we cited the equation 3 times but using the tag/numbering the same as that in our website/HTML file (i.e., (7.1)).

You can also see this issue clearly from the following screenshot, where I labelled the equation tags with red lines.

Look forward to your comments.

Other issues I noticed are coming soon.

When you scroll down a page, the sidebar behaves like normal content, scrolls up, and goes out of view. Which should not be the case.

Example: https://quantecon.github.io/lecture-python-programming.myst/python_by_example.html

We currently use a few pages 404.md, status.md ... that are built by sphinx but shouldn't be part of the _toc.yml. This is an issue as it doesn't appear to be possible with jupyter-book at the moment.

I think:

404 -> theme supported
status -> unsupported for now but will be added by a status/stats directive as a page.
troubleshooting -> List it at the end of a the Book (for now) and implement a hidden from toc tag
search -> theme supported (or reserved for sphinx use)

This is a good theme, is there a way to add a thebe button in the place below so that the code can be executed on the page

I tried setting thebe:true, but it didn't work

from @jstac

The figure below "Complete the program to show summary statistics and plot the result as a time series graph like this one:" is also the wrong size. But rather than fix it, I suggest use of the glue functionality instead -- you might like to try it just for fun.

• Fixing with glue is a manual post-processing step once auto conversion is complete

Morning @jstac , @mmcky and @AakashGfude ,

In all our QE lectures, we have two kinds of figures:

1. figures generated by code and
2. figures directly inserted into the contexts.

As we discussed a long time ago and elsewhere (in Feb and in some other places), we'd love to set the default size for most figures in the 1st category (possibly the 2nd category as well?).

• For some other special figures in the 1st category, we can specify their figure size directly in the plotting code.

Right now, we use the code plt.rcParams['figure.figsize'] = (10,6) explicitly in the imports part to set the default figure-plot size in a lecture.

• e.g., please see imports in the lecture python_by_example

As we discussed before, we may want to embed this setting into the conversion process.

Anyway, I think our current migration for programming lectures should be a good place for us to start this job.

I'd love to hear your comments.

(We could create another issue for setting default figure size for the 2nd category figures as mentioned above.)

To support migration we will reference the old notebooks in lecture-python-programming.notebooks for now.

How do we set this in the theme?

• fix nested list spacing from 4 spaces to 3 spaces using vscode global search and replace using _ _ _ _ #. and _ _ _ _ * patterns. The actual rule is to match the letter so this is 2 spaces following unordered list * and 3 spaces following ordered lists #.

Currently code-block:: none was used to denote code that wasn't to be executed.
The jupyter-book ecosystem pays no attention to this and includes these blocks as code.

pandas lecture has:

.. code-block:: none

    https://research.stlouisfed.org/fred2/series/UNRATE/downloaddata/UNRATE.csv

and produces the following md

```{code-cell} none
https://research.stlouisfed.org/fred2/series/UNRATE/downloaddata/UNRATE.csv
```

this is a correct translation but will need to adapt the sources (or get jupyter-book to respect multilanguage code-blocks)

• copy of _static folder is not present in _build/myst. It will also need to be put in the source location rather than the _build location.
• contents directive not being parsed correctly (See #3) [fixed by this PR]

It looks like index elements aren't being recognised by jupyter-book properly so need to look into this. It doesn't appear to alter the html output but produces a significant number of warnings -- which makes useful warnings hard to see.

single: SciPy; Linear Algebra
/Users/matthewmckay/repos-collab/lecture-python-programming.myst/source/writing_good_code.md:23: WARNING: Directive 'index': 1 argument(s) required, 0 supplied

single: Models; Code style

We need to either:

• translate without index elements parsed until jupyter-book supports them
• fix syntax to make compatible with jupyter-book

Github Source link isn't working -- how do we specify the repo?

Hi @mmcky ,

When list contains math, list number goes wrong like below.
[lecture:writing good code]

[lecture: OOP]

However, md does not seem to be wrong.

1. sets $k_0 = 1$
1. iterates using {eq}`gc_solmod` to produce a sequence $k_0, k_1, k_2 \ldots , k_T$
1. plots the sequence

1. Evaluate the polynomial {eq}`polynom`, returning $p(x)$ for any $x$.
1. Differentiate the polynomial, replacing the original coefficients with those of its derivative $p'$.

OK so there is an issue with pandas and api calls to a data service when running these in parallel.

We will need to:

• split the osx, linux and windows into separate workflows and then set different times.

Hi @mmcky and @AakashGfude ,

I noticed that unlike the numbering system for titles of lectures, sections, subsections, etc. in the website/HTML files, we do not have such numbering system in the downloaded jupyter notebook. For example, please see those for lecture about_py here:

Do you think it is a good idea to keep the numbering system for titles in IPYNB files consistent with those in the website?

In getting started lecture the following

``!conda install quantecon``

is getting translated to:

`> !conda install quantecon`

which suggests there is an extra block quote in the AST

@sayaikegawa do you know what is broken in the source file in this section?

https://5f5b2299083bb8c326dced64--epic-agnesi-957267.netlify.app/python_by_example.html#a-version-with-a-for-loop

The page not found / search is listed in the toc

To support migration we will reference the old single page pdf from lecture-python-programming for now.

What location are pdf files pulled from?

Hi @jstac and @mmcky , I find the x-axis label of the inserted figure in exercise 1 of lecture pandas is incomplete (please see following).

and I suggest that we should update this figure.