manual testing framework about docs HOT 14 OPEN

Extend this to example I/O. The introduction is riddled with references to stan_major=1, etc.

from docs.

I'll try to create a branch that uses the listings latex package.

from docs.

It looks like there are two main packages that are being used now. There's the listings and minted. I tried with minted because it does some useful things.

With minted, I was able to change the existing code blocks to be processed by minted.
Pros:

• full syntax highlighting; I think @jrnold was behind it!
• will be easy to include from files and specify start lines and end lines
• will be able to remove blank spaces up front easily
• there are options for syntax highlighting. Colors for keywords, etc.

Cons:

• relies on python and the Pygments package, which is straightforward to install
• it's slow. The manual currently builds in ~20 secs. With the minted package, it builds in ~2:20. That's without including files, etc. If we use a simpler lexer (just text instead of Stan), it still builds in ~2:00.

Thoughts?

I'm going to try listings as a comparison.

from docs.

If anyone wants to try minted, use this branch: feature/issue-319-manual-testing.

To change the style of how the code is rendered, change line 211 of stan-manuals.sty:
\usemintedstyle{friendly}

I tried with \usemintedstyle{bw} and it'll generate the Stan code like it is in the manual now.

from docs.

What's the output look like?

You mean it builds end to end from clean-all in 2 mins, or that each run of pdflatex takes 2 mins, so the whole thing takes like 10 mins+?

I'm not keen on dependencies, but then I added the fonts,
so I can hardly object if someone else thinks we need code
highlighting.

I really don't want too much highlighting---otherwise the code looks like an Xmas tree and I find it hard to read. Also, it might not print well if the colors aren't dark and high-contrast (not that I ever print anything). Also, the highlighting has to be correct. Inconsistent or incomplete highlighting is worse than none at all.

from docs.

With the bw style, it looks like the existing manual.

I checked in a version that'll make it look like a Christmas tree. (I just
did that to demonstrate that we could.)

It builds end to end from clean-all in 2 minutes and change.

On Fri, Sep 2, 2016 at 12:57 PM, Bob Carpenter [email protected]
wrote:

What's the output look like?

You mean it builds end to end from clean-all in 2 mins, or that each run
of pdflatex takes 2 mins, so the whole thing takes like 10 mins+?

I'm not keen on dependencies, but then I added the fonts,
so I can hardly object if someone else thinks we need code
highlighting.

I really don't want too much highlighting---otherwise the code looks like
an Xmas tree and I find it hard to read. Also, it might not print well if
the colors aren't dark and high-contrast (not that I ever print anything).
Also, the highlighting has to be correct. Inconsistent or incomplete
highlighting is worse than none at all.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/issues/319#issuecomment-244430457, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZ_F7csHaPLa4JIQdUXC3meBEV7WZnlks5qmFV0gaJpZM4BIN0F
.

from docs.

You committed a verison with \usemintedstyle{friendly}.

This is way too slow. It takes over 1s/page on my machine,
and it needs multiple pass, so I estimated over half an hour
and got bored waiting and gave up.

Maybe we could do this for release version, but for
work-in-progress this is going to be a huge drag on my
working on the manual.

I wrote this entire email between pages 110 and 297.

• Bob

On Sep 2, 2016, at 6:54 PM, Daniel Lee [email protected] wrote:

If anyone wants to try minted, use this branch: feature/issue-319-manual-testing.

To change the style of how the code is rendered, change line 211 of stan-manuals.sty:
\usemintedstyle{friendly}

I tried with \usemintedstyle{bw} and it'll generate the Stan code like it is in the manual now.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

from docs.

What are you running on? Did you have to do something
to juice Python?

• Bob

On Sep 2, 2016, at 7:00 PM, Daniel Lee [email protected] wrote:

With the bw style, it looks like the existing manual.

I checked in a version that'll make it look like a Christmas tree. (I just
did that to demonstrate that we could.)

It builds end to end from clean-all in 2 minutes and change.

On Fri, Sep 2, 2016 at 12:57 PM, Bob Carpenter [email protected]
wrote:

What's the output look like?

You mean it builds end to end from clean-all in 2 mins, or that each run
of pdflatex takes 2 mins, so the whole thing takes like 10 mins+?

I'm not keen on dependencies, but then I added the fonts,
so I can hardly object if someone else thinks we need code
highlighting.

I really don't want too much highlighting---otherwise the code looks like
an Xmas tree and I find it hard to read. Also, it might not print well if
the colors aren't dark and high-contrast (not that I ever print anything).
Also, the highlighting has to be correct. Inconsistent or incomplete
highlighting is worse than none at all.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/issues/319#issuecomment-244430457, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZ_F7csHaPLa4JIQdUXC3meBEV7WZnlks5qmFV0gaJpZM4BIN0F
.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

from docs.

No idea. I'm using python 2.7.10 and I just installed Pygments today.

On Sep 2, 2016, at 1:03 PM, Bob Carpenter [email protected] wrote:

What are you running on? Did you have to do something
to juice Python?

• Bob

On Sep 2, 2016, at 7:00 PM, Daniel Lee [email protected] wrote:

With the bw style, it looks like the existing manual.

I checked in a version that'll make it look like a Christmas tree. (I just
did that to demonstrate that we could.)

It builds end to end from clean-all in 2 minutes and change.

On Fri, Sep 2, 2016 at 12:57 PM, Bob Carpenter [email protected]
wrote:

What's the output look like?

You mean it builds end to end from clean-all in 2 mins, or that each run
of pdflatex takes 2 mins, so the whole thing takes like 10 mins+?

I'm not keen on dependencies, but then I added the fonts,
so I can hardly object if someone else thinks we need code
highlighting.

I really don't want too much highlighting---otherwise the code looks like
an Xmas tree and I find it hard to read. Also, it might not print well if
the colors aren't dark and high-contrast (not that I ever print anything).
Also, the highlighting has to be correct. Inconsistent or incomplete
highlighting is worse than none at all.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/issues/319#issuecomment-244430457, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZ_F7csHaPLa4JIQdUXC3meBEV7WZnlks5qmFV0gaJpZM4BIN0F
.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

from docs.

I really hate that output. Maybe if all the colors
were darker and less opposite each other and there was
no bold to mess up weight I might be able to learn to
accept it if everyone else really really wants highlighting.

Seems to have failed somehow at the end, but the real problem
is that it takes a long long time.

> make clean-all
> time make manual

...

make[1]: *** No rule to make target `stan-reference.w18'.  Stop.
Latexmk: All targets (stan-reference.pdf) are up-to-date
  building src/docs/stan-reference/stan-functions.txt
cp src/docs/stan-reference/stan-reference.pdf doc/stan-reference-2.11.0.pdf;  cp src/docs/stan-reference/stan-functions.txt doc/stan-functions-2.11.0.txt;

real    17m16.612s
user    15m28.673s
sys 1m39.578s

Running with make -j4 doesn't seem to be any faster.

• Bob

from docs.

I saw that error too.

Changing the style to bw helps, but yes, the time is a killer. I'll try the listings package out. Less dependencies is better.

On Sep 2, 2016, at 1:23 PM, Bob Carpenter [email protected] wrote:

I really hate that output. Maybe if all the colors
were darker and less opposite each other and there was
no bold to mess up weight I might be able to learn to
accept it if everyone else really really wants highlighting.

Seems to have failed somehow at the end, but the real problem
is that it takes a long long time.

> make clean-all
> time make manual

...

make[1]: *** No rule to make target `stan-reference.w18'. Stop.
Latexmk: All targets (stan-reference.pdf) are up-to-date
building src/docs/stan-reference/stan-functions.txt
cp src/docs/stan-reference/stan-reference.pdf doc/stan-reference-2.11.0.pdf; cp src/docs/stan-reference/stan-functions.txt doc/stan-functions-2.11.0.txt;

real  17m16.612s
user  15m28.673s
sys   1m39.578s

Running with make -j4 doesn't seem to be any faster.

• Bob
  —
  You are receiving this because you were assigned.
  Reply to this email directly, view it on GitHub, or mute the thread.

from docs.

Why do you want to do this? Is it that people want syntax
highlighting? If it's not subtle (by which I mean you can
barely see it at arm's length), I'm going to object.

• Bob

On Sep 2, 2016, at 7:30 PM, Daniel Lee [email protected] wrote:

I saw that error too.

Changing the style to bw helps, but yes, the time is a killer. I'll try the listings package out. Less dependencies is better.

On Sep 2, 2016, at 1:23 PM, Bob Carpenter [email protected] wrote:

I really hate that output. Maybe if all the colors
were darker and less opposite each other and there was
no bold to mess up weight I might be able to learn to
accept it if everyone else really really wants highlighting.

Seems to have failed somehow at the end, but the real problem
is that it takes a long long time.

> make clean-all
> time make manual

...

make[1]: *** No rule to make target `stan-reference.w18'. Stop.
Latexmk: All targets (stan-reference.pdf) are up-to-date
building src/docs/stan-reference/stan-functions.txt
cp src/docs/stan-reference/stan-reference.pdf doc/stan-reference-2.11.0.pdf; cp src/docs/stan-reference/stan-functions.txt doc/stan-functions-2.11.0.txt;

real    17m16.612s
user    15m28.673s
sys 1m39.578s

Running with make -j4 doesn't seem to be any faster.

• Bob
  —
  You are receiving this because you were assigned.
  Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

from docs.

I want to split out the Stan code chunks into valid Stan programs. We can
include bits of it in the manual. Then, we'd be able to test that the
chunks in the manual compile and we can generate output for the manual so
we can keep up to date with output.

On Fri, Sep 2, 2016 at 2:00 PM, Bob Carpenter [email protected]
wrote:

Why do you want to do this? Is it that people want syntax
highlighting? If it's not subtle (by which I mean you can
barely see it at arm's length), I'm going to object.

• Bob

On Sep 2, 2016, at 7:30 PM, Daniel Lee [email protected] wrote:

I saw that error too.

Changing the style to bw helps, but yes, the time is a killer. I'll try
the listings package out. Less dependencies is better.

On Sep 2, 2016, at 1:23 PM, Bob Carpenter [email protected]
wrote:

I really hate that output. Maybe if all the colors
were darker and less opposite each other and there was
no bold to mess up weight I might be able to learn to
accept it if everyone else really really wants highlighting.

Seems to have failed somehow at the end, but the real problem
is that it takes a long long time.

> make clean-all
> time make manual

...

make[1]: *** No rule to make target `stan-reference.w18'. Stop.
Latexmk: All targets (stan-reference.pdf) are up-to-date
building src/docs/stan-reference/stan-functions.txt
cp src/docs/stan-reference/stan-reference.pdf
doc/stan-reference-2.11.0.pdf; cp src/docs/stan-reference/stan-functions.txt
doc/stan-functions-2.11.0.txt;

real 17m16.612s
user 15m28.673s
sys 1m39.578s

Running with make -j4 doesn't seem to be any faster.

• Bob
  —
  You are receiving this because you were assigned.
  Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/issues/319#issuecomment-244445990, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZ_F7Y2oN8Su82oyWZeUKdq3ZUK2sViks5qmGQzgaJpZM4BIN0F
.

from docs.

OK. I could live with runtime for that functionality.
But the style has to be beter.

How would we markup the files that get included?

Also, this seems like a very low priority compared to all
the bugfixes and critical refactoring issues we have to do.

• Bob

On Sep 2, 2016, at 8:14 PM, Daniel Lee [email protected] wrote:

I want to split out the Stan code chunks into valid Stan programs. We can
include bits of it in the manual. Then, we'd be able to test that the
chunks in the manual compile and we can generate output for the manual so
we can keep up to date with output.

On Fri, Sep 2, 2016 at 2:00 PM, Bob Carpenter [email protected]
wrote:

Why do you want to do this? Is it that people want syntax
highlighting? If it's not subtle (by which I mean you can
barely see it at arm's length), I'm going to object.

• Bob

On Sep 2, 2016, at 7:30 PM, Daniel Lee [email protected] wrote:

I saw that error too.

Changing the style to bw helps, but yes, the time is a killer. I'll try
the listings package out. Less dependencies is better.

On Sep 2, 2016, at 1:23 PM, Bob Carpenter [email protected]
wrote:

I really hate that output. Maybe if all the colors
were darker and less opposite each other and there was
no bold to mess up weight I might be able to learn to
accept it if everyone else really really wants highlighting.

Seems to have failed somehow at the end, but the real problem
is that it takes a long long time.

> make clean-all
> time make manual

...

make[1]: *** No rule to make target `stan-reference.w18'. Stop.
Latexmk: All targets (stan-reference.pdf) are up-to-date
building src/docs/stan-reference/stan-functions.txt
cp src/docs/stan-reference/stan-reference.pdf
doc/stan-reference-2.11.0.pdf; cp src/docs/stan-reference/stan-functions.txt
doc/stan-functions-2.11.0.txt;

real 17m16.612s
user 15m28.673s
sys 1m39.578s

Running with make -j4 doesn't seem to be any faster.

• Bob
  —
  You are receiving this because you were assigned.
  Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/issues/319#issuecomment-244445990, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZ_F7Y2oN8Su82oyWZeUKdq3ZUK2sViks5qmGQzgaJpZM4BIN0F
.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

from docs.