Show executed code for JN during autograding about otter-grader HOT 5 CLOSED

There is a way to use Gradescope's debug via SSH (or otter grade or otter run) to do this. If you set "debug": true in the otter_config.json file in the autograder configuration zip file, this will tell otter to stop executing when it reaches an error and raise the error. To use this with Gradescope's debug via SSH feature (so you don't have to rebuild the grading image), you can just add this key to /autograder/source/otter_config.json and then run the run_autograder script. I don't think this is documented clearly anywhere, so I'll leave this issue open to track adding these steps to the documentation.

N.B. to do a similar thing with otter grade, you can create a docker container using the grading image it created for you (the value of the -n flag is the tag name, so otter-grade -n hw01 creates an image called otter-grade:hw01) with docker run -it otter-grade:hw01 bash. Then copy a submission into the container with docker cp subm.ipynb <container id>:/autograder/submission/subm.ipynb. Then in the bash prompt opened by the first command you can recreate the steps above as if you were SSHed into the Gradescope container.

from otter-grader.

this section describes the assignment names but doesn't actually mention that this is the -nflag, which is a pretty large oversight.

from otter-grader.

If you could kindly add to the documentation instructions for actually running the tests once you've logged in (you do "run_autograder" in that folder to run the autograder...) Seems stupid, but when you login there are multiple run files, that all kinda do the same thing, but it's confusing.

Is it true that if I set "show_stdout" to true, then this would have solved my problem (i.e., it would show the results of the JN calculation before the tests are run)?

from otter-grader.

If you could kindly add to the documentation instructions for actually running the tests once you've logged in (you do "run_autograder" in that folder to run the autograder...) Seems stupid, but when you login there are multiple run files, that all kinda do the same thing, but it's confusing.

will do

Is it true that if I set "show_stdout" to true, then this would have solved my problem (i.e., it would show the results of the JN calculation before the tests are run)?

No. show_stdout controls whether students are shown the autograder stdout when grades are published; instructors can always seen the autograder stdout. To debug this without using debug via SSH, you would've needed to set debug in the otter_config.json in the autograder zip file, reupload it to Gradescope and rebuild the image, and then rerun the autograder on the submission. (You would've also had to upload the old zip file after debugging to ensure that other submissions didn't run with the debug flag enabled.)

from otter-grader.

#759 adds a debugging guide to the docs

from otter-grader.