mkery / verdant Goto Github PK

View Code? Open in Web Editor NEW

146.0 6.0 22.0 35.97 MB

An experimental tool that stores and visualizes local versioning in JupyterLab

Home Page: https://marybethkery.com/Verdant/

License: MIT License

TypeScript 84.25% CSS 9.94% JavaScript 0.04% HTML 5.77%

jupyter-extension jupyterlab experimental version-control history-management

verdant's Introduction

Verdant

🌱🌿🌱 Verdant is a JupyterLab extension that automatically records history of all experiments you run in a Jupyter notebook, and stores them in a tidy .ipyhistory JSON file designed to be work alongside and compliment any other version control you use, like SVN or Git. Verdant also visualizes history of individual cells, code snippets, markdown, and outputs for you, for quick checks and references as you work.

Thoughts? We're super interested in making Verdant serve data scientists well. --> Feedback Survey

Figure alt text:

The history tab opens the sidebar for Verdant containing three tabs: Activity, Artifacts, and Search. In the currently open Artifacts tab, the user click on a large button titled "Version Inspector". Once clicked, the inspector is active, and now when the user mouses over different parts of their notebook, cells and output highlight blue. The use mouses over a code cell and clicks it. The inspector opens up a full history of that code cell with diff highlighting in the side bar of Verdant.

[Full demo on YouTube]

For design discussion and the research behind this check out our paper:

Mary Beth Kery, Bonnie E. John, Patrick O’Flaherty, Amber Horvath, and Brad A. Myers. 2019. Towards Effective Foraging by Data Scientists to Find Past Analysis Choices. In Proceedings of ACM SIGCHI, Glasgow, UK, May 2019 (CHI’19), 11 pages. DOI: 10.475/123 4

Install

Jupyterlab: Verdant works with > JupyterLab 3 (latest version). To install or update your JupyterLab:

with pip: pip install jupyterlab -U
with conda: conda install -c conda-forge jupyterlab
to check version: jupyter lab --version
for windows users!: due to a bug with lab extensions in earlier 3.0 releases, be sure you have >= 3.0.7 of JupyterLab

NodeJs: Jupyterlab needs node to configure and install extensions, not just Verdant

to check if you have node installed node --version
to install: Official NodeJS installers

Verdant:

command line jupyter labextension install verdant-history
OR open Jupyterlab with jupyter lab . and find Verdant in the extensions menu (side panel icon button with the puzzle piece) by searching verdant

🎉 If all has gone well, you'll see Verdant's log with a leaf icon in the left pane of Jupyterlab! 🎉

if not, please file an issue with what error you're getting and we'll fix it promptly

Develop

For a development install, do the following in the repository directory:

yarn
yarn build
yarn start

The last line installs Verdant as an extension and builds JupyterLab. It may take a minute. Upon a successful build, Jupyter Lab will launch in your browser window.

If you make modifications to the Verdant source code, you'll need to rebuild the package and the JupyterLab app:

yarn build
yarn start

If you plan on making lots of edits to Verdant, extension building is easier if you use:

jupyter lab --watch

and then each time you make a change, you'll only have to re-run:

yarn build

Acknowledgements

This research has been funded by Bloomberg L.P. and has been conducted at the Bloomberg L.P. and at the Natural Programming Group at the Human-Computer Interaction Institute at Carnegie Mellon University. Thank you to the JupyterLab project and also to all our awesome study participants for volunteering early design feedback!

verdant's People

Contributors

Stargazers

Watchers

verdant's Issues

On load, estimate best match for nodey when the notebook has significantly changed

Right now, assuming the cells are in the same position and, even with internal changes, are still the same cell. Need to drop that assumption.

Handle copy-paste of nodes

yarn start fails with conflicting dependencies

Hello, I would like to try this promising extension, but its installation fails with conflicting dependencies, such as here under Ubuntu 18

yarn start
yarn run v1.22.5
$ jupyter labextension install . && jupyter lab .
An error occured.
ValueError:
"[email protected]" is not compatible with the current JupyterLab
Conflicting Dependencies:
JupyterLab Extension Package

=3.0.2 <3.1.0 >=2.1.0 <3.0.0 @jupyterlab/application
=3.0.2 <3.1.0 >=2.1.0 <3.0.0 @jupyterlab/apputils
=3.0.2 <3.1.0 >=2.1.0 <3.0.0 @jupyterlab/codemirror
=5.0.1 <5.1.0 >=4.1.0 <5.0.0 @jupyterlab/coreutils
=3.0.2 <3.1.0 >=2.1.0 <3.0.0 @jupyterlab/docmanager
=3.0.2 <3.1.0 >=2.1.0 <2.2.0 @jupyterlab/notebook
=3.0.2 <3.1.0 >=2.1.0 <3.0.0 @jupyterlab/rendermime
=6.0.2 <6.1.0 >=5.1.0 <6.0.0 @jupyterlab/services
=17.0.1 <18.0.0 >=16.9.0 <16.10.0react
=17.0.1 <18.0.0 >=16.9.0 <16.10.0react-dom
See the log file for details: /tmp/jupyterlab-debug-3g59dvsp.log
error Command failed with exit code 1.

I tried it on Mac as well, same problem. Do you probably have a solution or could help me with installing the right versions? Thanks a lot, Doron

Weird bug in which closed ghostbook freezes the whole notebook

To reproduce:
open a ghost book
close it
open a different ghost book from the verdant checkpoint panel

Separate verdant view and lilgit into separate packages

Start by decoupling the views from lilgit
give each its own package

Installation fails at Jupyterlab build

Hello, thanks for the interesting package!

I was looking to try it out, but am having trouble with install, apparently due to versioning issues. Any chance you could provide assistance? Here are the full commands to reproduce from a clean conda environment:

conda create --name=verdant python=3 jupyterlab
conda activate verdant
git clone https://github.com/mkery/Verdant.git
cd Verdant/
pip install lilgit-parser
jupyter serverextension enable --py lilgit_parser
jupyter serverextension list
npm install
npm run build
jupyter labextension link .

At this point it dies with the following error:

$ jupyter labextension link .
An error occured.
ValueError: 
"[email protected]" is not compatible with the current JupyterLab
Conflicting Dependencies:
JupyterLab              Extension      Package
>=1.1.3 <1.2.0          >=0.19.1 <0.20.0@jupyterlab/application
>=1.1.3 <1.2.0          >=0.19.1 <0.20.0@jupyterlab/apputils
>=3.1.0 <3.2.0          >=2.2.1 <3.0.0 @jupyterlab/coreutils
>=1.1.3 <1.2.0          >=0.19.1 <0.20.0@jupyterlab/docmanager
>=1.1.3 <1.2.0          >=0.19.1 <0.20.0@jupyterlab/notebook
>=1.1.3 <1.2.0          >=0.19.1 <0.20.0@jupyterlab/rendermime
>=4.1.1 <4.2.0          >=3.2.1 <4.0.0 @jupyterlab/services
See the log file for details:  /var/folders/n9/8zbz_v_14ys5cdmn0vy0b9_w0000gn/T/jupyterlab-debug-qcke3yko.log

Looking in packages.json, it seems that a specific version for JupyterLab is specified, but I wasn't able to figure out how to install the specific version (the version numbers are different for different parts of the package, and it's not even possible to install 0.19.1 from pip at the moment). Would it be possible to get some directions about what to do to get this working? Thanks!

Inspector highlighting should work in ghost book

get inspector highlighting to work in both current notebook and ghost notebook

README updates

Incompatible with JupyterLab>=3.1.0

This extension looks amazing. However, it does not appear to be compatible with the latest versions of Jupyter Lab - I am getting the following errors when trying to install:

Conflicting Dependencies:
JupyterLab              Extension       Package
>=3.1.12 <3.2.0         >=3.0.0 <3.1.0  @jupyterlab/application
>=3.1.12 <3.2.0         >=3.0.0 <3.1.0  @jupyterlab/apputils
>=3.1.12 <3.2.0         >=3.0.0 <3.1.0  @jupyterlab/codemirror
>=3.1.12 <3.2.0         >=3.0.0 <3.1.0  @jupyterlab/docmanager
>=3.1.12 <3.2.0         >=3.0.0 <3.1.0  @jupyterlab/notebook
>=3.1.12 <3.2.0         >=3.0.0 <3.1.0  @jupyterlab/rendermime

Please upgrade this extension to be compatible with the latest version of Jupyter Lab. It would also be great it this could be installed via pip instead of relying on "jupyter labextension install".

Thanks!

Allow custom notes on versions of artifacts

From the paper, need to have

Some edits don't compile in python, which blocks the change from being recorded

String is one, maybe others?
Note sometimes this is not completely a problem because the user is just writing code that does not compile yet since it's not valid python yet. But we still need to know whats there even if we can't parse it!!!!

Update Readme to list Jupyterlab v3

JupyterLab v3 seems to be required to run Verdant. It might be nice to update README.md and link to how to install JupyterLab v3: https://blog.jupyter.org/jupyterlab-3-0-is-out-4f58385e25bb

Correctly log a run when multiple cells are run at once, instead of treating each as an individual run

waiting on reliable method to do this, besides (hacky?) comparing timestamps

Loose tokens in parse step are being attatched to the wrong nodey

Need to check out py2ast.py to figure out how to avoid this problem. The ast provided by python has a start location for each node, but not an end location, making this tricky.

Handle switching between notebooks

Model keeps reinitializing itself on notebooks it has already loaded

Verdant run list does not scroll

Add filters

Working on filters for the search tab

Get server extension working for the parser

fix parser so it's a server extension and not the hacky kernel thing it is now

Syntax error thrown when parsing code cells with certain characters

If a code cell contains the symbols """ (triple quote sign) or an escape character like \n inside of a string or comment then the parser throws an error. Have tried encoding string to utf-8 to avoid this but no luck so far.