Comments (15)
What do you get as git diff
when you hit the case of allegedly modified notebook files?
from nbstripout.
I generally see nothing when I run get diff on those files, but I haven't tested them all. (can't share them since it's my work notebooks anyways) As I mentioned, It's especially interesting that these files are gone when I uninstall nbstripout and install it again, which I weird.
from nbstripout.
Today I found that these changes show up after 'git pull', and I looked at a few other diffs and found that they are diffs between before) notebooks with output after) notebook with output stripped. So maybe the git maybe telling me to strip the output out from some notebooks which contains outputs from previous commits.
Does this make sense? But I still don't understand why these notebooks show up after 'git pull' and why they're gone after uninstall/reinstall.
from nbstripout.
So what exactly does git status
report in this situation? Are those notebooks modified? Or is it a mode change?
Also could you run a nbstripout --status
before and after the reinstall and report if there's a difference?
from nbstripout.
@jykim Any update?
from nbstripout.
This might be related to an issue I posted about here. I still haven't figured it out, so any help is welcome.
from nbstripout.
I have the same problem: when nbstripout is enabled and I checkout several files, the omitting of changes in these files is not reflected by git status (git status shows their presence, the changes are actually omitted). If I do 'nbstripout --uninstall' for the repo and then do 'nbstripout --install' again then the changes are reflected by git status.
from nbstripout.
@melaanya, try to change the configuration of the content filters as explained here and it'll help you debugging the issue. You will be able to tell when nbstripout is activated.
from nbstripout.
@stas00 I've looked at your issue and am about as puzzled as you are. Thanks for documenting it so thoroughly. Guess we'll have to find a better git plumber than I am to explain this behavior.
from nbstripout.
Currently, pretty much any time one of our developers forgets to enable the filter and commits w/o stripping the notebooks out, it breaks git pull
for anybody who already had a checked out version. And other than doing a new clone the only solution seems to be is to disable the filter, git pull
and reenable the filter.
What we really need is a way to enforce content filters repo-wide, except git doesn't support that due to security reasons (overriding user's config is unsafe). I'm not sure what to do about it. It's a constant problem for us. Perhaps there is a need to re-design content filters so that they aren't configured via a private .gitconfig and can be automatically installed for everybody (note that our repo carries its own version of the nbstripout filter, so the culprit is just not being able to enforce the configuration). Unless perhaps I'm missing something and you have some insights of another way of configuring it, so that the user doesn't need to do anything.
And of course, there are server-side hooks which could do the policing, but it doesn't sound like the best solution. In particular since last I looked github doesn't support those.
from nbstripout.
Yeah, to me this sounds like a Git issue rather than an nbstripout issue and I'm not sure what (if anything) we could fix in nbstripout to help with this. Have you considered raising the issue on the Git mailing list?
from nbstripout.
I already did so to no avail.
If it weren't for github and we were running our own git backend then server-side hooks could do the policing, but since github doesn't support that, there is nothing one can do to enforce that.
The best "enforcing" solution I found is to run a CI job that flags this problem as it appears, and hopefully the developers watch for [x]'s in their commit status on github.
echo "Check we are starting with clean git checkout"
if [ -n "$(git status -uno -s)" ]; then echo "git status is not clean"; false; fi
echo "Trying to strip out notebooks"
tools/fastai-nbstripout -d dev_nb/*ipynb dev_nb/*/*ipynb docs_src/*ipynb docs_src/*/*ipynb
echo "Check that strip out was unnecessary"
git status -s # display the status to see which nbs need cleaning up
if [ -n "$(git status -uno -s)" ]; then echo -e "!!! Detected unstripped out notebooks\n!!!Remember to run tools/run-after-git-clone"; false; fi
and the best resolution I found is:
disable the filters
git pull
fix the problems
enable the filters
git commit -a
git push
which in the fastai project is done with a script (-d
is for disable
):
tools/trust-origin-git-config -d
git pull
tools/trust-origin-git-config
from nbstripout.
FYI, we're discussing adding a dedicated nbstripout
command to help with this flow in #108. Could be worth taking inspiration from the fastai scripts.
from nbstripout.
I see the same problem, stopped using the tool after 2 days.
from nbstripout.
@DanielRudnicki Reproduction steps for the failures you're seeing would help.
from nbstripout.
Related Issues (20)
- Should be agnosting on trailing blank lines HOT 9
- [Feature Request] Process Folders (Batch / Bulk) HOT 1
- Doesn't strip out pycharm metadata HOT 9
- New release HOT 2
- Replace cram with prysk HOT 2
- Read config from `setup.cfg` HOT 1
- Option to error on cell outputs exceeding `--max-size` HOT 4
- It is recommended to remove pytest-runnner from setup_requires in setup.py HOT 4
- Strip output_type=stderr only, with keep_output? HOT 4
- Possible nbstripout-fast integration HOT 6
- Prevent committing notebooks with errors in cell outputs HOT 6
- Specifying Python executable path in `nbstripout --install`
- 'nbstripout' is not recognized as an internal or external command, operable program or batch file. HOT 2
- Not compatible with `pre-commit-hooks/pretty-format-json` hook HOT 4
- `git config filter.nbstripout.extrakeys ` support for `attachments`? HOT 4
- Support setting defaults for command line arguments via git config
- No valid notebook detected HOT 4
- required = true by default or make doc more explicit about it HOT 5
- Support git-filter-repo HOT 2
- Any examples for --keep-id option? HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nbstripout.