Running notebook w/ python 3 about forward HOT 22 CLOSED

Yes! Sorry for my delay - the command worked excellently.

from forward.

Yes, that is what I see in my config!

from forward.

Hi @vsoch, we weren't able to solve the issue, but have created a work around - finding when to run a separate pydeface.sh script. I think I will close the issue - it seems to be an issue separate from the jupyter notebook, which is working great!

from forward.

I’m on East Coast time so I’m off to bed, but if you can share the commands you’ve tried so far I can give you a script to try early tomorrow.

from forward.

Oh! And in case you weren’t aware, we also can offer Sherlock on demand https://www.sherlock.stanford.edu/docs/user-guide/ondemand/

from forward.

I've only tried loading the python3 module, and then running bash start.sh jupyter, but it hasn't been successful so far. Thank you for your help & go to sleep!

from forward.

okay here is what I just did. First I logged in with ssh sherlock (after setup of my ~/.ssh/config based on the instructions in this repository). This is to ensure that we can communicate cleanly.

ssh sherlock

Once that worked, I checked the latest version of Python on sherlock, which is python/3.6.1 (as a module). The version of jupyter hasn't changed either py-jupyter/1.0.0_py36.

So - we already have a script setup for that, I ran this:

$ bash start.sh sherlock/py3-jupyter 

And everything seemed ok - logs don't have any errors, but I think the admins on Sherlock (mainly Kilian) must have disabled this ability because I get a connection refused. I tried the same workflow with VPN, same response. Everyone is sleeping over there in CA but I'll check in with them and ping you with an update here with anything I learn. Unfortunately sometimes security standards get hardened so you can no longer use simple tools like forward :/

In the meantime, check out Sherlock On Demand, minimally I think you can request a notebook.

from forward.

Thank you! I must have missed that in the documentation. I've been able to use forward with the original jupyter command, so hopefully i will be able to use this one as well.

from forward.

Fantastic! Fingers crossed that it works ok - my issue might have been that I’m off campus.

from forward.

Did you get it working @henrymj? Can we close the issue?

from forward.

Hi,

I'm now running into a related issue. Though the notebook is running a python 3 kernel, we are struggling to use pydeface (via subprocess.call). It seems like we can't load python/3.6.1 to the compute node that the notebook is running on, which we need for pydeface to work.

We tried logging onto the compute node and loading python3 using ml python/3.6.1, but it did not change the results of python --version. Do you have any suggested work arounds?

Thank you!

from forward.

This particular script already has that module loaded. Take a look at the file to see. If you want to load other modules, edit the file.

from forward.

I see that - I guess I am confused why the jupyter terminal that I open in the compute node still says I am running python 2.7.5 when I try python --version

from forward.

python on sherlock is either python (2) or python3 (python 3) so you'd want to do python3 --version.

from forward.

@henrymj the module to load Jupyter (at least when I tested a long time ago!) should load python again (3), which is why the script does this:

module load py-jupyter/1.0.0_py36
jupyter notebook --no-browser --port=$PORT

but if it's the case that it's not loading python3, could you please try adding the ml python command to that script and try again?

I suspect if you are using subprocess you just need to target python3 instead of python though.

from forward.

Thank you, I will try that tomorrow and update you asap.

from forward.

okay sounds good! It might be helpful to login to an interactive node, and then load the jupyter module manually and check the python versions you have on your path. It could be that you have another loaded. Afaik jupyter should detect the kernels available but maybe that's not the case. Let me know what you learn! I can try to help more after that.

from forward.

hey @henrymj a quick question for you - are you using the version before or after this commit? https://github.com/vsoch/forward/pull/32/files (e.g., look at the ssh config generation and see if it uses a hash or not). We are having users report issues with the ssh forward and I want to figure out if this change did it. Thanks!

from forward.

Mine seems to be the newer version with %C ! I hope this is helpful

from forward.

yes thank you! And this is what you see in your ~/.ssh/config as well? (e.g., you could have generated it before pulling to get the updated version, which wouldn't touch your original generation).

from forward.

Following this issue as author of #32 — curious what issues that PR may have caused!

from forward.

@neutrinonerd3333, @henrymj reports that it worked previously with the change, so that suggests that it isn't it? But you never know...

from forward.