Figure out why explict LD_LIBRARY_PATH is needed about binder HOT 5 CLOSED

yeah, I was puzzled by that too, since clearly we don't have that issue when running the rocker images directly. Shooting from the hip here, but I'm guessing this is somehow a side-effect of bypassing the rserver process (which requires root) and just running rsession directly (which is how Ryan nicely bypasses the existing rserver config and the need for root). But could be way off. CC'ing @ryanlovett here.

What I think we want is the ld-paths created by $RHOME/etc/lpaths script ($RHOME is /usr/local/lib/R for us), e.g.

Sys.getenv("LD_LIBRARY_PATH")
[1] "/usr/local/lib/R/lib::/lib:/usr/local/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server"

on a standard rocker image.

Note that the script /usr/lib/rstudio-server/bin/r-ldpath has:

RHOME=$1
. $RHOME/etc/ldpaths
echo -n "${LD_LIBRARY_PATH}"

which sources in the correct script to set LD_LIBRARY_PATH; so I suspect that's the step that is getting bypassed when we don't call rserver?

from binder.

That's highly possible. I wonder if it's related to me finding out that PATH isn't respected by R?

# For some reason, R doesn't seem to want to interpret PATH from the environment!
#
#   rstudio@32b818e938c3:~$ echo $PATH
#   /srv/venv/bin:/usr/lib/rstudio-server/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#   rstudio@32b818e938c3:~$ R --quiet -e 'Sys.getenv("PATH")'
#   > Sys.getenv("PATH")
#   [1] "/usr/lib/rstudio-server/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
#
# This causes it to never find the `jupyter` script in ${VENV_DIR}/bin.
# This is the only hack that seems to work.
RUN R --quiet -e "Sys.setenv(PATH='${PATH}'); IRkernel::installspec(prefix='${VENV_DIR}')"

from binder.

Nope, the $PATH thing is a separate issue. R is getting $PATH from the system .Renviron file, not from the local environment. See https://github.com/rocker-org/rocker-versioned/blob/master/rstudio/Dockerfile#L52 we set the PATH as an ENV first and then manually write it into the system .Renviron :(

But for LD_LIBRARY_PATH; I'd wager a few peanuts that /usr/bin/rserver runs /usr/lib/rstudio-server/bin/r-ldpath, but that rsession executed by itself does not. (e.g. see https://support.rstudio.com/hc/en-us/community/posts/200645248-Setting-up-LD-LIBRARY-PATH-for-a-rsession) @ryanlovett is running this script something you want to add at your end?

from binder.

Technically I think just using /usr/local/lib/R/lib could have some trouble linking java libraries that should be part of LD_LIBRARY_PATH as detected by that library script, so some day we might want to have R source $RHOME/etc/ldpaths (e.g. by adding an R command to $RHOME/Rprofile.site maybe, since that R script gets run on startup under usual circumstances).

from binder.

I didn't investigate why LD_LIBRARY_PATH was needed, just observed that it was for rsession to start correctly. If rsession was built and shipped natively by Linux distributions, it'd probably have the right embedded linker path so the runtime path wouldn't be necessary.

I think nbrsessionproxy could probably do the equivalent of sourcing ${RHOME}/etc/ldpaths as you suggest, so the admin doesn't need to prep it.

from binder.