Add the ability to load R data into D-Tale about dtale HOT 16 CLOSED

The reticulate package in R automatically converts R objects into python objects when referencing them for use in python code from within R as well.

from dtale.

@Sebbyyy do you think it might be overkill for me to build a loader for R datasets into dtale using rpy2? I should just suggest that people use reticulate to convert their R data into pandas dataframes to pass to dtale?

I’m just trying to think of the best way for R users to use D-Tale without needed to dig into python that much.

I figure if I build the loader then all they’ll be tasked with on the python side is installing python and creating a virtualenv then using the rpy2 loader from the command line for any dataset they’ve saved off

Thanks for any feedback

from dtale.

@aschonfeld

I think anyone who is coming from the R side to use dtale should just use reticulate.

All they need to do is install python and the packages on the python-side.

From R side -
library('reticulate')
dtale <- import('dtale')
d = dtale$show(df)
d$open_browser()

It just seems super slow and sometimes can't really load. You may need someone with more experience on R and Python interface to test. You could ask the author of reticulate?

from dtale.

@Sebbyyy Thanks for the instructions, i may include them in the README for R users. I may still try the rpy2 loader, just to see if speed is still an issue. Although I think to get the full effect of D-Tale(loading data on the fly) you need to go your reticulate route

Thanks for your help!

from dtale.

@aschonfeld
Would you be able to try it from R? I want to see if it is just my pc struggling with dtale or there is some issue going through reticulate.

from dtale.

I just installed R on my machine and i was going to start hacking on it tomorrow morning. I’ll get back to you with my findings. I’m running linux and its a pretty beefy machine

from dtale.

@Sebbyyy was there anything special you had to do to allow your D-Tale process to be viewable from reticulate? I was able to run your code, but I can't view D-Tale from my browser.

I wonder if its because I didn't install miniconda but linked to a pre-existing python virtualenv?

from dtale.

@aschonfeld I had to install miniconda for python to work for me through RStudio.

Did you try running other python code through R?

try some basic code and see if it runs?
py_run_string('print('hello')')

from dtale.

The python code was definitely working because it was defaulting to printing the underlying pandas dataframe when doing “dtale$show(df)” i think its just something with either our proxy or my installation of R/python.

I’ll talk with the Core team tomorrow about getting miniconda installed

from dtale.

@aschonfeld Yeah this is beyond me. Even when I got it running in my browser, dtale stalled out or was v slow. There may be something in the background causing issues

from dtale.

Dont worry we’ll get it figured out. Even if rpy2 ends up being the way. R & python so closely knit there has to be a way

from dtale.

@Sebbyyy good news! D-Tale was working (kind of). So it looks like for some reason R is not allowing the Flask process to be run as a subprocess.

So I was able to run the following code:

library('reticulate')
dtale <- import('dtale')
df <- read.csv('https://vincentarelbundock.github.io/Rdatasets/csv/boot/acme.csv')
dtale$show(df, subprocess=FALSE, open_browser=TRUE)

So this will work fine, but the problem is that you won't be able to interact with the D-Tale session from your R session because it is blocking (since it's not a subprocess). Which is fine I guess if all you want to do is introspect your data. But if you want to add more pieces of data to it or alter the state of your data from the R console you're SOL.

I'll do some more digging on why D-Tale can't be run as a subprocess like when you're in python. I wonder if its something around how reticulate manages the python state.

from dtale.

@aschonfeld subprocess = FALSE makes it work much better on my data as well.

I do think there may be something wrong with translating data from python to R - Some of my data that was a negative, did not translate to python well

from dtale.

Unfortunately that seems to be a limitation of R converting its data to pandas data structures. You might have to dig into how it parses the data

from dtale.

@Sebbyyy this is now possible in v1.39.0 of D-Tale. I added the ability to load R datasets from the command-line as well as python console/jupyter notebook.

Command-line
dtale --r-path /home/jdoe/test.rda

Python console/notebook

import dtale

dtale.show_r(path='/home/jdoe/test.rda')

This is only available for python >= 3.0 and make sure you install rpy2. You could also simply do pip install -U dtale[r]

from dtale.

@Sebbyyy if you're looking for how I loaded the R datasets, here is the code:

  if not PY3:
      raise EnvironmentError("In order to use the R you must be on python3!")
  try:
      import rpy2.robjects as ro
      from rpy2.robjects.packages import importr
      from rpy2.robjects import pandas2ri

      from rpy2.robjects.conversion import localconverter
  except ImportError:
      raise ImportError("In order to use the R loader you must install rpy2!")

  pandas2ri.activate()
  base = importr("base")
  path = '/home/jsoe/test.rda'
  base.load(path)
  with localconverter(ro.default_converter + pandas2ri.converter):
      dataset = base.ls()[0]
      df = ro.conversion.rpy2py(base.get(dataset))
      return df

When you get a sec, please toss your ⭐ on the repo, thanks! 🙏

from dtale.