We should remove {devtools} from imports. It's a pretty heavy dependency, and we use it only to call {roxygen2}. We could do that directly.

See also: #88 (comment)

Adding, e.g., PATH="~/.cargo/bin:${PATH}" to your .Renviron file if rextendr/R can't find cargo, likely to here:
https://extendr.github.io/rextendr/articles/setting_up_rust.html

I'm filing an issue rather than a PR because I'm actually not sure how robust of a solution it is yet!

I would like to start generating some real examples of packages
if @Ilia-Kosenkov and @clauswilke could help.

We could, for example:

• Show memory mapped files with parallel parsing (such as JSON log files or VCF).
• Read/write data from/to an S3 data store.
• Plot some data in 3D/VR using Vulkan or OpenGL.
• Calculate and plot Linkage Disequilibrium from 1000 Genomes.

The module generation code expects #[extendr] immediately in front of a function or impl declaration. It would be good to make it more robust, e.g., allow additional attributes between #[extendr] and the declaration.

Also possibly generate a warning of no #[extendr] attributes are found.

While diagnosing my initial build issues on windows, see #2 , I found this thread describing the difference between cdylib and dylib. Turned out, it wasn't an issue for me to use dylib, but this might be a problem down the line. Also, r-rust/hellorust uses cdylib.

This is specifically concerning the generated Cargo.toml file in the source.R file.

Is it a good practice to use environment variables to configure tests? rextendr can be used for testing extendr (and in other situations), but for this, we need to configure patch.crates_io and, possibly, toolchain.
This can be achieved using env vars, but I am not sure if this is a good practice.

With extendr 0.2 on crates.io, we should consider a first CRAN release of rextendr. Is there anything currently missing that we would want to add before a first release? Any API issues that should be fixed now, before the first release?

In light of recent PRs (#29, #31, #34) I would like to suggest an overhaul of rust_source function that will simplify its usage, especially in the test environment.

• #44 Problematic parameters
  rust_source has a number of parameters that are very difficult to provide manually. I suggest creating a function (or use some package) that will convert list into toml-compatible representation. This will allow us to replace (at least) the following parameters
  a. dependencies = list(rand = "0.8.3", syn = "1.0.60")
  b. patch.crates_io = list(`extendr-api` = list( git = "..."), `extendr-macros` = list( git = "..."))
  I understand objections to list syntax, but it will allow usage of path or branch parameters, helpful while testing {extendr}/{rextendr}

• #36 options for default values.
  Instead of supplying complex default values, we should use options. I would suggest at least the following default options:
  a. options(rextendr.dependencies = NULL)
  b. options(rextendr.toolchain = NULL)
  c. options(rextendr.patch.crates_io = list(...))
  This will allow us updating function paramters to rust_source(..., patch.crates_io = getOption("rextendr.patch.crates_io")).
  Default options can be also chained with other patches simply using append(getOption("rextendr.patch.crates_io"), list(my_crate = list(path = "...")))

• #38 Referencing RTools on Windows
  When we build regular extendr-based packages, RTools gets included in the search path by R build tools.
  When we dynamically build rust libs using {rextendr}, no magic happens and the regular path gets used. By convention, RTools does not recommend exposing its mingwXX/bin paths to system's PATH, so it may not be present there. This creates testing problems due to the cross-compilation on Windows -- one cannot run rcmdcheck without first setting up system PATH. The inclusion of RTools can also be configured using options().
  I suggest to move system2 cargo call to a separate function and add the following preamble to it:

 # Append rtools path to the end of PATH
  if (.Platform$OS.type == "windows" && nzchar(Sys.getenv("RTOOLS40_HOME"))) {
    env_path <- Sys.getenv("PATH")
    r_tools_path <-
      normalizePath(
        file.path(
          Sys.getenv("RTOOLS40_HOME"), # {rextendr} targets R >= 4.0
          paste0("mingw", ifelse(R.version$arch == "i386", "32", "64")),
          "bin"
        )
      )
    Sys.setenv(PATH = paste(env_path, r_tools_path, sep = .Platform$path.sep))
    on.exit(Sys.setenv(PATH = env_path))
  }

Because RTools path is appended, any other mingw system on the path (configured by the user) will get prioritized, but if none is present, RTools will be used. With this patch, I am able to run rcmdcheck with --force-multiarch with a 'clean' system's PATH.

Usage of options should also help to address the usage of environment variables in test configurations -- there will be no need for formal arguments rewriting, the same will be achieved by simply setting options().

The documentation for rextendr::document() currently reads:

I propose the following modifications:

#' Compile Rust code and generate package documentation.
#' 
#' The function [`rextendr::document()`] updates the package documentation for an 
#' R package that uses `extendr` code, taking into account any changes that were made in the Rust code.
#' It is a wrapper for [`devtools::document()`], and it executes `extendr`-specific routines before
#' calling [`devtools::document()`]. Specifically, it ensures that Rust code is recompiled (when necessary)
#' and that up-to-date R wrappers are generated before re-generating the package documentation.

This is the TOML file generated by rextendr::use_extendr().

[package]
name = 'helloextendr'
version = '0.1.0'
edition = '2018'
[lib]
crate-type = [ 'staticlib' ]
[dependencies]
extendr-api = '*'

This might be just my preference, but I think it's more readable if there's an empty line between tables.

[package]
name = 'helloextendr'
version = '0.1.0'
edition = '2018'

[lib]
crate-type = [ 'staticlib' ]

[dependencies]
extendr-api = '*'

This can be done by concatenating each table to a single string, instead of flattening them. Here's my naive attempt (note that array of tables needs a bit more tweak):

yutannihilation@12611cd

As described on helloextendr's README, the current steps to produce the wrapper are:

# Compile the Rust code (If you are using RStudio, just run "Install and Restart")
devtools::install()

# Re-generate wrappers
rextendr::register_extendr()

# Re-generate documentation and NAMESPACE (If you are using RStudio, just run "Document")
devtools::document()

But, I come to wonder if explicit rextendr::register_extendr() is really needed. Can we generate it on compilation by tweaking some build.rs? (Or Makevars?)

In one of the latest PRs we brought in {cli}, which provides an exceptionally flexible tool for formatting messages.
It supports {glue}-like syntax with inlined styles.
It also provides CSS-like styles and themes, so the output, produced by {cli}, is highly configurable and does not depend on hard-coded constants in our code.
Windows R still does not support UTF8 out of the box, so {cli} handles special symbol display, e.g. v vs ✓.
The styling is done through ANSI escape sequences, which are part of a standard character string, but are interpreted by a terminal as colors/font weights when printed out (as cat, not print).

The reason I created this issue is that I found no (simple) way to capture the output of {cli} functions as ansi formatted strings. What I would like to have is the following:

cli::cli_alert_success("Function {.code package::function()} successfully wrote to file {.file dir/my_file.dat}.")
#> [1] "\033[32mv\033[39m Function \033[38;5;235m\033[48;5;253m\033[38;5;235m\033[48;5;253m`package::function()`\033[48;5;253m\033[38;5;235m\033[49m\033[39m successfully wrote to file \033[34m\033[34mdir/my_file.dat\033[34m\033[39m.\n"

This is plain text, which can be manipulated, most importantly concatenated, and passed to stop() to provide nicely styled error messages. And the style of these messages matches that of regular cli::cli_* functions.

So, when using rextendr::ui_throw, we could potentially have something like this:

ui_throw(
    "An error occured in function {.fun package:function}.", 
    cli::cli_alert_danger("This bad thing has happened."),
    cli::cli_alert_danger("That bad thing happened to {.file my_file}."),
    cli::cli_alert_info("Please ensure that {.val {21 * 2}} == {.val {22 + 20}}."),
    "Plain text note"
)

This is technically possible, but requires some unorthodox methods, hijacking the {cli} output generation. I doubt we can allow such a thing in {rextendr} (though it requires just two short methods and some quasi-quotation in ui_throw), so I wonder if there is something we can use to replace {cli}.

• If we choose {crayon}, we will have no access to inline styling syntax.
• {usethis} provides ui_* methods that throw errors, but we are getting rid of {usethis} in #70
• ??

We should rewrite rust_source() so it uses the new wrapper code generated by the Rust library.

This requires restructuring things a little bit but there's no fundamental technical barrier as far as I can see.

After the rewrite, rust_source() will require the Rust code to include an extendr_module! macro call. For rust_function(), we can generate the module macro call automatically as needed.

This is a tracking issue for further code cleanup regarding rextendr::document() etc. that was recently introduced. Let's file separate issues and then link them all here.

Things to consider/fix:

• Add usethis-like interface to generate messages #62
• Remove fs dependency #69
• Remove usethis dependency #70
• Don't generate .Rd pages for internal functions #71
• Improve documentation for rextendr::document() #72
• Document parameters compile in internal function make_wrappers_externally() #73
• Write vignette on how to create and update a package with extendr code #74
• Critical bug: Fix pretty_rel_path() for rust_source() use #79

The repository contains two LICENSE files:

• LICENSE, which appears to be a placeholder
• LICENSE.md, which seems to be a regular MIT license

This is confusing and does not allow GitHub to automatically pick up the repository license type.

In light of #53, I suggest adding a separate step that creates an empty package and adds extendr dependencies.
If we add a simple test (expect_equal(hello_world(), "Hello world!")), then R CMD check will reveal any potential problems with extendr setup.

testthat edition 3 requires that you opt-in. Since this project is young, I think it would be wise to do so now for the long-term health of the codebase. It's possible that will require some refactoring in the tests or code, but testthat 3 also comes with some nice features

In light of #67 we need to modify the testing workflow.
https://github.com/extendr/rextendr/blob/main/.github/workflows/test_pkg_gen.yaml
We do not test two major features, which were a source of pain in #67:

1. Non-trivial path. It is easy to call use_extendr() and document(), assuming path = ".", and having package root as the work directory. Everything works naturally. However, as soon as this changes, path resolution can break. I believe we pass path around correctly, but with future modifications, we need to track and verify this. The test script should, for example, setwd to one of the parent directories of the package root and then invoke use_extendr and document with explicit relative paths. Successful compilation ensures all paths are resolved correctly.
2. Package re-compilation. We document() test package and then run R CMD check to verify if the package is compiled and documented correctly.
   

   rextendr/.github/workflows/test_pkg_gen.yaml

   Lines 119 to 125 in 7481704

   	rextendr::document()
   	rcmdcheck::rcmdcheck(
   	path = ".",
   	args = c("--no-manual", "--as-cran"),
   	error_on = "warning",
   	check_dir = "check_use_extendr"
   	)

   
   What we do not test is re-compilation when Rust code gets modified. We need to be able to modify lib.rs, adding another {roxygen2} tag (e.g. @returns), and changing the returned value to a different string, then running document() and testing if everything has been updated in one go. This includes hello_world.Rd having new entry and a differrent return value of testpkg::hello_world().

I doubt this can be achieved using R CMD check, perhaps we should perform all checks manually and throw errors if something goes wrong (e.g., with stop). Rscript interrupted with stop should return a non-zero exit code, which should fail the test job.

I'm experiencing problems with the Markdown integration. For some reason the Rust return value is not captured anymore, and neither is any output generated by Rust.

For example, in the vignette I get this output (NULL instead of 35):

or this output (text printed by Rust is not output):

It looks like this on the website currently, so this must have been broken for a while:
https://extendr.github.io/rextendr/articles/rmarkdown.html

It worked in the past. Maybe something has changed in knitr package. As far as I can tell, it's not related to the rust_eval() function that evaluates the code. That function works fine.

Currently this parameter is not documented:

It seems we do not need msys2 dependency on Windows.
My local Windows machines have no msys2 or rtools on their PATHs, I only use RTOOLS40_HOME environment variable, yet I am able to run all tests.

When performing dynamic compilation, we inject path to rtools here

I removed these dependencies from CI here

• R CMD check
• Integration tests

I found it's not very pretty when it's outside of the current package directory.

build directory: /tmp/RtmpOxoyTp/file758976b349de

    Updating crates.io index
   Compiling rextendr3 v0.0.1 (/tmp/RtmpOxoyTp/file758976b349de)
    Finished dev [unoptimized + debuginfo] target(s) in 2.51s
✓ Writting wrappers to '../../../../tmp/RtmpOxoyTp/file758976b349de/target/extendr_wrappers.R'.

It seems there's no link to the pkgdown website. Can anyone add https://extendr.github.io/rextendr/ to the repository's description? (Or may I get the permission on this repo...?)

In a couple of places we say "writting" instead of "writing":

(The original discussion: #56 (comment))

Currently, we need to execute 3 lines of codes to reflect the changes on the Rust code. We might need some shortcut for this.

devtools::install(quick = TRUE)
rextendr::register_extendr()
devtools::document()

There are numerous internal functions that have little or no documentation. We should try to document each internal function with at least one sentence explaining what the function does, and a quick explanation of what the parameters do.

Examples (not exhaustive):

We currently only use usethis for usethis::write_over(). I think we can remove this dependency, in particular if we take the position that we don't write any files if src already exists.

It would be great if there could be a comparison between extender and Rcpp in terms of performance.

It looks like github actions have stopped running. Not sure why.

It would be nice to have a github pages site for this project. The site already exists, in the docs folder:
https://github.com/extendr/rextendr/tree/main/docs. It just needs to be turned on.

@andy-thomason Could you turn on github pages for this repo or give admin rights to the rextendr team so I can do it?

I've tried to use rextendr at another machine, and received this error:

> library(rextendr)
> install_libR_bindings(force = TRUE, quiet = FALSE)
build directory: C:\Users\tpb398\AppData\Local\Temp\Rtmp8uYQNw\file527c2f3c6a03
error: no such subcommand: `LIBRSYS_BINDINGS_DIR=C:/Users/tpb398/Documents/R/win-library/3.6/rextendr/rust/libR-sys/src`

Is there any documentation about writing Rust functions with vector and matrix as inputs/outputs? I am currently using Rcpp's NumericVector and NumericMatrix.

Thanks for this tool! I am trying to write a document using R markdown + rust. I am stuck when trying to write a code block using an external crate and cannot find any examples. Is this currently not possible?

My example:

# Test

```{r setup}
library(rextendr)
```

## Success

```{extender}
pub struct TestStruct {
    x: i32,
    y: String,
}
```

## Failure

How to do this?

```{extendr}
use tskit_rust;
let mut tables = tskit_rust::TableCollection::new(1000.).unwrap();
```

The example from here does not display output on my system when knit through Rstudio.

```{extendr}
rprintln!("Hello from Rust!");

let x = 5;
let y = 7;
let z = x*y;

z
```

I tried this with RStudio 1.2.5019 after running an "update all packages".

The session info are:

 sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Pop!_OS 20.10

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] compiler_4.0.2  htmltools_0.5.0 tools_4.0.2     xaringan_0.19   yaml_2.2.1      rmarkdown_2.6  
 [7] knitr_1.30      xfun_0.19       digest_0.6.27   rlang_0.4.10    evaluate_0.14  
> 

I am not familiar with how this website is built, but I noticed that the home page (https://extendr.github.io/rextendr/index.html) displays incorrect results, in particular, the last chunk:

rprintln!("Hello from Rust!");

let x = 5;
let y = 7;
let z = x*y;

z
#> NULL

Vignettes are built correctly though.

A few internal functions are currently documented using roxygen2, so man pages are generated. I think we should turn off man page generation. This generally means replacing #' with # at the beginning of each line.

These are the cases I'm aware of:

I'm happy to see the emergence of usethis-style functions, e.g. #53 and #60.

I always find it a bit strange, though, when usethis-style functions don't quite behave like usethis (e.g. the UI and other defaults). A simple example is using message() instead of ui_*() or cli.

One solution is to add usethis to Suggests. Another option is to create lighter-weight internal options using cli.

I'd be happy to work on this if it seems like a good idea.

I installed latest github version of the package and tried creating the following rust function from the README:

rust_function("fn add(a:f64, b:f64) -> f64 { a + b }")

I unfortuantely experienced a liking issue at the end, see attached log:

build directory: C:\Users\[redacted]\AppData\Local\Temp\Rtmp04R3wM\file46582e2c1e4d

    Updating git repository `https://github.com/extendr/extendr`
    Updating crates.io index
   Compiling winapi-build v0.1.1
   Compiling winapi v0.3.9
   Compiling winapi v0.2.8
   Compiling proc-macro2 v1.0.24
   Compiling unicode-xid v0.2.1
   Compiling syn v1.0.58
   Compiling extendr-engine v0.1.11 (https://github.com/extendr/extendr#f714309f)
   Compiling lazy_static v1.4.0
   Compiling kernel32-sys v0.2.2
   Compiling quote v1.0.8
   Compiling extendr-macros v0.1.11 (https://github.com/extendr/extendr#f714309f)
   Compiling libR-sys v0.2.1
   Compiling extendr-api v0.1.11 (https://github.com/extendr/extendr#f714309f)
   Compiling rextendr1 v0.0.1 (C:\Users\[redacted]\AppData\Local\Temp\Rtmp04R3wM\file46582e2c1e4d)
error: linking with `link.exe` failed: exit code: 1181
  |
  = note: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.28.29333\\bin\\HostX64\\x64\\link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.1109b7oki3iy8t63.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.147sjo61quy01gmx.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.18c95s8sfqsxquuj.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.1ibpcj3kfkp6aui2.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.1lqos2f8068mw4qh.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.26e1qwh59i1q5m7k.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.28pcu9knbknhvzkw.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2c9dqcehnk29gg5h.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2j9zzh0cm8tm34dr.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2lh12q8e2e0bj40g.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2mquew07qtassp2j.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2zt21rjm9icixcp2.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.328vlxktswt53wtg.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.33qkn1pkb117k5tg.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.3e9fnqq3jbqxzfk9.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.3hkoh4ve50ze28sj.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.3q25qttx9oje29z4.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.3rg0lshqq2jwm9lu.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.446tnh1hgeolf4fm.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.46z1eys4ytheu3u7.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.4fdhf75i23bk8vxa.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.4x2o391x3w6z5ml2.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.4zaytgzpph72bfw6.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.59upnde7k2408fk1.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.60phqctaujov6k4.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.br35nq041eolkor.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.f9qxssc7l1rmtgz.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.meaiqnde68um5p1.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.nykpq4yd0pxi0xe.rcgu.o" "/OUT:C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.dll" "/DEF:C:\\Users\\[redacted]\\AppData\\Local\\Temp\\rustcu1SLss\\lib.def" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.45yv15xwvrrq7lsi.rcgu.o" "/OPT:REF,NOICF" "/DLL" "/IMPLIB:C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.dll.lib" "/DEBUG" "/NATVIS:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis" "/LIBPATH:C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps" "/LIBPATH:C:/PROGRA~1/R/R-40~1.3\\bin\\x64" "/LIBPATH:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d\\target\\debug\\deps\\libextendr_api-e53357031a4dd514.rlib" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d\\target\\debug\\deps\\libextendr_engine-5644b2bd52a06f57.rlib" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d\\target\\debug\\deps\\liblibR_sys-f9732ac30d4d11f1.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd-f771f8d0374ceeb1.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libpanic_unwind-9049f341d2774a65.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_demangle-6a47dea777db971c.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libhashbrown-a3b06009982e3d9c.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_alloc-ebb710aea302de49.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunwind-4304b0c1525af3c3.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcfg_if-c70f620fc91130c5.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liblibc-3b06f844280b3802.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liballoc-86edfc7ab798fc86.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_core-3aa1c50c964075c2.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcore-3ec787c945bbba26.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-937aede5bfe8081e.rlib" "R.lib" "advapi32.lib" "ws2_32.lib" "userenv.lib" "msvcrt.lib"
  = note: LINK : fatal error LNK1181: cannot open input file 'R.lib'
          

error: aborting due to previous error

error: could not compile `rextendr1`

To learn more, run the command again with --verbose.
Error: Rust code could not be compiled successfully. Aborting.
Execution halted

This happened at least on Windows. One of the problems I see is that the linking is attempted agains R.lib, while it should be R.dll. It seems that either Makevars or something else is poorly configured.

With that said, maybe we should have 'real' tests attached to this repository? For instance, creating such add function and checking its output, because the package can be installed and testthat succeeds, yet I am unable to use it.

We should write a vignette explaining the recommended workflow.

I would like to understand what is the intended behavior if multiple Rust functions with the same name are emitted.
Consider the following scenario:

1. Generate first function named rust_inc

   rust_function("fn rust_inc(x : f64) -> f64 { x + 1f64 }")

2. Do the same but change the body:

   rust_function("fn rust_inc(x : f64) -> f64 { x + 2f64 }")

Expected behavior -- calling rust_inc(5) returns 7, actual behavior -- it returns 6.
The reason is in the wrapper, which uses .Call("wrap__rust_inc", ...).

Assuming these two functions are the first compiled in R session, the corresponding lib names are rextendr1 and rextendr2.
As a result,

.Call("wrap__rust_inc", 5, PACKAGE = "rextendr1")
# > 6

.Call("wrap__rust_inc", 5, PACKAGE = "rextendr2")
# > 7

Would it be possible to explicitly specify the library that exports the wrapper function to avoid such collisions?
This behavior is counter-intuitive to what R does, where a new function body can be bound to an old name:

r_inc <- function(x) x + 1
r_inc(5)
# > 6
r_inc <- function(x) x + 2
r_inc(5)
# > 7

UPD: This seems to be Windows-related

While investigating #2, I found out that the clean_build_dir doesn't remove the dll (on Windows) as it is already in use.
In order to remove it, it has to be unloaded.
I don't know when the contents of the temporary directory are dropped/unlinked/removed.

I tried this:

clean_build_dir <- function() {
  if (!is.null(the$shared_lib))
    dyn.unload(x = the$shared_lib)
  if (!is.null(the$build_dir)) {
    unlink(the$build_dir, recursive = TRUE, force = TRUE)
    the$build_dir <- NULL
  }
}

But this is not good, as it is called after the build is done, and the the R-session is unable to call, since the dll is deleted.

Any suggestions? Maybe this is a non-issue?

(Originally found here: extendr/extendr#190 (comment))

Currently it assumes the package root can be found, but I guess it might not be available in the case of rust_source().

As was discussed in #15, without proper tests it is impossible to catch errors like the one fixed in #16.
However, tests cannot be rust-dependent or otherwise compromise CRAN builds.
So tests can be either skipped on CRAN (or using other condition), or they can be implemented outside of the normal test framework and executed manually on CI.

I suggest to have skip_on_cran tests that can be executed when package is installed by the client -- to immediately reveal if client's system is misconfigured.

Apparently, under some circumstances, the default Rust on Windows needs to be gnu, not msvc.

See:
#15 (comment)
and
#7 (comment)

Is there any way to remove this requirement?

cc @Ilia-Kosenkov @CGMossa @yutannihilation

For Windows builds, the DLL is named without the "lib"-prefix.

E.g. in rust_source:

  # load shared library
  libfilename <- if (.Platform$OS.type == "windows") {
    paste0(libname, get_dynlib_ext())
  } else {
    paste0("lib", libname, get_dynlib_ext())
  }

  the$shared_lib <- file.path(dir, "target", "debug", libfilename)
  dyn.load(the$shared_lib, local = TRUE, now = TRUE)

Common CARGO_TARGET_DIR

A common "trick" to reduce cargo's disk usage, and compile time, is to set CARGO_TARGET_DIR or other things, to one common place for all cargo packages. This means that, when rextendr tries to locate the target, it might not be able to do so properly, as those would be in the common target directory, and not in the project (which is in a temporary location) directory.

Thus I've added:

  status <- system2(
    command = "cargo",
    args = c(
      "build",
      #"--release",  # release vs debug should be configurable at some point; for now, debug compiles faster
      sprintf("--lib --manifest-path=%s --target-dir %s/target/", file.path(dir, "Cargo.toml"), dir)
    ),
    stdout = stdout,
    stderr = stdout
  )

to the rust_source.

Edited: Moved previous points to #4 and #5.

I'll make a PR immediately as I get some comment on these points.

My understanding is there is no major reason to keep fs so let's remove it to keep dependencies low.

In extendr/libR-sys#54, we ended up discussing whether a find_cargo() helper might be useful, similar to blogdown's find_hugo()

If I understand @yutannihilation's point, it's that Rust, being a whole language, is inherently more complex tooling--including how it was installed--than a single binary like hugo, making this type of solution less obvious for Rust.

My point is that major methods of installation--particularly rustup--install cargo in predictable places, and that it's sensible to check those places if Sys.which("cargo") doesn't work. For instance, in my case, cargo is in ~/.cargo/bin, right where rustup put it.

I was hoping to find the deeper problem in the original thread, but it's still not clear to me why my PATH in R doesn't have cargo but I do otherwise (e.g in the terminal), given that my installation and workflow is similar to others. I seem not to be alone, though. Notably, #99 lets me use rextendr successfully, but it's not clear to me how robust it is.

Following up on comments from #12.

RMarkdown blocks are most useful when definitions are visible from downstream code fences, allowing one to mix code and prose. This currently works with R, Rcpp, and Python. (It does not, however, work with a c block and a few others, last I checked).

Working example (tested via vim + pandoc plugins):

```{r setup}
library(reticulate)
```

# Rcpp

```{Rcpp firstChunk}
#include <Rcpp.h>

//[[Rcpp::export]]
Rcpp::IntegerVector double2Me(Rcpp::IntegerVector x) {
  return x + x;
}
```

Blah blah blah.

```{r callFirstChunkInR}
double2Me(c(2, 2))
```

# Python

```{python}
import attr

@attr.s(auto_attribs=True)
class Test(object):
    a: int
    b: int
```

Blah blah blah--let's see this in action:

```{python}
t = Test(1, 2)
print(f"{t.a} {t.b}")
```

The analog fails with rust code:

```{r setup}
library(rextendr)
```

## Inline code

Here's a type we may want to work with.

```{extender}
pub struct TestStruct {
    x: i32,
    y: String,
}
```

More prose.

Code blocks are not persistent and this fails, being unaware of `TestStruct`:

```{extendr}
let t = TestStruct{ x: 1, y: "boo!".as_string() };
rprintln!("{} {}", t.x, y.y);
```

This is similar to the situation with C:

```{c}
typedef struct foo{
int x;
double y;
} Foo;
```

Let's use it:

```{c}
/* Uncomment the below to get this block to work */
/*
typedef struct foo{
int x;
double y;
} Foo;
*/
void x(Foo * f) {
}
```

Probably users want to set up GitHub Actions CI.

#94 adds principles.md to make explicit important internal coding conventions used in rextendr. That PR limited the contents of principles.md to user communication and errors. However, there are probably other conventions (or emerging conventions waiting to be codified) that would do well to be included.

Two particularly good examples of documents like this are in usethis and targets. These are both very well designed packages, and I think the principled approach to design helped get them there.

• Communicating with users
• Throwing errors
• to_toml()
• Working with paths, pretty_rel_path()
• Testing, including helper functions + snapshot testing