bazel-contrib / rules_ruby Goto Github PK

We had a red build on the Bazel Central Registry:
https://buildkite.com/bazel/bcr-presubmit/builds/2569#018c02af-083a-4eff-88ba-7f8ccbae9ff5

I'll disable Windows for now to land that PR, but I suspect this is trivial to fix for the next release?

Bazel 7 will be released in the next few weeks and enables bzlmod by default.

These rules should be tested with bzlmod and be published to the Bazel Central Registry with each release.

Without installing any ruby stuff on my machine, I'd like to be able to edit the Gemfile and Gemfile.lock files in a correct way by running sth like bazel run @ruby//:gem -- install rails, without worrying about version skew with a different version of gem than Bazel will use.

The runfiles produced by an rb_binary target ought to be self-contained, so that binary can be used as the tool in a subsequent action that uses a Ruby program to transform some input files into files under bazel-out.

I think this is currently not working, here's a PR with a demo that I'd expect to work:
#15

See #16 for original discussion

See #16 for original discussion

This would be useful to configure build flags for gems:

rb_bundle(
  env = {
    "BUNDLE_BUILD__GRPC": "--with-ldflags=-Wl,-undefined,dynamic_lookup"
  }
)

bundle install passes in PATH variable, which can head to non-hermetic builds.

In our case, we had $GEM_HOME/bin:$BUNDLE_PATH/gems/bin: in our PATH variable that resulted in gems getting downloaded in a different directory.

May be better to remove PATH altogether.

It seems like the ruleset requires the lib and spec directories to be at the root of the repository. We are looking at using the rules in a larger, mixed language monorepo where code typically lives in sub-directories based on the application name. In trying to adapt the gem example to fit our use case, we noted that the lib and spec directories are no longer available in the LOAD_PATH if these directories are not at the root. A simplified example can be found at: https://github.com/protocol7/rules_ruby/tree/subdirs/examples/gem

This uses a directory layout like:

├── BUILD
├── Gemfile
├── Gemfile.lock
├── MODULE.bazel
├── MODULE.bazel.lock
├── WORKSPACE.bzlmod
└── foo
    ├── lib
    │   ├── BUILD
    │   └── gem
    │       ├── BUILD
    │       └── add.rb
    └── spec
        ├── BUILD
        └── add_spec.rb

Is it possible to make the ruleset work for this type of use case?

There's a TODO introduced, see thread:
#17 (comment)

Follow-up for #41.

I've been using 0.6.0 for a bit, and the exposed Gem binaries seem follow weird patterns. Here are a few behaviors I noticed:

• The gem binary targets aren't immediately available. E.g. if run or depend on @ruby//:jekyll after cleaning my Bazel cache, I'll receive an error like:

ERROR: /home/dravesr/.cache/bazel/_bazel_dravesr/8e8f77e8714e317f7e314b9271c50fc0/external/rules_ruby~override~ruby~ruby/BUILD: no such target '@@rules_ruby~override~ruby~ruby//:jekyll': target 'jekyll' not declared in package '' defined by /home/dravesr/.cache/bazel/_bazel_dravesr/8e8f77e8714e317f7e314b9271c50fc0/external/rules_ruby~override~ruby~ruby/BUILD (Tip: use `query "@ruby//:*"` to see all the targets in that package)
ERROR: /home/dravesr/src/BUILD:10:12: no such target '@@rules_ruby~override~ruby~ruby//:jekyll': target 'jekyll' not declared in package '' defined by /home/dravesr/.cache/bazel/_bazel_dravesr/8e8f77e8714e317f7e314b9271c50fc0/external/rules_ruby~override~ruby~ruby/BUILD (Tip: use `query "@ruby//:*"` to see all the targets in that package) and referenced by '//:site'
ERROR: Analysis of target '//:site' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.541s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

if I instead first run bazel run @ruby//:bundle -- install, then the target is available. This seems like an issue with the hermeticity of the setup.

• More generally, all of the exposed gems run from somewhere in the Bazel cache / runfiles directories. This makes one command affect another and also makes it harder to export files; e.g. if I want to run jekyll new with bazel run @ruby//:jekyll, I need to fish the output files out of bazel-out/
• The exposed binaries are hard to depend on with how runfiles are managed. Maybe I'm not supposed to depend on them, but I set up a neat Bazel rule to wrap jekyll build and jekyll serve around @ruby//:jekyll, and to get the binary to run properly in a bazel run command I needed to wrap it in a script:

    executable = "export RUNFILES_DIR=$(readlink -f ../)\n"
    executable += ctx.attr.jekyll.files_to_run.executable.short_path + " " + " ".join(args) + " $@\n"

• bazel run @ruby//:bundle -- install doesn't respect bundle_fetch settings and defaults to //:Gemfile. It also doesn't play nicely with path arguments, e.g. if I run bazel run @ruby//:bundle -- install --gemfile dir/Gemfile, I'll get:

[!] There was an error parsing `Gemfile`: No such file or directory @ rb_sysopen - /home/dravesr/.cache/bazel/_bazel_dravesr/8e8f77e8714e317f7e314b9271c50fc0/execroot/_main/bazel-out/k8-fastbuild/bin/external/rules_ruby~override~ruby~ruby/bundle.runfiles/_main/dir/Gemfile. Bundler cannot continue.

I snooped through the Bazel cache a bit and found that bundle.runfiles/_main is empty and Bundle defaults to reaching out to ../../../../../../../Gemfile (7 parents, back in the execroot/_main which is symlinked to the original workspace). Similarly the Gemfile.lock is ignored.

For some extra details, here's a sample from my MODULE.bazel:

bazel_dep(name = "rules_ruby", version = "0.6.0")
# TODO: Wait for 0.6.0 to be published and remove Git override
git_override(
    module_name = "rules_ruby",
    remote = "https://github.com/bazel-contrib/rules_ruby",
    commit = "81bf18ecf7de001a6aa5b46e420f3a9b98866ad5",
)

ruby = use_extension("@rules_ruby//ruby:extensions.bzl", "ruby")
ruby.toolchain(
    name = "ruby",
    version = "3.0.6",
)
use_repo(ruby, "ruby")
ruby.bundle_fetch(
    name = "bundle",
    gemfile_lock = "//:Gemfile.lock",
    gemfile = "//:Gemfile",
)
use_repo(ruby, "bundle", "ruby_toolchains")
register_toolchains("@ruby_toolchains//:all")

I noticed that when running bazel build //some/non/ruby/target, the ruby interpreter is always downloaded even if the target does not depends on it.

It looks like rb_download always performs the download before registering the toolchain. I believe this is different from the recommended bazel toolchain setup, which only downloads the toolchain on a per-need basis (i.e. if rb_* rules are in the build action graph).

Akin to theoremlp/rules_multitool#29, as an addition to the existing support in binary.bzl, it would be great to also have a convenience target that makes it easier to bazel run the generated @bundle//bin:{tool} target in the current directory. This can be worked around with a custom script and runfiles.bash, but it would be much easier for me to just bazel run @bundle//bin/{tool}:cwd or something like that. Kind of like this.

Not sure if you're interested in this, but the support in multitool has already been very helpful to me 🙌🏼

Is there a way to support installing a specific version of bundler when setting up the ruby toolchain? Currently this is not configurable, afaik; instead the default version of bundler corresponding to the ruby version is used.

The psych gem (version 5.1.1.1) fails to install on my machine, log contains:

conftest.c:3:10: fatal error: yaml.h: No such file or directory
    3 | #include <yaml.h>
      |          ^~~~~~~~
compilation terminated.
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: #include <yaml.h>
/* end */

As documented here, for example
https://github.com/sparklemotion/nokogiri#native-gems-faster-more-reliable-installation
there's a way to start from a pre-compiled native gem instead. I think most Bazel users should be using this to achieve reproducible builds.

(Of course, you could also imagine a cc_library to build the C extension, but then we're swapping out part of the bundle installer which is a bigger project)

Current rb_binary implementation depends on Gemfile to be listed as a transitive source in order to set BUNDLE_GEMFILE env var. This is not super obvious. Given that we have already specified Gemfile in rb_bundle, a more explicit way to express this could be introducing a link_bundle_gems function for this, something like:

load("@bundle//:defs.bzl", "link_bundle_gems")
link_bundle_gems(name = "gems")

rb_binary(
    name = "add-numbers",
    args = ["1"], 
    main = "add.rb",
    deps = [
         ":gems",
    ],
)

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update dependency io_bazel_stardoc to v0.7.0

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

rb_binary() targets are not run via bundle exec so local gems are not visible to them. See https://github.com/jgao54/rules-ruby-issue-repro.

Hi,
I was using this, and bazel run //:whatever_binary works great, but if you try to run it manually via bazel-bin/whatever_binary.rb.sh, it will fail to find the relative path to the executable.

A simple patch will fix ithis.

index 4cd69dca..72771452 100644
--- ruby/private/binary.bzl
+++ ruby/private/binary.bzl
@@ -59,7 +59,7 @@ def generate_rb_binary_script(ctx, binary, bundler = False, args = [], env = {},
     toolchain_bindir = toolchain.bindir
 
     if binary:
-        binary_path = binary.short_path
+        binary_path = "${BUNDLE_BIN}/" + binary.basename
     else:
         binary_path = ""

I can fork and make a PR if that is desired, but its a simple change, so i decided to create it as an issue.

Thanks.

edit: Also i'm not sure if this change will actually break anything else.
Ultimately it would be nice to be able to run the command directly.

Ideally the ruleset should use rctx.download[_and_extract] to fetch dependency gems so that:

• they are cached in the bazel repository cache
• configuration of the Bazel downloader can be used, i.e. to ensure they are fetched from a corporate proxy
• bazel shows more information about the fetching phase than just "running bundler"
• we can apply patches to third-party gems in a similar way as other bazel fetches may be patched

Currently the approach for testing different interpreters is for CI to Run echo 'RUBY_VERSION = "3.1.4"' > ruby_version.bzl but under MODULE.bazel this .bzl file cannot be load()ed, so we have a fixed interpreter version.

A TODO for this was introduced in #48

https://github.com/bazel-contrib/rules_ruby/pull/48/files#diff-b803fcb7f17ed9235f1e5cb1fcd2f5d3b2838429d4368ae4c57ce4436577f03fR48

Try removing the exclusion and see if there's a way to fix it.

Not sure if the rules can already be used on a Rails App, but I think that this should be ultimately a goal because so many huge monolithic applications can benefit from partial Bazel builds (that would likely be mostly running tests on sub-sections of the app that changed).

Perhaps leave this as a placeholder for now, or comment why this isn't a goal, or what do you think about Rails support down the road.

Hello,

I have MODULE.bazel file that has the following contents

bazel_dep(name = "rules_ruby", version = "0.10.0")

ruby = use_extension("@rules_ruby//ruby:extensions.bzl", "ruby")
ruby.toolchain(
    name = "ruby",
    version = "3.3.1",
    # alternatively, load version from .ruby-version file
    # version_file = "//:.ruby-version",
)

use_repo(
    ruby,
    "ruby",
    "ruby_toolchains",
)

# Register ruby toolchain
register_toolchains("@ruby_toolchains//:all")

And I have a BUILD.bazel that defines a rb_binary(). When I try to build the target, I get the following failure

❯ bazel build -s //my/package/...
Starting local Bazel server and connecting to it...
INFO: Repository rules_ruby~~ruby~ruby instantiated at:
  <builtin>: in <toplevel>
Repository rule rb_download defined at:
  /Volumes/BazelOut/_bazel_bassam/49638b6383ca98bc7c45c8f9c0c8e8c7/external/rules_ruby~/ruby/private/download.bzl:190:30: in <toplevel>
ERROR: An error occurred during the fetch of repository 'rules_ruby~~ruby~ruby':
   Traceback (most recent call last):
        File "/Volumes/BazelOut/_bazel_bassam/49638b6383ca98bc7c45c8f9c0c8e8c7/external/rules_ruby~/ruby/private/download.bzl", line 53, column 32, in _rb_download_impl
                _install_via_ruby_build(repository_ctx, version)
        File "/Volumes/BazelOut/_bazel_bassam/49638b6383ca98bc7c45c8f9c0c8e8c7/external/rules_ruby~/ruby/private/download.bzl", line 182, column 13, in _install_via_ruby_build
                fail("%s\n%s" % (result.stdout, result.stderr))
Error in fail: 
ruby-build: definition not found: 3.3.1
ERROR: <builtin>: fetching rb_download rule //:rules_ruby~~ruby~ruby: Traceback (most recent call last):
        File "/Volumes/BazelOut/_bazel_bassam/49638b6383ca98bc7c45c8f9c0c8e8c7/external/rules_ruby~/ruby/private/download.bzl", line 53, column 32, in _rb_download_impl
                _install_via_ruby_build(repository_ctx, version)
        File "/Volumes/BazelOut/_bazel_bassam/49638b6383ca98bc7c45c8f9c0c8e8c7/external/rules_ruby~/ruby/private/download.bzl", line 182, column 13, in _install_via_ruby_build
                fail("%s\n%s" % (result.stdout, result.stderr))
Error in fail: 
ruby-build: definition not found: 3.3.1
ERROR: no such package '@@rules_ruby~~ruby~ruby//': 
ruby-build: definition not found: 3.3.1
ERROR: /Users/bkhouri/Documents/git/************/BUILD.bazel:23:10: ************************r depends on @@rules_ruby~~ruby~ruby//:toolchain in repository @@rules_ruby~~ruby~ruby which failed to fetch. no such package '@@rules_ruby~~ruby~ruby//': 
ruby-build: definition not found: 3.3.1
Target *********************r up-to-date (nothing to build)
ERROR: Analysis of target '***********************' failed; build aborted: Analysis failed
INFO: Elapsed time: 5.214s, Critical Path: 0.02s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

I looked at the rules source code, and it seems like it should work. Using version = "jruby-9.4.7.0", in the ruby.toolchain definitions does not generate an error.

Problem description

I set the following in my MODULE.bazel file

module(name = "myrepo", version = "0.0.0", repo_name = "myrepo")

bazel_dep(name = "rules_ruby", version = "0.10.0")

ruby = use_extension("@rules_ruby//ruby:extensions.bzl", "ruby")
ruby.toolchain(
    name = "ruby",
    version = "system",
)
use_repo(ruby, "ruby")

This yields 1002 targets!, and @ruby//:sudo is among them.

❯ bazel query @ruby//... | wc -l
Loading: 0 packages loaded
    1002

❯ bazel query @ruby//...  
@ruby//:AssetCacheLocatorUtil
<...SNIP...>
@ruby//:rmic
@ruby//:rmid
@ruby//:rmiregistry
@ruby//:rpcgen
@ruby//:rs
@ruby//:rsync
@ruby//:ruby
@ruby//:ruby_file
@ruby//:rview
@ruby//:rvim
<...SNIP...>
@ruby//:scp
@ruby//:screen
@ruby//:script
@ruby//:sdef
@ruby//:sdiff
@ruby//:sdp
@ruby//:sdx
@ruby//:security
@ruby//:sed
@ruby//:seeksize.d
<...SNIP...>
@ruby//:strip
@ruby//:su
@ruby//:sudo
@ruby//:sum
<...SNIP...>

However, we are unable to run the ruby target

❯ bazel run @ruby//:ruby -- --help
Starting local Bazel server and connecting to it...
INFO: Analyzed target @@rules_ruby~~ruby~ruby//:ruby (68 packages loaded, 1479 targets configured).
ERROR: /Volumes/BazelOut/_bazel_bassam/49638b6383ca98bc7c45c8f9c0c8e8c7/external/rules_ruby~~ruby~ruby/BUILD:14:10: @@rules_ruby~~ruby~ruby//:ruby: error reading file '@@rules_ruby~~ruby~ruby//:dist/bin/sudo': /Volumes/BazelOut/_bazel_bassam/49638b6383ca98bc7c45c8f9c0c8e8c7/external/rules_ruby~~ruby~ruby/dist/bin/sudo (Permission denied)
ERROR: /Volumes/BazelOut/_bazel_bassam/49638b6383ca98bc7c45c8f9c0c8e8c7/external/rules_ruby~~ruby~ruby/BUILD:14:10: 1 input file(s) are in error
Target @@rules_ruby~~ruby~ruby//:ruby failed to build
INFO: Elapsed time: 2.741s, Critical Path: 0.10s
INFO: 4 processes: 4 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target

Looking at the source code, it appears the rules are symlinking the parent directory returned by which ruby

https://github.com/bazel-contrib/rules_ruby/blob/main/ruby/private/download.bzl#L232-L236

Instead of symlinking the directory where ruby is located, can the repository rule symlink only the required binaries, or maybe exclude sudo and other critical utilities from being created?

Host Information

• Mac OS Sonoma 14.5

• Chip: Apple M1 Pro

• Ruby Locationa:

    ❯ which ruby
    /usr/bin/ruby
  
    ❯ which -a ruby
    /usr/bin/ruby
  

Have you seen something like this?

This error feels to me like the repository rule is trying to write to some global location on my disk, and maybe we need to set an environment variable like XRUBY_RUBYLIBDIR somewhere so the MRI build process is okay with running in some read-only sandbox?

I'm on Linux, tried ruby 3.0.6 and 3.2.2, get an error like the following

x86_64-linux-fake.rb updated
external command failed with status 2

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 19.5M  100 19.5M    0     0  7329k      0  0:00:02  0:00:02 --:--:-- 7329k
./tool/mkconfig.rb:272:in `write': Permission denied @ io_write - <STDOUT> (Errno::EACCES)
        from ./tool/mkconfig.rb:272:in `print'
        from ./tool/mkconfig.rb:272:in `<main>'
make: *** [uncommon.mk:879: .rbconfig.time] Error 1
make: *** Waiting for unfinished jobs....

BUILD FAILED (Ubuntu 20.04 on x86_64 using ruby-build 20231114)

I do have a system ruby already

$ ruby -v
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]

Note, this is "nice to have", maybe not a requirement for a 1.0 of rules_ruby.

Use case: I'm in a monorepo where one app has 1000 giant dependencies. I write a little ruby tool for all developers to run, and it only needs one small gem, call it "mygem".

I don't want users of my tool to have to wait to download all the giant dependencies. So the bundler install would have to be decomposed such that each gem goes in its own external/ repository, and then in my BUILD files instead of depending on "all gems" like @bundler//:all, I would depend on @bundler_mygem. That way the other gems aren't referenced in the build graph and so they aren't fetched.