Does not propose updates for dependent gems? about dependabot-core HOT 15 CLOSED

Hey @rdunlop,

I've been wondering when this was going to come up! At the moment, Dependabot doesn't bump sub-dependencies, except when bumping the parent dependency (which means #21 is the other side of this issue).

We decided to start out with the above approach because we were worried about the sheer number of PRs we'd be creating when bumping sub-dependencies. By their nature, they're things your less interested in than your primary dependencies, so they were a good candidate to ignore.

However, I can see arguments for updating them, too - the nokogiri example is a good one - 1.8.1 is a security patch, so you definitely want it.

I think the best compromise would be for Dependabot to continue to create single-dependency PRs for primary dependencies, but create an additional "update sub-dependencies" PR each day when required, which would combine many updates. Would that make sense to you? It's quite a bit of work on our side, so won't happen immediately, but I'm keen to get clarity on the best possible solution.

from dependabot-core.

From @rdunlop on September 22, 2017 12:7

Hi @greysteil,

I do like the idea of having ALL of my gems updated to the latest version, all the time. If I installed a gem fresh, all of its sub-dependencies would be "latest"...which is my goal for my projects...so I think that this is a good feature.

I currently really like the fact that each PR only contains a single gem in it. If you're proposing that the "sub-dependencies" PR would combine many unrelated gem version changes, doesn't feel right to me.

I am warming to your idea of batching them together though...because it sounds right that we care about certain gems more than we do others..... Perhaps we could do sub-dependencies after all of the other ones are done? (ie: if I have less than 5 PRs tonight...look for gems in the sub-dependencies group?).

Alternatively, perhaps it will require configuration/options to be allowed on a per-project basis. e.g. "check for updated dependencies of gems" option? This configuration page might become very useful, if you can use it to show those gem PRs which are currently open, closed/not-merged, etc.

I know I'm probably mixing many features together.

from dependabot-core.

Makes sense. I want to keep thinking on this one, so I'm not going to ship anything immediately, but it's really high up my list of problems to solve.

FWIW, we've tried to avoid having a configuration page because we quite like the interface for Dependabot to be entirely pull request based. That won't necessarily last forever, but I don't want to add a configuration page / file before thinking through all the other options.

from dependabot-core.

From @rdunlop on September 25, 2017 10:49

@greysteil Please take your time to think through this one. We have other means to track sub-dependencies right now, so this is not pressing...just would be really nice to have.

I can understand your desire to avoid a configuration page. I'll keep thinking about it too.

from dependabot-core.

From @rdunlop on September 28, 2017 13:10

Hi @greysteil I may be seeing a related (or unrelated) issue. I would like your opinion on whether it's the same issue, or a new bug.

Basically, it was my understanding that dependabot was doing the equivalent to bundle update <gem> for each of the gems specified in my Gemfile...and if it succeeded in resolving a gem update, it would open a PR. Is that a fairly accurate summary of the intended dependabot behavior?

I was expecting a PR last night, but I didn't get one:

On rdunlop/unicycling-registration my Gemfile specifies "gem 'carrierwave-aws'".

Currently, my Gemfile.lock says

    carrierwave-aws (1.2.0)
      aws-sdk (~> 2.1)
      carrierwave (>= 0.7, < 2.0)

Yet, last night I did not get a PR for upgrading that from 1.2.0 to 1.3.0?

Doing "bundle update carrierwave-aws" on my command-line results in

git diff
diff --git a/Gemfile.lock b/Gemfile.lock
index 93b0cfa8..eb938c74 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -105,16 +105,16 @@ GEM
     ast (2.3.0)
     autoprefixer-rails (7.1.4.1)
       execjs
-    aws-sdk (2.10.47)
-      aws-sdk-resources (= 2.10.47)
-    aws-sdk-core (2.10.47)
+    aws-sdk (2.10.54)
+      aws-sdk-resources (= 2.10.54)
+    aws-sdk-core (2.10.54)
       aws-sigv4 (~> 1.0)
       jmespath (~> 1.0)
     aws-sdk-rails (1.0.1)
       aws-sdk-resources (~> 2)
       railties (>= 3)
-    aws-sdk-resources (2.10.47)
-      aws-sdk-core (= 2.10.47)
+    aws-sdk-resources (2.10.54)
+      aws-sdk-core (= 2.10.54)
     aws-sigv4 (1.0.2)
     axiom-types (0.1.1)
       descendants_tracker (~> 0.0.4)

My settings:

bundle -v
  Bundler version 1.14.6
gem -v
  2.6.13

Thoughts? thanks.

from dependabot-core.

@rdunlop that's definitely related. You're right that Dependabot does the equivalent of bundle update <gem>, but it also has some logic to check whether the version of the dependency it's updating changes. If it doesn't (e.g., in this case), it doesn't generate a pull request, so you'll never get a "Bump sub-dependency of carrierwave-aws" pull request.

Having a general strategy for dealing with sub-dependencies would definitely fix the above - if they were getting updated anyway a PR like the above wouldn't ever be needed.

Unrelatedly, the reason you didn't get a carrierwave-aws PR is because the latest version requires aws-sdk-s3 ~> 1.0, which in turn requires aws-sdk-core ~> 3 (yes, the AWS team have put a ~> specifier on a major version. 🤷‍♂️), which conflicts with the version required by the version of aws-sdk-rails which is in your Gemfile.lock (and is in your Gemfile, so Bundler doesn't automatically bump it). The solution would be for Dependabot to be able to bump multiple dependencies at a time when required, which I'd really like to add but haven't had time to (yet).

To get to carrierwave-aws update locally, try bundle update carrierwave-aws aws-sdk-rails.

from dependabot-core.

From @rdunlop on September 28, 2017 14:1

@greysteil I knew you'd have an explanation for it.

I'm looking at the Gemfile.lock changes that I posted above....and I'm now seeing that it doesn't say what I thought it said. Oops.

You're right, this is totally explainable by #21.

I've seen people put gem "rails", "~> 4" recently.... 🤷‍♂️

from dependabot-core.

From @jalada on January 15, 2018 9:46

Ruby 2.5.0 introduced some bugs in our code that required sub-dependent gems to be updated (specifically around ruby_parser) but because haml-rails is the dependency, dependabot didn't notify us of them.

Just an extra datapoint for this feature; I think it would definitely be valuable.

from dependabot-core.

From @regentgal on January 31, 2018 18:16

I would expect sub-dependent gems to be updated with the parent. The strict 1 gem per PR rule is nice for isolating changes but updating a gem without updating its explicitly stated dependencies (and theirs), you end up with the issues listed here by @rdunlop and @jalada. I think moving to a 1 parent gem and its sub-dependencies per PR rule would be a nice way to limit the number of PRs and still keep changes isolated.

from dependabot-core.

Hey @regentgal, Dependabot does update sub-dependencies when it's updating a top-level dependency 🎉 . The issues above are because Dependabot never creates PRs for sub-dependencies on their own - in @jalada's case, for example, Dependabot would have included a ruby_parser update if it was updating haml-rails, it's just that haml-rails didn't need an update.

Hope that helps clarify!

from dependabot-core.

From @rdunlop on January 31, 2018 19:45

@greysteil If haml-rails doesn't need an update, but its dependent gem ruby_parser does need an update, I think that it would be great if we had a PR opened which was something like:

`haml-rails`'s dependent gems update:
- ruby_parser - from version 1 to version 2
- other sub-gem - from version 2.3 to 2.4

Which is the result when you run bundle update haml-rails. Even though haml-rails itself doesn't end up being changed, some other gems in my Gemfile.lock are updated.

I like this idea because it also helps to identify which parent-gem is using these dependencies, so that I can focus my manual testing/review before releasing changes to prod, if I deem them risky/worrisome.

This is not "creates PRs for sub-dependencies on their own", in that it's framing the new PR in terms of the parent/Gemfile-referenced gem.

My 2c.

from dependabot-core.

Interesting, thanks @rdunlop. Might see some movement on this one in the next couple of weeks as I’m hoping for some progress on #53 (meeting GutHub about it next week)

from dependabot-core.

From @rdunlop on March 27, 2018 14:9

@greysteil #53 appears to be focusing on updating sub-gems which have security alerts from github.

I agree that that's valuable work, but I still would like to have other sub-gems updated through automatic means if possible.

If we were to concentrate on security warnings as the goal for doing gem updates, here's some thoughts:

• Keeping gems updated, means that when a security-warning comes up, and we need to update the gem, we aren't many versions behind the tip, so we are less likely to have issues
• There aren't always "security warnings" posted for gems which actually have security issues discovered/resolved. It highly depends on the specific gem community/reach/perspective

In general, I just like keeping all my libraries up to date, and dependabot is doing a fantastic job doing that for most of them...but when I have to manually go through and create nokogiri, loofah, rack-protection, rails-html-sanitizer PRs...I just wish it was doing those for me too.

(I'd be happy with #53, I'd be thrilled with #22)

from dependabot-core.

👍. At this point it would be really straightforward for me to add a setting for this, so I'll do that this week.

from dependabot-core.

@rdunlop - I've just shipped the backend for this. If you ping me an email on [email protected] with the repos you'd like it set up on I'll do that now (same goes for anyone else who would like to receive individual PRs for sub-dependencies.

Assuming all goes well I'll add a front-end for this available to everyone in the next couple of weeks.

from dependabot-core.