Reactor builds? about capsule-maven-plugin HOT 28 CLOSED

Not sure exactly what you want?

With the thin jar build you would end up with just the 'app' module embedded in the capsule. So any dependencies such as 'app-models' and 'app-other' (and their dependencies would be downloaded at runtime).

If you want somewhere in between a thin jar and a fat jar where you have 'app' as well as 'app-models' and 'app-other' BUT not other dependencies (so basically you only want to package some of the dependencies in the capsule build), then this can be done also. You need to build a fat jar and for any dependencies you do not want included, just set the scope of that dependency to providedlike so:

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
  <version>17.0</version>
  <scope>provided</scope>
</dependency>

(this is because the capsule-maven-plugin will only add dependencies to the fat jar that are scoped in compile or runtime).

from capsule-maven-plugin.

Please note that there is currently an issue when building empty or thin capsules.

EDIT - FIXED

from capsule-maven-plugin.

So any dependencies such as 'app-models' and 'app-other' (and their dependencies would be downloaded at runtime).

That is what I am trying to avoid - downloading app-models and app-other. Since they are modules within the reactor build their sources (and generated classes) are co-located with the app module. I'm looking to build something in-between, as you said. I want a fat jar of my modules but a thin jar for all other dependencies.

If I set all my compile dependencies to provided will Capsule download them? I would not have expected so.

from capsule-maven-plugin.

Yes, its as I thought. Build a fat jar with the plugin (see <types>fat</types>) and then for any dependencies that you do not want embedded set the scope to anything other than compile or runtime, such as provided. Then the capsule will only have app-models and app-other embedded, and thus will not try and download them at runtime. The other dependencies, since not embedded in the capsule, will indeed be downloaded at runtime. (Yes Capsule will download these other dependencies because the plugin is clever enough to include them in the Dependencies attribute in the manifest, which Capsule reads at runtime).

I added info on this here.

Note that there was an issue with the above described in 0.10.7, so please checkout the 0.10.8-SNAPSHOT from github and test against that.

Let me know how it goes :)

from capsule-maven-plugin.

0.10.8 released.

from capsule-maven-plugin.

Tested. It works as you suggested, but not as I had hoped. The provided deps are excluded from the fat jar but the compile modules include their transitive dependencies making the fat jar still too fat and the thin jar still useless - for a reactor build.

It makes sense that it works this way, but I was looking for some way to treat reactor module dependencies differently than just being normal artifact dependencies. It seems possible since the parent pom knows about the module list, but I don't know the inner workings of Maven. Maybe it's alot of effort.

from capsule-maven-plugin.

So, basically you want to embed app-models and app-other without their dependencies?

This can be done, I'm thinking a flag such as <deep>false</deep> to turn off embedding the dependencies of dependencies...

from capsule-maven-plugin.

Yes, I want to embed app-models and app-other but I still want their dependencies downloaded for a thin capsule.

from capsule-maven-plugin.

Ok that can be done. Away at the moment but will aim to get this done on oct5.

from capsule-maven-plugin.

This is fixed in upcoming v1.0

from capsule-maven-plugin.

See here.

from capsule-maven-plugin.

Fixed in 1.0.0.

from capsule-maven-plugin.

I finally got around to giving your latest version a try. It does exclude transitives. But it still does not achieve what I was hoping for.

My goal is to embed reactor build module artifacts into a thin jar while excluding all non-module dependencies.

pom.xml
├── app (pom.xml, Capsule)
│    └── jetty
├── app-models (pom.xml)
├── app-utils (pom.xml)
│    ├── guava
│    └── commons.io
└── app-webui (pom.xml)

In the above example, I'd like to embed app, app-models, app-utils, and app-webui but none of their dependencies. On startup I'd like Capsule to download all missing dependencies of the embedded modules (i.e. jetty, guava, commons.io). You could think of this as building an uber jar from all the modules and generating a thin Capsule from the uber jar while specifying the aggregate dependency graph for Capsule to download.

The motivation would be, perhaps the modules in this project are not served by a public Maven repo. It's conceivable that the fat jar for the above example could easily be 40-100MB while the uber jar could be ridiculously small (10s of KB). So the Capsule is orders of magnitude larger than it could be if the Capsule just contained my code.

Maybe this can't be achieved. I really appreciate your efforts here so thanks for everything you've done so far.

from capsule-maven-plugin.

Possible solutions I see:

• use <fileSets> to manually include compiled classes from sibling modules. ick.
• plugin setting to specify modules to embed in a thin jar

  
  
  <modules>
  
  <module>app</module>
  
  <module>app-models</module>
  
  <module>app-webui</module>
  
  <module>app-utils</module>
  
  </modules>

• plugin setting to specify dependencies to embed in a thin jar

  
  
  <dependencies>
  
  <dependency>
  
  <groupId>group</groupId>
  
  <artifactId>app</artifactId>
  
  </dependency>
  
  </dependencies>

from capsule-maven-plugin.

Hmm, can I get an example project to play with please?

from capsule-maven-plugin.

You bet. I'll create an example and get back to you.

from capsule-maven-plugin.

Being able to embed reactor modules into a thin jar while leaving out the third-party dependencies for runtime download is something I'm also interested in; it simplifies internal deployment scenarios. I would like to throw one wrench into the discussion, though, by separating the resolution of a dependency and the download of the artifact.

The reason is that the resolution of all external transitive dependencies is something that really belongs to the build; the decisions you have to make about dependency exclusions, ordering of dependencies and what artifacts are available at the remote repositories all change the runtime behavior. To get a repeatable build, the build artifact should really have a flat list of artifacts, already resolved from the transitive dependencies. The capsule can then download these artifacts using its URL. For increased reliability, the capsule build stage may give the option to upload the artifacts to a stable location (e.g. S3); this prevents startup failures at runtime due to misbehaving remote repositories.

From what I see, Maven capsule resolves dependencies at runtime. Is there a way to make it work in the way I describe above?

from capsule-maven-plugin.

@chrischristo I've setup an example reactor project. The initial download may be a bit heavy, but it's a real-world baseline.

https://github.com/gitblit/reactor-example

The module reactor-app is the actual application and it's pom defines the capsule packaging. Right now it is setup for thin which we know fails. fat works fine but it includes everything. The goal is to generate something close to thin that includes the Maven output for reactor-models and reactor-utils. And obviously the capsule should include the list of aggregate dependencies so that everything is properly downloaded on initial startup.

from capsule-maven-plugin.

Ah yeah, there was a bug with fat capsule which included everything when transitive was set to false. I'm fixing... (my solution before was supposed to do as you desire).

from capsule-maven-plugin.

So essentially, what we want here is the following.

Just to clarify here, a thin capsule in Capsule speak is a jar with only the app itself embedded (with zero dependencies), so in this scenario the reactor-app project only (and not any of its dependencies and other modules such as reactor-models). A fat capsule is one with the app itself embedded as well as at least one dependency embedded.

So we want a fat capsule that includes the reactor-models and reactor-utils dependencies (without their transitive dependencies) but not any other dependency.

To do this with the plugin, we first set <scope>provided</scope> to all dependencies other than reactor-models and reactor-utils. Then we also set the <transitive>false</transitive> flag so that the transitive dependencies of reactor-models and reactor-utils are not included.

We can also set the <types>fat</types> so that the plugin only produces the fat capsule.

This will then create a fat jar with the following structure:

capsule/
Capsule.class
MavenCapsule.class
reactor-app-1.0.0-SNAPSHOT.jar
reactor-models-1.0.0-SNAPSHOT.jar
reactor-utils-1.0.0-SNAPSHOT.jar
META-INF/

So we only have our reactor-app, and its two dependencies reactor-models and reactor-utils. All other dependencies (and transitive dependencies) will be downloaded at runtime by Capsule, as they are listed in the META-INF/MANIFEST.MF file, thanks to the plugin.

Please await upcoming version 1.0.1 of the plugin that reflects all of the above.

from capsule-maven-plugin.

@chrischristo 👍 That's awesome! I also build more traditional packages with Stork so I'll have to play with the <provided> scope and see what that does to packages built using that tool.

from capsule-maven-plugin.

Cool @gitblit! Also, v1.0.1 has been released and is available on maven central. So give it a shot if you haven't already.

from capsule-maven-plugin.

@tinkerware Yes capsule (using capsule-maven) will resolve any missing dependencies at runtime and download them. Of course, Capsule does cache its resolutions in a local directory (so for example if the previous version of the app (or a different app) was run on the same machine that required the same dependency, its likely it will already be cached).

Hmm as far as I see, the only way to get a guaranteed repeatable runtime behaviour is by using a complete fat capsule (i.e has all the dependencies pre-packaged). Any other mechanism whether relying on repos (both local and remote) or even fixed URLs (that you suggested) will pose potential differences. The repo/url may be dead, or could change and provide inconsistent artifacts overtime. However I do understand that a fixed URL could perhaps be more reliable (but that could vary).

But you could also argue that a use case for letting Caspule download the dependencies at runtime is the fact that if there is an issue with resolution of dependency (at the repo/fixed URL), then the repo/URL can be checked and fixed, without needing to rebuild the capsule. Think of when you have the app deployed at various locations - you just need to re-run them.

Perhaps pose the feature request in Capsule, regarding downloading dependencies via URLs instead of repos. If its accepted there, then we can have the plugin provide Capsule URLs to dependencies (in the manifest), so that Capsule then uses those and avoids hitting repos.

from capsule-maven-plugin.

@chrischristo 1.0.1 handles dependencies exactly how I want. Perfect! Now if I could only get upstream Capsule to support relocating the .capsule directory without me writing custom code. see puniverse/capsule#59

from capsule-maven-plugin.

Btw, it turns out this issue was basically a duplicate of #20 but the OP never followed-up with details. So now you've officially closed them both. 👍

from capsule-maven-plugin.

Ah great on the duplicate issue and glad its solved your issue. I'll have a think about puniverse/capsule#59.

from capsule-maven-plugin.

@gitblit is the ides that Capsule should resolve the transitive dependencies at runtime? Or somehow you will provide these dependencies?

from capsule-maven-plugin.

from capsule-maven-plugin.