Is it possible to set an app default for the cache dir ? about capsule HOT 33 CLOSED

How would that option be expressed? The location of the cache directory should be up to the user of the capsule, not its author.

BTW, current behavior is to put the cache in the home dir, or, if that fails, in /tmp; if the latter is chosen, the cache is automatically deleted when the application terminates.

from capsule.

I'm not suggesting removing control from the user. I'm requesting the option of specifying the default location. The location could still be overridden by the user with -Dwhatever or env something just like you currently have it.

I use Capsule through the maven plugin so I guess it could be a config node in the build phase. Maybe something like:

<defaultCacheLocation>${user.dir}/lib</defaultCacheLocation>

from capsule.

OK, but I still don't understand the use case. Why should the capsule care where its cache directory is?

from capsule.

The location is relevant when supporting users who have received Capsule-packaged apps. Not all users are technically savvy. From the support perspective it's easier to delete (or to instruct deletion of) a self-contained folder than to have them navigate and find hidden folders in their local user directory.

Or if for whatever reason the cache needs to be dumped and re-downloaded or re-extracted, it's much easier to explain deleting the lib folder co-located with the Capsule.

from capsule.

I see. Well, I'm a little conflicted with the proposed solution, because capsules should all really behave in the same way, and this behavior might be surprising.

Today, you can add the -Dcapsule.reset flag to delete the cache before launching. Perhaps we can add a -Dcapsule.delete flag to dimply delete the (app's) cache dir. Would that be an acceptable solution?

from capsule.

Would that be an acceptable solution?

Not for my needs, but I can respect your position.

Surprising behavior depends on your perspective. Capsule has some really compelling deployment features. My opinion is the target audience of Capsule is the developer, not the end-user. If everything comes together cleanly, the end-user has no idea that Capsule is the deployment container; it's a developer's deployment implementation detail and the behaviors of other Capsules is irrelevant to the end-user.

As soon as the end-user is required to have some technical knowledge of Capsule to tweak a deployment then some of the magic is gone and the focus has shifted away from the deployed product.

from capsule.

Just to make it clear: The behavior you desire can already be achieved today with a caplet that overrides buildAppCacheDir to return the location of a directory of your choosing to be used as the cache.

I am just trying to understand your need because it might be something that is worth putting in the default caplet (i.e. Capsule itself). Why are you saying the user will need to tweak deployment? How will the user even be aware of the capsule cache? How will specifying the cache location in the capsule manifest help?

from capsule.

The goal is to have an executable jar that Does The Right Thing (tm) based on simplicity of support. For my team, Does The Right Thing (tm) means a self-contained deployment directory which can be deleted to remove all traces of the app or a cache subdirectory that can be deleted to reset.

Goal: java -jar myapp.jar
Not: java -Dcapsule.dir=%CD% -jar myapp.jar

Specifying the cache dir manually with ENV (Unixy solution) or a command-line arg trickles down to all launch entry points for the capsule: command-line, double-click, Windows service definition, Unix service script.

I was not aware that I could achieve my goal with my own caplet. I'll take a look at that, if needs be.

from capsule.

How about a manifest attribute that specifies that the cache be deleted after each run?

Short of that, I really think the solution should remain a caplet.

An attribute specifying the cache location no longer describes the capsule itself but how it should be run. The problem is that there is no way of knowing how much the application will depend on this behavior, and so it's hard to decide what to do if a cache is not used at all or takes a different form. We have (or will have) caplets that create don't use a virtual file system into the JAR instead of a cache directory (and caches might be made altogether unnecessary by Java 9), caplets that run the application in a Linux container etc. If there were an attribute specifying the cache location, it will be very hard to generalize.

from capsule.

How about a manifest attribute that specifies that the cache be deleted after each run?

I can see that as being useful, though I doubt I would use it for my immediate need.

An attribute specifying the cache location no longer describes the capsule itself but how it should be run

You're on a slippery slope; you already support attributes that define how a capsule should run/behave (e.g. repositories, dependencies, extract jars, etc). Isn't that the point of a capsule?

If there were an attribute specifying the cache location, it will be very hard to generalize.

So by that logic should -Dcapsule.dir=x be considered deprecated?

I find a caplet as an acceptable alternative, but I was hoping to just configure something instead of coding something.

from capsule.

So by that logic should -Dcapsule.dir=x be considered deprecated?

No, as the application is not aware of it and therefore does not depend on it, so Capsule is free to ignore this directive if no cache directory is needed.

You're on a slippery slope; you already support attributes that define how a capsule should run/behave (e.g. repositories, dependencies, extract jars, etc).

You are partly right (at least about repositories), which is why this behavior has changed in 1.0 (to be released next week, probably).

Dependencies are now completely general and don't imply any resolution mechanism. The default caplet will resolve them by looking for matching JARs within the capsule. To get dependencies to resolve against a Maven repository, you'll need to add the Maven caplet (you just list it in the Caplets attribute and import the classes into the JAR). The Maven caplet does indeed allow you to change the default repositories in an attribute (caplets are, of course, are free to read attributes), but even then, the Maven dependency model guarantees that the application does not care where the artifact comes from, as all artifacts with the same coordinates must resolve to the same binary. Capsule is therefore free to disregard that directive, too.

As to Extract, you are absolutely right. It is going away. The correct behavior should be that the capsule only touches the local filesystem if it has to.

Isn't that the point of a capsule?

The point is that a capsule is the complete description of everything the application requires in order to run. How it runs (simply launched, in a container, or turned into a native executable; pulls configuration from etcd or in a Java sandbox) is entirely up to caplets. The capsule's cache is an (unfortunate) implementation detail, necessitated only by the JREs current inability to embed JARs and native libraries within JARs. It can go away any second. It therefore does not make sense to me that the application will require all launching configurations to create a cache directory in a certain location.

I find a caplet as an acceptable alternative, but I was hoping to just configure something instead of coding something.

Good. You can have your caplet read the location of the directory from an attribute, if you like. But if so, make sure to use the dev version (1.0), as this API (for accessing attributes) has changed.

from capsule.

Sorry to confuse you: I've looked at the code again, and simply overriding buildAppCacheDir won't cut it, as the original implementation also extracts the capsule in that method. Let me think about it some more.

from capsule.

I would very much appreciate the option to extract or download into the folder where capsule.jar has been started.

From my point of view, when the user is interested in an application, she usually has no idea about capsule being used at all. It's just a vehicle for easily distributable applications - for me as a developer - and for easily executable applications - for the user.
The user downloads the application and copies it to the folder it should be installed to. Then she runs the application. Because everything is extracted where she started the application, this makes portable applications with user-editable configuration files possible. Maybe that's not the goal of capsule, but it would be perfect for my needs.

I didn't use capsule yet, but today I came across it, because I'm in desperate need of a tool that:

• creates one deployable artifact as part of the build-process (gradle or maven)
• is easily executable for the end-user (ideally a native executable, but capsules possibilities also seem to fit very well)
• extracts (or downloads) dependencies and configuration files to a configurable folder (thus behaves like a very limited installer).
• allows portable applications

My needs would be perfectly matched, if the extraction-target-folder (cache-folder) would be the parent of the capsule.jar or if the folder would be configurable by the developer. Ideally, I would just need to configure override-cache-folder='./'.

I would very much appreciate a feature like that, but certainly I do understand your view as well.

from capsule.

I think your request is different from @gitblit's. You are asking for a semi-permanent directory where user-edited configuration can be placed. While you can write a caplet to do that, that is entirely the application's concern. The best solution is to use the capsule to configure the application and tell it where to place configuration files.

The fact that capsule extracts anything is an implementation detail which is a (hopefully temporary) workaround for a current Java limitation that will partly or entirely be fixed in Java 9 (namely, the ability to embed JARs and native libraries within a JAR). That temporary cache directory is not intended for configuration files at all, may be deleted at any time -- even after each run or while the application is still running -- or not even exist at all (the current version of capsule creates it only if it absolutely must). Nothing is allowed to depend on its foggy existence, except when it is briefly exposed during the application launch process.

If you want to control its location from within the capsule manifest (i.e. not as a system administrator who wants control over filesystem use, and therefore care about any file -- be it temporary on some code's implementation detail), that means that you are about the directory and possibly want to use it. We can't have that :)

In short, forget about it, because as far as anyone is concerned -- be it the capsule author or user -- it doesn't exist. Hopefully, Java 9 would make it truly gone.

OTOH, @gitblit was concerned (I believe) that the efforts at complete transparency would fail, and therefore needed a "manual override" that would easily let the user fix a mistake. I, on the other hand, fear that making this ghost directory visible would make capsule authors depend on it. Even in the worst case of a forgotten cache, it is no worse than thousands of applications that leave their temporary data lying around.

Nevertheless, you are encouraged to write a caplet that creates a configuration directory where you think is appropriate and configures the application to use it. That would be a directory of a completely different nature from the capsule cache. The mere fact you are even asking to use the cache for what it's not intended to do, shows me I was right in resisting this feature request.

from capsule.

If Capsule read the capsule.dir from the System-Properties manifest entry (instead of only from -D args) then this wouldn't be a problem, as using the build tools such as the maven plugin, you could list the capsule.dir=x as a system property. See #99.

from capsule.

That's probably not going to happen unless someone comes up with a really good argument in favor.

from capsule.

Well look at it from this perspective.

I totally agree with this, as you said @pron - "The app says in the manifest: this is what I need to run."

Does Capsule need a default log level to run? yes.
Does Capsule need a default cache directory to run? yes.

Does Capsule make an opinionated assumption on what these defaults are? Yes.

...Should the developer have the right to his own opinion on what these defaults are? YES.

from capsule.

The problem is, I don't see why the application would need a default for the cache location. The application must not depend on the location of the cache directory; in fact, it must not count on its existence at all (even if you specify a location on the command line, Capsule may choose not to create a cache directory).

So far, the requests for this feature only mentioned the current capsule JAR directory as a possible alternative location. I'm willing to consider such a boolean attribute (Cache-In-Jar-Dir). Is that what you're suggesting, too? Can you see reasons for other locations?

from capsule.

Each of us will see different value for Capsule.

At present my interest in Capsule is primarily due to the availability of runtime dependency resolution. Java 9's projected capability to embed jars is of less interest to me as I am more interested in small, tight deliverables with automated dependency download. I'm not particularly interested in mega jar distribution; I already have deployment solutions for that. This may be short-sighted on my part as there are many other compelling features to Capsule - but that's my long-standing initial attraction.

From that perspective I would say that Capsule is a tool for deployment of my app and as such I should be able to control the cache location for my app. If I approached it as Capsule is a deployment system and my app can be distributed as a Capsule, then having the non-app-specific, "system-level" Capsule config makes more sense.

Personally, I do not see any technical reason to prohibit the application from specifying a default location for it's Capsule cache. The developer should be empowered.

from capsule.

The developer should be empowered.

1. The developer is empowered. You can place the cache directory wherever you like with a caplet, just not with a manifest attribute, as we don't want to make it too easy.
2. We don't want to make it easy because the cache directory may disappear completely in future versions (in fact, we want it to disappear if possible), and I don't want applications to depend on its existence. If you insist, well, that's what caplets are for. Caplets let you do pretty much anything you want. Standard manifest attributes, OTOH, should not encourage "bad" practices.

from capsule.

Your last comment about the caplet approach was this:

Sorry to confuse you: I've looked at the code again, and simply overriding buildAppCacheDir won't cut it, as the original implementation also extracts the capsule in that method. Let me think about it some more.

Have things changed? Can I write and embed a caplet to control the cache location? It's not my first-choice approach, but it would suffice for now.

from capsule.

Can I write and embed a caplet to control the cache location?

Yes, although things have not changed -- I simply didn't think them through before. You can write a trivial caplet that simply sets the CAPSULE_CACHE_DIR environment variable in its static initialization block.

from capsule.

Let's drill into this a little more:

(in fact, we want it to disappear if possible),

Let's say it disappears in a future version. What would that mean for the maven-caplet and my use-case of runtime dependency downloads? Would it instead download directly to ~/.m2/ ? Would that become a maven-caplet detail? Or would runtime downloads no longer be a supported capability?

from capsule.

Oh, we're certainly not going to take away capabilities. What I meant by that was that we'll try to provide as many features as possible without polluting the file system. For example, if there was a way to put dependency JARs in memory and load them from there, we'd want that (although probably not for remotely-downloaded dependencies, as we also want to cache those).

from capsule.

You can write a trivial caplet that simply sets the CAPSULE_CACHE_DIR environment variable in its static initialization block.

That sounded ok until I tried to do it. I don't think you can change an environment variable from within Java.

This caplet relocates the cache to the working directory using reflection, but since everything is so locked down in Capsule it's difficult to achieve a clean implementation or a result better than literally relocating the cache root. e.g. it might be nice to flatten the dir structure in this one-app cache case

public class LocalCache extends Capsule {

    static {
       setLocalCache();
    }

   public static void setLocalCache() {
        try {
            Path capsuleJarPath = Paths.get(Capsule.class.getProtectionDomain().getCodeSource().getLocation().toURI());
            File localCapsuleDir = new File(capsuleJarPath.toFile().getParentFile(), ".capsule");
            String cacheDir = Optional.ofNullable(System.getProperty("cache.dir")).orElse(localCapsuleDir.getAbsolutePath());

            Path cachePath = new File(cacheDir).toPath();
            Field field = Capsule.class.getDeclaredField("CACHE_DIR");
            field.setAccessible(true);
            field.set(null, cachePath);
            Path readPath = (Path) field.get(null);
            if (readPath.equals(cachePath)) {
                log(LOG_VERBOSE, LocalCapsule.class.getName() + " set " + field.getName() + "=" + readPath);
            } else {
                log(LOG_VERBOSE, LocalCapsule.class.getName() + " FAILED TO SET " + field.getName() + "=" + readPath);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public LocalCache(Capsule pred) {
        super(pred);
    }
}

from capsule.

I've refined the caplet a few times. What I have will co-locate the cache with the capsule and should allow a system property override that could be specified by the Maven plugin. It still relies on reflection.

from capsule.

Good! You're right about setting environment variables from within Java, of course.

I now remember that there was another reason for making this possible only through reflection (and in general having everything in Capsule "so locked down") is absolutely intentional. Capsule is also a security mechanism, and we have an experimental secure caplet (a wrapper caplet, of course) that launches the app in a sandbox. It allows (or supposed to; it's experimental) writing to the cache directory if it exists, so we had to make sure that we can block the app capsule from setting the location of the cache directory. As we also block setAccessible, making sure this could only be done through reflection.

from capsule.

Ok, next snafu.

I tried creating a re-usable caplet jar file but this is a fail. This won't work because now there is a chicken-and-egg situation: you can't embed a caplet jar that relocates the cache directory because the caplet jar has be extracted before it can be loaded.

So at this point if you want to co-locate the cache directory your only recourse is to add Capsule as a provided dependency to your app, replicate the above snippet of code in the default package of your app, and properly configure your Capsule definition to load the replicated caplet snippet.

I don't suppose you'd be willing to add my proposed LocalCache caplet (or some variation) into Capsule?

from capsule.

Don't embed the caplet as a JAR within the capsule. That's only useful (if at all) for caplets that require more than one class. Yours is just a single class. Just embed your class directly in any capsule you create when you build. Actually, that used to be the only way to embed caplets before we added the ability to embed some caplets as JARs just as another option.

There's no need to replicate anything.

from capsule.

This may be a dumb question, but it is not immediately obvious to me. If I have 5 unrelated projects that I want to distribute as capsules each with independent local caches, how do I have one, re-usable LocalCache caplet implementation that I can reference when I build my 5 independent capsules?

from capsule.

The way caplets are applied is as follows. In your project build, do the following:

1. Place LocalCache.class in the root of your capsule JAR. I.e. explode your caplet JAR into the capsule JAR.
2. List LocalCache in the capsule manifest's Caplets attribute.

The build-tool plugin you use should do this automatically for you. If not, this is quite easy in most build tools. Don't hesitate to ask again if this is not clear.

Recently we also added the ability to add a caplet in an unexploded JAR to a capsule and list the JAR name -- rather than the class name -- instead, but this doesn't work for caplets that mess with the cache.

from capsule.

What is the reason it's not possible to load jars from memory?

from capsule.

The java command only takes a classpath of JARs (or classfiles) that reside on the filesystem (although you can configure your capsule cache to point to a RAM filesystem). It is possible -- through class loader games -- to load embedded JARs directly, but it doesn't always work (e.g. if the application itself also does some class loader games), and it doesn't work at all for agents or boot-classpath JARs.

from capsule.