Exporting images to a specific image name about toast HOT 4 CLOSED

👋 Thank you for this feature request!

I wonder, however, if perhaps Toast doesn't actually behave in the way you'd want for your use case. Toast computes a topological sort of tasks based on their dependencies and then runs them in that order, with each task essentially inheriting the entire filesystem left over from the previous task. This might not be desirable for generating minimal production images, because there is no way to have a task that simply copies a fixed set of files from a previous task (which is the typical use case for multi-stage builds) rather than inheriting the whole filesystem (with all the build tools etc.).

So even if there were a --tag option, I'm not sure how useful it would be!

In the past, I've considered adding functionality to Toast that would help with this. The main idea would be to allow you to specify an image on a per-task basis, rather than executing tasks all in the same conceptual filesystem. Such tasks could still have dependencies on other tasks, but they would have to explicitly copy files from their dependencies rather than just implicitly inheriting the whole filesystem. Unfortunately, this would require some more design work and would be a bigger change than just adding a way to tag the image produced by a task.

As a solution for your use case, it is possible to actually use Docker within a Toast task (more info here). So you could use Toast to define all your tasks, and then in the final build task you could actually just do docker build ... -t ... inside the task. That way you have complete control over what ends up in your production image, and you can still use Toast to produce the artifacts that get baked into it.

Does my understanding of your situation seem right to you?

from toast.

Hi. Thanks for looking into this.

Multi-stage Dockerfiles are mostly used for creating minimal output images, but they can also be used sort of as an alternative to Makefiles. See for example the annotated Dockerfile example in this repository: https://github.com/garethr/multi-stage-build-example.

That method looks quite similar to how Toast works, but means you can reuse the Dockerfile for both ad-hoc jobs and for building the resulting output image. That's the workflow I'm envisioning using, but I would ideally build it up using Toast and a YAML file rather than having one big Dockerfile.

Does the feature request make more sense now?

from toast.

Thanks for the explanation. However, I'm still not sure whether I totally understand the use case (pardon my being dense here). From my perspective, I still think Toast is not a great fit for generating output images (at least not the images that Toast itself generates), because usually you don't want your output images to contain all your build tools and various intermediate artifacts that are typically used in Toast tasks. Toast already supports exporting files from a task, and to me that is the right way to use Toast for producing build artifacts for use in production. Alternatively, when you need the final artifact to be a Docker image, Docker can be used directly within Toast (by exposing the Docker daemon socket via Toast's mount_paths feature) to produce tagged images. But the images that Toast generates are not ideal as output images for a few reasons:

1. In a typical Toast use case, the images generated by Toast contain lots of things that you don't want in your output images (build tools, etc.).
2. Toast doesn't give you complete control over the order in which tasks are run (unless you structure your dependencies in a linear chain rather than a more general DAG). It does compute topological sorts in a deterministic way, but that's probably not something to rely on. This could introduce some unpredictability in the way the output images are produced.
3. Toast doesn't give you complete control over the resulting images. For example, there is no way to set the entrypoint.

So I'm essentially trying to determine if this feature request is specific to your use case and less generally applicable to everyone, or maybe there is a useful pattern here that many people would appreciate. I don't want Toast to turn into a general Docker image builder, because that direction would likely require future features that would make Toast more complicated. I am not opposed to this specific feature necessarily, but each feature comes with a maintenance burden that can complicate further development (though in this case, the maintenance burden would probably be small).

I will leave this issue open for any further discussion, and if any passers-by discover this and also want this feature, please chime in. It's totally possible that this would help a lot of people and I have misjudged the situation.

from toast.

It's wholly possible I'm the one who have an odd idea about how Toast might could fit into a setup I've got in my mind.

I suggest we close this ticket for now and then I'll come back and reopen it when I have some more concrete code where I may or may not need this feature. Thanks again for listening to me and for great explanations!

from toast.