[Epic] Agent dynamic attach about cryostat-agent HOT 3 OPEN

As seen done by another project recently, the Agent JAR could be built into a simple JAR-carrier container image, ex:

FROM scratch
COPY ./target/cryostat-agent.jar /cryostat/

This would be an easy way for application images to consume the Agent JAR - rather than manually downloading the JAR to the build machine and copying it into the container build, they could use a multi-stage container build like:

FROM cryostat/agent-carrier:latest as cryostat-agent
RUN echo '' # no-op, can this be removed?

FROM base-image
COPY --from=cryostat-agent /cryostat/cryostat-agent.jar /app/deployment/lib
... copy application artifacts in

Alternatively, this JAR-carrier image can be used as an init container in the application's Deployment to achieve the same result dynamically, if there is a volume shared between the init container and the application container:

        initContainers:
        - name: download-agent
          image: agent-carrier:latest
          imagePullPolicy: Always
          command: [ 'cp', '/cryostat/cryostat-agent.jar','/etc/agent/ ]
          volumeMounts: # this must also be defined on the container that runs the host JVM which the Agent should attach to, so that the JAR is available on its filesystem as well
            - name: shared
              mountPath: /etc/agent/

from cryostat-agent.

https://github.com/open-telemetry/opentelemetry-operator/blob/16e11344ae27db6f2bfb0a8953d908827c7b9019/pkg/instrumentation/javaagent.go#L28

This is a good reference for the same/similar technique, which also includes the technique of modifying the target container's JAVA_TOOL_OPTIONS environment variable to include -javaagent to launch the agent via static attach. This would necessitate a target container restart for agent instrumentation to begin.

Discussion on dynamic attach mechanisms over here: cryostatio/cryostat-operator#784

from cryostat-agent.

Since the init container needs at least cp in order to copy the carried JAR into the shared volume, FROM scratch alone won't be quite enough. It could be a FROM scratch that also has a standalone cp binary copied into it, or else a very minimal base image with some coreutils in it instead:

FROM docker.io/library/busybox:latest
COPY ./target/cryostat-agent.jar /cryostat/

from cryostat-agent.