Comments (13)
As we discussed today on planning, the right solution would be if we introduce a dedicated component to handle projects cloning (which eventually would also update DWT regards file system, like .che/editor.yaml).
But as intermediate step, we may use just init container with bash, to prototype the flow faster and check different aspect of this approach on DWO and components side.
No private repos are supported at this point.
from devworkspace-operator.
We may want to use https://github.com/kubernetes/git-sync
from devworkspace-operator.
We should git clone all projects before a DevWorkspace is started.
Depending on how big the repository[ies] are, it may influences workspace start-up time essentially. Some examples I got locally:
On RHPDS OpenShift 4.6 cluster it's better but still seems to move us back from faster workspace start-up goal:
from devworkspace-operator.
It seems to be related to
Start an editor only workspace and asynchronously update the CR and "recreate" the workspace pod
item from #209
from devworkspace-operator.
I'm not sure this is a DevWorkspace Operator issue. Git cloning before the workspace is started can be implemented by whoever is creating a DevWorkspace by either applying an appropriate init container, or by using something like https://github.com/amisevsk/workspace-bootstrap. In general, I don't think it's on the DevWorkspace operator to think about whether the PVC for a workspace contains the appropriate files, as this is potentially more error prone due to factors outside of our control.
from devworkspace-operator.
I'm not sure this is a DevWorkspace Operator issue. Git cloning before the workspace is started can be implemented by whoever is creating a DevWorkspace by either applying an appropriate init container, or by using something like amisevsk/workspace-bootstrap. In general, I don't think it's on the DevWorkspace operator to think about whether the PVC for a workspace contains the appropriate files, as this is potentially more error prone due to factors outside of our control.
That means that the operator would only manage/reconcile the components
of a DevWorkspace
. If projects
, commands
or events
in the CR are updated something else should take care of "reconciliation". That doesn't sound right.
from devworkspace-operator.
That means that the operator would only manage/reconcile the
components
of aDevWorkspace
. Ifprojects
,commands
orevents
in the CR are updated something else should take care of "reconciliation". That doesn't sound right.
What do you mean? The DevWorkspaceOperator will reconcile whenever you put in your DevWorkspace, but reconciling doesn't extend to populating PVCs with data -- that's out of scope. If you apply a new preStart container to your DevWorkspace, the operator will already handle it.
from devworkspace-operator.
Yes sorry, events
are not ignored, but commands
or projects
are right? I am providing more details in #273.
from devworkspace-operator.
Currently, we do process projects and commands, in the sense that we construct the runtime-annotations configmap and mount it into the che REST API server emulator. In general, we've planned to move away from that, and having consumers of commands/projects directly process that info by reading the DevWorkspace (whether from the cluster or directly mounted into the container).
In general, the barrier I draw mentally for whether something should be explicitly handled by the DevWorkspace Operator is basically Kubernetes objects vs. interactive elements. As a result:
- The DevWorkspaceOperator processes commands as necessary, in order to correctly apply initContainers (prestart event + apply commands)
- The DevWorkspaceOperator will automatically make any changes to k8s objects where required by an updated DevWorkspace (you can add containers, endpoints, etc. and the involved k8s objects will be created)
The second point means we do handle updated commands, etc., since we currently propagate those to a configmap, and I expect we will continue to do some work in propagating that information downwards. I also had a PR (#268) open to automatically apply postStart events to a devworkspace, but Serhii and I decided the concept needed more discussion.
For projects, I think our implementation is well-designed enough that it can be enabled or disabled based on how a devfile gets turned into a DevWorkspace. For example, the "plugin" below would be sufficient to automatically clone projects on-demand:
kind: DevWorkspaceTemplate
apiVersion: workspace.devfile.io/v1alpha2
metadata:
name: project-cloner
spec:
components:
- name: project-cloner
container:
image: '<an-appropriate-image>'
mountSources: true
args:
- '/bin/bash'
- '-c'
- 'read devfile projects, clone them if necessary'
commands:
- id: clone-projects
apply:
component: project-cloner
events:
preStart:
- clone-projects
This could be added to any devworkspace by including
spec:
components:
- name: cloner
plugin:
kubernetes:
name: project-cloner
in a devworkspace.
The DWO could apply such a component automatically, but that blurs the line between what you write in a devfile and what you get ultimately. We would have to, in the operator, address the edge cases of devfiles without projects, or devfiles that don't want PVCs mounted at all (e.g. web terminal)
from devworkspace-operator.
After disucssing this with Mario briefly, we've come up with two potential approaches to managing projects in a devworkspace. I think both have pros and cons, to the extent that I'm pretty 50-50 between the two.
There are of course other options possible, these are just the two that came to me readily
Option 1: Projects are cloned via a plugin
With this approach, we define a plugin DWT (similar to the one above) that can be added to a DevWorkspace by either the Dashboard, Theia, etc. This plugin would define an init container that reads the DevWorkspace and ensures all projects in the Devfile are cloned.
Pros
- Simple, standalone definition; could be added/disabled directly and potentially handle edge cases more easily (dirty branches, etc.)
- More statically defined; could be updated/modified by simply changing the plugin's definition
Cons
- Harder to manage certain cases; if the projects in a devworkspace CR are updated, the change won't be reflected until the DW restarts. This means e.g. Theia still has to clone repos sometimes
Option 2: DevWorkspace Operator adds init container that syncs projects
Here, the DevWorkspace Operator would automatically add an init container to the workspace deployment that clones the projects from the DevWorkspace. Functionally similar to Option 1 (an init container that clones projects), but the definition lives in the operator -- effectively, this makes devfile projects
resolve to initContainers in the pod spec. A devfile attribute could disable automatic cloning for a DevWorkspace if necessary.
Pros
- Moves more devfile support into the DevWorkspace Operator, rather than external components. With Option 1, to fully "resolve" a Devfile you have to include a plugin.
- Would support automatically restarting a workspace to clone projects added to the DW: if e.g. the entrypoint to a container is
git clone <myproject>
, then updating projects would change the pod spec and cause a restart
Cons
- Somewhat more complex, as definition lives in the operator. Debugging/improving this functionality requires debugging/improving the operator itself.
- No way to customize functionality (the init container you get is defined in the version of DWO)
- It's not strictly necessary to restart a workspace to add a project, but this approach forces it.
from devworkspace-operator.
Another detail for consideration here is how this interplays with async storage. When async storage is used, volumes mounted to the workspace are all ephemeral, and are filled after startup by the async sidecar. This means that git cloning does unnecessary work (except for the very first time its run) for async storage.
from devworkspace-operator.
@amisevsk we briefly discussed that during today Cabal. The plugin approach looks more flexible and should be preferred IMHO. But I would also expect the plugin to be implicitly appended to any DevWorkspace object that has projects.
In other words, as a user, I would expect the projects to be cloned when I kubectl apply
a DevWorkspace
like the following one:
kind: DevWorkspace
apiVersion: workspace.devfile.io/v1alpha2
metadata:
name: java-sample
spec:
started: true
template:
projects:
- name: frontend
git:
remotes:
origin: https://github.com/spring-projects/spring-petclinic
components:
- name: maven
container:
image: quay.io/eclipse/che-java8-maven:nightly
You are right, in the case of async
storage the behavior should be different. We may need a different plugin in this case.
from devworkspace-operator.
The second point is moved to a separate issue #400
from devworkspace-operator.
Related Issues (20)
- Update to Devfile API v2.2.2
- Home persistence should be disabled if ephemeral storage strategy is used
- Add support for dependentProjects in DevWorkspaces
- Add restrictions for automount volumes when restricted-access annotation is used
- Move to v1alpha3 DevWorkspaces HOT 1
- Missing operatorframework.io/arch label in DWO CSV HOT 1
- Add some way of defining readiness/liveness checks for DevWorkspaces
- Allow pod/container overrides for volumes/volumeMounts HOT 1
- invalid metadata.labels being added HOT 4
- Webhook Certificate not generated during OLM installation HOT 4
- persistUserHome does not work with per-workspace storage strategy
- per-workspace storage PVC size calculation is incorrect when no volumes defined & mountSources used
- Check sources PR GitHub Action failing to setup envtest
- Controller env test fails if ginkgo is not installed.
- Fall-back to username for DevWorkspace creator label if UID is not present. HOT 4
- Unable to resolve parent devworkspace when using self-signed certs on disconnected/airgap cluster
- Ensure all user/workspace PVCs have a required PVC type label
- Mounting files that conflict with stow directory files causes persistent home to not work
- Deleted common/per-user PVC causes workspaces to be stuck until timeout
- Add initContainer for initializing persistentHome when $HOME persistence enabled
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from devworkspace-operator.