googlecontainertools / google-container-tools-intellij Goto Github PK

Sets up plugin.xml, initial directory structure, build files etc.

Create Skaffold Gradle module to add all Skaffold related functionality and features.

• bash script to be executed by kokoro release job
• gradle release plugin
• gradle publishPlugin task configuration (called by bash script)
• setup backend kokoro config (not part of this repo)

Default target is 1.6, not compatible with some of the required features such as UI DSL.

When adding a new change into Skaffold YAML file, for example, new profile or an image change, or even adding a new Skaffold YAML file, tools that rely on file content - Skaffold file search or YAML content may not see the change right away unless file is explicitly saved by Ctrl-S.

We need to find a way to commit the writes or flush the cache before the actions that need up-to-date Skaffold YAML contents.

Single run (deployment) mode does not stream logs by default, which can be useful to assess initial progress of the deployment. Stopping the process will then stop the logs streaming, but leave the original deployment intact. We can add a simple option for single run mode to enable log tailing.

If project is opened or new files appear, detect if Skaffold configuration YAML is present, and prompt a user to create Skaffold run configuration, using pop-up notification.

Create ContainerToolsRule similar to what CloudToolsRule has but probably faster and simpler, with better accomodation for Kotlin test framework .
The test rule should support:

• test services
• easy mocks
• test projects via heavyweight fixture

Conceivably we can follow what this blog suggests (https://blog.philipphauer.de/best-practices-unit-testing-kotlin/) and also consult Kotlin group on this.

As of now, Skaffold streams all logs from all pods/deployments to console, without additional highlighting (at least in IDEA console) and filtering. Consider implementing:

• effective search across the logs output
• filtering by pod name/pattern, using it to find logs for specific pod/deployment/version
• additional highlighting and hyperlinking in the IDE console window, i.e. to jump to the code/YAML file related to the log line

Originally posted by @ivanporty in #85

This will allow downstream unexpected exceptions to bubble up to an IDE error for reporting. Currently, these are swallowed and shown to the user as a red error notification.

Currently @TestFile annotation uses actual temp file on disk and requires a few manual steps, including mocking or defining PsiManager per project and converting sequence of File -> Virtual File -> Psi File. We might benefit from using @PsiFile with a given text for quicker and easier unit testing.

I created a new skaffold dev type run configuration and gave it some name, something like my Skaffold config.

A few moments later I went in to edit it and I couldn't remember if it was a dev or run type.

Perhaps we should have a label in the UI to make this clear

i.e. the opt-in 'Report exception to Google' notification when something really bad happens.

(this depends on Skaffold exposing an event api)

Currently as part of the Kubernetes deployment and continuous development features we do not provide any insight to the user about the state of the action beyond just printing the skaffold log output.

For example, if the build fails, the user will see the run process window complete, with no visual indication or action suggestions for the user (beyond just showing the logs):

At a minimum, with proper events exposed, we could show the completion status of the operation similar to how we do for app engine deployment in the cloud tools plugin. Here is an example of a failed app engine deployment, followed by a successful one:

Furthermore, we use the event knowledge here to print out additional information for the user to the output console in case of a failure.

todo: investigate the what can be done with the standard intellij run configuration output window - i.e. how much control do we have for displaying different completion states (@ivanporty if you have any ideas here).

When a user opts in, send pings when live templates are used, such as typing "skaffold" to create initial Skaffold template.

Originally part of #110.

Make sure it's fully compatible given issues with Cloud Tools.

Skaffold now supports concept of default image repository, the one that is used for all Skaffold deployments regardless of what is configured in skaffold configuration for build stage. This allows users to try out other developer pipelines without having to worry about changing build step and image/repo name.

We can add this to individual run configuration or as a global setting.

See GoogleContainerTools/skaffold#1057 for more details.

Currently Jetbrains officially provides Kubernetes Plugin which offers autocompletion and code inspection for Kubernetes resources yaml files.

Given following skaffold.yaml file,

apiVersion: skaffold/v1alpha2
kind: Config
build:
  artifacts:
  - imageName: joelhandwell/kotlin-jvm-hello
deploy:
  kubectl:
    manifests:
    - k8s-*
profiles:
  - name: gcb
      googleCloudBuild:
        projectId: k8s-skaffold

As skaffold.yaml file syntax is very similar to kubernetes resource yaml file the file inspections defined for kubernetes resources triggered against skaffold.yaml too with inspection warnings.

Value 'skaffold/v1alpha2' is not expected here less... (Ctrl+F1) 
Inspection info: This inspection finds unrecognized values in Kubernetes resource definition files and highlights them in the editor.

Value 'Config' is not expected here less... (Ctrl+F1) 
Inspection info: This inspection finds unrecognized values in Kubernetes resource definition files and highlights them in the editor.

Key 'deploy' is not expected here less... (Ctrl+F1) 
Inspection info: This inspection finds unrecognized keys in Kubernetes resource definition files and highlights them in the editor.

Key 'profiles' is not expected here less... (Ctrl+F1) 
Inspection info: This inspection finds unrecognized keys in Kubernetes resource definition files and highlights them in the editor.

Screenshot looks like following:

Possible solution could be adding inspection which is only applied to skaffold.yaml at the same time disabling Kubernetes plugin inspection.

P.S. for those who want to suppress Intellij kubernetes plugin warning against skaffold.yaml file, you can click the Hector icon and change inspection level to syntax only on the file.

It seems that test sources cannot access internal classes and functions if called in Gradle sub-modules (at least for our project). Root Gradle project sources and test sources work when set up in a simple one module project.

Run configuration allows a user to run Skaffold continuously (development mode, skaffold dev), and stream the logs into the console window until stopped by a user.

Includes:

• declaration of run configuration
• factory to create configuration
• settings
• run profile support to execute Skaffold itself and show logs in the console view
• support to properly stop Skaffold in dev mode (SIGINT instead of kill)
• UI editor.

Part of #47.

The following events will be supported:

• Skaffold continuous development run
• Skaffold single deployment run
• Auto-creation of Skaffold run targets
• Manual creation of Skaffold run targets (development/deploy)
• (Optional) Usage of Skaffold live template to create new Skaffold file, if possible to detect

Instead of hard-coding in build scripts.

Currently we assume Skaffold to be present in the system in order to run its deployment and development configurations. If it's not present, error will be shown (skaffold executable not found or similar) after run and nothing else is proposed.

We need to validate presence of Skaffold using current (default system based) executor service and show run configuration as invalid if Skaffold is not present in the system, suggesting to install Skaffold in order to fix it. Later this is going to be further enhanced with managed Skaffold installation.

Skaffold profiles support (#95) introduced YAML format basic parsing. It can be re-used to better identify valid Skaffold YAML file by parsing the content and detecting apiVersion instead of just checking the first line for regular expression.

Umbrella issue for dealing with obsolete/deprecated runtimes.

• Code inspections + quickfix on old runtime
• Project generation of old runtime removal (already complete)
• (possible) warning on deployment of application with obsolete runtime
• Eventually remove supporting logic of obsolete runtime (e.g. whitelisting)

The following icons, aside from the original Skaffold icon (made from logo) are needed:

• Skaffold continuous run image, separating it from single run configuration
• Skaffold action icon for manual trigger to re-deploy

Iterate project files, without excluded directories such as module compile output, and use the list of found files to populate drop-down with available Skaffold files in run configurations UI settings.

Part of #7, #12, #13.

Currently running fresh version of Skaffold on an older version/schema of skaffold.yaml results in a warning that schema has changed and suggestion to run skaffold fix on it. This is not quite IDE friendly way and we should research and implement a better way to detect schema change and ask a user about upgrading it before actual deployment so the warning won't be necessary.

If project opened has dependencies that need long indexing or IDE is opened for first time and needs indexing of a core SDK, Skaffold files detection does not work yet. It need to only be run after initial project indexing is done and files are available.

We should probably have our own "Hello World" style example for SpringBoot RESTful application, and use it as a reference for quick start with the plugin and its features in the tutorial / getting started documentation, rather than creating everything from scratch or using external getting started project.

It should definitely use Jib and can be based on https://github.com/GoogleContainerTools/skaffold/tree/master/examples/jib with a few enhancements.

Based on the following Skaffold functionality:
GoogleContainerTools/skaffold#1098

When GKE cluster is used for deployment with Skaffold, labels will identify Container Tools IntelliJ plugin is used for Skaffold pipeline with additional details, such as Skaffold and Plugin version, and IDE type and version.

Just ran a skaffold dev config, then stopped it, and noticed this IDE error:

java.lang.Throwable: Write-unsafe context! Model changes are allowed from write-safe contexts only. Please ensure you're using invokeLater/invokeAndWait with a correct modality state (not "any"). See TransactionGuard documentation for details.
  current modality=ModalityState:{[com.intellij.openapi.ui.impl.DialogWrapperPeerImpl$MyDialog[dialog0,734,1028,1092x698,invalid,layout=java.awt.BorderLayout,APPLICATION_MODAL,title=Run/Debug Configurations,defaultCloseOperation=DO_NOTHING_ON_CLOSE,rootPane=com.intellij.openapi.ui.impl.DialogWrapperPeerImpl$MyDialog$DialogRootPane[,5,25,1082x668,invalid,layout=javax.swing.JRootPane$RootLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=449,maximumSize=,minimumSize=,preferredSize=],rootPaneCheckingEnabled=true]]}
  known modalities:
    ModalityState.NON_MODAL, writingAllowed=true;
    ModalityState:{}, writingAllowed=true;
    ModalityState:{[com.intellij.openapi.ui.impl.DialogWrapperPeerImpl$MyDialog[dialog0,734,1028,1092x698,invalid,layout=java.awt.BorderLayout,APPLICATION_MODAL,title=Run/Debug Configurations,defaultCloseOperation=DO_NOTHING_ON_CLOSE,rootPane=com.intellij.openapi.ui.impl.DialogWrapperPeerImpl$MyDialog$DialogRootPane[,5,25,1082x668,invalid,layout=javax.swing.JRootPane$RootLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=449,maximumSize=,minimumSize=,preferredSize=],rootPaneCheckingEnabled=true]]}, writingAllowed=true
	at com.intellij.openapi.diagnostic.Logger.error(Logger.java:134)
	at com.intellij.openapi.application.TransactionGuardImpl.assertWriteActionAllowed(TransactionGuardImpl.java:232)
	at com.intellij.openapi.fileEditor.impl.FileDocumentManagerImpl.saveAllDocuments(FileDocumentManagerImpl.java:289)
	at com.intellij.openapi.fileEditor.impl.FileDocumentManagerImpl.saveAllDocuments(FileDocumentManagerImpl.java:281)
	at com.google.container.tools.skaffold.run.ui.SkaffoldFilesComboBox$setProject$1$1.run(SkaffoldFilesComboBox.kt:49)
	at com.intellij.openapi.application.impl.ApplicationImpl.invokeAndWait(ApplicationImpl.java:670)
	at com.intellij.openapi.application.impl.ApplicationImpl.invokeAndWait(ApplicationImpl.java:683)
	at com.google.container.tools.skaffold.run.ui.SkaffoldFilesComboBox.setProject(SkaffoldFilesComboBox.kt:49)
	at com.google.container.tools.skaffold.run.ui.BaseSkaffoldSettingsEditor.resetEditorFrom(BaseSkaffoldSettingsEditor.kt:108)
	at com.google.container.tools.skaffold.run.ui.BaseSkaffoldSettingsEditor.resetEditorFrom(BaseSkaffoldSettingsEditor.kt:43)
	at com.intellij.openapi.options.SettingsEditor.lambda$resetFrom$0(SettingsEditor.java:73)
	at com.intellij.openapi.options.SettingsEditor.bulkUpdate(SettingsEditor.java:80)
	at com.intellij.openapi.options.SettingsEditor.resetFrom(SettingsEditor.java:73)
	at com.intellij.execution.impl.ConfigurationSettingsEditor$ConfigToSettingsWrapper.resetEditorFrom(ConfigurationSettingsEditor.java:287)
	at com.intellij.execution.impl.ConfigurationSettingsEditor$ConfigToSettingsWrapper.resetEditorFrom(ConfigurationSettingsEditor.java:271)
	at com.intellij.openapi.options.CompositeSettingsEditor.resetEditorFrom(CompositeSettingsEditor.java:52)
	at com.intellij.execution.impl.ConfigurationSettingsEditorWrapper.resetEditorFrom(ConfigurationSettingsEditorWrapper.java:98)
	at com.intellij.execution.impl.ConfigurationSettingsEditorWrapper.resetEditorFrom(ConfigurationSettingsEditorWrapper.java:25)
	at com.intellij.openapi.options.SettingsEditor.lambda$resetFrom$0(SettingsEditor.java:73)
	at com.intellij.openapi.options.SettingsEditor.bulkUpdate(SettingsEditor.java:80)
	at com.intellij.openapi.options.SettingsEditor.resetFrom(SettingsEditor.java:73)

High level features:

1. Available from drop-down list of run/debug configurations
2. Exposes basic configuration UI (conceivably with skaffold.yaml location, environment variables)
3. Compatible with maximum number of IDEs (possibly with all of them)
4. Extensible to support language specific features (build image with JIB for Java etc.)

While testing Skaffold run configurations support for various IDEs, I found out non-Java IDEs versions 2018.1 and earlier do not have UI DSL classes packaged with them hence causing exceptions when using them for our UI. We either need to decrease range of supported IDEs or change UI back to form builder to allow older IDE versions support.

Since project is using Kotlin, it could make sense to include simple RESTful application based on Kotlin Ktor framework to lure Kotlin growing backend developers. Functionality could be similar to one in #133 and it should use Jib to build images.

Which additional flags if any should we surface? Should we surface them as a structured UI or a freeform textfield etc. ...

Set up all the usual CI targets. Can use Cloud Tools for IntelliJ as a template

Create JB IDE plugin.xml that will contain all extension points, services and components for all future features of the plugin.

time="2018-11-01T12:46:38-04:00" level=fatal msg="exiting dev mode because the first build failed: building [gcr.io/**/my-proj]: tagging: pushing: getting auth config for gcr.io/**/my-proj:3c275201-dirty-50f870d: getting auth config: error getting credentials - err: exec: \"docker-credential-gcr\": executable file not found in $PATH, out: ``"

possibly related: https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000464664-IDEA-sandbox-problems-with-shell-command

It could be useful to support specific cluster (kubectl context) for a run configuration, in order to support fine-grained configurations to deploy and test in a local cluster (minikube or docker) vs. remote GKE cluster without having to check which cluster is active now and switching it manually.

Possible ways to do this:

• Call kubectl config set-context before deployment. This changes the global context after each run.
• Use Skaffold support and pass --context to kubectl command to deploy to specified cluster.
  Related Skaffold issue: GoogleContainerTools/skaffold#511

Run configuration allows a user to run Skaffold one time (deploy mode, skaffold run), and optionally, stream the logs into the console window.
Includes:

• declaration of run configuration
• factory to create configuration
• settings
• run profile support to execute Skaffold itself and show logs in the console view
• UI editor.

This is more of a tracking issue since the implementation does not take place here.

When a developer is working with a project that has a Skaffold config, they should be prompted to install this plugin. The detection should not be based off of the file name (since the name is arbitrary), but rather on its contents (either apiVersion, or kind)

Skaffold supports multiple profiles in a single configuration file, for example, one for docker build, and one for Cloud Build, see https://github.com/GoogleContainerTools/skaffold/blob/master/examples/getting-started/skaffold.yaml with two additional profiles (gcb, test).

We can parse the configuration file and find all profiles. If more than one profile (default) exists, we then provide a drop down with the list of profiles and allow a user to select which profile needs to be run. Chosen profile is then passed on to skaffold with a -p --profile flag.

For example, this could be based on the manifests, or usage of the JB k8s plugin etc.

Currently, we detect Skaffold files only when we initially open a project. When a new file is created, we don't detect it and do not prompt to create Skaffold configurations automatically.