kubermatic / docs Goto Github PK

https://docs.loodse.com/kubermatic/master/requirements/cluster_requirements/
Page is pretty much outdated what is required.
This should also explain how a default setup looks like (kubeone + kubermatic)

The most important is here the Network Ranges, LB's, Firewalls, Certificates,
as input you could take a look at: https://forms.gle/aLvRGfSApFejm5jQ7

add the output of govc tool to it. see output of @chrkl

 govc role.ls cus_seed_k8c-storage-vmfolder-propagate
Folder.Create
Folder.Delete
System.Anonymous
System.Read
System.View
VirtualMachine.Config.AddExistingDisk
VirtualMachine.Config.AddNewDisk
VirtualMachine.Config.AddRemoveDevice
VirtualMachine.Config.RemoveDisk
[kleineweber@ck-mbp ~] govc role.ls cus_seed_k8c-storage-datastore-propagate
Datastore.AllocateSpace
Datastore.FileManagement
System.Anonymous
System.Read
System.View
[kleineweber@ck-mbp ~] govc role.ls cus_user_k8c-user-vcenter
System.Anonymous
System.Read
System.View
VirtualMachine.Provisioning.ModifyCustSpecs
VirtualMachine.Provisioning.ReadCustSpecs
[kleineweber@ck-mbp ~] govc role.ls cus_user_k8c-user-datacenter
Datastore.AllocateSpace
Datastore.Browse
Datastore.DeleteFile
Datastore.FileManagement
System.Anonymous
System.Read
System.View
VApp.ApplicationConfig
VApp.InstanceConfig
VirtualMachine.Config.CPUCount
VirtualMachine.Config.Memory
VirtualMachine.Config.Settings
VirtualMachine.Inventory.CreateFromExisting
[kleineweber@ck-mbp ~] govc role.ls cus_user_k8c-user-cluster-propagate
Host.Config.SystemManagement
Host.Local.ReconfigVM
Resource.AssignVMToPool
Resource.ColdMigrate
Resource.HotMigrate
System.Anonymous
System.Read
System.View
VApp.ApplicationConfig
VApp.InstanceConfig
[kleineweber@ck-mbp ~] govc role.ls cus_user_k8s-network-attach
Network.Assign
System.Anonymous
System.Read
System.View
[kleineweber@ck-mbp ~] govc role.ls cus_user_k8c-user-datastore-propagate
Datastore.AllocateSpace
Datastore.Browse
Datastore.FileManagement
System.Anonymous
System.Read
System.View
[kleineweber@ck-mbp ~] govc role.ls cus_user_k8c-user-folder-propagate
Folder.Create
Folder.Delete
Global.SetCustomField
System.Anonymous
System.Read
System.View
VirtualMachine.Config.AddExistingDisk
VirtualMachine.Config.AddNewDisk
VirtualMachine.Config.AddRemoveDevice
VirtualMachine.Config.AdvancedConfig
VirtualMachine.Config.Annotation
VirtualMachine.Config.CPUCount
VirtualMachine.Config.ChangeTracking
VirtualMachine.Config.DiskExtend
VirtualMachine.Config.DiskLease
VirtualMachine.Config.EditDevice
VirtualMachine.Config.HostUSBDevice
VirtualMachine.Config.ManagedBy
VirtualMachine.Config.Memory
VirtualMachine.Config.MksControl
VirtualMachine.Config.QueryFTCompatibility
VirtualMachine.Config.QueryUnownedFiles
VirtualMachine.Config.RawDevice
VirtualMachine.Config.ReloadFromPath
VirtualMachine.Config.RemoveDisk
VirtualMachine.Config.Rename
VirtualMachine.Config.ResetGuestInfo
VirtualMachine.Config.Resource
VirtualMachine.Config.Settings
VirtualMachine.Config.SwapPlacement
VirtualMachine.Config.ToggleForkParent
VirtualMachine.Config.UpgradeVirtualHardware
VirtualMachine.GuestOperations.Execute
VirtualMachine.GuestOperations.Modify
VirtualMachine.GuestOperations.ModifyAliases
VirtualMachine.GuestOperations.Query
VirtualMachine.GuestOperations.QueryAliases
VirtualMachine.Interact.AnswerQuestion
VirtualMachine.Interact.Backup
VirtualMachine.Interact.ConsoleInteract
VirtualMachine.Interact.CreateScreenshot
VirtualMachine.Interact.CreateSecondary
VirtualMachine.Interact.DefragmentAllDisks
VirtualMachine.Interact.DeviceConnection
VirtualMachine.Interact.DisableSecondary
VirtualMachine.Interact.DnD
VirtualMachine.Interact.EnableSecondary
VirtualMachine.Interact.GuestControl
VirtualMachine.Interact.MakePrimary
VirtualMachine.Interact.Pause
VirtualMachine.Interact.PowerOff
VirtualMachine.Interact.PowerOn
VirtualMachine.Interact.PutUsbScanCodes
VirtualMachine.Interact.Record
VirtualMachine.Interact.Replay
VirtualMachine.Interact.Reset
VirtualMachine.Interact.SESparseMaintenance
VirtualMachine.Interact.SetCDMedia
VirtualMachine.Interact.SetFloppyMedia
VirtualMachine.Interact.Suspend
VirtualMachine.Interact.TerminateFaultTolerantVM
VirtualMachine.Interact.ToolsInstall
VirtualMachine.Interact.TurnOffFaultTolerance
VirtualMachine.Inventory.Create
VirtualMachine.Inventory.CreateFromExisting
VirtualMachine.Inventory.Delete
VirtualMachine.Inventory.Move
VirtualMachine.Inventory.Register
VirtualMachine.Inventory.Unregister
VirtualMachine.Provisioning.Clone
VirtualMachine.Provisioning.CloneTemplate
VirtualMachine.Provisioning.CreateTemplateFromVM
VirtualMachine.Provisioning.Customize
VirtualMachine.Provisioning.DeployTemplate
VirtualMachine.Provisioning.DiskRandomAccess
VirtualMachine.Provisioning.DiskRandomRead
VirtualMachine.Provisioning.FileRandomAccess
VirtualMachine.Provisioning.GetVmFiles
VirtualMachine.Provisioning.MarkAsTemplate
VirtualMachine.Provisioning.MarkAsVM
VirtualMachine.Provisioning.ModifyCustSpecs
VirtualMachine.Provisioning.PromoteDisks
VirtualMachine.Provisioning.PutVmFiles
VirtualMachine.Provisioning.ReadCustSpecs

Documentation for master branch should be updated similarly to what we already did for v2.14.

For instance we should:

• Add the step to install cert-manager CRDs that have been extracted from the Helm chart.

• Show how to download latest weekly tarball instead of the latest release that points to latest release branch (currently 2.14)

• Get Started

• How to create Cluster
• how to manage a cluster

• System requirements
• Installation
• Advanced

• etcd Backup
• ...

• Check if docs theme is compatible with latest hugo release v0.44
• Update Netflify Build Pipeline

Relevant discussion from slack:

Regarding the long kubectl delete cluster xxxxx call, @artiom mentioned.
The latest versions of kubectl use PropagationPolicy: foreground which lets kubectl wait until the object is deleted.
This policy also lets to the GarbageCollector delete dependents before the Owner is deleted (For example: etcd StatefulSets).
This leads to a broken state, as we require the etcd to cleanup machines (for which we use a finalizer on the cluster).

When you need to use kubectl delete cluster, please add the flag --cascade=false.
This will make the GarbageCollector wait until the owner is gone. So we can do the machine-cleanup

• Write a guide how to manage an existing kubermatic cluster

The install kubermatic section starts a bit abrupt. The helm lines for installing the charts reference some manifests in some charts/ directory. Personally I just used config/ directory from the respective branch of the kubermatic git repository. But this could be made explicit (starting with a downloaded zip file from releases, or git, or xyz).

Customers have less context and are less willing to guess.

It would be very useful to have support for tabs, so we can easily split the content without having to create additional pages. The Kubernetes docs website has this feature and they use it a lot. For example https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl

Here's the markdown source of the mentioned document which shows how they use it https://raw.githubusercontent.com/kubernetes/website/master/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md

I've also found the source for those shortcodes:

• https://github.com/kubernetes/website/blob/master/layouts/shortcodes/tabs.html
• https://github.com/kubernetes/website/blob/master/layouts/shortcodes/tab.html

Porting those shortcodes doesn't work out of the box, so some additional modifications are needed to make them work with our template.

After discovering the upgrade issue when upgrading from 2.11 to 2.12 with Flannel and Network Policy being used, this PR has been created: #379

It needs now to be applied to 2.14, 2.13, 2.12 documentation versions.

Use the OpenAPI Spec and create a proper documentation for the API

Enhance the spec with more user content.

Let us use tools like
https://github.com/lord/slate
https://github.com/Mermade/widdershins

I guess this should be: https://cloud.kubermatic.io/

We should remove the GUI installer from the docs, it's outdated and not supported anymore.
https://docs.kubermatic.io/installation/install_kubermatic/_installer/

Maybe we could let then entry stay and just write a hint, that we are working on a Kubermatic Operator and a new version of the installer

Document migration steps from v2.13 to v2.14. Particular care should be dedicated to the nodeport proxy that should move from Helm chart based installation to operator based.

• Create Get Started Guide, explaining how to create and manage a cluster with Kubermatic

In Kubermatic v2.15 we will deprecate datacenter configuration through datacenters.yaml, and then remove it entirely in Kubermatic v2.16.

For that purpose we must first update the documentation with the following:

• in docs master ensure that all documentation related to datacenters is based on CRDs and not datacenters.yaml
• ensure migration of Kubermatic installations from datacenter.yaml to CRD-based DCs is documented (if necessary)
• in docs master add warning boxes whenever datacenters are mentioned:
  • informing about the deprecation of datacenter.yaml in v2.15
  • informing about the removal of datacenter.yaml in v2.16
  • with a link to the migration guide

Currently there a a lot of different place where a user need to find out, how a Kubermatic setup behind a proxy could look like:

• https://docs.kubermatic.io/concepts/seeds/
• https://docs.kubermatic.io/advanced/offline_mode/
• #166

This should be centralized and described in end user perspective way. Currently only people with deeper insight knowledge are able to set this up.

Cross ref: kubermatic/kubermatic#4399

Currently for a user it is hard to understand how our user management reflect in the user cluster RBAC. At least in combination with OIDC share cluster function it's hard to understand what's happen. To avoid questions from customers we should describe the following points in our documentation:

• How can user manage cluster permissions
• How does the RBAC generator reflect this inside of the user cluster
• What different roles(groups) and permission do they have to resources in our system
• How dos the OIDC share cluster option defer

I think we could create therefor a chapter in the Architecture section. In OIDC share cluster](https://docs.kubermatic.io/advanced/oidc_auth/) section we should mention the basic concept as well.

cc @p0lyn0mial @kron4eg feel free to add your thoughts ;-)

Create and overview and architecture docu for users:

Let's start with the following topics:

• What is Kubermatic
• Architecture
• Features
• Usage
  • Create/delete cluster
  • Add/remove nodes
• Launch Stages (similar https://cloud.google.com/terms/launch-stages or https://dcos.io/docs/1.8/overview/feature-maturity/)

Examples for similar documentations are:
http://docs.rancher.com/rancher/v1.3/en/
https://dcos.io/docs/1.8/overview/
https://docs.docker.com/datacenter/ucp/2.0/guides/

Migrate Documentation from http://github.com/kubermatic/kubermatic-installer to this repository (without deleting the content in http://github.com/kubermatic/kubermatic-installer)

Document the reboot issues when using vSphere.

Result of kubermatic/kubermatic#1571

From the original issue:

When using a vsphere seed and rebooting a node that runs one or more pods with attached PVs, that > node can not be started anymore because the cloud provider does not remove the volume binding from > the old node even thought the pod will get rescheduled to another node.

This then means that the vsphere instance can not be started anymore until an operator manually > > > removes the binding to the node inside vsphere.

Upstream issue: kubernetes/kubernetes#63577

This issue will be resolved with kubernetes 1.12: kubernetes/kubernetes#63413 (comment)

Explain the concepts behind:

• Master Cluster
• Seed Cluster
• Customer Cluster

use uml diagrams

• Adding .drone.yml
• Configure Netlify deployment

A new section is necessary, that will document the behavior of MachineDeployments on AWS in absence of a specified subnet. AWS will select the default subnet in a given VPC/AZ and the user needs to understand that.

Don't use real names in our docs. initech.com is a company use example.com instead

Since our docs are open source we should fill out also the contribution page:
https://docs.loodse.com/kubermatic/master/contributing/

After a talk with Henrik, Matthias and Sebastian, a public documentary is planned for Kubermatic.
We can use the following as content ideas:

https://github.com/openshift/openshift-docs
https://github.com/coreos/tectonic-docs
https://github.com/coreos/etcd/tree/v3.2.17/Documentation/op-guide

https://cloud.google.com/container-engine/docs/
https://github.com/MicrosoftDocs/azure-docs
https://github.com/kubernetes/kubernetes.github.io

We moved from the Prometheus Operator to simple Statefulsets.
The migration from the old setup to the new one introduces some manual steps we want to document.

@cbrgm where do we put this? Is this something for the docs? I might actually fit better into a changelog.

To get a better overview how the machine Controller works and what fields are usable in the Cluster API spec, we should transfer this docs to here as well:

• https://github.com/kubermatic/machine-controller/tree/master/docs
• https://github.com/kubermatic/machine-controller/tree/master/examples

When using non-default VPC:

• Private and public networks have to be created (including gateway and routes)
• At least public network has to be tagged for LB usage (kubernetes.io/role/elb)

found at:
https://docs.kubermatic.io/kubermatic/v2.12/advanced/custom_links/

"Check the [Creating the Master Cluster] values.yaml"

That link points to: https://docs.kubermatic.io/installation/install_kubermatic/_manual/#creating-the-master-cluster-values-yaml
... which goes 404.

Besides this small readme we don't have any proper documentation how to use the kubevirt cloudprovider.

• https://github.com/kubermatic/machine-controller/blob/master/docs/kubevirt.md

• kubeconfig: seviceaccount + RBAC

• storage class requirements

• kubevirt operator version and pot. requirements to machine nodes

• machine spec options

For testing e.g. i used the following images

• Ubuntu Image URL for kubevirt: https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img
• CoreOS (uploaded to public gcloud bucket): https://storage.googleapis.com/ps-run/kubevirt/coreos/coreos_production_qemu_image.img

https://docs.kubermatic.io/installation/install_kubermatic/_installer/

you cannot get the installer container from quay without admins giving you the rights, this should be made clear in the docs

All components should be covered

• Kubermatic
• k8sniff
• Dex
• Prometheus
• etc.

According to customer feedback, the following permissions are missing:

  elasticloadbalancing:DescribeLoadBalancerAttributes
  elasticloadbalancing:DescribeLoadBalancers
  elasticloadbalancing:CreateLoadBalancer
  elasticloadbalancing:AttachLoadBalancerToSubnets
  elasticloadbalancing:RegisterInstancesWithLoadBalancer
  elasticloadbalancing:ConfigureHealthCheck
  elasticloadbalancing:DeleteLoadBalancer
  elasticloadbalancing:ModifyLoadBalancerAttributes

I created a new AWS policy in the cloudifire AWS account and figured that documented AWS policy won't work: https://docs.kubermatic.io/requirements/cloud_provider/_aws/
Maybe this policy is just for the seed cluster? If so we should add least what (minimum) policies the cluster creator user should have to our docu.

The policy I extracted from loodse account of the group cluster-creator are:

AmazonEC2FullAccess
IAMFullAccess

Then the cluster will come up, but this roles are may to open. @pkavajin please check this in our testing setup also.
Also the cloudifire thoughts are helpful https://github.com/CloudiFire/caas/issues/42

The described data structure of the presets in the docs doesn't match to the implementation.
https://docs.kubermatic.io/advanced/presets/

https://github.com/kubermatic/kubermatic/blob/master/api/pkg/crd/kubermatic/v1/preset.go#L25

https://kubernetes.io/docs/setup/pick-right-solution/ and
https://github.com/ramitsurana/awesome-kubernetes

• Write a guide how to set up a cluster with kubermatic

The current installation documentation relies on Helm2, the goal of this issue is to document the installation with Helm3 that is supposed to work already.

SImilarly to https://docs.kubermatic.io/requirements/cloud_provider/_aws/ we need to document Azure-specific setup steps, like preparing access credentials with permissions assigned.