/kind bug

The current architecture page in user-guide does not actually describe the KubeVirt architecture. Please rebuild this doc.

Update: Architecture doc is [here|https://github.com/kubevirt/kubevirt/blob/master/docs/architecture.md]. This story just becomes a matter of fixing the web page link which is significantly easier.

hostDisk is not yet documented at https://kubevirt.io/user-guide/#/workloads/virtual-machines/disks-and-volumes?id=volumes

From kubevirt/kubevirt#906 (comment)

Conclusion for the moment:

Inside Windows guests, the full FQDN must be used to resolve cluster services
Windows does not support search domains via DHCP

Because windows diverges from Linux we should document it.

Do the latest docs still intentionally reference v0.17.0? Or should these references continue to change as newer versions of Kubevirt are released?

Latest Release Notes is not up to date. Maybe rather link to the releases page?

since there is no central place where virtctl is documented, will add under: VM exposure, and under OVM and VMRS sections

Some examples zuse registry disks with devel tag, but these disks are not publicly available.

The user guide should note how the cpou model can be set and it's default

If to follow yaml [1, 2], VM fails to start with an error:

Generated from virtualmachine-controller 4 times in the last 31 minutes (combined from similar events): Error creating pod: Pod "virt-launcher-cnv-vm-wind-pvc-9x5wc" is invalid: spec.containers[0].name: Invalid value: "volumevirtioContainerDisk": a DNS-1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name', or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?')

[1] https://github.com/petrkotas/user-guide/blame/extend-virtio-win-info/creating-virtual-machines/virtio-win.adoc#L125
[2] https://github.com/petrkotas/user-guide/blame/extend-virtio-win-info/creating-virtual-machines/virtio-win.adoc#L137

https://kubevirt.io/user-guide/#/templates/templates has two links. One is internal https://kubevirt.io/user-guide/#/installation/README ; the other is external https://docs.openshift.org/latest/dev_guide/templates.html . Both of them are broken.

We should continuously find and correct broken links in this repository. Internal ones can be avoided by a static analyzer; external ones require periodic crawling.

/cc @brianredbeard

We need to write a hook developer guide.

Currently there is only hook protocol description https://github.com/kubevirt/kubevirt/blob/master/pkg/hooks/v1alpha1/api.proto and an example hook https://github.com/kubevirt/kubevirt/tree/master/cmd/example-hook-sidecar.

KubeVirt version v0.20.1
Kubernetes version 1.16.1

$ kubectl get vmi -oyaml
apiVersion: v1
items:
- apiVersion: kubevirt.io/v1alpha3
  kind: VirtualMachineInstance
  metadata:
    annotations:
      kubevirt.io/latest-observed-api-version: v1alpha3
      kubevirt.io/storage-observed-api-version: v1alpha3
    creationTimestamp: "2019-10-06T21:11:03Z"
    finalizers:
    - foregroundDeleteVirtualMachine
    generateName: debian9
    generation: 8
    labels:
      kubevirt.io/nodeName: stardust
      special: key
    name: debian9
    namespace: default
    ownerReferences:
    - apiVersion: kubevirt.io/v1alpha3
      blockOwnerDeletion: true
      controller: true
      kind: VirtualMachine
      name: debian9
      uid: f3ee797d-4276-4411-823c-7aaab35f0936
    resourceVersion: "252101"
    selfLink: /apis/kubevirt.io/v1alpha3/namespaces/default/virtualmachineinstances/debian9
    uid: 8a57aa21-fdde-4abc-894e-7c746c3ba5aa
  spec:
    domain:
      cpu:
        cores: 2
      devices:
        disks:
        - disk:
            bus: virtio
          name: disk0
        - cdrom:
            bus: sata
            readonly: true
            tray: closed
          name: cloudinitdisk
        interfaces:
        - bridge: {}
          name: default
      features:
        acpi:
          enabled: true
      firmware:
        uuid: 12c1df43-db95-576d-8351-c45480ab4bc9
      machine:
        type: q35
      resources:
        requests:
          cpu: 100m
          memory: 1024M
    networks:
    - name: default
      pod: {}
    volumes:
    - name: disk0
      persistentVolumeClaim:
        claimName: debian-stretch
    - cloudInitNoCloud:
        userData: |
          #cloud-config
          hostname: debian9
          ssh_pwauth: True
          disable_root: false
          ssh_authorized_keys:
          - ssh-rsa ***
      name: cloudinitdisk
  status:
    conditions:
    - lastProbeTime: null
      lastTransitionTime: null
      message: cannot migrate VMI with non-shared PVCs
      reason: DisksNotLiveMigratable
      status: "False"
      type: LiveMigratable
    - lastProbeTime: null
      lastTransitionTime: null
      message: cannot migrate VMI with a bridge interface connected to a pod network
      reason: InterfaceNotLiveMigratable
      status: "False"
      type: LiveMigratable
    - lastProbeTime: null
      lastTransitionTime: "2019-10-06T21:11:29Z"
      status: "True"
      type: Ready
    interfaces:
    - ipAddress: 10.32.0.12
      mac: 7e:eb:23:3c:0d:9c
      name: default
    migrationMethod: BlockMigration
    nodeName: stardust
    phase: Running
    qosClass: Burstable
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

$ virtctl expose vmi debian9 --port=12000 --target-port=22 --name=debian9-ssh --type=NodePort

$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)           AGE
debian9-ssh     NodePort    10.101.118.117   <none>        12000:30000/TCP   6m42s

$ kubectl get ep
debian9-ssh     <none>               7m28s

$ ssh [email protected] -p 30000
ssh: connect to host 192.168.99.91 port 30000: Connection refused

But on the node, with its private ip, ssh works fine:

debian@stardust:~$ ssh [email protected]
Linux debian9 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sun Oct  6 20:45:12 2019 from 10.32.0.1

In the disks-and-volumes guide there is a Volumes section with links to the different types of volumes (cloudInitNoCloud, etc); unfortunately clicking these links in the guide result in a 404 error.

https://kubevirt.io/user-guide/docs/latest/release-notes/changelog.html mentions 0.15.0 whereas 0.16.0 is out...

Hope this helps

In the intro/installation docs.. it mentions having to need the the privileged SCC in order to run on Openshift:

Deploying on OpenShift
The following SCC needs to be added prior KubeVirt deployment:

$ oc adm policy add-scc-to-user privileged -n kubevirt -z kubevirt-operator

Is this still necessary as of v0.20.0 (as I believe it mentions creating specific SCC's now versus requiring privileged)? Should this note be removed or updated to say it only needs this as of < v0.20.0?

Section URL: https://kubevirt.io/user-guide/#/misc/monitoring?id=integrating-with-the-openshift-cluster-monitoring-operator

While the Kubernetes section works without any issue, the integration for OpenShift is not clear. I've done few tests with different settings and nothing really seems to work.

• Created the ServiceMonitor shown in the Kubernetes section within the openshift-monitoring namespace, I can see it being logged on the cluster-monitoring-operator logs (increasing verbosity) but nothing is being scrapped.

• Created the ServiceMonitor within the kube-system namespace, no improvement.

• Tried to create it again in the openshift-monitoring namespace but adding namespaceSelector.matchNames set to kube-system, doesn't work. I know that field is not required but just for the sake of getting it to work.

• I've added openshift.io/cluster-monitoring: "" label to the selector.matchLabels section, also added it to the service kubevirt-prometheus-metrics and even to the virt-api deployment. No improvement.

For all the tests, I never saw any KubeVirt related target being configured on the targets section of the prometheus UI. I'm currently out of options, but willing to test any suggestion.

I think it's worth noting the following from OCP docs [0]:

Explicitly unsupported cases include:

Creating additional ServiceMonitor objects in the openshift-monitoring namespace, thereby extending the targets the cluster monitoring Prometheus instance scrapes. This can cause collisions and load differences that cannot be accounted for, therefore the Prometheus setup can be unstable.

Creating additional ConfigMap objects, that cause the cluster monitoring Prometheus instance to include additional alerting and recording rules. Note that this behavior is known to cause a breaking behavior if applied, as Prometheus 2.0 will ship with a new rule file syntax.

And also, from the cluster monitoring operator [1]:

The deployed Prometheus Operator is intended to be used only for cluster-level monitoring. As such, the deployed Prometheus instance (prometheus-k8s) is responsible for monitoring and alerting on cluster and OpenShift components; it should not be extended to monitor user applications. Important: The Prometheus Operator managed by the Cluster Monitoring Operator will by default only look for ServiceMonitor resources in openshift-monitoring namespace.

Users interested in leveraging Prometheus for application monitoring on OpenShift should consider using OLM to easily deploy a Prometheus Operator and setup new Prometheus instances to monitor and alert on their applications.

[0] Prometheus cluster monitoring - supported configuration

[1] Cluster monitoring operator - README

We should STATE that only N-1 to N is supported in big red bold text :)

https://kubevirt.io/user-guide/docs/latest/administration/intro.html#update

Follow up on kubevirt/kubevirt#1112.

It is not obvious for everyone how k8s objects work and also not how exactly our workload and our controllers tie in there. Let's add more detailed descriptions, including links to the official k8s documentation where we align with k8s.

Currently KuebVirt can not run on the CRIO runtime.

We should mention this.

While going through the docs, I found a couple of broken links, probably because the whole tree got reorganized:

architecture/virtualmachine.html :

• Link to "Run Strategies": can be found in Run Strategies
• Link to "Expose Services": Exposing VirtualMachineInstance

creating-virtual-machines/intro.html:

All the links at the bottom of the page:

• Link to volumes: More information about persistent and ephemeral volumes
• Link to graphical & serial access: How to access a VirtualMachineInstance via console or vnc
• Link to cloud init: How to customize VirtualMachineInstances with cloud-init

/kind bug

It is hard to determine if specific features exist in a kubevirt version X.

Hi, based on the pr for #259 at #292 , I got this answer:

redhataccess/ascii_binder#152 (comment)

Seems that Fedora Doc (which was working on ascii_binder before) is now using 'Antora' (https://docs.antora.org/antora/2.1/install/install-antora/)

Should we consider it instead 'too' ?

Must be changed here: https://github.com/kubevirt/user-guide/tree/master/installation

The NodePort example is mentioning a couple of ports and is confusing to understand.

It should be more straight forward to udnerstand.

See for instances https://kubevirt.io/user-guide/docs/latest/creating-virtual-machines/disks-and-volumes.html#emptyDisk.

@fabiand who would be the one to fix this?

https://kubevirt.io/user-guide/docs/latest/administration/intro.html contains "RELEASE=v0.14.0" or "VERSION=v0.14.0"... which tend to be obsoleted by new releases.

May I suggest something like referring to the latest issued release, for instance:

export KUBEVIRT_RELEASE=$(curl --silent "https://api.github.com/repos/kubevirt/kubevirt/releases/latest" | grep '"tag_name":'| sed -E 's/.*"([^"]+)".*/\1/')

Hope this helps,

Improve image-upload guide to include necessary step to get rid of --uploadproxy-url

PROXY="https://10.88.0.8:30000"
kubectl patch -n cdi cdiconfig.cdi.kubevirt.io/config --type "json" --patch '[{"op":"replace","path":"/spec/uploadProxyURLOverride","value":"'${PROXY}'" }]'

Ref: https://github.com/kubevirt/containerized-data-importer/blob/master/doc/cdi-config.md
Ref: https://kubevirt.io/user-guide/#/installation/image-upload
code: https://github.com/kubevirt/kubevirt/blame/736b4aec6b2e92f2ab09ccfa6c5eb79366e88e5a/pkg/virtctl/imageupload/imageupload.go#L536

The example provided in the DataVolume section results in an error if you try and use it as written:

$ cluster/kubectl.sh create -f ~/alpine-dv.yaml
The  "" is invalid: : spec.dataVolumeTemplates.status in body is required

The field definition in the Kubevirt code dos not provide the omitempty tag, so this is a required field for this object.

We did not have static pages for quite a while.

With #174 we moved to asciidoc.
Now it's a small gap to add static page generation for this content.

I use the user-guide very often to point people to examples and explanations. However, since the search box disappeared, even I can't find anything. I normally then go to the github repo and use the github search to find the page.

We should add a search box.

/kind enhancement

On an iPad Air form factor, the docs are unusable because there's no way to bring up the navigation bar. It's pretty easy to emulate this, just make the browser window smaller until the navbar disappears. There's no way to bring it back.

I assume the menu was once available from a hamburger menu or a hover zone, but I can't find the latter if it's there.

Please let me know if I can provide further info!

Changelog last entry is still 0.11.0.

The startup scripts doc section does not contain information on how a Windows VM can leverage cloudbase-init (https://github.com/cloudbase/cloudbase-init) for achieving the same level of provisioning capabilities of the Linux machines.

Cloudbase-Init is cloud-init's counterpart for Windows, the defacto provisioner for Windows on OpenStack, CloudStack, OpenNebula, packet.net, with support for AWS, Azure, GCE.

From KubeVirt perspective, it supports ConfigDrive and NoCloud metadata support is coming soon.

Cloudbase-Init supports a subset of cloud-config directives, well suited for Windows environments: write_files, set_timezone, set_hostname, ntp, groups, users, runcmd.

More information on the cloud-config support can be found here: https://cloudbase-init.readthedocs.io/en/latest/userdata.html#cloud-config

Example userdata file for adding user / groups and running various commands:

#cloud-config
groups:
  - cloud-users
users:
  - name: Admin
    primary_group: cloud-users
    groups: Administrators
    passwd: StrongPassw0rd
    ssh_authorized_keys:
      - ssh first key
      - ssh second key
runcmd:
 - 'dir C:\\'
 - ['echo', '1']

Based on the information from your labs, I made a comprehensive tutorial on how to deploy Kubernetes and all the glue necessary to successfully import, boot, provision and customize a Window VM, using Cloudbase-Init:

https://github.com/ader1990/windows-openstack-imaging-tools/tree/add_kubevirt_example/Examples/config/kubevirt

workloads/templates/vm-creation.md seems to be missing:

1. running: false
2. ${{MEMORY}} appears as a parameter which should be passed, but its not defined in the parameters section

The documentation link (https://kubevirt.io/user-guide) which is linked from the homepage seems to give a 404.

Add node anti-/affinity to the docs.

The creation example is invalid and outdated:

http://www.kubevirt.io/user-guide/#/workloads/virtual-machines/creation

It's using the iscsi volume which is not present anymore.

Can we follow a little bit more the structure like we have for https://kubevirt.io/user-guide/#/workloads/virtual-machines/disks-and-volumes? The structure there is lent from k8s, which I personally find pretty easy to follow and user-focused.

So to have

• Examples per configs
• For instance also show where and how to set the model
• For a comprehensive and up-to-date list of supported values, link to the api-reference and make sure that the api-reference comment contains supported values.

It may just be me, but I find the structure of this document very confusing and it is almost impossible for me to parse it and extract the necessary information. It also looks a little bit like this documentation tries to be a mixture of api-reference and user-guide. We should leverage the fact that we have an api-reference.

from #89 (comment)

I've been trying to replicate a cloud-init config for a VMI following:
https://github.com/kubevirt/user-guide/blob/master/creating-virtual-machines/startup-scripts.adoc#cloud-init-network-config-as-clear-text

From my experience, it seems that the Ubuntu 18.04 cloud-init wouldn't like:

      - cloudInitNoCloud:
          networkData: |
            network:
              version: 1
              config:
              - type: physical
              name: enp1s0
              subnets:
                - type: dhcp

But that one worked:

      - cloudInitNoCloud:
          networkData: |
            version: 2
            ethernets:
              enp1s0:
                dhcp4: true

I have the impression that networkData: shouldn't contain a network entry.

This is a bit confusing wrt other docs instructing how to construct contents of the noCloud.iso... but still...

Hope this helps,

Let's add a section which is focusing on the what do I have to use to solve a problem view on things.

What happened: API Reference at home page seems broken.

What you expected to happen: It points to correct link, and I can open to browse all APIs.

For better performance a host should be configured with the "tuned profile latency-performance`.

    # Implicit cascade delete (first deletes the virtual machine and then the virtual machine)
    kubectl delete virtualmachine myvm

    # Explicit cascade delete (first deletes the virtual machine and then the virtual machine)
    kubectl delete virtualmachine myvm --cascade=true

which should be virtual machine instance

VMIs can have a watchdog.

This issue is about adding documentation for the watchdog device.

The test case can be used for reference:

• https://github.com/kubevirt/kubevirt/blob/master/tests/utils.go#L965
• https://github.com/kubevirt/kubevirt/blob/master/tests/vmi_monitoring_test.go#L56

Reasoning is that asciidoc hsa some more primitves to generate a good user-guide, i.e. emblems for warnings etc