k3s-io / docs Goto Github PK
View Code? Open in Web Editor NEWk3s Documentation
Home Page: http://docs.k3s.io
k3s Documentation
Home Page: http://docs.k3s.io
Instructions for installation of CRD for system-upgrade-controller is missing.
Following the instructions will lead to a message like this when applying the plans:
resource mapping not found for name: "server-plan" namespace: "system-upgrade" from "system-upgrade-plan.yaml": no matches for kind "Plan" in version "upgrade.cattle.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "agent-plan" namespace: "system-upgrade" from "system-upgrade-plan.yaml": no matches for kind "Plan" in version "upgrade.cattle.io/v1"
ensure CRDs are installed first
This note here on firewalld
should also mention that this apply to Fedora as well. It will be easier find-able if one use search and put in Fedora
.
We wrote up a deprecation policy (defined here: https://github.com/k3s-io/k3s/blob/master/docs/adrs/deprecating-and-removing-flags.md) that's been discussed and agreed upon within the team -- let's write up a page on the docs and make sure it's publicly stated.
This should be noted that it is as of May 2023 patch releases per k3s-io/k3s#7335.
There is a section for "Single-stack IPv6 installation", however there's no instructions on how to actually do it - the section is empty except for warnings.
In "docs/installation/private-registry.md" in the section "Adding Images to the Private Registry" it says to:
"obtain the k3s-images.txt file from GitHub", but that file does not seem to exist.
However, a file "scripts/airgap/image-list.txt" does exist.
Perhaps that is the intended file?
As mentioned in some k3s issues, https://docs.k3s.io/helm#helmchart-field-definitions is out of date and needs to be updated with all the latest fields. We need to cover everything in https://github.com/k3s-io/helm-controller/blob/master/pkg/apis/helm.cattle.io/v1/types.go#L19 and https://github.com/k3s-io/helm-controller/blob/master/pkg/apis/helm.cattle.io/v1/types.go#L49.
Hi
I just started reading the K3s documentation. I found that the most important letter was missing in the description of our K3s and opened PR. #283
Per k3s-io/k3s#6581 the --secrets-encryption
flag is no longer experimental, and the docs for managing secrets encryption doesn't mention it as experimental. But the reference for the Server Configuration still lists it under Experimental Options https://docs.k3s.io/reference/server-config#experimental-options
I can't quite figure out this page:
https://docs.k3s.io/cli/secrets-encrypt
I'm trying to figure out how to enable secrets encryption on my cluster. I didn't do anything special to turn it on, so I presume it is off.
Under 'Encryption key rotation', it says "Starting K3s without encryption and enabling it at a later time is currently not supported." Does this mean that it is impossible to enable secrets encryption on a cluster that didn't have it on from the start? Or does it mean that you have to restart the k3s
server in order to enable secrets encryption?
There is also a section called "Secrets Encryption Disable/Enable" - but this seems to presuppose that the --secrets-encrypt
flag is already set, I've disabled secrets encryption and I now want to re-enable it.
The release notes on the docs page are about 2 months out of date vs actual releases
For example it's missing 1.30 entirely: https://github.com/k3s-io/docs/tree/main/docs/release-notes
https://docs.k3s.io/release-notes/v1.29.X
A bot used to make PRs, maybe it's broken?
See last PR: #249
Is there any reason why Certificate Rotation is hidden away under Advanced Options ?
Surely its a fairly critical bit of sysadmin and not an advanced option ? Maybe it could go under "Installation" instead ?
Also, perhaps a bit of clarity on what breaks if the certificates go unrotated would not go amiss ? Equally I'm unclear why k3s needs a hard reboot and not a reload (SIGHUP) ?
There is no info on how to use NATS datastore with k3s in https://docs.k3s.io/datastore
As this is about Private Registry Configuration, the https://github.com/k3s-io/docs/blob/main/docs/installation/private-registry.md#with-tls and https://github.com/k3s-io/docs/blob/main/docs/installation/private-registry.md#without-tls examples might be misleading. In particular,
mirrors:
docker.io:
should be
mirrors:
mycustomreg.com:
A reader may skip over the first mirrors example and use an example from either TLS section. The result is an error "PullImage from image service failed" err="rpc error: code = NotFound desc = failed to pull and unpack image \"
that the image cannot be found.
Networking information is divided into two different sections:
I think that separation is strange and confusing. As a consequence we have related content spread across these two. We should think of the reasons why we need two different sections and have a clear separation for each of them. Right now it is not clear what goes where
[Moved from https://github.com/k3s-io/k3s/issues/8805, which I am now closing]
Thank you for the great work, and in advance for your consideration!
As the documentation says, nm-cloud-setup
must be disabled for installation to proceed. I didn't see this, and therefore struggled to understand why the service would not start. I inspected the startup commands, and found the reference to nm-cloud-setup
, and that led me to the conclusion it should be deleted.
"Well, problem solved, RTFM, right?" Indeed. And yet. I would like to suggest that installation prerequisites should not go into a section called Advanced Options / Configuration. I was just trying to do some quick and dirty testing, with nothing "advanced" on my mind! It would therefore not occur to me to read anything in a section with Advanced in the title. There is nothing to suggest I'd need to do this in the Quick Start Guide, nor in the Requirements section.
I would like to suggest the following:
nm-cloud-setup
check in the systemctl
script into the get.k3s.io
install script as well, perhaps along with checks for ufw
and firewalld
.EDIT: Oh, one more add: I discovered the SELinux enforcement with the default k3s-selinux
ruleset is not compatible with local-path-provisoner
; that should probably be called out as well. See here: rancher/local-path-provisioner#362
Thank you again!
I was chatting with someone last night about k3s, and I said, "well I'm pretty sure k3s is a shortening of KISS (keep it simple,)" and they said, "wouldn't that be k2s?" This made me wonder, why is it called "k3s", and I couldn't find any information about it on the website or documentation. Would you consider adding it to the FAQ?
Thanks!
Environmental Info:
K3s Version:
Node(s) CPU architecture, OS, and Version:
Cluster Configuration:
Describe the bug:
Browsing the documentation at
https://docs.k3s.io/faq
https://docs.k3s.io/cli/server
https://docs.k3s.io/cli/agent
Does not link to, or define what a server or agent is.
However, this is described in https://docs.k3s.io/architecture
The faq and cli documentation should link to the architecture page for words like agent and server, as the CLI and FAQ are more prominent entry places for a user than the architecture overview.
Steps To Reproduce:
Expected behavior:
Actual behavior:
Additional context / logs:
data-dir
and other flags need to be given "after" all subcommands for them to used properly. We need to warn the user about this in the docs.
i.e. k3s --data-dir=/tmp/k3s etcd-snapshot save
is invalid. You need k3s etcd-snapshot save --data-dir=/tmp/k3s
Page: https://www.rancher.com/products/k3s
There are 2 errors on this page:
In the text block under "Why K3s and Rancher?->A CNCF Sandbox Project" the description for this is missing the word "to", it should read:
"K3s was originally a Rancher project. In June 2020 Rancher donated K3s to the CNCF as a testament to the Rancher's commitment to the open source community"
In the text block under "Why K3s and Rancher?->Perfect for Edge" the word "available" is misspelled "avalible".
Aswome job for make kubernetes easier to all
I want to be able to find easy on k3s documentation how to auto redirect to https with traefik without modifing manifest
Like this way
kubectl apply -f httpstraefik.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
ports:
websecure:
tls:
enabled: true
web:
redirectTo: websecure
Its a common issue why people don't use the bundled traefik because they can't find how to make http to https redirection
Like we see in some forums
https://stackoverflow.com/questions/68575472/k3s-redirect-http-to-https/71989847#71989847
At present the docs do a good job of explaining how to harden k3s by using PSPs.
However, it appears that PSPs are deprecated as of Kubernetes 1.21
This likely means that the docs should be updated accordingly.
Unfortunate I'm far from a subject-matter expert in this area, so I can't viably propose a PR myself, hence I am just flagging this here.
When I read the existing documentation at https://docs.k3s.io/datastore about setting up a High Availability External DB there isn't any detail about how the database should be created and what permissions the user provided to k3s needs to have. I would suggest the Drupal database setup page as an excellent example of the details that should go into this documentation for both MySQL and PostgreSQL: https://www.drupal.org/docs/getting-started/installing-drupal/create-a-database#create-a-database-using-mysqlmariadb-commands
The requirements for k3s might actually be the exact same as for Drupal but I don't know that. Details like collation order and character set should be provided and an example database creation command are very helpful. Similarly providing a list of the specific permissions required for the database userid are also needed so the user doesn't end up assigning all permissions which has security implications.
Using the Drupal docs as a start an making any necessary modifications would be a great improvement to the k3s docs on the topic.
Environmental Info:
K3s Version:
v1.27.4+k3s1
Node(s) CPU architecture, OS, and Version:
N/A
Cluster Configuration:
N/A
Describe the bug:
As of kubernetes-dashboard v3.0.0-alpha0 there is a breaking change that causes the instructions on Kubernetes Dashboard to fail with the following error:
error: resource mapping not found for name: "selfsigned" namespace: "kubernetes-dashboard" from "https://raw.githubusercontent.com/kubernetes/dashboard/v3.0.0-alpha0/charts/kubernetes-dashboard.yaml": no matches for kind "Issuer" in version "cert-manager.io/v1"
ensure CRDs are installed first
See v3.0.0-alpha0 quoted below:
Starting from the release v3 of the Kubernetes Dashboard, the underlying architecture has changed, and it requires a clean installation. Please remove the previous installation first.
Kubernetes Dashboard now uses cert-manager and nginx-ingress-controller by default to work properly. Please make sure you have them installed in your cluster if you want to use a manifest-based installation path. The helm-based approach can install all required dependencies automatically for you if needed.
Expected Results
Update the documentation to add the required setups and configuration to deploy the latest version of the kubernetes-dashboard.
Workaround:
Deploy v2.7.0 of the kubernetes-dashboard.
Hi,
I would like to start translating the docs of K3s into Arabic, to allow Arabic speakers to understand better this project.
Best regards.
When creating the traefik-config.yaml in /var/lib/rancher/k3s/server/manifests/traefik-config.yaml
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
image:
name: traefik
tag: v2.8.5
forwardedHeaders:
enabled: true
trustedIPs:
- 10.0.0.0/8
ssl:
enabled: true
permanentRedirect: false
official guide needs to be updated as I get an image pull error and back off.
There must be an overlooked step that is missing from that guide.
Originally posted by @gavan1 in k3s-io/k3s#1313 (comment)
Need to document the improved templating functionality from
My understanding is that the default for --default-local-storage-path
is /var/lib/rancher/k3s/storage
and so if you want to change where local-path-provisioner saves the PVs you need to pass that to server instances, is that correct? If so, can that be added to the docs (I'm happy to make a PR if that helps), and perhaps the option of changing the storage path should be given on https://docs.k3s.io/storage#setting-up-the-local-storage-provider, given its a k3s-specific option?
https://docs.k3s.io/installation/kube-dashboard#deploying-the-kubernetes-dashboard
The dashboard of k8s has released the latest version v3.0.0-alpha0
, and the installation method has been updated. Using the current document method to install will prompt a 404 error.Maybe you should limit the installation to a specific version instead of getting it from the github release page.
The below is copied from https://github.com/rancher/docs/issues/3995.
Request Summary:
I found the page explaining private registry configuration confusing. It would help to change some of the examples and perhaps add some explanation so that people don't get confused.
Details:
In the With TLS section we have the following example:
mirrors:
docker.io:
endpoint:
- "https://mycustomreg.com:5000"
configs:
"mycustomreg:5000":
auth:
username: xxxxxx # this is the registry username
password: xxxxxx # this is the registry password
tls:
cert_file: # path to the cert file used in the registry
key_file: # path to the key file used in the registry
ca_file: # path to the ca file used in the registry
From this, I understand that a mirror, docker.io, is being configured. Image names that reference this mirror use the base URL https://mycustomreg.com:5000 to find and download the images. However, it is not clear to me how the configs section applies to the mirror. Presumably this section applies to the docker.io mirror, but I don't see anything that references docker.io in this section. All we have is "mycustomreg:5000", which doesn't occur anywhere in the mirrors section. Was "mycustomreg:5000" supposed to be "mycustomreg.com:5000"? Or was it supposed to be docker.io? Or does it somehow relate to the mirrors section in a way I'm not seeing? Or not at all? This is the first point of confusion for me.
The second point of confusion is about the use of the name docker.io in the examples. As far as I know, docker.io is the default registry. So by defining a docker.io mirror in the private registries, are we changing the default registry? If so, this should be described in the docs. If not, perhaps a less confusing name than docker.io could be used in the examples.
These comments also apply to the Without TLS section.
I was trying out the curl
command to download the airgap images for k3s from the documentation, and it threw this error:
$ sudo curl -L -O /var/lib/rancher/k3s/agent/images/k3s-airgap-images-amd64.tar.zst https://github.com/k3s-io/k3s/releases/download/v1.29.1-rc2%2Bk3s1/k3s-airgap-images-amd64.tar.zst
curl: (3) URL using bad/illegal format or missing URL
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
I have fixed this in a PR #243 . I'd appreciate if the awesome team at k3s would take a look at it.
Thank you
Hello,
Following the discussion linked above, I think this whole documentation page should be moved and redone : https://docs.k3s.io/upgrades/killall
There is no sense having this page in the upgrades section (as it is not needed for upgrades), maybe it could be a new docs section, or a paragraph in the FAQ ?
This page could also be expanded to give general instructions about stopping and starting the k3s service (without killing the pods) with a paragraph dedicated to the killall script as a special case.
https://docs.k3s.io/installation/configuration#configuration-file
i was looking at the ways we can configure a k3s node and i would like to give a try to the config.yaml method
however it's not exactly clear what can i include on this file,
there is a good example about the merging of configuration of different sources
but i am having some doubts about if every flag or setting is available:
for example:
can i specify k3s version? (i believe not but that would be cool)
can i put flannel related settings, and specify ip addresses?
can i specify the node is a server or an agent or is it assumed? [server: true; agent:true]?
does the file allows to configure server only flags? (i guess its possible due to have the cluster init example)
does it support envs or subshelling from other sources? (like $(ip address | grep....))
my actual question is, can i have every k3s setting in this file?
I wanted to suggest to add to the table the corresponding config keys in the following page,
or since they all have the same names, if these are available as a config.yaml setting
https://docs.k3s.io/reference/env-variables
There is no reference to the k3s-ansible repository in the documentation.
This could be added in the Quick-Start page. This would allow easy bootstrap of multi-node cluster. Since the k3s-ansible repo is already part of the k3s-io system, i don't see how it would be a problem.
https://docs.k3s.io/upgrades/automated
I wanted to suggest adding the well known toleration for control plane by default
node-role.kubernetes.io/control-plane
https://kubernetes.io/docs/reference/labels-annotations-taints/#node-role-kubernetes-io-control-plane-taint
and update the examples to provide a more recent kubernetes version..
i can make a PR if this decision was not on purpose, i mean, i guess by default k3s doesn't have this toleration
but having it so causes no harm?
Original issue: https://github.com/rancher/docs/issues/2351
Copying over relevant bits from original issue since K3s docs have been moved off the old rancher/docs repo.
During the validation for this issue: rancher/rancher#25495 (comment)
I found there are some improvements we can do for the documentation for k3s air-gap installation: https://rancher.com/docs/k3s/latest/en/installation/airgap/
Prepare Images Directory
, add some explanation for why we still need this when we have a private registryCreate Registry YAML
, add some explanation for when and how the private registry will be used by k3s, why the private registry is not used for k3s' own imagesInstall K3s
, add a sample command when installing k3s with an external databaseI’ve just set up a new machine with the latest, stable Ubuntu 22.04. This release has nftables enabled by default, so requires the same fix as Raspberry Pi OS to revert to legacy iptables. I would suggest updating the requirements page to reflect this and perhaps updating the known issues section.
Note that this issue surfaced when trying to define Ingress resources to access a simple HTTP service (whoami) on a default k3s installation (i.e. as per the quick start instructions). Matching requests to the public traefik ports (80 & 443) were failing with a 504 timeout error because it couldn’t route the network traffic, even though the service was accessible via the private k3s network address associated with the service. Enabling traefik debug didn’t shed any light on the problem, and I spent lots of hours trying to track down the issue.
I’ve attached a file containing deployment, service and ingress definitions to reproduce, assuming whoami
is added to /etc/hosts
as an alternative hostname for the machine.
whoami.txt
Ran kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/latest/download/system-upgrade-controller.yaml --insecure-skip-tls-verify
Output:
kubectl get deploy -n system-upgrade --insecure-skip-tls-verify
NAME READY UP-TO-DATE AVAILABLE AGE
system-upgrade-controller 1/1 1 1 21m
Now ran my plans.tf file:
resource "kubernetes_manifest" "server_plan" {
manifest = {
apiVersion = "upgrade.cattle.io/v1"
kind = "Plan"
metadata = {
name = "server-plan"
namespace = "system-upgrade"
}
spec = {
concurrency = 1
cordon = true
nodeSelector = {
matchExpressions = [
{
key = "node-role.kubernetes.io/control-plane"
operator = "Exists"
}
]
}
tolerations = [
{
operator = "Exists"
effect = "NoSchedule"
}
]
serviceAccountName = "system-upgrade"
upgrade = {
image = "rancher/k3s-upgrade"
}
channel = "https://update.k3s.io/v1-release/channels/stable"
}
}
}
resource "kubernetes_manifest" "agent_plan" {
manifest = {
apiVersion = "upgrade.cattle.io/v1"
kind = "Plan"
metadata = {
name = "agent-plan"
namespace = "system-upgrade"
}
spec = {
concurrency = 1
cordon = true
nodeSelector = {
matchExpressions = [
{
key = "node-role.kubernetes.io/control-plane"
operator = "DoesNotExist"
}
]
}
tolerations = [
{
operator = "Exists"
effect = "NoSchedule"
}
]
prepare = {
args = ["prepare", "server-plan"]
image = "rancher/k3s-upgrade"
}
serviceAccountName = "system-upgrade"
upgrade = {
image = "rancher/k3s-upgrade"
}
channel = "https://update.k3s.io/v1-release/channels/stable"
}
}
}
On apply I get the error:
Error: Failed to determine GroupVersionResource for manifest
│
│ with kubernetes_manifest.server_plan,
│ on plans.tf line 1, in resource "kubernetes_manifest" "server_plan":
│ 1: resource "kubernetes_manifest" "server_plan" {
│
│ no matches for kind "Plan" in group "upgrade.cattle.io"
╵
╷
│ Error: Failed to determine GroupVersionResource for manifest
│
│ with kubernetes_manifest.agent_plan,
│ on plans.tf line 35, in resource "kubernetes_manifest" "agent_plan":
│ 35: resource "kubernetes_manifest" "agent_plan" {
│
│ no matches for kind "Plan" in group "upgrade.cattle.io"
Downgrades are NOT SUPPORTED and are not something that we recommend anyone to do. We have made some updates when using system-upgrade-controller now that will fail earlier in the process if a user attempts a downgrade on the cluster, but should document the potential pitfalls when having cordon: true
set in the upgrade plans.
Due to the drop of PSPs in v1.25, there needs to be clear documentation around how to perform an upgrade to this minor.
Similar to rancher/rke2-docs#111, we need to update the https://docs.k3s.io/installation/requirements#large-clusters with newer information. Additionally, the page should be reworked to consolidate the "large cluster" info with the "single node" information in the upper portion of the page.
On https://docs.k3s.io/installation/packaged-components it says:
K3s comes with a number of packaged components that are deployed as AddOns via the manifests directory:
coredns
,traefik
,local-storage
, andmetrics-server
.
And later:
Care should also be taken not to conflict with names in use by the default K3s packaged components, even if those components are disabled.
This requires that the list of packaged components is complete, but it is missing several names, like ccm
, rolebindings
, runtimes
.
The slack link on this page: https://docs.k3s.io/faq#im-having-an-issue-where-can-i-get-help, which is https://rancher-users.slack.com/archives/CGGQEHPPW takes me to a page where I needed to have suse or rancher email address to sign in.
The slack link on the main rancher.com page works. It sends me to: https://slack.rancher.io/
For air-gap installation, provide instructions on how to download and install container-selinux and k3-selinux package manually/offline.
[INFO] Skipping k3s download and verify
[INFO] Skipping installation of SELinux RPM
[ERROR] Failed to find the k3s-selinux policy, please install:
dnf install -y container-selinux
dnf install -y https://rpm.rancher.io/k3s/stable/common/centos/8/noarch/
Request Summary:
In current hardening guide example, we don't have any --audit-policy-file
parameter defined in the k3s server
command example.
Details:
If we don't provide any --audit-policy-file
, the audit logging is disabled (default behaviour of k8s).
Having some kind of "default" audit.k8s.io/v1:Policy
file and the parameter in the command example could be a better description of "how to enable it", especially when we enable every other parameters about audit-log
.
Thank you
BTW, if you provide me some information about what we should do on this one, I would be able to do a PR.
NOTE: Creation requested by @jtravee in this comment, as a clone of https://github.com/rancher/docs/issues/3656
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.