little help needed about hetzner-k3s HOT 27 CLOSED

I think you need to add an annotation to the ingress resource to specify that you want to use the Nginx ingress as ingress controller. This was changed recently. So change your ingress resource like this:

...
metadata:
  name: hello-world
  annotations:
    "kubernetes.io/ingress.class": "nginx"
...

This try and let me know.

from hetzner-k3s.

Hi! Thanks for the reply.
Ive added the annotations lines to the ingress resource, but unfortunately it didn't change anything, i still get an 400 bad request when I go to "hello-world.162.55.152.65.nip.io" in this case. (I change this to the nip.io instead my own domain for troubleshooting purposes)

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello-world
  namespace: ingress-nginx
  annotations:
    kubernetes.io/ingress.class: ingress-nginx
spec:
  rules:
  - host: hello-world.162.55.152.65.nip.io
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: hello-world
            port:
              number: 80

When I describe the hello-world service, it gives me:

PS C:\kluster> kubectl describe service hello-world -n ingress-nginx
Name:              hello-world
Namespace:         ingress-nginx
Labels:            <none>
Annotations:       <none>
Selector:          app=hello-world
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.43.229.204
IPs:               10.43.229.204
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.4.7:80
Session Affinity:  None
Events:            <none>

When I describe the ingress of hello-world it gives me the correct host:

PS C:\kluster> kubectl describe ingress hello-world -n ingress-nginx
Name:             hello-world
Namespace:        ingress-nginx
Address:          10.0.0.8,162.55.152.65,2a01:4f8:1c1d:201::1
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                              Path  Backends
  ----                              ----  --------
  hello-world.162.55.152.65.nip.io  
                                    /   hello-world:80 (10.244.4.7:80)
Annotations:                        kubernetes.io/ingress.class: ingress-nginx
Events:
  Type    Reason  Age                    From                      Message
  ----    ------  ----                   ----                      -------
  Normal  Sync    3m41s (x3 over 4m56s)  nginx-ingress-controller  Scheduled for sync
  Normal  Sync    3m41s (x3 over 4m56s)  nginx-ingress-controller  Scheduled for sync

the pod hello-world and ingress are running both:

PS C:\kluster> kubectl get pods -n ingress-nginx
NAME                             READY   STATUS    RESTARTS   AGE
hello-world-678c699476-zlmhl     1/1     Running   0          24h
ingress-nginx-controller-m4rd4   1/1     Running   0          24h
ingress-nginx-controller-qsvpc   1/1     Running   0          24h

When I portforward it shows me this, so the pod is healthy..

hmm what am I missing here, I don't get it...

from hetzner-k3s.

Did you enable proxy protocol with the load balancer annotations? If yea you also need to enable it in the Mainz configmap.

from hetzner-k3s.

Hm I've added 3 hetzner Annotations, the cloud/location, cloud/name and use-private-ip
I don't think you mean this?

PS C:\kluster> kubectl -n ingress-nginx describe svc ingress-nginx-controller
Name:                     ingress-nginx-controller
Namespace:                ingress-nginx
Labels:                   app.kubernetes.io/component=controller
                          app.kubernetes.io/instance=ingress-nginx
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=ingress-nginx
                          app.kubernetes.io/version=1.0.5
                          helm.sh/chart=ingress-nginx-4.0.9
Annotations:              load-balancer.hetzner.cloud/location: nbg1
                          load-balancer.hetzner.cloud/name: kluster-ingress-nginx
                          load-balancer.hetzner.cloud/use-private-ip: true
                          meta.helm.sh/release-name: ingress-nginx
                          meta.helm.sh/release-namespace: ingress-nginx
Selector:                 app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.
kubernetes.io/name=ingress-nginx
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.43.131.110
IPs:                      10.43.131.110
LoadBalancer Ingress:     162.55.152.65, 2a01:4f8:1c1d:201::1, 10.0.0.8
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  30584/TCP
Endpoints:                10.244.2.4:80,10.244.4.5:80
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  31524/TCP
Endpoints:                10.244.2.4:443,10.244.4.5:443
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

from hetzner-k3s.

Can you share the full manifests of the various resources you've created?

from hetzner-k3s.

When you visit the IP directly, not the domain, what do you see?

from hetzner-k3s.

the error is the same as going to "hello-world.162.55.152.65.nip.io" or "162.55.152.65"

400 Bad Request
nginx

This is my ingress-nginx.yaml file:
https://file.re/2021/11/22/ingress-nginx/

and the hello-world yaml (including the deployment, service and ingress):
https://file.re/2021/11/22/hello-world/

So what I did are these steps (nothing more):

1. helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
2. helm repo update
3. helm upgrade --install --namespace ingress-nginx --create-namespace -f C:\kluster\ingress-nginx.yaml ingress-nginx ingress-nginx/ingress-nginx
4. kubectl apply -f hello-world.yaml

from hetzner-k3s.

Can you check the nginx logs? 400 status code happens when for example the proxy protocol is enabled on the load balancer but not in the nginx configmap, or when you are making https requests to an http backend or the other way around.

I am wondering if for some reason the proxy protocol is enabled on the load balancer.

Can you share the output of kubectl -n ingress-nginx get cm ingress-nginx-controller -oyaml?

You can try adding the setting use-proxy-protocol: "true" to that config map to see if the problem is related to the proxy protocol as I am suspecting. If you update the config map with that setting the nginx config will be automatically reloaded and you can try accessing the ingress resource again from the domain.

from hetzner-k3s.

This is the output:

PS C:\kluster> kubectl -n ingress-nginx get cm ingress-nginx-controller -oyaml
apiVersion: v1
data:
  allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: ingress-nginx
    meta.helm.sh/release-namespace: ingress-nginx
  creationTimestamp: "2021-11-21T19:59:16Z"
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/version: 1.0.5
    helm.sh/chart: ingress-nginx-4.0.9
  name: ingress-nginx-controller
  namespace: ingress-nginx
  resourceVersion: "1694"
  uid: 144f0834-f352-47f1-b167-74b644e9d2be

yes I will try to add:
load-balancer.hetzner.cloud/uses-proxyprotocol: 'true'
to the ingress-nginx.yaml.

from hetzner-k3s.

Make sure you edit the config map also, like I explained in the previous comment. Also check in the Hetzner Console if the proxy protocol is enabled on the load balancer, so to confirm.

from hetzner-k3s.

i think i need a little explaination to edit the config map, so it is not the 'uses-proxyprotocol' line?
This is what I have now:

controller:
  kind: DaemonSet
  service:
    annotations:
      load-balancer.hetzner.cloud/location: nbg1
      load-balancer.hetzner.cloud/name: kluster-ingress-nginx
      load-balancer.hetzner.cloud/use-private-ip: "true"
      load-balancer.hetzner.cloud/uses-proxyprotocol: 'true'

In the console, I cant find the proxy protocol
Edit: When I click on Service on the Loadbalancer, it shows me: enabled.

from hetzner-k3s.

Ok, the proxy protocol is enabled. Now run kubectl -n ingress-nginx edit cm ingress-nginx-controller to edit the config map I mentioned, and add use-proxy-protocol: "true" to the data section. BTW these issues are not related to the tool :)

from hetzner-k3s.

OK it is working! you've just made my day. thanks for your time.
for my own learning; is the proxy protocol only needed for exposing the real IP address of the client in applications that I deploy to the cluster?

from hetzner-k3s.

Nice! Glad you got it working. Yes, the proxy protocol is required to ensure you always get the actual IP of the user and not the IP of a load balancer in between (either the load balancer for the ingress or some internal loadbalacing in kubernetes).

If you don't care about that you can disable the proxy protocol but be sure you do so both in the load balancer annotations and in the config map.

from hetzner-k3s.

thanks for the explaination!
in the meanwhile Iam trying to deploy an oauth service in the cluster to be able to add some authentication later to the apps.
but when I search the internet for examples, it's still complex for me
May I ask what kind of authentication you are using (for private apps)? Do you mind sharing a good example of the deployment?

I notice that it is very difficult to find good documentation about this in general...
I'm coming from Docker compose and swarm and there's plenty of documentation about that! I'm a little disappointed with Kubernetes.

from hetzner-k3s.

What kid of applications?

from hetzner-k3s.

What kid of applications?

Like jackett (torrent tracker) for instance or homer (dashboard), they dont have their own authentication so i need something to make it private and protected.

from hetzner-k3s.

You can enable http basic authentication with the ingress resource. See https://kubernetes.github.io/ingress-nginx/examples/auth/basic/

from hetzner-k3s.

hm when I try to execute this command: htpasswd -c auth foo
the result is:

htpasswd : The term 'htpasswd' is not recognized as the name of a cmdlet, function, script file, or operable program.
 Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

I cannot use htpasswd in the cluster I guess?
I'm using Windows 11 which I connect to the cluster, and htpasswd is a Linux thing right? maybe this has something to do with it.
but the VMs in hetnzer are ubuntu right?

from hetzner-k3s.

htpasswd is only needed to generate the encoded string. If it's not available on Windows you can try some other utility, e.g. a quick googling showed this at the top https://htpasswdgenerator.com/download_htpasswd_generator.html

from hetzner-k3s.

ah nice it is working!
this is basic authentication right, it would be nicer to have something in place to use oauth2-proxy
i was hoping you had an example for deployment and ingress for an test application

Iam trying for days now to get oauth2-proxy working and created a ticket for help
kubernetes/ingress-nginx#7987
I just don't think I'll get a response anytime soon.

from hetzner-k3s.

At ingress level you can only do basic http authentication. Anything else must be implemented at application level I'm afraid. At least AFAIK.

from hetzner-k3s.

As far as I know the oauth2-proxy consists of a deployment, service and ingress.
if you have a particular application that you want to add authentication to, you need to add annotations to that ingress.

example ( but not 100% working yet ): https://github.com/jboesh/test

from hetzner-k3s.

Cool, I haven't come across that before :)

from hetzner-k3s.

basically this page. but its not working yet in my case
https://kubernetes.github.io/ingress-nginx/examples/auth/oauth-external-auth/

from hetzner-k3s.

this has nothing to do with the creating of the cluster,I was just wondering if you had something in place already
so you can close the ticket if you want, thanks for all the help!

from hetzner-k3s.

No problem. I'll close this one but let me know if you run into any issues. Cheers

from hetzner-k3s.