I installed fabric8 on top of kubernetes1.4.7 and I can't reach the URL for fabric8. I

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Kubernetes on private network,about nginxinc/kubernetes-ingress

Comments (38)

r4j4h commented on August 28, 2024 1

Incase it is not from the security context, are you using CNI for your cluster? If so the hostPort entries are ignored right now by CNI implementations (see this issue). If that is indeed your problem, try hostNetworking: true to make the container use the host's network instead of just the ports. This works in the meantime over CNI.

from kubernetes-ingress.

fattahmahran commented on August 28, 2024 1

Thanks a lot Michael you already helped a lot :-) Thanks Best regards,

On Wed, Dec 21, 2016 at 12:41 Michael Pleshakov ***@***.***> wrote: @abdoo <https://github.com/abdoo> That's great that you found out the root cause. I'm not sure If I can help — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz-D_o7gPRMmTI-pW0fH2mTZRmPxjks5rKWTVgaJpZM4LPvNH> . --

Thanks, Best regards,

from kubernetes-ingress.

fattahmahran commented on August 28, 2024 1

Hi Michael, It is working after I added this parameter. thanks again I have the same on ubuntu server and it is running perfectly without this parameter. Thanks again.

…

On Wed, Jan 4, 2017 at 9:47 AM, Michael Pleshakov ***@***.***> wrote: @abdoo <https://github.com/abdoo> Sorry for the confusion. By the controller I meant the NGINX Ingress controller. Could you change its deployment by adding hostNetwork: true to the Pod spec? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhzxwOb4a5QDQfOfvE7Nm927MecYC4ks5rO7D0gaJpZM4LPvNH> .

-- Thanks Best Regards, AbdelFattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo
My apologies for the delayed response. From my understanding, this file is used to deploy the controller https://github.com/fabric8io/fabric8-devops/blob/master/ingress-nginx/src/main/fabric8/deployment.yml As you can see, ports 80 and 443 on the container are mapped to ports 80 and 443 on the host machine. NGINX can't start probably because one or both of those ports are taken by some other application or a container running on that machine.

Unfortunately, we don't report the exact error when NGINX isn't able start. This will be fixed.

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

…

On Mon, Dec 19, 2016 at 8:42 AM, Michael Pleshakov ***@***.*** > wrote: @abdoo <https://github.com/abdoo> My apologies for the delayed response. From my understanding, this file is used to deploy the controller https://github.com/fabric8io/ fabric8-devops/blob/master/ingress-nginx/src/main/fabric8/deployment.yml As you can see, ports 80 and 443 on the container are mapped to ports 80 and 443 on the host machine. NGINX can't start probably because one or both of those ports are taken by some other application or a container running on that machine. Unfortunately, we don't report the exact error when NGINX isn't able start. This will be fixed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz4_l7jwLtoIwt1o-lmUuZtM28WWJks5rJonIgaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

also I checked these two ports 80 and 443 and they not being used on the server. root@kuber-master:~# telnet 0 80 Trying 0.0.0.0... Connected to 0. Escape character is '^]'. Connection closed by foreign host. root@kuber-master:~# telnet 0 443 Trying 0.0.0.0... Connected to 0. Escape character is '^]'. Connection closed by foreign host. root@kuber-master:~# On Mon, Dec 19, 2016 at 10:21 AM, Abd El-Fattah Mahran < [email protected]> wrote:

…

Thanks for the response. I try to check what's using these ports. I have another question related to this. my servers are running on private Env. so I need proxy to connect to Internet. I configured all applications on these servers to use this proxy (docker, curl..etc). does NGINX needs to be configured to use this proxy as well? Thanks, best regards, On Mon, Dec 19, 2016 at 8:42 AM, Michael Pleshakov < ***@***.***> wrote: > @abdoo <https://github.com/abdoo> > My apologies for the delayed response. From my understanding, this file > is used to deploy the controller https://github.com/fabric8io/f > abric8-devops/blob/master/ingress-nginx/src/main/fabric8/deployment.yml > As you can see, ports 80 and 443 on the container are mapped to ports 80 > and 443 on the host machine. NGINX can't start probably because one or both > of those ports are taken by some other application or a container running > on that machine. > > Unfortunately, we don't report the exact error when NGINX isn't able > start. This will be fixed. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#101 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAMhz4_l7jwLtoIwt1o-lmUuZtM28WWJks5rJonIgaJpZM4LPvNH> > . > -- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo

I try to check what's using these ports. I have another question related to
this. my servers are running on private Env. so I need proxy to connect to
Internet. I configured all applications on these servers to use this proxy
(docker, curl..etc). does NGINX needs to be configured to use this proxy as
well?

NGINX doesn't require outbound Internet connectivity so there's no need to configure it for NGINX.

also I checked these two ports 80 and 443 and they not being used on the
server.

I think it's the issue when your Kubernetes environment doesn't allow to create containers that expose privileged ports on the host. Could you try to add

   securityContext:
      privileged: true

to the NGINX deployment yaml file and then redeploy the controller, similar to https://github.com/nginxinc/kubernetes-ingress/pull/53/files ?

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

Hi Michael, this is the information I have from the container spec: containers: - args: - -v=3 - -nginx-configmaps=fabric8-system/nginx-config image: nginxdemos/nginx-ingress:0.3.1 imagePullPolicy: IfNotPresent name: nginx-ingress ports: - containerPort: 80 hostPort: 80 name: http protocol: TCP - containerPort: 443 hostPort: 443 name: https protocol: TCP resources: {} terminationMessagePath: /dev/termination-log dnsPolicy: ClusterFirst nodeSelector: fabric8.io/externalIP: "true" restartPolicy: Always securityContext: {} terminationGracePeriodSeconds: 30 I tried to add the information like that spec: containers: - args: - -v=3 - -nginx-configmaps=fabric8-system/nginx-config image: nginxdemos/nginx-ingress:0.3.1 imagePullPolicy: IfNotPresent name: nginx-ingress ports: - containerPort: 80 hostPort: 80 name: http protocol: TCP securityContext: privileged: true - containerPort: 443 hostPort: 443 name: https protocol: TCP securityContext: privileged: true resources: {} terminationMessagePath: /dev/termination-log dnsPolicy: ClusterFirst nodeSelector: fabric8.io/externalIP: "true" restartPolicy: Always securityContext: {} terminationGracePeriodSeconds: 30 I got this error A copy of your changes has been stored to "/tmp/kubectl-edit-fplln.yaml" found invalid field securityContext for v1.ContainerPort found invalid field securityContext for v1.ContainerPort found invalid field privileged for v1.ContainerPort Can you guide is this is the correct place to add? Thanks, best regards,

…

On Mon, Dec 19, 2016 at 12:07 PM, Michael Pleshakov < ***@***.***> wrote: @abdoo <https://github.com/abdoo> I try to check what's using these ports. I have another question related to this. my servers are running on private Env. so I need proxy to connect to Internet. I configured all applications on these servers to use this proxy (docker, curl..etc). does NGINX needs to be configured to use this proxy as well? NGINX doesn't require outbound Internet connectivity so there's no need to configure it for NGINX. also I checked these two ports 80 and 443 and they not being used on the server. I think it's the issue when your Kubernetes environment doesn't allow to create containers that expose privileged ports on the host. Could you try to add securityContext: privileged: true to the NGINX deployment yaml file and then redeploy the controller, similar to https://github.com/nginxinc/kubernetes-ingress/pull/53/files ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhzz3yA2pWY2LQWr8UyCLDdYLAwRRWks5rJrm_gaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

hi Jasmine, I'm not sure if Im using it or not. I just checked ubuntu/util.sh and I found this code. function create-kubelet-opts() { if [ -n "$7" ] ; then cni_opts=" --network-plugin=cni --network-plugin-dir=/etc/cni/net.d" else cni_opts="" fi cat <<EOF > ~/kube/default/kubelet KUBELET_OPTS="\ --hostname-override=${1} \ --api-servers=http://${2}:8080 \ --logtostderr=true \ --cluster-dns=${3} \ --cluster-domain=${4} \ --config=${5} \ --allow-privileged=true $cni_opts" EOF } I'm not sure which condition happened?

…

On Mon, Dec 19, 2016 at 1:38 PM, Jasmine Hegman ***@***.***> wrote: Incase it is not from the security context, are you using CNI for your cluster? If so the hostPort entries are ignored right now by CNI implementations (see this issue <kubernetes/kubernetes#31307 (comment)>). If that is indeed your problem, try hostNetworking: true to make the container use the host's network instead of just the ports. This works in the meantime over CNI. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz_4Ow370rzAtjDOfzCEm_rCSPHo8ks5rJs80gaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo
securityContext must be put one level up into the Container spec, not the Port spec.

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

Thanks a million Michael Pod is Running now :-) I1219 19:29:14.620935 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1219 19:29:14.620984 1 utils.go:70] Syncing kube-system/kube-scheduler I1219 19:29:14.621007 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1219 19:29:14.645287 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1219 19:29:14.645334 1 utils.go:70] Syncing kube-system/kube-controller-manager I1219 19:29:14.645344 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1219 19:29:16.634414 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1219 19:29:16.634486 1 utils.go:70] Syncing kube-system/kube-scheduler I1219 19:29:16.634498 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1219 19:29:16.657874 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1219 19:29:16.657916 1 utils.go:70] Syncing kube-system/kube-controller-manager I1219 19:29:16.657926 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1219 19:29:18.642477 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1219 19:29:18.642554 1 utils.go:70] Syncing kube-system/kube-scheduler I1219 19:29:18.642575 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1219 19:29:18.687874 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1219 19:29:18.687969 1 utils.go:70] Syncing kube-system/kube-controller-manager I1219 19:29:18.687992 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1219 19:29:20.651936 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1219 19:29:20.652033 1 utils.go:70] Syncing kube-system/kube-scheduler I1219 19:29:20.652053 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1219 19:29:20.699236 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1219 19:29:20.699307 1 utils.go:70] Syncing kube-system/kube-controller-manager I1219 19:29:20.699325 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1219 19:29:22.668951 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1219 19:29:22.669028 1 utils.go:70] Syncing kube-system/kube-scheduler I1219 19:29:22.669039 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1219 19:29:22.710568 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1219 19:29:22.710606 1 utils.go:70] Syncing kube-system/kube-controller-manager I1219 19:29:22.710616 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1219 19:29:24.679679 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1219 19:29:24.679790 1 utils.go:70] Syncing kube-system/kube-scheduler I1219 19:29:24.679962 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1219 19:29:24.722250 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1219 19:29:24.722304 1 utils.go:70] Syncing kube-system/kube-controller-manager I1219 19:29:24.722338 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager

…

On Mon, Dec 19, 2016 at 2:14 PM, Michael Pleshakov ***@***.*** > wrote: @abdoo <https://github.com/abdoo> securityContext must be put one level up into the Container spec, not the Port spec. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz4Oa7BsSPdEV0BgXgGeE73oqh-XSks5rJteigaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

Hi Michael, one more thing, now when I'm trying to access http://fabric8.default.10.103.12.20.nip.io/ it returns 502 bad gateway (nginx/1.11.4) On Mon, Dec 19, 2016 at 2:33 PM, Abd El-Fattah Mahran < [email protected]> wrote:

…

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo

NGINX returns the bad gateway error when there is no healthy upstream servers available. Please double check that the containers of the service you are exposing via NGINX are successfully created and running properly. Checking the controller logs also night help to determine the problem.

Additionally, I suggest to try to use the latest version of the controller, which is nginxdemos/nginx-ingress:0.6.0

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

hi Michael, all services are running normally. root@kuber-master:~# kubectl get pods NAME READY STATUS RESTARTS AGE configmapcontroller-3703242456-s914e 1/1 Running 0 5h exposecontroller-1115949590-fuxxw 1/1 Running 0 5h fabric8-3192130638-0fue6 2/2 Running 0 5h fabric8-docker-registry-1981087684-rr84r 1/1 Running 0 5h fabric8-forge-3730694965-ldqb6 1/1 Running 0 5h gogs-1853259817-dxkd0 1/1 Running 0 5h jenkins-1661962136-9ugls 1/1 Running 0 5h nexus-3628846063-ejfcx 1/1 Running 0 5h root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6 Error from server: a container name must be specified for pod fabric8-3192130638-0fue6, choose one of: [fabric8 jenkinshift] root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6 fabric8 2016/12/19 20:59:01 Creating config file: /site/osconsole/config.basic.js.tmpl => /site/osconsole/config.js 2016/12/19 20:59:01 2016/12/19 20:59:52 Creating service proxy: /k8s/ => https://192.168.3.1:443/ 2016/12/19 20:59:52 Listening on :9090 2016/12/19 20:59:52 ^C root@kuber-master:~# kubectl logs -f fabric8-3192130638-0fue6 jenkinshift 2016-12-19 20:59:02.092742 I | Invoking Jenkins on URL http://jenkins/ 2016-12-19 20:59:02.092919 I | About to query Kubernetes 2016-12-19 20:59:02.133939 I | Using namespace default 2016-12-19 20:59:02.134879 I | jenkinshift start listening on localhost:9191 ^C root@kuber-master:~# kubectl logs -f gogs-1853259817-dxkd0 2016/12/19 20:58:12 Creating admin user: true 2016/12/19 20:58:12 [T] Custom path: /opt/gogs/custom 2016/12/19 20:58:12 [T] Log path: /app/gogs/data/logs 2016/12/19 20:58:12 [I] Gogs: Go Git Service 2016/12/19 20:58:15 Custom path: /opt/gogs/custom 2016/12/19 20:58:15 Admin account created: gogsadmin 2016/12/19 20:58:15 [T] Custom path: /opt/gogs/custom 2016/12/19 20:58:15 [T] Log path: /app/gogs/data/logs 2016/12/19 20:58:15 [I] Gogs: Go Git Service 0.9.97.0901 ^C *I tried to use 0.6.0 and it didn't work* I1220 02:31:54.941108 1 main.go:37] Starting NGINX Ingress controller Version 0.6.0 I1220 02:31:54.942405 1 nginx.go:251] Writing NGINX conf to /etc/nginx/nginx.conf user nginx; worker_processes 1; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; server_names_hash_max_size 512; map $http_upgrade $connection_upgrade { default upgrade; '' close; } include /etc/nginx/conf.d/*.conf; } I1220 02:31:54.943303 1 nginx.go:269] The main NGINX configuration file had been updated I1220 02:31:54.943508 1 nginx.go:224] executing nginx F1220 02:31:54.959240 1 nginx.go:207] Failed to start nginx spec: containers: - args: - -v=3 - -nginx-configmaps=fabric8-system/nginx-config image: nginxdemos/nginx-ingress:0.6.0 imagePullPolicy: IfNotPresent name: nginx-ingress ports: - containerPort: 80 hostPort: 80 name: http protocol: TCP - containerPort: 443 hostPort: 443 name: https protocol: TCP resources: {} securityContext: privileged: true terminationMessagePath: /dev/termination-log dnsPolicy: ClusterFirst nodeSelector: fabric8.io/externalIP: "true" restartPolicy: Always securityContext: {} Thanks, best regards,

…

On Mon, Dec 19, 2016 at 4:32 PM, Michael Pleshakov ***@***.*** > wrote: @abdoo <https://github.com/abdoo> NGINX returns the bad gateway error when there is no healthy upstream servers available. Please double check that the containers of the service you are exposing via NGINX are successfully created and running properly. Checking the controller logs also night help to determine the problem. Additionally, I suggest to try to use the latest version of the controller, which is nginxdemos/nginx-ingress:0.6.0 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz30UqpXJ9KOxj737SnTr0qV6ksxXks5rJvfUgaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

root@kuber-master:~# kubectl get services NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE fabric8 192.168.3.182 <none> 80/TCP 5m fabric8-docker-registry 192.168.3.113 <none> 80/TCP 5m fabric8-forge 192.168.3.134 <none> 80/TCP 5m gogs 192.168.3.252 <none> 80/TCP 5m gogs-ssh 192.168.3.123 <none> 22/TCP 5m jenkins 192.168.3.194 <none> 80/TCP 5m jenkins-jnlp 192.168.3.32 <none> 50000/TCP 5m jenkinshift 192.168.3.8 <none> 80/TCP 5m kubernetes 192.168.3.1 <none> 443/TCP 3d mongo 192.168.3.201 <none> 27017/TCP 3d nexus 192.168.3.111 <none> 80/TCP 5m root@kuber-master:~# root@kuber-master:~# kubectl get endpoints NAME ENDPOINTS AGE fabric8 172.16.87.3:9090 5m fabric8-docker-registry 172.16.49.4:5000 5m fabric8-forge 172.16.97.4:8080 5m gogs 172.16.49.6:3000 5m gogs-ssh 172.16.49.6:2222 5m jenkins 5m jenkins-jnlp 5m jenkinshift 172.16.87.3:9191 5m kubernetes 10.103.12.20:6443 3d mongo <none> 3d nexus 172.16.49.2:8081 5m root@kuber-master:~# On Mon, Dec 19, 2016 at 10:02 PM, Abd El-Fattah Mahran < [email protected]> wrote:

…

hi Michael, all services are running normally. ***@***.***:~# kubectl get pods NAME READY STATUS RESTARTS AGE configmapcontroller-3703242456-s914e 1/1 Running 0 5h exposecontroller-1115949590-fuxxw 1/1 Running 0 5h fabric8-3192130638-0fue6 2/2 Running 0 5h fabric8-docker-registry-1981087684-rr84r 1/1 Running 0 5h fabric8-forge-3730694965-ldqb6 1/1 Running 0 5h gogs-1853259817-dxkd0 1/1 Running 0 5h jenkins-1661962136-9ugls 1/1 Running 0 5h nexus-3628846063-ejfcx 1/1 Running 0 5h ***@***.***:~# kubectl logs -f fabric8-3192130638-0fue6 Error from server: a container name must be specified for pod fabric8-3192130638-0fue6, choose one of: [fabric8 jenkinshift] ***@***.***:~# kubectl logs -f fabric8-3192130638-0fue6 fabric8 2016/12/19 20:59:01 Creating config file: /site/osconsole/config.basic.js.tmpl => /site/osconsole/config.js 2016/12/19 20:59:01 2016/12/19 20:59:52 Creating service proxy: /k8s/ => https://192.168.3.1:443/ 2016/12/19 20:59:52 Listening on :9090 2016/12/19 20:59:52 ^C ***@***.***:~# kubectl logs -f fabric8-3192130638-0fue6 jenkinshift 2016-12-19 20:59:02.092742 I | Invoking Jenkins on URL http://jenkins/ 2016-12-19 20:59:02.092919 I | About to query Kubernetes 2016-12-19 20:59:02.133939 I | Using namespace default 2016-12-19 20:59:02.134879 I | jenkinshift start listening on localhost:9191 ^C ***@***.***:~# kubectl logs -f gogs-1853259817-dxkd0 2016/12/19 20:58:12 Creating admin user: true 2016/12/19 20:58:12 [T] Custom path: /opt/gogs/custom 2016/12/19 20:58:12 [T] Log path: /app/gogs/data/logs 2016/12/19 20:58:12 [I] Gogs: Go Git Service 2016/12/19 20:58:15 Custom path: /opt/gogs/custom 2016/12/19 20:58:15 Admin account created: gogsadmin 2016/12/19 20:58:15 [T] Custom path: /opt/gogs/custom 2016/12/19 20:58:15 [T] Log path: /app/gogs/data/logs 2016/12/19 20:58:15 [I] Gogs: Go Git Service 0.9.97.0901 ^C *I tried to use 0.6.0 and it didn't work* I1220 02:31:54.941108 1 main.go:37] Starting NGINX Ingress controller Version 0.6.0 I1220 02:31:54.942405 1 nginx.go:251] Writing NGINX conf to /etc/nginx/nginx.conf user nginx; worker_processes 1; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; server_names_hash_max_size 512; map $http_upgrade $connection_upgrade { default upgrade; '' close; } include /etc/nginx/conf.d/*.conf; } I1220 02:31:54.943303 1 nginx.go:269] The main NGINX configuration file had been updated I1220 02:31:54.943508 1 nginx.go:224] executing nginx F1220 02:31:54.959240 1 nginx.go:207] Failed to start nginx spec: containers: - args: - -v=3 - -nginx-configmaps=fabric8-system/nginx-config image: nginxdemos/nginx-ingress:0.6.0 imagePullPolicy: IfNotPresent name: nginx-ingress ports: - containerPort: 80 hostPort: 80 name: http protocol: TCP - containerPort: 443 hostPort: 443 name: https protocol: TCP resources: {} securityContext: privileged: true terminationMessagePath: /dev/termination-log dnsPolicy: ClusterFirst nodeSelector: fabric8.io/externalIP: "true" restartPolicy: Always securityContext: {} Thanks, best regards, On Mon, Dec 19, 2016 at 4:32 PM, Michael Pleshakov < ***@***.***> wrote: > @abdoo <https://github.com/abdoo> > > NGINX returns the bad gateway error when there is no healthy upstream > servers available. Please double check that the containers of the service > you are exposing via NGINX are successfully created and running properly. > Checking the controller logs also night help to determine the problem. > > Additionally, I suggest to try to use the latest version of the > controller, which is nginxdemos/nginx-ingress:0.6.0 > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#101 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAMhz30UqpXJ9KOxj737SnTr0qV6ksxXks5rJvfUgaJpZM4LPvNH> > . > -- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo
Could you also list Ingress resources?

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

Hi Michael, root@kuber-master:~# kubectl get ing NAME HOSTS ADDRESS PORTS AGE fabric8 fabric8.default.10.103.12.20.nip.io 80 16h fabric8-docker-registry fabric8-docker-registry.default.10.103.12.20.nip.io 80 16h fabric8-forge fabric8-forge.default.10.103.12.20.nip.io 80 16h gogs gogs.default.10.103.12.20.nip.io 80 16h gogs-ssh gogs-ssh.default.10.103.12.20.nip.io 80 16h jenkins jenkins.default.10.103.12.20.nip.io 80 16h jenkins-jnlp jenkins-jnlp.default.10.103.12.20.nip.io 80 16h jenkinshift jenkinshift.default.10.103.12.20.nip.io 80 16h nexus nexus.default.10.103.12.20.nip.io 80 16h root@kuber-master:~# Thanks, best regards,

…

On Tue, Dec 20, 2016 at 12:16 PM, Michael Pleshakov < ***@***.***> wrote: @abdoo <https://github.com/abdoo> Could you also list Ingress resources? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz76AqZwQbpO_pjhzAWrHFAlzx4qIks5rKA1rgaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo The problematic one is http://fabric8.default.10.103.12.20.nip.io ?

Could you do the following:

$ kubectl get -o yaml ing fabric8
$ kubectl describe svc fabric8

Could you also looks through NGINX logs and find the generated config related to the fabric8 ingress?

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

Hi Michael, actually all of them giving "502 bad gateway". root@kuber-master:~# kubectl get -oyaml ing fabric8 apiVersion: extensions/v1beta1 kind: Ingress metadata: creationTimestamp: 2016-12-20T02:55:38Z generation: 1 labels: provider: fabric8 name: fabric8 namespace: default resourceVersion: "549512" selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/fabric8 uid: c9b294b7-c65f-11e6-b3d9-0050560116bd spec: rules: - host: fabric8.default.10.103.12.20.nip.io http: paths: - backend: serviceName: fabric8 servicePort: 80 status: loadBalancer: {} root@kuber-master:~# root@kuber-master:~# kubectl describe svc fabric8 Name: fabric8 Namespace: default Labels: expose=true group=io.fabric8.apps project=fabric8-console provider=fabric8 version=2.2.199 Selector: expose=true,group=io.fabric8.platform.console,project=fabric8-console,provider=fabric8 Type: ClusterIP IP: 192.168.3.182 Port: <unset> 80/TCP Endpoints: 172.16.87.3:9090 Session Affinity: None No events.root@kuber-master:~# Attached logs for ingress pod while starting. Thanks, best regards,

…

On Tue, Dec 20, 2016 at 3:12 PM, Michael Pleshakov ***@***.*** > wrote: @abdoo <https://github.com/abdoo> The problematic one is http://fabric8.default.10.103.12.20.nip.io ? Could you do the following: $ kubectl get -o yaml ing fabric8 $ kubectl describe svc fabric8 Could you also looks through NGINX logs and find the generated config related to the fabric8 ingress? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz-8He6BnmSDsBeReNG1QeEdrwQzBks5rKDbLgaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran root@kuber-master:~# kubectl -n fabric8-system get podslogs -f ingress-nginx-2015555637-tayf7 I1220 20:19:43.107886 1 nginx.go:240] Writing NGINX conf to /etc/nginx/nginx.conf user nginx; worker_processes 1; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; server_names_hash_max_size 512; map $http_upgrade $connection_upgrade { default upgrade; '' close; } include /etc/nginx/conf.d/*.conf; } I1220 20:19:43.112387 1 nginx.go:258] The main NGINX configuration file had been updated I1220 20:19:43.112593 1 nginx.go:213] executing nginx I1220 20:19:43.257849 1 controller.go:143] Adding endpoints: kube-scheduler I1220 20:19:43.257883 1 controller.go:143] Adding endpoints: jenkins I1220 20:19:43.257894 1 controller.go:143] Adding endpoints: kube-controller-manager I1220 20:19:43.257904 1 controller.go:143] Adding endpoints: kube-dns I1220 20:19:43.257914 1 controller.go:143] Adding endpoints: kubernetes-dashboard I1220 20:19:43.257927 1 controller.go:143] Adding endpoints: gogs-ssh I1220 20:19:43.257936 1 controller.go:143] Adding endpoints: jenkins-jnlp I1220 20:19:43.257948 1 controller.go:143] Adding endpoints: nexus I1220 20:19:43.257972 1 controller.go:143] Adding endpoints: mongo I1220 20:19:43.257984 1 controller.go:143] Adding endpoints: kubernetes I1220 20:19:43.257993 1 controller.go:143] Adding endpoints: fabric8 I1220 20:19:43.258002 1 controller.go:143] Adding endpoints: fabric8-docker-registry I1220 20:19:43.258011 1 controller.go:143] Adding endpoints: fabric8-forge I1220 20:19:43.258036 1 controller.go:143] Adding endpoints: gogs I1220 20:19:43.258049 1 controller.go:143] Adding endpoints: jenkinshift I1220 20:19:43.258064 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:43.258077 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:43.258090 1 utils.go:70] Syncing default/jenkins I1220 20:19:43.258099 1 controller.go:275] Syncing endpoints default/jenkins I1220 20:19:43.258107 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:43.258113 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:19:43.258129 1 utils.go:70] Syncing kube-system/kube-dns I1220 20:19:43.258138 1 controller.go:275] Syncing endpoints kube-system/kube-dns I1220 20:19:43.258146 1 utils.go:70] Syncing kube-system/kubernetes-dashboard I1220 20:19:43.258155 1 controller.go:275] Syncing endpoints kube-system/kubernetes-dashboard I1220 20:19:43.258162 1 utils.go:70] Syncing default/gogs-ssh I1220 20:19:43.258171 1 controller.go:275] Syncing endpoints default/gogs-ssh I1220 20:19:43.258178 1 utils.go:70] Syncing default/jenkins-jnlp I1220 20:19:43.258183 1 controller.go:275] Syncing endpoints default/jenkins-jnlp I1220 20:19:43.258190 1 utils.go:70] Syncing default/nexus I1220 20:19:43.258195 1 controller.go:275] Syncing endpoints default/nexus I1220 20:19:43.258203 1 utils.go:70] Syncing default/mongo I1220 20:19:43.258208 1 controller.go:275] Syncing endpoints default/mongo I1220 20:19:43.258215 1 utils.go:70] Syncing default/kubernetes I1220 20:19:43.258220 1 controller.go:275] Syncing endpoints default/kubernetes I1220 20:19:43.258227 1 utils.go:70] Syncing default/fabric8 I1220 20:19:43.258232 1 controller.go:275] Syncing endpoints default/fabric8 I1220 20:19:43.258239 1 utils.go:70] Syncing default/fabric8-docker-registry I1220 20:19:43.258244 1 controller.go:275] Syncing endpoints default/fabric8-docker-registry I1220 20:19:43.258251 1 utils.go:70] Syncing default/fabric8-forge I1220 20:19:43.258256 1 controller.go:275] Syncing endpoints default/fabric8-forge I1220 20:19:43.258263 1 utils.go:70] Syncing default/gogs I1220 20:19:43.258268 1 controller.go:275] Syncing endpoints default/gogs I1220 20:19:43.258275 1 utils.go:70] Syncing default/jenkinshift I1220 20:19:43.258280 1 controller.go:275] Syncing endpoints default/jenkinshift I1220 20:19:43.263305 1 controller.go:117] Adding service: kubernetes I1220 20:19:43.263369 1 controller.go:368] ignoring service kubernetes: No ingress for service kubernetes I1220 20:19:43.263402 1 controller.go:117] Adding service: kube-dns I1220 20:19:43.263410 1 controller.go:368] ignoring service kube-dns: No ingress for service kube-dns I1220 20:19:43.263420 1 controller.go:117] Adding service: fabric8 I1220 20:19:43.263426 1 controller.go:368] ignoring service fabric8: No ingress for service fabric8 I1220 20:19:43.263440 1 controller.go:117] Adding service: gogs I1220 20:19:43.263446 1 controller.go:368] ignoring service gogs: No ingress for service gogs I1220 20:19:43.263458 1 controller.go:117] Adding service: jenkins-jnlp I1220 20:19:43.263465 1 controller.go:368] ignoring service jenkins-jnlp: No ingress for service jenkins-jnlp I1220 20:19:43.263474 1 controller.go:117] Adding service: kubernetes-dashboard I1220 20:19:43.263480 1 controller.go:368] ignoring service kubernetes-dashboard: No ingress for service kubernetes-dashboard I1220 20:19:43.263497 1 controller.go:117] Adding service: fabric8-docker-registry I1220 20:19:43.263508 1 controller.go:368] ignoring service fabric8-docker-registry: No ingress for service fabric8-docker-registry I1220 20:19:43.263517 1 controller.go:117] Adding service: fabric8-forge I1220 20:19:43.263523 1 controller.go:368] ignoring service fabric8-forge: No ingress for service fabric8-forge I1220 20:19:43.263545 1 controller.go:117] Adding service: gogs-ssh I1220 20:19:43.263555 1 controller.go:368] ignoring service gogs-ssh: No ingress for service gogs-ssh I1220 20:19:43.263570 1 controller.go:117] Adding service: jenkins I1220 20:19:43.263586 1 controller.go:368] ignoring service jenkins: No ingress for service jenkins I1220 20:19:43.263596 1 controller.go:117] Adding service: jenkinshift I1220 20:19:43.263602 1 controller.go:368] ignoring service jenkinshift: No ingress for service jenkinshift I1220 20:19:43.263611 1 controller.go:117] Adding service: nexus I1220 20:19:43.263617 1 controller.go:368] ignoring service nexus: No ingress for service nexus I1220 20:19:43.263626 1 controller.go:117] Adding service: mongo I1220 20:19:43.263632 1 controller.go:368] ignoring service mongo: No ingress for service mongo I1220 20:19:43.264525 1 controller.go:178] Adding ConfigMap: nginx-config I1220 20:19:43.264555 1 utils.go:70] Syncing fabric8-system/nginx-config I1220 20:19:43.264562 1 controller.go:297] Syncing configmap fabric8-system/nginx-config I1220 20:19:43.264891 1 nginx.go:240] Writing NGINX conf to /etc/nginx/nginx.conf user nginx; worker_processes 1; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; sendfile on; #tcp_nopush on; keepalive_timeout 65; #gzip on; server_names_hash_max_size 1024; server_names_hash_bucket_size 256; map $http_upgrade $connection_upgrade { default upgrade; '' close; } include /etc/nginx/conf.d/*.conf; } I1220 20:19:43.265559 1 nginx.go:258] The main NGINX configuration file had been updated I1220 20:19:43.268010 1 controller.go:85] Adding Ingress: jenkins I1220 20:19:43.268038 1 controller.go:85] Adding Ingress: fabric8-docker-registry I1220 20:19:43.268049 1 controller.go:85] Adding Ingress: jenkins-jnlp I1220 20:19:43.268060 1 controller.go:85] Adding Ingress: gogs I1220 20:19:43.268072 1 controller.go:85] Adding Ingress: fabric8 I1220 20:19:43.268083 1 controller.go:85] Adding Ingress: fabric8-forge I1220 20:19:43.268092 1 controller.go:85] Adding Ingress: gogs-ssh I1220 20:19:43.268101 1 controller.go:85] Adding Ingress: jenkinshift I1220 20:19:43.268115 1 controller.go:85] Adding Ingress: nexus I1220 20:19:43.268128 1 utils.go:70] Syncing default/jenkins I1220 20:19:43.268138 1 controller.go:334] Syncing default/jenkins I1220 20:19:43.268146 1 controller.go:349] Adding or Updating Ingress: default/jenkins I1220 20:19:43.268286 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.268860 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-jenkins.conf upstream default-jenkins-jenkins.default.10.103.12.20.nip.io-jenkins { server 172.16.97.5:8080; } server { listen 80; server_name jenkins.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-jenkins-jenkins.default.10.103.12.20.nip.io-jenkins; } } I1220 20:19:43.281185 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.281221 1 nginx.go:213] executing nginx -t I1220 20:19:43.301186 1 nginx.go:213] executing nginx -s reload 2016/12/20 20:19:43 [notice] 18#18: signal process started I1220 20:19:43.317017 1 utils.go:70] Syncing default/fabric8-docker-registry I1220 20:19:43.317071 1 controller.go:334] Syncing default/fabric8-docker-registry I1220 20:19:43.317093 1 controller.go:349] Adding or Updating Ingress: default/fabric8-docker-registry I1220 20:19:43.317260 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.317999 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-fabric8-docker-registry.conf upstream default-fabric8-docker-registry-fabric8-docker-registry.default.10.103.12.20.nip.io-fabric8-docker-registry { server 172.16.49.4:5000; } server { listen 80; server_name fabric8-docker-registry.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-fabric8-docker-registry-fabric8-docker-registry.default.10.103.12.20.nip.io-fabric8-docker-registry; } } I1220 20:19:43.320623 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.320651 1 nginx.go:213] executing nginx -t I1220 20:19:43.343128 1 nginx.go:213] executing nginx -s reload 2016/12/20 20:19:43 [notice] 23#23: signal process started I1220 20:19:43.354240 1 utils.go:70] Syncing default/jenkins-jnlp I1220 20:19:43.354379 1 controller.go:334] Syncing default/jenkins-jnlp I1220 20:19:43.354481 1 controller.go:349] Adding or Updating Ingress: default/jenkins-jnlp I1220 20:19:43.354781 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.355683 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-jenkins-jnlp.conf upstream default-jenkins-jnlp-jenkins-jnlp.default.10.103.12.20.nip.io-jenkins-jnlp { server 172.16.97.5:50000; } server { listen 80; server_name jenkins-jnlp.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-jenkins-jnlp-jenkins-jnlp.default.10.103.12.20.nip.io-jenkins-jnlp; } } I1220 20:19:43.358216 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.358343 1 nginx.go:213] executing nginx -t I1220 20:19:43.374987 1 nginx.go:213] executing nginx -s reload 2016/12/20 20:19:43 [notice] 27#27: signal process started upstream default-gogs-gogs.default.10.103.12.20.nip.io-gogs { server 172.16.49.6:3000; } server { listen 80; server_name gogs.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-gogs-gogs.default.10.103.12.20.nip.io-gogs; } } I1220 20:19:43.388641 1 utils.go:70] Syncing default/gogs I1220 20:19:43.388686 1 controller.go:334] Syncing default/gogs I1220 20:19:43.388703 1 controller.go:349] Adding or Updating Ingress: default/gogs I1220 20:19:43.388835 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.389547 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-gogs.conf I1220 20:19:43.390482 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.390515 1 nginx.go:213] executing nginx -t I1220 20:19:43.412396 1 nginx.go:213] executing nginx -s reload I1220 20:19:43.414080 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:43.414227 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:43.414244 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:43.432724 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:43.432778 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:43.434289 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager 2016/12/20 20:19:43 [notice] 31#31: signal process started I1220 20:19:43.452114 1 utils.go:70] Syncing default/fabric8 I1220 20:19:43.452241 1 controller.go:334] Syncing default/fabric8 I1220 20:19:43.452310 1 controller.go:349] Adding or Updating Ingress: default/fabric8 I1220 20:19:43.452487 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.453202 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-fabric8.conf upstream default-fabric8-fabric8.default.10.103.12.20.nip.io-fabric8 { server 172.16.87.3:9090; } server { listen 80; server_name fabric8.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-fabric8-fabric8.default.10.103.12.20.nip.io-fabric8; } } I1220 20:19:43.454848 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.454940 1 nginx.go:213] executing nginx -t I1220 20:19:43.476880 1 nginx.go:213] executing nginx -s reload 2016/12/20 20:19:43 [notice] 36#36: signal process started I1220 20:19:43.502286 1 utils.go:70] Syncing default/fabric8-forge I1220 20:19:43.502448 1 controller.go:334] Syncing default/fabric8-forge I1220 20:19:43.502532 1 controller.go:349] Adding or Updating Ingress: default/fabric8-forge I1220 20:19:43.502791 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.503600 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-fabric8-forge.conf upstream default-fabric8-forge-fabric8-forge.default.10.103.12.20.nip.io-fabric8-forge { server 172.16.97.4:8080; } server { listen 80; server_name fabric8-forge.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-fabric8-forge-fabric8-forge.default.10.103.12.20.nip.io-fabric8-forge; } } I1220 20:19:43.505642 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.505764 1 nginx.go:213] executing nginx -t I1220 20:19:43.524679 1 nginx.go:213] executing nginx -s reload 2016/12/20 20:19:43 [notice] 40#40: signal process started I1220 20:19:43.544801 1 utils.go:70] Syncing default/gogs-ssh I1220 20:19:43.544848 1 controller.go:334] Syncing default/gogs-ssh I1220 20:19:43.544899 1 controller.go:349] Adding or Updating Ingress: default/gogs-ssh I1220 20:19:43.545085 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.546166 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-gogs-ssh.conf upstream default-gogs-ssh-gogs-ssh.default.10.103.12.20.nip.io-gogs-ssh { server 172.16.49.6:2222; } server { listen 80; server_name gogs-ssh.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-gogs-ssh-gogs-ssh.default.10.103.12.20.nip.io-gogs-ssh; } } I1220 20:19:43.548603 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.548632 1 nginx.go:213] executing nginx -t I1220 20:19:43.568165 1 nginx.go:213] executing nginx -s reload 2016/12/20 20:19:43 [notice] 45#45: signal process started I1220 20:19:43.584852 1 utils.go:70] Syncing default/jenkinshift I1220 20:19:43.584939 1 controller.go:334] Syncing default/jenkinshift I1220 20:19:43.585024 1 controller.go:349] Adding or Updating Ingress: default/jenkinshift I1220 20:19:43.585142 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.585842 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-jenkinshift.conf upstream default-jenkinshift-jenkinshift.default.10.103.12.20.nip.io-jenkinshift { server 172.16.87.3:9191; } server { listen 80; server_name jenkinshift.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-jenkinshift-jenkinshift.default.10.103.12.20.nip.io-jenkinshift; } } I1220 20:19:43.590415 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.590463 1 nginx.go:213] executing nginx -t I1220 20:19:43.607452 1 nginx.go:213] executing nginx -s reload 2016/12/20 20:19:43 [notice] 49#49: signal process started I1220 20:19:43.620424 1 utils.go:70] Syncing default/nexus I1220 20:19:43.620455 1 controller.go:334] Syncing default/nexus I1220 20:19:43.620465 1 controller.go:349] Adding or Updating Ingress: default/nexus I1220 20:19:43.620514 1 nginx.go:108] Updating NGINX configuration I1220 20:19:43.620824 1 nginx.go:154] Writing NGINX conf to /etc/nginx/conf.d/default-nexus.conf upstream default-nexus-nexus.default.10.103.12.20.nip.io-nexus { server 172.16.49.2:8081; } server { listen 80; server_name nexus.default.10.103.12.20.nip.io; location / { proxy_http_version 1.1; proxy_connect_timeout 500s; proxy_read_timeout 500s; client_max_body_size 2000m; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://default-nexus-nexus.default.10.103.12.20.nip.io-nexus; } } I1220 20:19:43.621904 1 nginx.go:174] NGINX configuration file had been updated I1220 20:19:43.621933 1 nginx.go:213] executing nginx -t I1220 20:19:43.631064 1 nginx.go:213] executing nginx -s reload 2016/12/20 20:19:43 [notice] 53#53: signal process started I1220 20:19:45.416818 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:45.416876 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:45.416888 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:45.440454 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:45.440494 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:45.440501 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:19:47.424786 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:47.424831 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:47.424841 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:47.446089 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:47.446132 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:47.446142 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:19:49.433960 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:49.434015 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:49.434027 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:49.457482 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:49.457526 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:49.457537 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:19:51.454488 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:51.454604 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:51.454617 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:51.471905 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:51.471951 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:51.471966 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:19:53.471170 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:53.471224 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:53.471235 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:53.485512 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:53.485548 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:53.485558 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:19:55.477402 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:55.477442 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:55.477472 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:55.499894 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:55.499964 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:55.499976 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:19:57.487559 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:57.487624 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:57.487635 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:57.508453 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:57.508537 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:57.508549 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:19:59.492359 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:19:59.492400 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:19:59.492410 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:19:59.514079 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:19:59.514119 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:19:59.514128 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:01.504817 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:01.504878 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:01.504891 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:01.521787 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:01.521831 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:01.521844 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:03.511644 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:03.511694 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:03.511704 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:03.529068 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:03.529142 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:03.529153 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:05.519338 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:05.519401 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:05.519414 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:05.536118 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:05.536154 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:05.536199 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:07.527061 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:07.527111 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:07.527157 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:07.545302 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:07.545371 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:07.545381 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:09.537279 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:09.537772 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:09.537857 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:09.555983 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:09.556044 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:09.556055 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:12.283176 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:12.283603 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:12.284866 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:12.285009 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:12.285296 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:12.286194 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:13.555725 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:13.555807 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:13.555825 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:13.568339 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:13.568412 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:13.568427 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:15.562319 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:15.562356 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:15.562365 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:15.578309 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:15.578370 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:15.578380 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager I1220 20:20:17.573304 1 controller.go:154] Endpoints kube-scheduler changed, syncing I1220 20:20:17.573341 1 utils.go:70] Syncing kube-system/kube-scheduler I1220 20:20:17.573349 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler I1220 20:20:17.588338 1 controller.go:154] Endpoints kube-controller-manager changed, syncing I1220 20:20:17.588404 1 utils.go:70] Syncing kube-system/kube-controller-manager I1220 20:20:17.588414 1 controller.go:275] Syncing endpoints kube-system/kube-controller-manager

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo The generated configuration looks good.

When you get the 502 error, could you past the related log entry from the NGINX logs?
Could you connect to the pod, install curl and try to connect to any of the endpoints?

$ kubectl exec <nginx-pod> -it -- bash
# apt-get update -y
# apt-get install -y curl
# curl http://172.16.87.3:9090

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

@michael: I couldn't connect with bash root@kuber-master:~# kubectl -n fabric8-system get pods NAME READY STATUS RESTARTS AGE ingress-nginx-2015555637-tayf7 1/1 Running 0 16m root@kuber-master:~# kubectl -n fabric8-system exec ingress-nginx-2015555637-tayf7 -it -- bash error: unable to upgrade connection: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html><head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>ERROR: The requested URL could not be retrieved</title> <style type="text/css"></style> </head><body id=ERR_INVALID_URL> <div id="titles"> <h1>ERROR</h1> <h2>The requested URL could not be retrieved</h2> </div> <hr> <div id="content"> <p>The following error was encountered while trying to retrieve the URL: <a href="/api/v1/namespaces/fabric8-system/pods/ingress-nginx-2015555637-tayf7/exec?command=bash&container=nginx-ingress&container=nginx-ingress&stdin=true&stdout=true&tty=true">/api/v1/namespaces/fabric8-system/pods/ingress-nginx-2015555637-tayf7/exec?command=bash&container=nginx-ingress&container=nginx-ingress&stdin=true&stdout=true&tty=true</a></p> <blockquote id="error"> <p><b>Invalid URL</b></p> </blockquote> <p>Some aspect of the requested URL is incorrect.</p> <p>Some possible problems are:</p> <ul> <li><p>Missing or incorrect access protocol (should be <q>http://</q> or similar)</p></li> <li><p>Missing hostname</p></li> <li><p>Illegal double-escape in the URL-Path</p></li> <li><p>Illegal character in hostname; underscores are not allowed.</p></li> </ul> <p>Your cache administrator is <a href="mailto:root ?subject=CacheErrorInfo%20-%20ERR_INVALID_URL&body=CacheHost%3A% 20www-proxy.naladmz.ericsson.ca %0D%0AErrPage%3A%20ERR_INVALID_URL%0D%0AErr%3A%20%5Bnone%5D%0D%0ATimeStamp%3A%20Tue,%2020%20Dec%202016%2020%3A39%3A41%20GMT%0D%0A%0D%0AClientIP%3A%2010.103.12.20%0D%0A%0D%0AHTTP%20Request%3A%0D%0A%0D%0A%0D%0A">root</a>.</p> <br> </div> <hr> <div id="footer"> <p>Generated Tue, 20 Dec 2016 20:39:41 GMT by www-proxy.naladmz.ericsson.ca (squid/3.1.16)</p>  </div> </body></html> root@kuber-master:~# Thanks, best regards,

…

On Tue, Dec 20, 2016 at 3:33 PM, Michael Pleshakov ***@***.*** > wrote: @abdoo <https://github.com/abdoo> The generated configuration looks good. When you get the 502 error, could you past the related log entry from the NGINX logs? Could you connect to the pod, install curl and try to connect to any of the endpoints? $ kubectl exec <nginx-pod> -it -- bash # apt-get update -y # apt-get install -y curl # curl http://172.16.87.3:9090 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz867_k1uzFZtKKvDFTAgzxfUHK5vks5rKDuVgaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo Could you setup kubectl to bypass the squid proxy when connecting to the API server?

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

I don't understand what do you mean by squid proxy? you mean company's proxy? If this is the case so I can't do that because we will not be able to use Internets for downloading images..etc Thanks, best regards,

…

On Tue, Dec 20, 2016 at 3:57 PM, Michael Pleshakov ***@***.*** > wrote: @abdoo <https://github.com/abdoo> Could you setup kubectl to bypass the squid proxy when connecting to the API server? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz-lexGoJDubfIsmhtJsiQ55Lo1lDks5rKEFFgaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo
In your case, kubectl using a proxy to connect to the API server, which resides on the same host from my understanding, so using a proxy for that isn't necessary. Could make it bypass the proxy temporary?

Also, could you try to run the commands like below fist and see if they will work:

$ kubectl exec <nginx-pod> -- apt-get update -y
...

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

Hi Michael, this command didnt work as well. we need the proxy during installation of kubernetes to be able to connect to Internet. If there is a way to stop using it now it would be great. If you can share how to make kubernetes skip it. Thanks, best regards,

…

On Tue, Dec 20, 2016 at 4:31 PM, Michael Pleshakov ***@***.*** > wrote: @abdoo <https://github.com/abdoo> In your case, kubectl using a proxy to connect to the API server, which resides on the same host from my understanding, so using a proxy for that isn't necessary. Could make it bypass the proxy temporary? Also, could you try to run the commands like below fist and see if they will work: $ kubectl exec <nginx-pod> -- apt-get update -y ... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhzzLgYfY4Bue649b95RmeKOh_qIrpks5rKEkggaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo
Could you try to connect to any of the servers like http://fabric8.default.10.103.12.20.nip.io and then find the error in the NGINX logs? It will tell why NGINX couldn't connect to the endpoint.

This might work to bypass the proxy: $ HTTPS_PROXY='' kubectl exec ...

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

hi Michael, I tried to bypass the proxy but it didnt work and I got the same error. root@kuber-master:~# HTTP_PROXY='' kubectl -n fabric8-system exec ingress-nginx-2015555637-7zhrh -- apt-get update -y error: unable to upgrade connection: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html><head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>ERROR: The requested URL could not be retrieved</title> <style type="text/css"><!-- /* Stylesheet for Squid Error pages Adapted from design by Free CSS Templates http://www.freecsstemplates.org Released for free under a Creative Commons Attribution 2.5 License */ /* Page basics */ * { font-family: verdana, sans-serif; } html body { margin: 0; padding: 0; background: #efefef; font-size: 12px; color: #1e1e1e; } /* Page displayed title area */ #titles { margin-left: 15px; padding: 10px; padding-left: 100px; background: url('http://www.squid-cache.org/Artwork/SN.png') no-repeat left; } root@kuber-master:~# curl http://fabric8.default.10.103.12.20.nip.io <html> <head><title>502 Bad Gateway</title></head> <body bgcolor="white"> <center><h1>502 Bad Gateway</h1></center> <hr><center>nginx/1.11.4</center> </body> </html> root@kuber-master:~# *these logs found on nginx* I1221 15:42:04.276871 1 controller.go:275] Syncing endpoints kube-system/kube-scheduler 172.16.82.1 - - [21/Dec/2016:15:42:04 +0000] "GET / HTTP/1.1" 502 173 "-" "curl/7.35.0" "10.103.12.20" 2016/12/21 15:42:04 [error] 177#177: *5 connect() failed (110: Connection timed out) while connecting to upstream, client: 172.16.82.1, server: fabric8.default.10.103.12.20.nip.io, request: "GET / HTTP/1.1", upstream: " http://172.16.5.2:9090/", host: "fabric8.default.10.103.12.20.nip.io" I1221 15:42:04.957281 1 controller.go:154] Endpoints kube-controller-manager changed, syncing so from above nginx did the correct mapping but it cant reach the server itself? Thanks, best regards,

…

On Wed, Dec 21, 2016 at 9:57 AM, Michael Pleshakov ***@***.*** > wrote: @abdoo <https://github.com/abdoo> Could you try to connect to any of the servers like http://fabric8.default.10.103.12.20.nip.io and then find the error in the NGINX logs? It will tell why NGINX couldn't connect to the endpoint. This might work to bypass the proxy: $ HTTPS_PROXY='' kubectl exec ... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz9jVzskfdID4CdlZhOOkyG6Zi1msks5rKT5mgaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo

so from above nginx did the correct mapping but it cant reach the server
itself?

That's correct

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

I managed to remove the proxy from my shell and I can do ssh to the containers. any container in the same internal subnet with ingress can be seen. I can see (gogs and jenkins). others I cant see. there is a network issue in the cluster.

…

On Wed, Dec 21, 2016 at 11:50 AM, Michael Pleshakov < ***@***.***> wrote: @abdoo <https://github.com/abdoo> so from above nginx did the correct mapping but it cant reach the server itself? That's correct — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz9-QWKd_tjVRWw5LVcyoCfspfW6nks5rKVjlgaJpZM4LPvNH> .

-- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo That's great that you found out the root cause. I'm not sure If I can help

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

hi Guys, I installed kubernetes using kubeadm. after that I installed Fabric8 and I can see ingress is not working. it provides the below logs [root@centos-master ~]# kubectl -n fabric8-system describe pod ingress-nginx-2015555637-9bvlf Name: ingress-nginx-2015555637-9bvlf Namespace: fabric8-system Node: / Labels: group=io.fabric8.devops.apps pod-template-hash=2015555637 project=ingress-nginx provider=fabric8 version=2.2.298 Status: Pending IP: Controllers: ReplicaSet/ingress-nginx-2015555637 Containers: nginx-ingress: Image: nginxdemos/nginx-ingress:0.3.1 Ports: 80/TCP, 443/TCP Args: -v=3 -nginx-configmaps=fabric8-system/nginx-config Volume Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-0zwau (ro) Environment Variables: <none> Conditions: Type Status PodScheduled False Volumes: default-token-0zwau: Type: Secret (a volume populated by a Secret) SecretName: default-token-0zwau QoS Class: BestEffort Tolerations: <none> Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 13s 13s 1 {default-scheduler } Warning FailedScheduling pod (ingress-nginx-2015555637-9bvlf) failed to fit in any node fit failure on node (centos-slave02): MatchNodeSelector fit failure on node (centos-master): PodToleratesNodeTaints fit failure on node (centos-slave01): MatchNodeSelector 12s 12s 1 {default-scheduler } Warning FailedScheduling pod (ingress-nginx-2015555637-9bvlf) failed to fit in any node fit failure on node (centos-slave02): MatchNodeSelector fit failure on node (centos-slave01): MatchNodeSelector fit failure on node (centos-master): PodToleratesNodeTaints 10s 10s 1 {default-scheduler } Warning FailedScheduling pod (ingress-nginx-2015555637-9bvlf) failed to fit in any node fit failure on node (centos-master): PodToleratesNodeTaints fit failure on node (centos-slave01): MatchNodeSelector fit failure on node (centos-slave02): MatchNodeSelector 13s 6s 2 {default-scheduler } Warning FailedScheduling pod (ingress-nginx-2015555637-9bvlf) failed to fit in any node fit failure on node (centos-slave01): MatchNodeSelector fit failure on node (centos-slave02): MatchNodeSelector fit failure on node (centos-master): PodToleratesNodeTaints [root@centos-master ~]# help is really needed. On Wed, Dec 21, 2016 at 12:44 PM, Abd El-Fattah Mahran < [email protected]> wrote:

…

Thanks a lot Michael you already helped a lot :-) Thanks Best regards, On Wed, Dec 21, 2016 at 12:41 Michael Pleshakov ***@***.***> wrote: > @abdoo <https://github.com/abdoo> That's great that you found out the > root cause. I'm not sure If I can help > > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#101 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAMhz-D_o7gPRMmTI-pW0fH2mTZRmPxjks5rKWTVgaJpZM4LPvNH> > . > > > > > > > > > > > > > > > > > > > -- Thanks, Best regards,

-- Thanks Best Regards, AbdelFattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo
The Ingress controller is deployed on the nodes with the following label -- fabric8.io/externalIP: true -- according to https://github.com/fabric8io/fabric8-devops/blob/master/ingress-nginx/src/main/fabric8/deployment.yml It looks like in your case there are no nodes with that label. You can check it by running kubectl get nodes -l fabric8.io/externalIP=true. Not sure why those labels are not assigned to the nodes as I'm not familiar with the fabric8 installation.

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

hi Michael, thanks for your resposne. I used kubectl taint nodes --all dedicated- then I can see it working now. my issue now with Centos setup I did. all pods are running and ingress is running and I added the securityContext but still port 80 and 443 are down. [root@centos-master ~]# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE default configmapcontroller-3703242456-mzu44 1/1 Running 0 1d default exposecontroller-1027083163-pm14g 1/1 Running 0 1d default fabric8-3192130638-yyedg 2/2 Running 0 1d default fabric8-docker-registry-1981087684-cva4j 1/1 Running 0 1d default fabric8-forge-3730694965-wq4xe 1/1 Running 0 1d default gogs-2230157356-i2qm5 1/1 Running 0 1d default jenkins-1661962136-bu1k2 1/1 Running 0 1d default nexus-3628846063-nf77v 1/1 Running 0 1d fabric8-system ingress-nginx-2015555637-kjhdr 1/1 Running 2 1d kube-system dummy-2088944543-fyflm 1/1 Running 5 4d kube-system etcd-centos-master 1/1 Running 4 3d kube-system kube-apiserver-centos-master 1/1 Running 0 16h kube-system kube-controller-manager-centos-master 1/1 Running 670 1d kube-system kube-discovery-3670776889-9uiq0 1/1 Running 5 4d kube-system kube-dns-2924299975-zabz7 4/4 Running 20 4d kube-system kube-flannel-ds-9qle9 2/2 Running 0 1d kube-system kube-flannel-ds-clsc0 2/2 Running 5 1d kube-system kube-flannel-ds-g3pku 2/2 Running 0 1d kube-system kube-proxy-8d4qx 1/1 Running 5 4d kube-system kube-proxy-mwzat 1/1 Running 0 3d kube-system kube-proxy-o2ce4 1/1 Running 0 3d kube-system kube-scheduler-centos-master 1/1 Running 673 1d kube-system kubernetes-dashboard-3095304083-81nso 1/1 Running 5 4d [root@centos-master ~]# [root@centos-master ~]# telnet 0 80 Trying 0.0.0.0... telnet: connect to address 0.0.0.0: Connection refused [root@centos-master ~]# telnet 0 443 Trying 0.0.0.0... telnet: connect to address 0.0.0.0: Connection refused [root@centos-master ~]# [root@centos-master ~]# kubectl get ing NAME HOSTS ADDRESS PORTS AGE fabric8 fabric8.default.10.103.12.23.nip.io 80 3d fabric8-docker-registry fabric8-docker-registry.default.10.103.12.23.nip.io 80 3d fabric8-forge fabric8-forge.default.10.103.12.23.nip.io 80 3d gogs gogs.default.10.103.12.23.nip.io 80 3d gogs-ssh gogs-ssh.default.10.103.12.23.nip.io 80 3d jenkins jenkins.default.10.103.12.23.nip.io 80 3d jenkins-jnlp jenkins-jnlp.default.10.103.12.23.nip.io 80 3d jenkinshift jenkinshift.default.10.103.12.23.nip.io 80 3d nexus nexus.default.10.103.12.23.nip.io 80 3d [root@centos-master ~]# NOTE:- I had to change the red line in this file "/etc/kubernetes/manifests/kube-apiserver.json" to reach kubenetes UI from browser from outside and not from localhost. "containers": [ { "name": "kube-apiserver", "image": "gcr.io/google_containers/kube-apiserver-amd64:v1.4.7", "command": [ "kube-apiserver", "--insecure-bind-address=0.0.0.0", "--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota", "--service-cluster-ip-range=10.96.0.0/12", "--service-account-key-file=/etc/kubernetes/pki/apiserver-key.pem", "--client-ca-file=/etc/kubernetes/pki/ca.pem", "--tls-cert-file=/etc/kubernetes/pki/apiserver.pem", "--tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem", "--token-auth-file=/etc/kubernetes/pki/tokens.csv", "--secure-port=6443", "--allow-privileged=true", "--advertise-address=10.103.12.23", "--etcd-servers=http://127.0.0.1:2379" ], Thanks, best regards,

…

On Tue, Jan 3, 2017 at 1:18 PM, Michael Pleshakov ***@***.***> wrote: @abdoo <https://github.com/abdoo> The Ingress controller is deployed on the nodes with the following label -- fabric8.io/externalIP: true -- according to https://github.com/fabric8io/fabric8-devops/blob/master/ ingress-nginx/src/main/fabric8/deployment.yml It looks like in your case there are no nodes with that label. You can check it by running kubectl get nodes -l fabric8.io/externalIP=true. Not sure why those labels are not assigned to the nodes as I'm not familiar with the fabric8 installation. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhzwiIBEqlyqKUk9Au9h7V38QlRcg_ks5rOpDugaJpZM4LPvNH> .

-- Thanks Best Regards, AbdelFattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

for the command you sent. [root@centos-master ~]# kubectl get nodes -l fabric8.io/externalIP=true NAME STATUS AGE centos-master Ready,master 4d [root@centos-master ~]# On Tue, Jan 3, 2017 at 2:34 PM, Abd El-Fattah Mahran <[email protected]

…

wrote: hi Michael, thanks for your resposne. I used kubectl taint nodes --all dedicated- then I can see it working now. my issue now with Centos setup I did. all pods are running and ingress is running and I added the securityContext but still port 80 and 443 are down. ***@***.*** ~]# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE default configmapcontroller-3703242456-mzu44 1/1 Running 0 1d default exposecontroller-1027083163-pm14g 1/1 Running 0 1d default fabric8-3192130638-yyedg 2/2 Running 0 1d default fabric8-docker-registry-1981087684-cva4j 1/1 Running 0 1d default fabric8-forge-3730694965-wq4xe 1/1 Running 0 1d default gogs-2230157356-i2qm5 1/1 Running 0 1d default jenkins-1661962136-bu1k2 1/1 Running 0 1d default nexus-3628846063-nf77v 1/1 Running 0 1d fabric8-system ingress-nginx-2015555637-kjhdr 1/1 Running 2 1d kube-system dummy-2088944543-fyflm 1/1 Running 5 4d kube-system etcd-centos-master 1/1 Running 4 3d kube-system kube-apiserver-centos-master 1/1 Running 0 16h kube-system kube-controller-manager-centos-master 1/1 Running 670 1d kube-system kube-discovery-3670776889-9uiq0 1/1 Running 5 4d kube-system kube-dns-2924299975-zabz7 4/4 Running 20 4d kube-system kube-flannel-ds-9qle9 2/2 Running 0 1d kube-system kube-flannel-ds-clsc0 2/2 Running 5 1d kube-system kube-flannel-ds-g3pku 2/2 Running 0 1d kube-system kube-proxy-8d4qx 1/1 Running 5 4d kube-system kube-proxy-mwzat 1/1 Running 0 3d kube-system kube-proxy-o2ce4 1/1 Running 0 3d kube-system kube-scheduler-centos-master 1/1 Running 673 1d kube-system kubernetes-dashboard-3095304083-81nso 1/1 Running 5 4d ***@***.*** ~]# ***@***.*** ~]# telnet 0 80 Trying 0.0.0.0... telnet: connect to address 0.0.0.0: Connection refused ***@***.*** ~]# telnet 0 443 Trying 0.0.0.0... telnet: connect to address 0.0.0.0: Connection refused ***@***.*** ~]# ***@***.*** ~]# kubectl get ing NAME HOSTS ADDRESS PORTS AGE fabric8 fabric8.default.10.103.12.23.nip.io 80 3d fabric8-docker-registry fabric8-docker-registry. default.10.103.12.23.nip.io 80 3d fabric8-forge fabric8-forge.default.10.103.12.23.nip.io 80 3d gogs gogs.default.10.103.12.23.nip.io 80 3d gogs-ssh gogs-ssh.default.10.103.12.23.nip.io 80 3d jenkins jenkins.default.10.103.12.23.nip.io 80 3d jenkins-jnlp jenkins-jnlp.default.10.103.12.23.nip.io 80 3d jenkinshift jenkinshift.default.10.103.12.23.nip.io 80 3d nexus nexus.default.10.103.12.23.nip.io 80 3d ***@***.*** ~]# NOTE:- I had to change the red line in this file "/etc/kubernetes/manifests/kube-apiserver.json" to reach kubenetes UI from browser from outside and not from localhost. "containers": [ { "name": "kube-apiserver", "image": "gcr.io/google_containers/kube-apiserver-amd64:v1.4.7", "command": [ "kube-apiserver", "--insecure-bind-address=0.0.0.0", "--admission-control=NamespaceLifecycle, LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass, ResourceQuota", "--service-cluster-ip-range=10.96.0.0/12", "--service-account-key-file=/etc/kubernetes/pki/apiserver- key.pem", "--client-ca-file=/etc/kubernetes/pki/ca.pem", "--tls-cert-file=/etc/kubernetes/pki/apiserver.pem", "--tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem", "--token-auth-file=/etc/kubernetes/pki/tokens.csv", "--secure-port=6443", "--allow-privileged=true", "--advertise-address=10.103.12.23", "--etcd-servers=http://127.0.0.1:2379" ], Thanks, best regards, On Tue, Jan 3, 2017 at 1:18 PM, Michael Pleshakov < ***@***.***> wrote: > @abdoo <https://github.com/abdoo> > The Ingress controller is deployed on the nodes with the following label > -- fabric8.io/externalIP: true -- according to > https://github.com/fabric8io/fabric8-devops/blob/master/ingr > ess-nginx/src/main/fabric8/deployment.yml It looks like in your case > there are no nodes with that label. You can check it by running kubectl > get nodes -l fabric8.io/externalIP=true. Not sure why those labels are > not assigned to the nodes as I'm not familiar with the fabric8 installation. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#101 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAMhzwiIBEqlyqKUk9Au9h7V38QlRcg_ks5rOpDugaJpZM4LPvNH> > . > -- Thanks Best Regards, AbdelFattah Mahran http://www.linkedin.com/in/abdoomahran

-- Thanks Best Regards, AbdelFattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo

[root@centos-master ~]# telnet 0 80

Are you sure that the controller is running on the master node?

Could you also add the hostNetwork: true to the Pod spec of the controller and see if that helps?

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

@michael: what do you mean by the controller. I still have the issue with flannel so I closed two slaves and I have only one master node. root@kuber-master:~# kubectl get nodes NAME STATUS AGE 10.103.12.20 Ready 12d 10.103.12.21 NotReady 12d 10.103.12.22 NotReady 12d root@kuber-master:~# Thanks, best regards,

…

On Tue, Jan 3, 2017 at 6:03 PM, Michael Pleshakov ***@***.***> wrote: @abdoo <https://github.com/abdoo> ***@***.*** ~]# telnet 0 80 Are you sure that the controller is running on the master node? Could you also add the hostNetwork: true to the Pod spec of the controller and see if that helps? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMhz8oSDJro08kno_hsXPtIdJ0egiazks5rOtO0gaJpZM4LPvNH> .

-- Thanks Best Regards, AbdelFattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

fattahmahran commented on August 28, 2024

I checked below [root@centos-master manifests]# pwd /etc/kubernetes/manifests [root@centos-master manifests]# grep "hostNetwork" kube-controller-manager.json "hostNetwork": true [root@centos-master manifests]# On Tue, Jan 3, 2017 at 10:19 PM, Abd El-Fattah Mahran < [email protected]> wrote:

…

@michael: what do you mean by the controller. I still have the issue with flannel so I closed two slaves and I have only one master node. ***@***.***:~# kubectl get nodes NAME STATUS AGE 10.103.12.20 Ready 12d 10.103.12.21 NotReady 12d 10.103.12.22 NotReady 12d ***@***.***:~# Thanks, best regards, On Tue, Jan 3, 2017 at 6:03 PM, Michael Pleshakov < ***@***.***> wrote: > @abdoo <https://github.com/abdoo> > > ***@***.*** ~]# telnet 0 80 > > Are you sure that the controller is running on the master node? > > Could you also add the hostNetwork: true to the Pod spec of the > controller and see if that helps? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#101 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAMhz8oSDJro08kno_hsXPtIdJ0egiazks5rOtO0gaJpZM4LPvNH> > . > -- Thanks Best Regards, AbdelFattah Mahran http://www.linkedin.com/in/abdoomahran

-- Thanks Best Regards, AbdelFattah Mahran http://www.linkedin.com/in/abdoomahran

from kubernetes-ingress.

pleshakov commented on August 28, 2024

@abdoo
Sorry for the confusion. By the controller I meant the NGINX Ingress controller.
Could you change its deployment by adding hostNetwork: true to the Pod spec?

from kubernetes-ingress.

Kubernetes on private network about kubernetes-ingress HOT 38 CLOSED

Comments (38)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent