After following the docs on getting everything installed, I'm unable to login with the default credentials: An error occurred while connecting to server: Network Error
I'm not using any external services on AWS/Azure which should make debugging this slightly easier, however there aren't any red flags in the logs that I notice.
Everything in the namespace is running and healthy:
elasticsearch-master-0 1/1 Running 0 36m
elasticsearch-master-1 1/1 Running 0 36m
elasticsearch-master-2 1/1 Running 0 36m
rp-minio-779cbdd9d5-mj26z 1/1 Running 0 27m
rp-psql-postgresql-0 1/1 Running 0 29m
rp-reportportal-analyzer-0 1/1 Running 0 26m
rp-reportportal-api-6f6dc4c96f-6px22 1/1 Running 1 26m
rp-reportportal-index-744974dbb5-zsktq 1/1 Running 0 26m
rp-reportportal-uat-79d596bdd9-v459r 1/1 Running 0 26m
rp-reportportal-ui-5f678496d8-ttmn5 1/1 Running 0 26m
rp-rmq-rabbitmq-ha-0 1/1 Running 0 36m
Logs in the uat pod look normal:
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/health'; against '/sso/login/**'
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/health'; against '/epam/**'
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/health'; against '/info'
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.u.matcher.AntPathRequestMatcher : Checking match of request : '/health'; against '/health'
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.a.i.FilterSecurityInterceptor : Secure object: FilterInvocation: URL: /health; Attributes: [permitAll]
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.a.i.FilterSecurityInterceptor : Previously Authenticated: org.springframework.security.authentication.AnonymousAuthenticationToken@1b7d758f: Principal: anonymousUser; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@ffff8868: RemoteIpAddress: 10.43.54.144; SessionId: null; Granted Authorities: ROLE_ANONYMOUS
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.access.vote.AffirmativeBased : Voter: org.springframework.security.web.access.expression.WebExpressionVoter@77c004d7, returned: 1
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.a.i.FilterSecurityInterceptor : Authorization successful
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.a.i.FilterSecurityInterceptor : RunAsManager did not change Authentication object
2020-04-07 20:10:10.856 DEBUG 7 --- [nio-9999-exec-3] o.s.security.web.FilterChainProxy : /health reached end of additional filter chain; proceeding with original chain
2020-04-07 20:10:10.859 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.header.writers.HstsHeaderWriter : Not injecting HSTS header since it did not match the requestMatcher org.springframework.security.web.header.writers.HstsHeaderWriter$SecureRequestMatcher@3e1b9fec
2020-04-07 20:10:10.859 DEBUG 7 --- [nio-9999-exec-3] o.s.s.w.a.ExceptionTranslationFilter : Chain processed normally
2020-04-07 20:10:10.859 DEBUG 7 --- [nio-9999-exec-3] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed
Logs in the API pod look normal:
2020-04-07 19:44:47.177 INFO 7 --- [ main] org.quartz.core.QuartzScheduler : Scheduler reportportal_$_api:988c14df17f284b540d07fd82cb66a68 started.
2020-04-07 19:44:47.183 INFO 7 --- [_MisfireHandler] o.s.s.quartz.LocalDataSourceJobStore : Handling 5 trigger(s) that missed their scheduled fire-time.
2020-04-07 19:44:48.445 INFO 7 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8585 (http) with context path ''
2020-04-07 19:44:48.449 INFO 7 --- [ main] c.e.t.r.core.configs.ReportPortalApp : Started ReportPortalApp in 47.296 seconds (JVM running for 48.975)
2020-04-07 19:44:52.465 INFO 7 --- [nio-8585-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2020-04-07 19:44:52.465 INFO 7 --- [nio-8585-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2020-04-07 19:44:52.477 INFO 7 --- [nio-8585-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 12 ms
2020-04-07 19:44:52.586 INFO 7 --- [nio-8585-exec-1] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: rp-rmq-rabbitmq-ha.rp.svc.cluster.local:5672
2020-04-07 19:44:52.589 INFO 7 --- [nio-8585-exec-1] o.s.a.r.c.CachingConnectionFactory : Created new connection: analyzerConnectionFactory#3204e238:0/SimpleConnection@5fe2e9dc [delegate=amqp://[email protected]:5672/analyzer, localPort= 45424]
2020-04-07 19:59:48.561 WARN 7 --- [ ConnectionPool] okhttp3.OkHttpClient : A connection to http://rp-minio.rp.svc.cluster.local:9000/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE: Logger.getLogger(OkHttpClient.class.getName()).setLevel(Level.FINE);
Logs in the UI pod look strange:
2020/04/07 20:08:11 "POST http://localhost:8080/uat/sso/oauth/token HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 33.997ยตs
2020/04/07 20:08:11 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 27.118ยตs
2020/04/07 20:08:11 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 25.612ยตs
2020/04/07 20:08:11 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 19.416ยตs
2020/04/07 20:08:11 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 22.081ยตs
2020/04/07 20:08:11 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 20.383ยตs
2020/04/07 20:08:11 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 23.832ยตs
2020/04/07 20:08:11 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 19.722ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 20.838ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 22.687ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 20.573ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 18.556ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 20.172ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 18.482ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 20.428ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 21.73ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 22.183ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 28.027ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 33.315ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 18.116ยตs
2020/04/07 20:08:12 "POST http://localhost:8080/ui/ HTTP/1.1" from 127.0.0.1:46406 - 307 0B in 40.53ยตs
Good to note: I followed each step as it was written in the README, minio, elasticsearch and postgres are all running in the cluster with default configurations (no crazy passwords, etc) so I'm extremely confused what's going wrong here. I'm also exposing the UI using kubectl port-forward
since exposing the service with an ingress causes my browser to redirect too many times (as shown in the logs)
Any tips or pointers on how to debug this or figure out what is going on would be great!
P.S: yes, I've looked at this however it was extremely unhelpful in figuring out root-cause