Comments (12)
gtudor
commented on July 26, 2024
If you get problems with odfe not starting using the sample docker-compose.yml file, you may need to increase the mmaps value in your host operating system. After hours of debugging with different config variables, I found this on the elastic.co website. After I executed the command sysctl -w vm.max_map_count=262144 and then did a docker-compose up using the sample docker-compose file everything started normally.
From the elastic.co website:
Elasticsearch uses a mmapfs directory by default to store its indices. The default operating system limits on mmap counts is likely to be too low, which may result in out of memory exceptions.
On Linux, you can increase the limits by running the following command as root:
sysctl -w vm.max_map_count=262144
To set this value permanently, update the vm.max_map_count setting in /etc/sysctl.conf. To verify after rebooting, run sysctl vm.max_map_count.
from for-elasticsearch-docs.
agadelshin
commented on July 26, 2024
azure log says auth operations are successful.
from for-elasticsearch-docs.
aetter
commented on July 26, 2024
Hi @pondohva, a couple thoughts:
- I'm only seeing one authentication domain, but you need two. Also, you want the Open ID domain to have an
orderof1: https://opendistro.github.io/for-elasticsearch-docs/docs/security/openid-connect/#elasticsearch-configuration - Are you running
securityadmin.shto reload settings after making tweaks to the YML files? We've had a few people in the past who had things configured properly, but weren't reloading settings, thought they needed to change variables, etc.: https://opendistro.github.io/for-elasticsearch-docs/docs/security/security-admin/
Given the presence of an external provider (and in this case, a proxy), these sorts of issues are always really hard to troubleshoot, but let me know if you're able to get it working.
from for-elasticsearch-docs.
agadelshin
commented on July 26, 2024
@aetter oh.
- Thank you, I forgot it.
- No. I'm using docker image amazon/opendistro-for-elasticsearch-kibana:0.7.1.
there is no tools directory in /usr/share/kibana/plugins/opendistro_security, whoami shows me "kibana" and there is no JAVA in image.
it seems securityadmin.sh will fix everything.
from for-elasticsearch-docs.
agadelshin
commented on July 26, 2024
Also I'm using elasticsearch-oss instead of opendistro image (because opendistro image doesn't work with helm chart).
So I'll fix it and try again (I thought security is a kibana plugin, not elasticsearch)
from for-elasticsearch-docs.
aetter
commented on July 26, 2024
Hi @pondohva, gotcha. Yeah, Security is really two plugins: one for Elasticsearch, one for Kibana. You can use the Elasticsearch plugin by itself, but the Kibana plugin relies on a secured Elasticsearch cluster.
from for-elasticsearch-docs.
agadelshin
commented on July 26, 2024
updated config to plugin on elasticsearch node
opendistro_security:
dynamic:
http:
anonymous_auth_enabled: false
xff:
enabled: false
authc:
basic_internal_auth_domain:
http_enabled: true
transport_enabled: true
enabled: true
order: 0
http_authenticator:
type: basic
challenge: false
authentication_backend:
type: internal
openid_auth_domain:
http_enabled: true
transport_enabled: true
order: 1
http_authenticator:
type: openid
challenge: false
config:
subject_key: preferred_username
openid_connect_url: https://login.microsoftonline.com/{tenant}/v2.0/.well-known/openid-configuration
authentication_backend:
type: noop
kibana.yaml
server.name: kibana
server.host: "0"
elasticsearch.ssl.verificationMode: none
opendistro_security.auth.type: "openid"
opendistro_security.openid.connect_url: https://login.microsoftonline.com/{tenant}/v2.0/.well-known/openid-configuration
opendistro_security.openid.client_id: "{app_id}"
opendistro_security.openid.client_secret: "{secret}"
opendistro_security.openid.base_redirect_url: "{url}"
elasticsearch.requestHeadersWhitelist: ["Authorization", "security_tenant", "x-forwarded-for", "x-forwarded-by"]
elasticsearch.username: "kibanaserver"
elasticsearch.password: "kibanaserver"
and I'm getting error in es logs on each request.
[2019-04-04T16:51:38,782][WARN ][c.a.o.s.h.HTTPBasicAuthenticator] [elasticsearch-master-0] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2019-04-04T16:51:38,784][WARN ][o.a.c.r.s.j.j.JwsUtils ] [elasticsearch-master-0] No signature algorithm was defined
[2019-04-04T16:51:38,784][WARN ][c.a.o.s.a.BackendRegistry] [elasticsearch-master-0] Authentication finally failed for null from 10.2.66.9:54226
from for-elasticsearch-docs.
agadelshin
commented on July 26, 2024
securityadmin.sh works too
[root@elasticsearch-master-1 elasticsearch]# "/usr/share/elasticsearch/plugins/opendistro_security/tools/securityadmin.sh" -cd "/usr/share/elasticsearch/plugins/opendistro_security/securityconfig" -icl -key "/usr/share/elasticsearch/config/kirk-key.pem" -cert "/usr/share/elasticsearch/config/kirk.pem" -cacert "/usr/share/elasticsearch/config/root-ca.pem" -nhnv
Open Distro Security Admin v6
Will connect to localhost:9300 ... done
Elasticsearch Version: 6.5.4
Open Distro Security Version: 0.7.0.1
Connected as CN=kirk,OU=client,O=client,L=test,C=de
Contacting elasticsearch cluster 'elasticsearch' and wait for YELLOW clusterstate ...
Clustername: elasticsearch
Clusterstate: GREEN
Number of nodes: 3
Number of data nodes: 3
.opendistro_security index already exists, so we do not need to create one.
Populate config from /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/
Will update 'security/config' with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/config.yml
SUCC: Configuration for 'config' created or updated
Will update 'security/roles' with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/roles.yml
SUCC: Configuration for 'roles' created or updated
Will update 'security/rolesmapping' with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/roles_mapping.yml
SUCC: Configuration for 'rolesmapping' created or updated
Will update 'security/internalusers' with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/internal_users.yml
SUCC: Configuration for 'internalusers' created or updated
Will update 'security/actiongroups' with /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/action_groups.yml
SUCC: Configuration for 'actiongroups' created or updated
from for-elasticsearch-docs.
aetter
commented on July 26, 2024
Hi @pondohva, everything looks pretty good to me. Are you setting elasticsearch.url in docker-compose.yml? Is Kibana successfully connecting to the cluster in its startup logs? Something like:
odfe-kibana | {"type":"log","@timestamp":"2019-04-03T17:18:03Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from red to green - Ready","prevState":"red","prevMsg":"Service Unavailable"}
odfe-kibana | {"type":"log","@timestamp":"2019-04-03T17:18:04Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0:5601"}
from for-elasticsearch-docs.
agadelshin
commented on July 26, 2024
@aetter yep
{"type":"log","@timestamp":"2019-04-04T17:05:42Z","tags":["status","plugin:[email protected]","info"],"pid":1,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2019-04-04T17:05:42Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0:5601"}
Also, I'm using distro images in GKE cluster. Kibana is behind google cloud load balancer, but kibana and elastic in the same cluster.
There are messages on each failed request (I enabled DEBUG)
[2019-04-04T20:37:15,387][DEBUG][c.a.o.s.a.BackendRegistry] [elasticsearch-master-2] Rest authentication request from 10.2.66.18:56934 [original: /10.2.66.18:56934]
[2019-04-04T20:37:15,387][WARN ][c.a.o.s.h.HTTPBasicAuthenticator] [elasticsearch-master-2] No 'Basic Authorization' header, send 401 and 'WWW-Authenticate Basic'
[2019-04-04T20:37:15,388][WARN ][o.a.c.r.s.j.j.JwsUtils ] [elasticsearch-master-2] No signature algorithm was defined
[2019-04-04T20:37:15,388][DEBUG][c.a.o.s.a.BackendRegistry] [elasticsearch-master-2] 'org.apache.cxf.rs.security.jose.jws.JwsException: ALGORITHM_NOT_SET' extracting credentials from jwt-key-by-oidc http authenticator
org.apache.cxf.rs.security.jose.jws.JwsException: ALGORITHM_NOT_SET
at org.apache.cxf.rs.security.jose.jws.JwsUtils.getPublicKeySignatureVerifier(JwsUtils.java:177) ~[cxf-rt-rs-security-jose-3.2.2.jar:3.2.2]
at org.apache.cxf.rs.security.jose.jws.JwsUtils.getSignatureVerifier(JwsUtils.java:146) ~[cxf-rt-rs-security-jose-3.2.2.jar:3.2.2]
at org.apache.cxf.rs.security.jose.jws.JwsUtils.getSignatureVerifier(JwsUtils.java:138) ~[cxf-rt-rs-security-jose-3.2.2.jar:3.2.2]
from for-elasticsearch-docs.
aetter
commented on July 26, 2024
@pondohva, everything still looks good to me, so I'm going to move this issue over to the security repo so that engineering can check it out.
from for-elasticsearch-docs.
aetter
commented on July 26, 2024
Reopened here: https://github.com/opendistro-for-elasticsearch/security/issues/29
from for-elasticsearch-docs.
Related Issues (20)
- 403 - Acess denied error while installing elasticsearch alerting plugin
- Should be POST request when searching with body?
- ODBC Driver 1.9.00
- Forward to OpenSearch? HOT 1
- Access rights to management page HOT 1
- PPL Command Doc Type
- Opensearch crashing on Macbook Pro M1 Max (Docker) HOT 3
- No handler for type [knn_vector]
- Update docs to reflect 1.13.3 HOT 4
- opendistro-for-elasticsearch-kibana version 1.13.3 HOT 1
- opendistroforelasticsearch yum repo not updated with 1.13.3 HOT 1
- mitigate vulnerability log4j
- Failing to use fetch_size parameter with query
- openid not working with opendistro kibana. As per steps which are provided by openditsro. kindly help us
- [Urgent] I can't make notebooks with recently created visualizations.
- mapped users option is missing
- alerting index permission
- Unable to use window functions in version 1.13.0
- OpenDistro Index Management
- opendistro-alerting-alert-history alerts disable HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from for-elasticsearch-docs.