Comments (7)
I did a quick check of the previous CRC release
[localhost crc-linux-1.16.0-amd64]$ ./crc version
CodeReady Containers version: 1.16.0+bf72d3a
OpenShift version: 4.5.9 (embedded in binary)
[localhost crc-linux-1.16.0-amd64]$
The node tuning operator is running
Using project "openshift-cluster-node-tuning-operator".
[localhost crc-linux-1.16.0-amd64]$ oc get all
NAME READY STATUS RESTARTS AGE
pod/cluster-node-tuning-operator-576df548b-jmz8z 1/1 Running 0 26d
pod/tuned-d5frc 1/1 Running 0 26d
and vm.max_map_count is what it's supposed to be
[localhost crc-linux-1.16.0-amd64]$ oc debug node/crc-zqfk6-master-0
Starting pod/crc-zqfk6-master-0-debug ...
To use host binaries, run `chroot /host`
Pod IP: 192.168.126.11
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# cat /proc/sys/vm/max_map_count
262144
Definitely a regression.
from elasticsearch-operator.
@huikang this is specifically talking about the node's vm.max_map_count
setting. There is normally an operator that updates this setting, I'm not sure if running this on CRC is different, I would need to defer to someone who has stood up a cluster on CRC in the past.
@vimalk78 @syedriko @periklis ?
from elasticsearch-operator.
@huikang For future issues, could you describe your environment and the steps that led to the issue you're reporting?
I repro'd what you're reporting on
[localhost crc-linux-1.17.0-amd64]$ ./crc version
CodeReady Containers version: 1.17.0+99f5c87
OpenShift version: 4.5.14 (embedded in binary)
[localhost crc-linux-1.17.0-amd64]$
with
apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
name: "instance"
namespace: "openshift-logging"
spec:
managementState: "Managed"
logStore:
type: "elasticsearch"
elasticsearch:
nodeCount: 1
storage: {}
redundancyPolicy: "ZeroRedundancy"
resources:
requests:
cpu: "100m"
memory: "2Gi"
...
The fix for now is
[localhost crc]$ oc get nodes
NAME STATUS ROLES AGE VERSION
crc-j55b9-master-0 Ready master,worker 8d v1.18.3+970c1b3
[localhost crc]$ oc debug node/crc-j55b9-master-0
Starting pod/crc-j55b9-master-0-debug ...
To use host binaries, run `chroot /host`
Pod IP: 192.168.126.11
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sh-4.4# sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144
sh-4.4# exit
exit
sh-4.2# exit
exit
Removing debug pod ...
[localhost crc]$
You'll need to reapply it every time you recreate the CRC VM.
As far as I recall, this used to be broken, then got fixed and now seems to have regressed.
from elasticsearch-operator.
@syedriko , thanks for the solution. Sure, I will report the crc version
if I find anything in the future (the original post has been updated to include the crc version).
from elasticsearch-operator.
BTW, shall we report this to the CRC repo?
from elasticsearch-operator.
@huikang you're welcome to take this up with the CRC follks.
Here's the back story on this, including the node tuning operator @ewolinetz mentioned: https://developers.redhat.com/blog/2019/11/12/using-the-red-hat-openshift-tuned-operator-for-elasticsearch . In my CRC instance, I'm not seeing a separate openshift-node-es profile, but rather that vm.max_map_count=262144 applies to all cluster nodes. I'm not seeing a node tuning operator running in my CRC instance though ( https://docs.openshift.com/container-platform/4.5/scalability_and_performance/using-node-tuning-operator.html )
from elasticsearch-operator.
I filed an issue with CRC: crc-org/crc#1590
from elasticsearch-operator.
Related Issues (20)
- Please prepare bundle for OCP 4.7 HOT 3
- Not launching kibana instance from elasticsearch operator HOT 1
- Unable to launch a Kibana CR instance using latest ElasticSearch operator, Steps needed HOT 12
- Kibana POD is not getting created in OCP 4.6 cluster (Elasticsearch instance + POD got created, Kibana instance created but POD is not created) HOT 6
- Index creation permission HOT 9
- Feature request: Allow setting shard allocation awareness HOT 4
- Elasticsearch-operator scaling info/documentation/manual HOT 2
- Avoid copying all file to the image HOT 1
- OLM doesn't trigger CSV update even though catalog has newer version HOT 2
- Unable to read /etc/elasticsearch/secret/searchguard-key.p12 HOT 4
- memoryleak HOT 11
- elastic-search pod not starting up HOT 8
- keytool error: java.io.IOException: parseAlgParameters failed: PBE AlgorithmParameters not available HOT 6
- Search query with bearer token returning no results HOT 5
- index rollover cronjob fails on openshift-logging operator HOT 7
- DNS flood coming from elasticsearch-operator pod HOT 4
- CrashLoopBackoff when installing on kind 0.18 (GOMAXPROCS?)
- How to install from the command line HOT 7
- OpenShift Elasticsearch Operator (v5.7.6) error (Received an empty response from elasticsearch -- server may not be ready) HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elasticsearch-operator.