Giter Club home page Giter Club logo

Comments (7)

syedriko avatar syedriko commented on September 24, 2024 1

I did a quick check of the previous CRC release

[localhost crc-linux-1.16.0-amd64]$ ./crc version
CodeReady Containers version: 1.16.0+bf72d3a
OpenShift version: 4.5.9 (embedded in binary)
[localhost crc-linux-1.16.0-amd64]$ 

The node tuning operator is running

Using project "openshift-cluster-node-tuning-operator".
[localhost crc-linux-1.16.0-amd64]$ oc get all
NAME                                               READY   STATUS    RESTARTS   AGE
pod/cluster-node-tuning-operator-576df548b-jmz8z   1/1     Running   0          26d
pod/tuned-d5frc                                    1/1     Running   0          26d

and vm.max_map_count is what it's supposed to be

[localhost crc-linux-1.16.0-amd64]$ oc debug node/crc-zqfk6-master-0
Starting pod/crc-zqfk6-master-0-debug ...
To use host binaries, run `chroot /host`
Pod IP: 192.168.126.11
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# cat /proc/sys/vm/max_map_count
262144

Definitely a regression.

from elasticsearch-operator.

ewolinetz avatar ewolinetz commented on September 24, 2024

@huikang this is specifically talking about the node's vm.max_map_count setting. There is normally an operator that updates this setting, I'm not sure if running this on CRC is different, I would need to defer to someone who has stood up a cluster on CRC in the past.

@vimalk78 @syedriko @periklis ?

from elasticsearch-operator.

syedriko avatar syedriko commented on September 24, 2024

@huikang For future issues, could you describe your environment and the steps that led to the issue you're reporting?

I repro'd what you're reporting on

[localhost crc-linux-1.17.0-amd64]$ ./crc version
CodeReady Containers version: 1.17.0+99f5c87
OpenShift version: 4.5.14 (embedded in binary)
[localhost crc-linux-1.17.0-amd64]$

with

apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
  name: "instance"
  namespace: "openshift-logging"
spec:
  managementState: "Managed"
  logStore:
    type: "elasticsearch"
    elasticsearch:
      nodeCount: 1
      storage: {}
      redundancyPolicy: "ZeroRedundancy"
      resources:
        requests:
          cpu: "100m"
          memory: "2Gi"
...

The fix for now is

[localhost crc]$ oc get nodes
NAME                 STATUS   ROLES           AGE   VERSION
crc-j55b9-master-0   Ready    master,worker   8d    v1.18.3+970c1b3
[localhost crc]$ oc debug node/crc-j55b9-master-0
Starting pod/crc-j55b9-master-0-debug ...
To use host binaries, run `chroot /host`
Pod IP: 192.168.126.11
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sh-4.4# sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144
sh-4.4# exit
exit
sh-4.2# exit
exit

Removing debug pod ...
[localhost crc]$ 

You'll need to reapply it every time you recreate the CRC VM.

As far as I recall, this used to be broken, then got fixed and now seems to have regressed.

from elasticsearch-operator.

huikang avatar huikang commented on September 24, 2024

@syedriko , thanks for the solution. Sure, I will report the crc version if I find anything in the future (the original post has been updated to include the crc version).

from elasticsearch-operator.

huikang avatar huikang commented on September 24, 2024

BTW, shall we report this to the CRC repo?

from elasticsearch-operator.

syedriko avatar syedriko commented on September 24, 2024

@huikang you're welcome to take this up with the CRC follks.
Here's the back story on this, including the node tuning operator @ewolinetz mentioned: https://developers.redhat.com/blog/2019/11/12/using-the-red-hat-openshift-tuned-operator-for-elasticsearch . In my CRC instance, I'm not seeing a separate openshift-node-es profile, but rather that vm.max_map_count=262144 applies to all cluster nodes. I'm not seeing a node tuning operator running in my CRC instance though ( https://docs.openshift.com/container-platform/4.5/scalability_and_performance/using-node-tuning-operator.html )

from elasticsearch-operator.

syedriko avatar syedriko commented on September 24, 2024

I filed an issue with CRC: crc-org/crc#1590

from elasticsearch-operator.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.