Giter Club home page Giter Club logo

chef-ulimit's Introduction

ulimit Cookbook

delivery Cookbook Version

This cookbook provides resources for managing ulimits configuration on nodes.

  • user_ulimit resource for overriding various ulimit settings. It places configured templates into /etc/security/limits.d/, named for the user the ulimit applies to.
  • ulimit_domain which allows for configuring complex sets of rules beyond those supported by the user_ulimit resource.

The cookbook also includes a recipe (default.rb) which allows ulimit overrides with the 'su' command on Ubuntu.

Requirements

Platforms

  • Debian/Ubuntu and derivatives
  • RHEL/Fedora and derivatives

Chef

  • Chef 12.7+

Cookbooks

  • none

Attributes

  • node['ulimit']['pam_su_template_cookbook'] - Defaults to nil (current cookbook). Determines what cookbook the su pam.d template is taken from
  • node['ulimit']['users'] - Defaults to empty Mash. List of users with their limits, as below.

Default Recipe

Instead of using the user_ulimit resource directly you may define user ulimits via node attributes. The definition may be made via an environment file, a role file, or in a wrapper cookbook. Note: The preferred way to use this cookbook is by directly defining resources as it is much easier to troubleshoot and far more robust.

Example role configuration:

"default_attributes": {
   "ulimit": {
      "users": {
         "tomcat": {
            "filehandle_limit": 8193,
               "process_limit": 61504
             },
            "hbase": {
               "filehandle_limit": 32768
             }
       }
    }
 }

To specify a change for all users change specify a wildcard resource or user name like so user_ulimit "*"

Resources

user_ulimit

The user_ulimit resource creates individual ulimit files that are installed into the /etc/security/limits.d/ directory.

Actions:

  • create
  • delete

Properties

  • username - Optional property to set the username if the resource name itself is not the username. See the example below.
  • filename - Optional filename to use instead of naming the file based on the username
  • filehandle_limit -
  • filehandle_soft_limit -
  • filehandle_hard_limit -
  • process_limit -
  • process_soft_limit -
  • process_hard_limit -
  • memory_limit -
  • virt_limit -
  • core_limit -
  • core_soft_limit -
  • core_hard_limit -
  • stack_soft_limit -
  • stack_hard_limit -
  • rtprio_limit -
  • rtprio_soft_limit -
  • rtprio_hard_limit -

Examples

Example of a resource where the resource name is the username:

user_ulimit "tomcat" do
  filehandle_limit 8192 # optional
  filehandle_soft_limit 8192 # optional; not used if filehandle_limit is set)
  filehandle_hard_limit 8192 # optional; not used if filehandle_limit is set)
  process_limit 61504 # optional
  process_soft_limit 61504 # optional; not used if process_limit is set)
  process_hard_limit 61504 # optional; not used if process_limit is set)
  memory_limit 1024 # optional
  core_limit 2048 # optional
  core_soft_limit 1024 # optional
  core_hard_limit 'unlimited' # optional
  stack_soft_limit 2048 # optional
  stack_hard_limit 2048 # optional
  rtprio_limit 60 # optional
  rtprio_soft_limit 60 # optional
  rtprio_hard_limit 60 # optional
end

Example where the resource name is not the username:

user_ulimit 'set filehandle ulimits for our tomcat user' do
  username 'tomcat'
  filehandle_soft_limit 8192
  filehandle_hard_limit 8192
end

ulimit_domain

Note: The ulimit_domain resource creates files named after the domain with no modifiers by default. To override this behavior, specify the filename parameter to the resource.

Actions:

  • create
  • delete

Examples:

ulimit_domain 'my_user' do
  rule do
    item :nofile
    type :hard
    value 10000
  end
  rule do
    item :nofile
    type :soft
    value 5000
  end
end

chef-ulimit's People

Contributors

abhiyerra avatar backslasher avatar bmhatfield avatar brianbianco avatar chrisroberts avatar dwradcliffe avatar jjlimepoint avatar kpumuk avatar markgibbons avatar moperacz avatar nhajratw avatar portertech avatar pushrax avatar szymonpk avatar tas50 avatar tomdoherty avatar troyready avatar vt0r avatar wolf31o2 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

chef-ulimit's Issues

Use of domain * results in a tricky filename

Not a hard bug per se, but if I wanted to set limits on all users for a given system, I'd typically use * domain.

ulimit_domain '*' do
  rule do
    item :nofile
    type :hard
    value 10000
  end
  rule do
    item :nofile
    type :soft
    value 5000
  end
end

Creates the file that I think I want, but it creates: /etc/security/limits.d/\*.conf - not the best filename I've seen this week.

I think the issue is that the domain_name attribute for the ulimit_domain is largely unseen, so that this would work just fine:

ulimit_domain 'systemwide' do
  domain_name '*'
...
end

So really a documentation bug. ๐Ÿ˜‰

ulimit_domain wrong filename extension

It seems that, at least on Ubuntu 12.04, files in limits.d must have .conf extension in order to work.

I have learned that the very hard way, after spending whole day trying to figure out why it is not applying limits per user.

I am guessing this is all the change needed.

# providers/domain.rb
def load_current_resource
  new_resource.filename new_resource.name unless new_resource.filename
  new_resource.filename "#{new_resource.filename}.conf" # added
...

I've tried and it works fine. If you want, I can add a PR.

providers/domain.rb typo: ufile -> utemplate

Saw a problem setting some limits:

  * ulimit_domain[pifs4] action create[2013-07-18T13:06:33+01:00] INFO: Processing ulimit_domain[pifs4] action create (pifs4::default line 36)
Recipe: <Dynamically Defined Resource>
  * ulimit_rule[ulimit_rule[pifs4:nofile-hard-524280]] action create[2013-07-18T13:06:33+01:00] INFO: Processing ulimit_rule[ulimit_rule[pifs4:nofile-hard-524280]] action create (dynamically defined)
 (up to date)
  * ulimit_rule[ulimit_rule[pifs4:nofile-soft-262140]] action create[2013-07-18T13:06:33+01:00] INFO: Processing ulimit_rule[ulimit_rule[pifs4:nofile-soft-262140]] action create (dynamically defined)
 (up to date)

================================================================================
Error executing action `create` on resource 'ulimit_domain[pifs4]'
================================================================================


NameError
---------
Cannot find a resource for ufile on centos version 6.4


Cookbook Trace:
---------------
/var/chef/cache/cookbooks/ulimit/providers/domain.rb:30:in `block in class_from_file'

Just a typo in metadata.rb:

diff --git a/metadata.rb b/metadata.rb
index 6daf2c7..969fb5e 100644
--- a/metadata.rb
+++ b/metadata.rb
@@ -4,4 +4,4 @@ license          "Apache 2.0"
 description      "Installs/Configures ulimit"
 long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
 name             "ulimit"
-version          "0.3.0"
+version          "0.3.1"
diff --git a/providers/domain.rb b/providers/domain.rb
index 829be90..6c78ab4 100644
--- a/providers/domain.rb
+++ b/providers/domain.rb
@@ -27,7 +27,7 @@ action :create do
   end

   unless(self.respond_to?(:use_inline_resources))
-    new_resource.updated_by_last_action(ufile.updated_by_last_action?)
+    new_resource.updated_by_last_action(utemplate.updated_by_last_action?)
   end

 end

template uses incorrect source

On Ubuntu 14.04 running Chef 11.12.4 get the following error:

Recipe: ulimit::default
  * template[/etc/pam.d/su] action create
================================================================================
Error executing action `create` on resource 'template[/etc/pam.d/su]'
================================================================================


Chef::Exceptions::FileNotFound
------------------------------
Cookbook 'ulimit' (0.3.3) does not contain a file at any of these locations:
  templates/ubuntu-14.04/service.erb
  templates/ubuntu/service.erb
  templates/default/service.erb

This cookbook _does_ contain: ['ulimit.erb','domain.erb','su.erb']


Resource Declaration:
---------------------
# In /root/.chef/local-mode-cache/cache/cookbooks/ulimit/recipes/default.rb

 22:     template "/etc/pam.d/su" do
 23:       cookbook ulimit['pam_su_template_cookbook']
 24:     end
 25: end



Compiled Resource:
------------------
# Declared in /root/.chef/local-mode-cache/cache/cookbooks/ulimit/recipes/default.rb:22:in `from_file'

template("/etc/pam.d/su") do
  provider Chef::Provider::Template
  action "create"
  retries 0
  retry_delay 2
  guard_interpreter :default
  path "/etc/pam.d/su"
  backup 5
  atomic_update true
  source "service.erb"
  variables {:conf_lines=>{"pam_rootok"=>{"interface"=>"auth", "control_flag"=>"sufficient", "name"=>"pam_rootok.so"}, "pam_env"=>{"interface"=>"session", "control_flag"=>"required", "name"=>"pam_env.so", "args"=>"readenv=1"}, "pam_env_locate"=>{"interface"=>"session", "control_flag"=>"required", "name"=>"pam_env.so", "args"=>"readenv=1 envfile=/etc/default/locale"}, "pam_mail"=>{"interface"=>"session", "control_flag"=>"optional", "name"=>"pam_mail.so", "args"=>"nopen"}, "pam_limits"=>{"interface"=>"session", "control_flag"=>"required", "name"=>"pam_limits.so"}}, :includes=>["common-auth", "common-account", "common-session"]}
  cookbook_name "ulimit"
  recipe_name "default"
  owner "root"
  group "root"
  mode 420
end




Running handlers:
[2015-05-15T01:37:49+00:00] ERROR: Running exception handlers
Running handlers complete

[2015-05-15T01:37:49+00:00] ERROR: Exception handlers complete
[2015-05-15T01:37:49+00:00] FATAL: Stacktrace dumped to /root/.chef/local-mode-cache/cache/chef-stacktrace.out

No attributes are overridden.
If you look at the compiled resource section, you'll notice that the source is service.erb instead of su.erb
Looking at the Chef 11.12.4 source code, the source should be su.erb: https://github.com/chef/chef/blob/11.12.4/lib/chef/resource/template.rb#L39

uninitialized constant Chef::Resource::UlimitDomain

Recipe Compile Error in /tmp/vagrant-chef-1/chef-solo-1/cookbooks/ulimit/resources/rule.rb

NameError

uninitialized constant Chef::Resource::UlimitDomain

Cookbook Trace:

/tmp/vagrant-chef-1/chef-solo-1/cookbooks/ulimit/resources/rule.rb:7:in `class_from_file'

Relevant File Content:

/tmp/vagrant-chef-1/chef-solo-1/cookbooks/ulimit/resources/rule.rb:

1: actions :create, :delete
2: default_action :create
3:
4: attribute :type, :kind_of => [Symbol,String], :required => true
5: attribute :item, :kind_of => [Symbol,String], :required => true
6: attribute :value, :kind_of => [String,Numeric], :required => true
7: attribute :domain, :kind_of => [Chef::Resource::UlimitDomain, String], :required => true
8:

Not working with Ubuntu

I tried everything, Chef run runs successfully, files are in /etc/security/limits.d

I include the recipe ulimit to overwrite the pam file under Ubuntu, rebooted, etc. I tried everything and ulimit -n is always returns 1024!

Here's my snippet

include_recipe 'ulimit'

# Boost up ulimit

user_ulimit "root" do
  filehandle_soft_limit 10000
  filehandle_hard_limit 30000
end

ulimit_domain 'wildcard' do
  domain_name '*'

  rule do
    item :nofile
    type :soft
    value 10000
  end

  rule do
    item :nofile
    type :hard
    value 30000
  end
end

Any ideas?

uninitialized constant Chef::Resource::UlimitRule when using ulimit_domain

Cookbook version

v0.4.0 from Supermarket

Chef-client version

12.17.44 (inside Docker via kitchen-dokken)

Platform Details

CentOS 7.4.1708
Running under kitchen-dokken via ChefDK

$ chef --version
Chef Development Kit Version: 2.0.28
chef-client version: 13.2.20
delivery version: master (17c1b0fed9be4c70f69091a6d21a4cbf0df60a23)
berks version: 6.2.0
kitchen version: 1.16.0
inspec version: 1.31.1

Scenario:

Attempting to set limits for HDFS (nofile, nproc) via this cookbook.

Steps to Reproduce:

.kitchen.dokken.yml:

driver:
  name: dokken
  privileged: true # systemd, docker, and sysctl, oh my!
  # Set this to configure the Chef version to test
  chef_version: <%= ENV['CHEF_VERSION'] || 'current' %>

transport:
  name: dokken

provisioner:
  name: dokken
  client_rb:
    treat_deprecation_warnings_as_errors: true
    resource_cloning: false
  # chef_log_level: debug

platforms:
  - name: centos-7
    driver:
      image: centos:7
      platform: rhel
      pid_one_command: /usr/lib/systemd/systemd
      intermediate_instructions:
        - RUN yum -y install lsof which systemd-sysv initscripts wget net-tools libselinux-utils make
suites:
  - name: default
    run_list:
    - recipe[cookbook::limits]

cookbook::limits.rb

include_recipe 'ulimit::default'

# loosen limits on files and processes
ulimit_domain 'hdfs' do
  rule do
    item 'nofile'
    type '-'
    value 'unlimited'
  end
  rule do
    item 'nproc'
    type '-'
    value 32768
  end
end

Steps to reproduce:

KITCHEN_LOCAL_YAML=.kitchen.dokken.yml kitchen converge default

Expected Result:

Convergence! A file hdfs.conf with my limits set.

Actual Result:

  * ulimit_domain[hdfs] action create

    ================================================================================
    Error executing action `create` on resource 'ulimit_domain[hdfs]'
    ================================================================================

    NameError
    ---------
    uninitialized constant Chef::Resource::UlimitRule

    Cookbook Trace:
    ---------------
    /opt/kitchen/cache/cookbooks/ulimit/providers/domain.rb:6:in `block in load_current_resource'
    /opt/kitchen/cache/cookbooks/ulimit/providers/domain.rb:5:in `map!'
    /opt/kitchen/cache/cookbooks/ulimit/providers/domain.rb:5:in `load_current_resource'

    Resource Declaration:
    ---------------------
    # In /opt/kitchen/cache/cookbooks/cookbook/recipes/limits.rb

     20: ulimit_domain 'hdfs' do
     21:   rule do
     22:     item 'nofile'
     23:     type '-'
     24:     value 'unlimited'
     25:   end
     26:   rule do
     27:     item 'nproc'
     28:     type '-'
     29:     value 32768
     30:   end
     31: end

    Compiled Resource:
    ------------------
    # Declared in /opt/kitchen/cache/cookbooks/cookbook/recipes/limits.rb:20:in `from_file'

    ulimit_domain("hdfs") do
      subresource_rules [[nil, #<Proc:0x0000000349f680@/opt/kitchen/cache/cookbooks/cookbook/recipes/limits.rb:21>], [nil, #<Proc:0x0000000349f900@/opt/kitchen/cache/cookbooks/cookbook/recipes/limits.rb:26>]]
      action [:create]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      declared_type :ulimit_domain
      cookbook_name "cookbook"
      recipe_name "limits"
      filename "hdfs.conf"
    end

    Platform:
    ---------
    x86_64-linux


Running handlers:
[2017-11-09T19:49:28+00:00] ERROR: Running exception handlers
Running handlers complete
[2017-11-09T19:49:28+00:00] ERROR: Exception handlers complete
Chef Client failed. 1 resources updated in 24 seconds
[2017-11-09T19:49:28+00:00] FATAL: Stacktrace dumped to /opt/kitchen/cache/chef-stacktrace.out
[2017-11-09T19:49:28+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2017-11-09T19:49:28+00:00] ERROR: ulimit_domain[hdfs] (cookbook::limits line 20) had an error: NameError: uninitialized constant Chef::Resource::UlimitRule
[2017-11-09T19:49:28+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <default-centos-7>.  Please see .kitchen/logs/default-centos-7.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

Convert the definition to a custom resource

The definition should be replaced by a custom resource. This can be done while maintaining the existing name, but should also use a standardized name. Moving to a custom resource solves resource cloning issues and also introduces reporting for Chef automate users

Undefined method error w/ user_ulimit on Chef 12

Cookbook version

370b4b0

Chef-client version

12.21.4

Platform Details

Ubuntu 16.04

Scenario:

Use the user_ulimit resource on Chef 12

Steps to Reproduce:

  1. Add a user_ulimit resource to a cookbook
  2. Attempt to converge using Chef 12

Expected Result:

Successful convergence

Actual Result:

Failure:

================================================================================
Recipe Compile Error in /tmp/kitchen/cache/cookbooks/ulimit/providers/domain.rb
================================================================================

NoMethodError
-------------
undefined method `action' for Chef::Provider::UlimitRule:Class

Cookbook Trace:
---------------
  /tmp/kitchen/cache/cookbooks/ulimit/providers/rule.rb:8:in `<class:UlimitRule>'
  /tmp/kitchen/cache/cookbooks/ulimit/providers/rule.rb:1:in `<top (required)>'
  /tmp/kitchen/cache/cookbooks/ulimit/providers/domain.rb:1:in `require_relative'
  /tmp/kitchen/cache/cookbooks/ulimit/providers/domain.rb:1:in `class_from_file'

Relevant File Content:
----------------------
/tmp/kitchen/cache/cookbooks/ulimit/providers/rule.rb:

  1:  class Chef::Provider::UlimitRule < Chef::Provider
  2:    def load_current_resource
  3:      new_resource.domain new_resource.domain.domain_name if new_resource.domain.is_a?(Chef::Resource)
  4:      node.run_state[:ulimit] ||= Mash.new
  5:      node.run_state[:ulimit][new_resource.domain] ||= Mash.new
  6:    end
  7:
  8>>   action :create do # ~FC017
  9:      node.run_state[:ulimit][new_resource.domain][new_resource.item] ||= Mash.new
 10:      node.run_state[:ulimit][new_resource.domain][new_resource.item][new_resource.type] = new_resource.value
 11:    end
 12:
 13:    action :delete do
 14:      # NOOP
 15:    end
 16:  end
 17:

This works well under Chef 13.

verify fields exist before setting them

I encountered a problem today where all my filehandle_limits were not being set by the attributes of my role. After some research, it turned out I had written file_handle_limit. It would be lovely if the cookbook would verify the names of the limits being set and throw an error if they do not exist.

License specifies "All rights reserved"

@bmhatfield tks for the great lwrp!

The license for this cookbook specifcies that all rights are reserved and that it shouldn't be redistributed.

Is that intentional or an oversight? I assume not since you put it on github

new tag?

It's been a while (October 2013) since you have made a release/tag. Now that chef-dk and berkshelf are the standard chef cookbook managment tools this sha: 936d3c8 fixes your cookbook from breaking in berkshelf per this issue: berkshelf/berkshelf#1019.

Thanks for all your contributions and apologies if this is not appropriate as an "issue".

Add a test recipe that uses all the resources

Right now we test the default recipe that uses the definition. We should create a test recipe that tests a few different scenarios so we have greater confidence before releasing the cookbook

The default.rb requires ulimit['users'] to be set

I am on Ubuntu and want the updated pam.d/su from template, but do not need to set ulimits via Role. I use the LWRP elsewhere, and some cookbooks provide their own files for limits.d...

Can we simply wrap the ulimit['users'].each in a check for the key?

if (node.has_key? 'ulimit' && node['ulimit'].has_key? 'users')
... (second resource) ...
end

wrong number of arguments on ulimit_domain (breaks with chef 16?)

Cookbook version

1.1.0

Chef-client version

16.1.16

Platform Details

Centos 7.7 - Azure

Scenario:

Using ulimit_domain

Steps to Reproduce:

Resource declaration:

ulimit_domain 'elasticsearch' do
  rule do
    item :nofile
    type :hard
    value 65_535
  end
  rule do
    item :nofile
    type :soft
    value 65_535
  end
  rule do
    item :memlock
    type :hard
    value 'unlimited'
  end
  rule do
    item :memlock
    type :soft
    value 'unlimited'
  end
end

Running it using kitchen-azurerm

Expected Result:

Expected to apply the resource correctly

Actual Result:

================================================================================
Error executing action `create` on resource 'ulimit_domain[elasticsearch]'
================================================================================

ArgumentError
-------------
wrong number of arguments (given 0, expected 1..3)

Cookbook Trace:
---------------
/tmp/kitchen/cache/cookbooks/ulimit/libraries/domain.rb:20:in `instance_eval'
/tmp/kitchen/cache/cookbooks/ulimit/libraries/domain.rb:20:in `block (2 levels) in <class:UlimitDomain>'
/tmp/kitchen/cache/cookbooks/ulimit/libraries/domain.rb:16:in `map!'
/tmp/kitchen/cache/cookbooks/ulimit/libraries/domain.rb:16:in `block in <class:UlimitDomain>'

Some insight

I tested with chef-client version 15.9.17 and still was working, maybe something in chef 16 breaks this cookbook ๐Ÿ˜•

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.