Giter Club home page Giter Club logo

mixlib-shellout's Introduction

Mixlib::ShellOut

Build Status Gem Version

Umbrella Project: Chef Foundation

Project State: Active

Issues Response Time Maximum: 14 days

Pull Request Response Time Maximum: 14 days

Provides a simplified interface to shelling out while still collecting both standard out and standard error and providing full control over environment, working directory, uid, gid, etc.

Example

Simple Shellout

Invoke find(1) to search for .rb files:

  require 'mixlib/shellout'
  find = Mixlib::ShellOut.new("find . -name '*.rb'")
  find.run_command

If all went well, the results are on stdout

  puts find.stdout

find(1) prints diagnostic info to STDERR:

  puts "error messages" + find.stderr

Raise an exception if it didn't exit with 0

  find.error!

Advanced Shellout

In addition to the command to run there are other options that can be set to change the shellout behavior. The complete list of options can be found here: https://github.com/chef/mixlib-shellout/blob/master/lib/mixlib/shellout.rb

Run a command as the www user with no extra ENV settings from /tmp with a 1s timeout

  cmd = Mixlib::ShellOut.new("apachectl", "start", :user => 'www', :environment => nil, :cwd => '/tmp', :timeout => 1)
  cmd.run_command # etc.

STDIN Example

Invoke crontab to edit user cron:

  # :input only supports simple strings
  crontab_lines = [ "* * * * * /bin/true", "* * * * * touch /tmp/here" ]
  crontab = Mixlib::ShellOut.new("crontab -l -u #{@new_resource.user}", :input => crontab_lines.join("\n"))
  crontab.run_command

Windows Impersonation Example

Invoke "whoami.exe" to demonstrate running a command as another user:

  whoami = Mixlib::ShellOut.new("whoami.exe", :user => "username", :domain => "DOMAIN", :password => "password")
  whoami.run_command

Invoke "whoami.exe" with elevated privileges:

  whoami = Mixlib::ShellOut.new("whoami.exe", :user => "username", :domain => "DOMAIN", :password => "password", :elevated => true)
  whoami.run_command

NOTE: The user 'admin' must have the 'Log on as a batch job' permission and the user chef is running as must have the 'Replace a process level token' and 'Adjust Memory Quotas for a process' permissions.

Platform Support

Mixlib::ShellOut does a standard fork/exec on Unix, and uses the Win32 API on Windows. There is not currently support for JRuby.

See Also

Contributing

For information on contributing to this project see https://github.com/chef/chef/blob/master/CONTRIBUTING.md

License

  • Copyright:: Copyright (c) 2011-2016 Chef Software, Inc.
  • License:: Apache License, Version 2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

mixlib-shellout's People

Contributors

akshaykarle avatar backslasher avatar btm avatar chef-ci avatar chef-expeditor[bot] avatar coderanger avatar danielsdeleo avatar dheerajd-msys avatar docwhat avatar hosh avatar jaym avatar lamont-granquist avatar maxlinc avatar mdkent avatar mwrock avatar phiggins avatar piyushawasthi avatar poorndm avatar prajaktapurohit avatar ryancragun avatar schisamo avatar skeshari12 avatar smurawski avatar stevenproctor avatar tas50 avatar tduffield avatar thcipriani avatar thommay avatar tyler-ball avatar zenspider avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mixlib-shellout's Issues

TypeError in reap_errant_child

I occasionally run into this error:

/opt/boxen/rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:350:in `kill': no implicit conversion from nil to integer (TypeError)
    from /opt/boxen/rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:350:in `reap_errant_child'
    from /opt/boxen/rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:97:in `ensure in run_command'
    from /opt/boxen/rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:105:in `run_command'
    from /opt/boxen/rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout.rb:227:in `run_command'

It's trying to send a kill to child_pgid which is nil. Since the ensure block already called should_read? it seems that @child_pid is not nil, but child_pgid is.

I am testing on OSX, so while I'm not 100% sure what causes this situation and don't have an easy way to reproduce it on demand I suspect the handling of zombie processes on OSX:

      def get_child_pgid
        # The behavior of Process.getpgid (see also getpgid(2) ) when the
        # argument is the pid of a zombie isn't well specified. On Linux it
        # works, on OS X it returns ESRCH (which ruby turns into Errno::ESRCH).
        #
        # If the child dies very quickly, @child_pid may be a zombie, so handle
        # ESRCH here.
        @child_pgid = -Process.getpgid(@child_pid)
      rescue Errno::ESRCH
        @child_pgid = nil
      end

I'm looking for either a fix that avoids this error, or at least to catch TypeError and re-raise something more specific so I don't have to rescue TypeError. I'm currently rescuing SystemCallError (which covers all the Errno::* exceptions) and Mixlib::ShellOut::RuntimeError.

CreateProcessW(): Invalid access to memory location

I'm getting these errors randomly (and hard to reproduce) on my boxen:

Chef::Exceptions::MultipleFailures: Multiple failures occurred:
* SystemCallError occurred in chef run: powershell_script[ssl-cert: permissions] (bigdatalab-role-windows::_ssl_cert line 47) had an error: SystemCallError: Invalid access to memory location. - CreateProcessW
* SystemCallError occurred in delayed notification: windows_task[\chef-client] (chef-client::task line 32) had an error: SystemCallError: Invalid access to memory location. - CreateProcessW


C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/mixlib-shellout-2.1.0-universal-mingw32/lib/mixlib/shellout/windows/core_ext.rb:346:in `create'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/mixlib-shellout-2.1.0-universal-mingw32/lib/mixlib/shellout/windows.rb:86:in `run_command'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/mixlib-shellout-2.1.0-universal-mingw32/lib/mixlib/shellout.rb:259:in `run_command'
C:/opscode/chef/embedded/apps/chef/lib/chef/mixin/shell_out.rb:97:in `shell_out_command'
C:/opscode/chef/embedded/apps/chef/lib/chef/mixin/shell_out.rb:50:in `shell_out'
C:/opscode/chef/embedded/apps/chef/lib/chef/mixin/shell_out.rb:55:in `shell_out!'
C:/opscode/chef/embedded/apps/chef/lib/chef/provider/powershell_script.rb:79:in `block in validate_script_syntax!'
C:/opscode/chef/embedded/lib/ruby/2.0.0/tempfile.rb:324:in `open'
C:/opscode/chef/embedded/apps/chef/lib/chef/provider/powershell_script.rb:64:in `validate_script_syntax!'
C:/opscode/chef/embedded/apps/chef/lib/chef/provider/powershell_script.rb:33:in `action_run'
C:/opscode/chef/embedded/apps/chef/lib/chef/provider.rb:140:in `run_action'
C:/opscode/chef/embedded/apps/chef/lib/chef/resource.rb:584:in `run_action'
C:/opscode/chef/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
C:/opscode/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
C:/opscode/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
C:/opscode/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
C:/opscode/chef/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:83:in `block in execute_each_resource'
C:/opscode/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
C:/opscode/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
C:/opscode/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
C:/opscode/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
C:/opscode/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
C:/opscode/chef/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:81:in `execute_each_resource'
C:/opscode/chef/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:654:in `block in converge'
C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:649:in `catch'
C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:649:in `converge'
C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:688:in `converge_and_save'
C:/opscode/chef/embedded/apps/chef/lib/chef/client.rb:269:in `run'
C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:252:in `run_with_graceful_exit_option'
C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:229:in `block in run_chef_client'
C:/opscode/chef/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:212:in `run_chef_client'
C:/opscode/chef/embedded/apps/chef/lib/chef/application/client.rb:375:in `run_application'
C:/opscode/chef/embedded/apps/chef/lib/chef/application.rb:60:in `run'
C:/opscode/chef/embedded/apps/chef/bin/chef-client:26:in `<top (required)>'
C:/opscode/chef/bin/chef-client:67:in `load'
C:/opscode/chef/bin/chef-client:67:in `<main>'

They don't happen at any particular recipe for any particular cookbook. That is, they are pretty random.

What I think is happening is that CreateProcessW() is passing in a "constant" version of cmd. According to the MSDN Documentation's for lpCommandLine:

The Unicode version of this function, CreateProcessW, can modify the contents of this string. Therefore, this parameter cannot be a pointer to read-only memory (such as a const variable or a literal string). If this parameter is a constant string, the function may cause an access violation.

The error message "Invalid access to memory location" is consistent with the second argument (lpCommandLine) being within read-only memory.

I think the fix is to change core_ext.rb line 226 to use an allocated bit of memory:

    if hash['command_line']
      cmd = FFI::MemoryPointer.new(:string, 1024)
      cmd.write_string hash['command_line'].to_wide_string
    end

I'm not sure 1024 is the right value. Other examples I have seen use things like (reverting to C/C++) the following:

WCHAR lpCommandLine[MAX_PATH * 2];
/* Make a copy of cmd so CreateProcessW() can modify it if it needs to. */
_tcscpy_s(lpCommandLine, MAX_PATH *2, (WCHAR *)cmd);

or

// http://stackoverflow.com/questions/4514027/createprocessw-acess-violation-in-rtlinitunicodestring

//I'm copying the string here because CreateProcessW mutates its arguments
wchar_t *lpCommandLine = _wcsdup(cmd);
...
free(lpCommandLine);

If it helps, there is a Microsoft blog post about why CreateProccesW() has historically had this behavior: http://blogs.msdn.com/b/oldnewthing/archive/2009/06/01/9673254.aspx

Chef recepe create Windows_Task throw Error "SystemCallError: Invalid access to memory location. - CreateProcessW"

Hi,

We have chef running on 100 machines every 30mins and constantly see random instance failed on "SystemCallError: Invalid access to memory location. - CreateProcessW" error. Totally random no easy way to replicate this issue locally.

The recipe is to create a windows scheduled job:
windows_task 'Schedule_Job' do
action :create
task_name 'Schedule_Job'
command 'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe C:\scripts\Schedule_Job.ps1'
frequency :hourly
end

Error:
windows_task[Schedule_Job] (server::Schedule_Job line 28) had an error: SystemCallError: Invalid access to memory location. - CreateProcessW

I have googled how to fix this issue. But the following links ended up in no solution.
#111
https://bugs.ruby-lang.org/issues/11515

Could anyone help me or give me some insight why this happens?

Thank you
Vanessa

clean_parent_file_descriptors not working when parent shell is closed

Was at https://tickets.opscode.com/browse/MIXLIB-29

When launching chef-client in daemon mode, and then closing the ssh session to launch it, I get these errors :

Apr 24 19:32:37 vty-admin1 chef-client[6013]: [2014-04-24T19:32:37+02:00] INFO: ######## /usr/local/lib/ruby/gems/1.9/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:231: [BUG] rb_update_max_fd: invalid fd (5) given.
ruby 1.9.3p484 (2013-11-22 revision 43786) [amd64-freebsd9]

-- Control frame information -----------------------------------------------
c:0043 p:---- s:0152 b:0152 l:000151 d:000151 CFUNC  :for_fd
c:0042 p:0045 s:0148 b:0148 l:000137 d:000147 BLOCK  /usr/local/lib/ruby/gems/1.9/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:231
c:0041 p:---- s:0144 b:0144 l:000143 d:000143 FINISH
c:0040 p:---- s:0142 b:0142 l:000141 d:000141 CFUNC  :upto
c:0039 p:0014 s:0138 b:0138 l:000137 d:000137 METHOD /usr/local/lib/ruby/gems/1.9/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:225
c:0038 p:0035 s:0135 b:0135 l:000126 d:000134 BLOCK  /usr/local/lib/ruby/gems/1.9/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:311
c:0037 p:---- s:0132 b:0132 l:000131 d:000131 FINISH
c:0036 p:---- s:0130 b:0130 l:000129 d:000129 CFUNC  :fork 
c:0035 p:0021 s:0127 b:0127 l:000126 d:000126 METHOD /usr/local/lib/ruby/gems/1.9/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:300
c:0034 p:0011 s:0124 b:0124 l:000123 d:000123 METHOD /usr/local/lib/ruby/gems/1.9/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/unix.rb:52
c:0033 p:0091 s:0120 b:0120 l:000119 d:000119 METHOD /usr/local/lib/ruby/gems/1.9/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout.rb:227
c:0032 p:0033 s:0116 b:0116 l:000115 d:000115 METHOD /usr/local/lib/ruby/gems/1.9/gems/ohai-7.0.2/lib/ohai/mixin/command.rb:33
c:0031 p:0011 s:0111 b:0111 l:002368 d:000110 BLOCK  /usr/local/lib/ruby/gems/1.9/gems/ohai-7.0.2/lib/ohai/plugins/freebsd/platform.rb:23
c:0030 p:---- s:0108 b:0108 l:000107 d:000107 FINISH
c:0029 p:---- s:0106 b:0106 l:000105 d:000105 CFUNC  :instance_eval
c:0028 p:0059 s:0103 b:0103 l:000102 d:000102 METHOD /usr/local/lib/ruby/gems/1.9/gems/ohai-7.0.2/lib/ohai/dsl/plugin/versionvii.rb:88

The easiest way to reproduce is to open an ssh session to a FreeBSD server, then run :

$ sudo pkg install rubygem-chef
$ chef-client -z -d -i 10 -s 0 -L /tmp/chef-client.log

You can tail /tmp/chef-client.log, and see the run going fine. Then close the SSH session, and the run will hang.

I have discussed this with Brian, and we dont have yet an smaller repro case.

May be related to https://bugs.ruby-lang.org/issues/8594, and tied to ruby/ruby@2bcd502
Alternatives to patching the ruby VM would be great, to have a quick fix.

Windows Shellout Fails with wrong variable type (expect boolean)

I am trying to use mixlib::shellout inside a ruby block and I keep getting the following error:

TypeError: wrong argument type (expected a boolean parameter)

Stacktrace:
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/mixlib-shellout-2.2.5-universal-mingw32/lib/mixlib/shellout/windows/core_ext.rb:293:in CreateProcessAsUserW' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/mixlib-shellout-2.2.5-universal-mingw32/lib/mixlib/shellout/windows/core_ext.rb:293:increate'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/mixlib-shellout-2.2.5-universal-mingw32/lib/mixlib/shellout/windows.rb:80:in run_command' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/mixlib-shellout-2.2.5-universal-mingw32/lib/mixlib/shellout.rb:259:inrun_command'
C:/chef/cache/chef/cookbooks/pw_ad_config/recipes/server_groups.rb:46:in block (2 levels) in from_file' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/provider/ruby_block.rb:35:incall'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/provider/ruby_block.rb:35:in block in action_run' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/mixin/why_run.rb:52:incall'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/mixin/why_run.rb:52:in add_action' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/provider.rb:175:inconverge_by'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/provider/ruby_block.rb:34:in action_run' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/provider.rb:144:inrun_action'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/resource.rb:596:in run_action' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/runner.rb:74:inrun_action'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/runner.rb:106:in block (2 levels) in converge' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/runner.rb:106:ineach'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/runner.rb:106:in block in converge' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/resource_collection/resource_list.rb:83:inblock in execute_each_resource'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:116:in call' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:116:incall_iterator_block'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:85:in step' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:104:initerate'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:55:in each_with_index' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/resource_collection/resource_list.rb:81:inexecute_each_resource'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/runner.rb:105:in converge' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/client.rb:658:inblock in converge'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/client.rb:653:in catch' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/client.rb:653:inconverge'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/client.rb:692:in converge_and_save' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/client.rb:271:inrun'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/application.rb:243:in run_with_graceful_exit_option' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/application.rb:220:inblock in run_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/local_mode.rb:44:in with_server_connectivity' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/application.rb:203:inrun_chef_client'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/application/client.rb:386:in run_application' C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/lib/chef/application.rb:58:inrun'
C:/opscode/chef/embedded/lib/ruby/gems/2.0.0/gems/chef-12.6.0-universal-mingw32/bin/chef-client:26:in `<top (required)>

My Code looks like:
ruby_block 'Adding a host to the right Computer Group' do
block do
res_cmd = "dsmod group "CN=#{update_grp},OU=Global,OU=Groups"....
res_grp = Mixlib::ShellOut.new(res_cmd,
user: ad_user,
domain: 'mgmt.cloud.ds.ge.com',
password: ad_pass)
res_grp.run_command
res_grp.stdout
end
end

Note: I have removed part of the res_cmd string.

If I manually run the command in a windows cmd shell, it works and when i manually run chef-client.bat it works, but running in the chef-client (as a service) it fails.

If in: https://github.com/chef/mixlib-shellout/blob/2.2.5/lib/mixlib/shellout/windows/core_ext.rb

In my testing I core_ext: https://gist.github.com/jwitrick/e87bc127ea6f1918da77

diff --git a/core_ext.rb b/core_ext.rb
index d889778..0094559 100644
--- a/core_ext.rb
+++ b/core_ext.rb
@@ -288,6 +288,13 @@ module Process
end

       token = token.read_ulong
  •      if inherit.is_a?(Integer)
    
  •        inherit = if inherit == 1
    
  •                    true
    
  •                  else
    
  •                    false
    
  •                  end
    
  •      end
    
       begin
         bool = CreateProcessAsUserW(
    

My setup:
Windows2012R2
Chef-12.6.0
chef is running as a service.

DEFAULT_READ_TIMEOUT should be exposed higher

Consider this a feature request. Currently, Mixlib::Shellout will kill a long running command after 600 seconds (see: https://github.com/chef/mixlib-shellout/blob/master/lib/mixlib/shellout.rb#L29).

In order to account for long running compiles during Chef runs or downloads, (vagrant box add, for example) it might make sense to expose DEFAULT_READ_TIMEOUT as an ENV variable so that the user can configure it to suit their needs without having to go hacking the gems that call shell_out().

Array form of command passing doesn't work on Windows

The windows backend assumes in several places that command is a string, but it can be an array too. The simplest fix would be to use Shellwords.join or something similar to convert to a string, but supporting the same semantics as unix would be nice. On unix, passing an array instead of a string bypasses the /bin/sh -c wrapper on the command which helps avoid bugs/security issues from injecting shell parsing/handling where it isn't expected.

Mixlib::Shellout secondary groups not set in USER session

Hi,

I'm testing on RHEL 5,6,7 and require a valid OS login in order to complete some configuration. I need to have the target OS user session fully instantiated as my subsequent processing fails with
"CAUSE: The installation user account must be a member of all groups required for installation."

I have tried setting :with_logon => true and :login => true in my new() but this fails with

error: Mixlib::ShellOut::InvalidCommandOption: option ':with_logon' is not a valid option for Mixlib::ShellOut

I'm running CHEF client 11.x and Mixlib::ShellOut version 1.6.1. As far as I can see, this option should be valid.

I cant find any documentation or examples where there are complex uses for Mixlib::ShellOut. If there are examples anywhere of more complex uses of this library I would love to see them.

Thanks,
Goran

Preventing environment variable expansion in windows displays unexpected output

I tried to track down the root cause of this but was unable to in the time I had available.

In cmd.exe, running this echo %^COMPUTERNAME% outputs %COMPUTERNAME%

In mixlib/shellout, running

blah = Mixlib::ShellOut.new("echo %^COMPUTERNAME%")
blah.run_command.stdout

outputs %^COMPUTERNAME%. I would expect the carat to be removed like it is in cmd.exe.

The end result of this behavior is that I can't run commands that expected unexpanded environment variables as arguments.

mixlib-shellout requires Ruby version >= 1.9.3.

however ,my ruby version is 1.9.3.

sudo gem install mixlib-shellout
ERROR: Error installing mixlib-shellout:
mixlib-shellout requires Ruby version >= 1.9.3.

os is ubuntu 10.04.4 LTS .how to resolve it

Dependabot can't parse your Gemfile

Dependabot couldn't parse the Gemfile found at /Gemfile.

The error Dependabot encountered was:

Dependabot only supports uninterpolated string arguments to eval_gemfile. Got `__FILE__ + ".local"`

Foodcritic whining for FC048: Prefer Mixlib::ShellOut:

My code block:

cmd = Mixlib::ShellOut.new("/usr/bin/echo test",
    user: (node['webhook_api']['app_username']).to_s)
    cmd.run_command

Then foodcritic stops whining when change to Mixlib::Shellout: but my code converge will fail.

Help.. Thank you

Doesn't take array form on windows

PS C:\Users\vagrant\Desktop> irb -rmixlib/shellout
irb(main):001:0> Mixlib::ShellOut.new(*%w[echo hello world]).run_command
Traceback (most recent call last):
        8: from C:/opscode/chefdk/embedded/bin/irb.cmd:31:in `<main>'
        7: from C:/opscode/chefdk/embedded/bin/irb.cmd:31:in `load'
        6: from C:/opscode/chefdk/embedded/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
        5: from (irb):1
        4: from C:/opscode/chefdk/embedded/lib/ruby/gems/2.6.0/gems/mixlib-shellout-2.4.4-universal-mingw32/lib/mixlib/shellout.rb:267:in `run_command'
        3: from C:/opscode/chefdk/embedded/lib/ruby/gems/2.6.0/gems/mixlib-shellout-2.4.4-universal-mingw32/lib/mixlib/shellout/windows.rb:69:in `run_command'
        2: from C:/opscode/chefdk/embedded/lib/ruby/gems/2.6.0/gems/mixlib-shellout-2.4.4-universal-mingw32/lib/mixlib/shellout/windows.rb:200:in `command_to_run'
        1: from C:/opscode/chefdk/embedded/lib/ruby/gems/2.6.0/gems/mixlib-shellout-2.4.4-universal-mingw32/lib/mixlib/shellout/windows.rb:269:in `should_run_under_cmd?'
NoMethodError (undefined method `each_char' for ["echo", "hello", "world"]:Array)
Did you mean?  each_cons

mixlib-shellout v2.1.0 breaks vagrant-docker

Not sure exactly what happened, but vagrant-docker stopped working when mixlib-shellout, goes from v2.0.1 to v2.1.0:
https://github.com/chef/mixlib-shellout/blob/master/CHANGELOG.md#release-210

I could do more digging, but adding this would seem to be the most communicative and effective way of fixing this for now:

gem "mixlib-shellout", "< 2.1.0"

This was the error that appeared:

~/repos/chef-confluence (master โœ˜)โœนโœญ แ… kitchen list
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::UserError
>>>>>> Message: You must first install the Docker CLI tool http://www.docker.io/gettingstarted/
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

I have docker installed and can use it fine and soon as I add that constraint :)

EDIT: updated incorrect error paste

`should_run_under_cmd?` fails on windows when argument is an array

This issue comes from something discovered in chef-cookbooks/habitat PR.

When using similar to the following:

shell_out_with_timeout!(clean_array('hab', ['pkg', 'path', 'skylerto/splunkforwarder']))
NoMethodError
-------------
undefined method `each_char' for ["hab", "pkg", "path", "skylerto/splunkforwarder"]:Array
Did you mean?  each
    each_cons

and debug stacktrace:

C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7-universal-mingw32/lib/mixlib/shellout/windows.rb:266:in `should_run_under_cmd?'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7-universal-mingw32/lib/mixlib/shellout/windows.rb:191:in `command_to_run'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7-universal-mingw32/lib/mixlib/shellout/windows.rb:59:in `run_command'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/mixlib-shellout-2.2.7-universal-mingw32/lib/mixlib/shellout.rb:259:in `run_command'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/mixin/shell_out.rb:191:in `shell_out_command'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/mixin/shell_out.rb:112:in `shell_out'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/mixin/shell_out.rb:117:in `shell_out!'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/provider/package.rb:637:in `shell_out_with_timeout!'
  C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/cookbooks/habitat/libraries/provider_hab_package.rb:90:in `hab'
  C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/cookbooks/habitat/libraries/provider_hab_package.rb:144:in `installed_version'
  C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/cookbooks/habitat/libraries/provider_hab_package.rb:139:in `block in current_versions'
  C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/cookbooks/habitat/libraries/provider_hab_package.rb:138:in `map'
  C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/cookbooks/habitat/libraries/provider_hab_package.rb:138:in `current_versions'
  C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/cookbooks/habitat/libraries/provider_hab_package.rb:59:in `load_current_resource'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/provider.rb:128:in `run_action'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/resource.rb:622:in `run_action'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/runner.rb:69:in `run_action'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/runner.rb:97:in `block (2 levels) in converge'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/runner.rb:97:in `each'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/runner.rb:97:in `block in converge'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/resource_collection/resource_list.rb:94:in `block in execute_each_resource'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:114:in `call_iterator_block'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:103:in `iterate'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/resource_collection/resource_list.rb:92:in `execute_each_resource'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/runner.rb:96:in `converge'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/client.rb:715:in `block in converge'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/client.rb:710:in `catch'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/client.rb:710:in `converge'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/client.rb:749:in `converge_and_save'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/client.rb:286:in `run'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application.rb:277:in `run_with_graceful_exit_option'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application.rb:253:in `block in run_chef_client'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/local_mode.rb:44:in `with_server_connectivity'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application.rb:236:in `run_chef_client'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application/client.rb:450:in `loop'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application/client.rb:450:in `interval_run_chef_client'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application/client.rb:434:in `run_application'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/lib/chef/application.rb:59:in `run'
  C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.20.3-universal-mingw32/bin/chef-client:26:in `<top (required)>'
  C:/opscode/chef/bin/chef-client:68:in `load'
  C:/opscode/chef/bin/chef-client:68:in `<main>'

shellout on windows may fail when non ASCII chars are in the path

see chef-boneyard/windows#332 where a resource fails and a shell out is involved and a non ascii characher (like รผ) is in the path. It is raising:

Encoding::CompatibilityError: windows_package[Ruby 2.1.7-p400] (windev::packages line 24) had an error: Encoding::CompatibilityError: incompatible character encodings: IBM437 and ASCII-8BIT
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.3-universal-mingw32/lib/mixlib/shellout/windows/core_ext.rb:132:in `join'
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.3-universal-mingw32/lib/mixlib/shellout/windows/core_ext.rb:132:in `create'
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.3-universal-mingw32/lib/mixlib/shellout/windows.rb:80:in `run_command'
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.3-universal-mingw32/lib/mixlib/shellout.rb:259:in `run_command'
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1-universal-mingw32/lib/chef/mixin/shell_out.rb:97:in `shell_out_command'
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1-universal-mingw32/lib/chef/mixin/shell_out.rb:50:in `shell_out'
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1-universal-mingw32/lib/chef/mixin/shell_out.rb:55:in `shell_out!'
l:/scripts/local-mode-cache/cache/cookbooks/windows/libraries/windows_package.rb:102:in `install_package'
l:/scripts/local-mode-cache/cache/cookbooks/windows/libraries/windows_package.rb:29:in `block in <class:WindowsCookbookPackage>'
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1-universal-mingw32/lib/chef/provider/lwrp_base.rb:86:in `instance_eval'
C:/opscode/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1-universal-mingw32/lib/chef/provider/lwrp_base.rb:86:in `block in action'

Dependabot can't evaluate your Ruby dependency files

Dependabot can't evaluate your Ruby dependency files.

As a result, Dependabot couldn't check whether any of your dependencies are out-of-date.

The error Dependabot encountered was:

Bundler::Dsl::DSLError with message: 
[!] There was an error parsing `Gemfile`: 
[!] There was an error while loading `mixlib-shellout-universal-mingw32.gemspec`: (eval):1: unexpected fraction part after numeric literal
0.0.1
^~~. Bundler cannot continue.

 #  from /home/dependabot/dependabot-updater/dependabot_tmp_dir/mixlib-shellout-universal-mingw32.gemspec:1
 #  -------------------------------------------
 >  gemspec = instance_eval("0.0.1")
 #  
 #  -------------------------------------------
. Bundler cannot continue.

 #  from /home/dependabot/dependabot-updater/dependabot_tmp_dir/Gemfile:3
 #  -------------------------------------------
 #  
 >  gemspec name: "mixlib-shellout"
 #  
 #  -------------------------------------------

View the update logs.

Regression on OS X in 2.1.0

Hi!

I recently noticed an odd new behavior in mixlib-shellout on OS X and eventually traced it back to a place a bit out of my depth and this change that was released in 2.1.0.

In OS X (10.10.5 if it makes a difference), the shell out to hdiutil attach in the dmg cookbook no longer works, always exiting with status 1.

As easy-ish demonstration, download a .dmg package...

wget -P /tmp/ https://steamcdn-a.akamaihd.net/client/installer/steam.dmg

...and try to mount it with a shell out to hdiutil...

require 'mixlib/shellout'

sh = Mixlib::ShellOut.new("echo Y | PAGER=true hdiutil attach '/tmp/steam.dmg' -quiet")
sh.run_command
sh.error!
puts sh.stdout

This code fails under 2.1.0, but works under 2.0.1 or if you just change Process.setsid back to Process.setpgrp.

Oddly, it passes in 2.1.0 if the -quiet at the end is removed. According to the hdiutil man page:

-quiet   close stdout and stderr, leaving only hdiutil's exit status to indicate success or failure.
            No /dev entries or mount points will be printed.
            -debug and -verbose disable -quiet.

Anyone with more experience with this project have any thoughts what might be going on? I can submit a tiny patch to the dmg cookbook and move along, but wanted to let folks know in case this is really a larger problem than just that use case.

Thanks!

Mixlib::ShellOut strips color from output

See output below. Let me know if there's any more information I can provide.

2017-04-19-122531_1276x1353_scrot

Transcript of the irb session:

irb(main):001:0> require 'mixlib/shellout'
=> true
irb(main):002:0> c = Mixlib::ShellOut.new("fortune | cowsay | lolcat")
=> <Mixlib::ShellOut#18198020: command: 'fortune | cowsay | lolcat' process_status: nil stdout: '' stderr: '' child_pid: nil environment: {} timeout: 600 user:  group:  working_dir:  >
irb(main):003:0> c.run_command; nil
=> nil
irb(main):004:0> c.stdout
=> " _________________________________________ \n/ \"Can you program?\" \"Well, I'm literate, \\\n\\ if that's what you mean!\"               /\n ----------------------------------------- \n        \\   ^__^\n         \\  (oo)\\_______\n            (__)\\       )\\/\\\n                ||----w |\n                ||     ||\n"
irb(main):005:0> puts c
#<Mixlib::ShellOut:0x000000022b5c08>
=> nil
irb(main):006:0> puts c.stdout
 _________________________________________ 
/ "Can you program?" "Well, I'm literate, \
\ if that's what you mean!"               /
 ----------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
=> nil
irb(main):007:0>

Add gemspec for 'x64-mingw32' platform

Just ran into this with Ruby 2.0.0p481 (64 bits) on Win7:

D:\Repos\_github\_cookbooks\sample-application-cookbook>rake test
knife cookbook test sample-application-cookbook -o ..
X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/windows.rb:21:in `require': cannot load such file -- win32/process (LoadError)
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout/windows.rb:21:in `<top (required)>'
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout.rb:33:in `require'
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout.rb:33:in `<class:ShellOut>'
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout.rb:26:in `<module:Mixlib>'
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/mixlib-shellout-1.4.0/lib/mixlib/shellout.rb:24:in `<top (required)>'
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/chef-11.12.8/lib/chef/shell_out.rb:1:in `require'
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/chef-11.12.8/lib/chef/shell_out.rb:1:in `<top (required)>'
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/chef-11.12.8/lib/chef/mixin/shell_out.rb:20:in `require'
        from X:/tools/ruby-2.0.0/lib/ruby/gems/2.0.0/gems/chef-11.12.8/lib/chef/mixin/shell_out.rb:20:in `<top (required)>'

It seems to be because mixlib-shellout (as of 1.4.0) is only published for the x86-mingw32 platform, but not for x64-mingw32 on rubygems:
https://rubygems.org/gems/mixlib-shellout

For now I added win32-process to my Gemfile to fix it, but I think there should really be an additional gemspec for x64 akin to https://github.com/opscode/mixlib-shellout/blob/master/mixlib-shellout-x86-mingw32.gemspec

mixlib-shellout v 1.6.0 pulled by Chef 11.8.2 is failing with an error message (in the body of the ticket)

ubuntu@ip-10-39-63-94:$ sudo chef-client
/usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require': /var/lib/gems/1.8/gems/mixlib-shellout-1.6.0/lib/mixlib/shellout/unix.rb:294: syntax error, unexpected tSYMBEG, expecting tAMPER (SyntaxError) ..._of?(Array) ? exec(*command, :close_others=>true) : exec(com... ^ from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
from /var/lib/gems/1.8/gems/mixlib-shellout-1.6.0/lib/mixlib/shellout.rb:36
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require' from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
from /var/lib/gems/1.8/gems/chef-11.8.2/bin/../lib/chef/shell_out.rb:1
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require' from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
from /var/lib/gems/1.8/gems/chef-11.8.2/bin/../lib/chef/mixin/shell_out.rb:20
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require' from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
from /var/lib/gems/1.8/gems/chef-11.8.2/bin/../lib/chef/util/selinux.rb:23
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require' from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
from /var/lib/gems/1.8/gems/chef-11.8.2/bin/../lib/chef/config.rb:25
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require' from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
from /var/lib/gems/1.8/gems/chef-11.8.2/bin/../lib/chef.rb:24
from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:in gem_original_require' from /usr/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
from /var/lib/gems/1.8/gems/chef-11.8.2/bin/chef-client:23
from /usr/local/bin/chef-client:19:in `load'
from /usr/local/bin/chef-client:19
ubuntu@ip-10-39-63-94:
$ ruby -v
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]
ubuntu@ip-10-39-63-94:~$

How can we fix Chef installation on our server?

Reap failure can show red herring error message

If an exception is raised during run_command, we try to reap things, it will show a red herring error log that "Command exceeded allowed execution time" even though the timeout wasn't involved. We should differentiate the log message on the two places we use reap_errant_child.

In this case the exception was being raised by a faulty live_stream implementation, but there are others.

Lets pull shell_out/shell_out! and friends into mixlib-shellout from chef/chef

ohai has a completely different version mangled together with the old run_command here:

https://github.com/chef/ohai/blob/master/lib/ohai/mixin/command.rb

chef has its version here:

https://github.com/chef/chef/blob/master/lib/chef/mixin/shell_out.rb

we're probably missing the ability to inject the default_locale into the ohai one so that if we run ohai on DE_de.UTF-8 it'll give incorrect results as it fails to parse localized strings that are returned translated into German.

the ohai one is also sprouting the need to debug log the output. so the codebase is starting to fork.

we need to consolidate this, and it might as well go into this library.

it needs to be decoupled so we need to be able to dependency inject the logger and the internal_locale into this mixin. there probably needs to be one class variable which is global and is the default logger/internal_locale for all of the classes that get the mixin. then there needs to be a class instance variable in each mixed in class which can be tweaked by the class.

then Chef::Config or Chef::Application could be responsible for injecting Ohai::Mixin::Command.internal_locale = Chef::Config[:internal_locale] that would remove the dependency on Chef::Config which would ease a lot of circular dependency issues. similarly the logger could be injected into the mixin globally.

i doubt chef or ohai would use the ability to override on a class-by-class basis, but if we merge it into here, then we should probably build that in because someone will wind up wanting to do that...

Cannot launch process with elevated token on Windows

When providing alternate credentials and the alternate user is in the administrators group, the process being shelled out to does not have an administrative token and cannot perform actions that require elevation.

Repro:

Requires a user in the administrators group. In the example, there is a local user account tester which is in the local Administrators group.

admin_check = <<-EOH
ZgB1AG4AYwB0AGkAbwBuACAAVABlAHMAdAAtAEEAZABtAGkAbgBpAHMAdAByAGEAdABvAHIAIAAgAAoAewAgACAACgAgACAAIAAgACQAdQBzAGUAcgAgAD0AIABbAFMAZQBjAHUAcgBpAHQAeQAuAFAAcgBpAG4AYwBpAHAAYQBsAC4AVwBpAG4AZABvAHcAcwBJAGQAZQBuAHQAaQB0AHkAXQA6ADoARwBlAHQAQwB1AHIAcgBlAG4AdAAoACkAOwAKACAAIAAgACAAKABOAGUAdwAtAE8AYgBqAGUAYwB0ACAAUwBlAGMAdQByAGkAdAB5AC4AUAByAGkAbgBjAGkAcABhAGwALgBXAGkAbgBkAG8AdwBzAFAAcgBpAG4AYwBpAHAAYQBsACAAJAB1AHMAZQByACkALgBJAHMASQBuAFIAbwBsAGUAKABbAFMAZQBjAHUAcgBpAHQAeQAuAFAAcgBpAG4AYwBpAHAAYQBsAC4AVwBpAG4AZABvAHcAcwBCAHUAaQBsAHQAaQBuAFIAbwBsAGUAXQA6ADoAQQBkAG0AaQBuAGkAcwB0AHIAYQB0AG8AcgApACAAIAAKAH0ACgB0AGUAcwB0AC0AYQBkAG0AaQBuAGkAcwB0AHIAYQB0AG8AcgA=
EOH

whoami = Mixlib::ShellOut.new("powershell.exe -noprofile -encodedcommand \"#{admin_check}\"", :user => "tester", :password => "P2ssw0rd!")
whoami.run_command.stdout

This will return "true" if elevated, "false" if not.

Exceptions should share a common parent

The shellout exceptions should all share a common parent for ease of rescue.

e.g.

module Mixlib
  class ShellOut
    class Error < RuntimeError; end
    class ShellCommandFailed < Error; end
    class CommandTimeout < Error; end
    class InvalidCommandOption < Error; end
  end
end

This allows code like...

begin
  Mixlib::ShellOut.new('some command').run_command
rescue Mixlib::ShellOut::Error => err
  puts "Whups! While running 'some command' I got #{err}"
  raise
end

Set supplementary groups by default, allow users to modify supplementary groups

In addition to the real and effective group ID of a process, every process has a list of supplementary groups IDs. From the credentials(7) man page on Linux:

Supplementary group IDs. This is a set of additional group IDs that are used for permission checks when accessing files and other shared resources. On Linux kernels before 2.6.4, a process can be a member of up to 32 supplementary groups; since kernel 2.6.4, a process can be a member of up to 65536 supplementary groups. The call sysconf(_SC_NGROUPS_MAX) can be used to determine the number of supplementary groups of which a process may be a member. A process can obtain its set of supplementary group IDs using getgroups(2), and can modify the set using setgroups(2).

See the related Chef bug here: https://tickets.opscode.com/browse/CHEF-3510

Programs such as sudo and su typically set the list of supplementary groups to the groups of which the EUID is a member.

In C, the relevant system calls to modify the group list are initgroups and setgroups

In Ruby, the group list can be modified with the Process.groups= method.

Could not find mixlib-shellout-1.6.0 in any of the sources

I am getting this error, and when I look at rubygems it says it has been yanked, but it's still available. We upload our Gemfile.lock's into scm, because we want to avoid configuration drift. This is currently breaking some of our cookbooks. Why was this yanked, can you replace it, and will this be a common occurrence?

Support no-tty and redirect stdout/stderr to file use cases

Describe the Enhancement:

  1. Use Chef resources in recipes that rely on shell command IO contents in a no-tty environment, e.g. Clouds, Dokken and other such use cases.
  2. Run shells and have their output directed to files: e.g. SO

Describe the Need:

  1. Users who employ no-tty environments for production or testing (Dokken) can have resources that don't rely on .stdout, .stderr or .stdin
  2. Users who want to pipe shell command output to file and process can do that naturally from shell_out

Current Alternative

Issue chef/chef/issues/8855 suggests there is currently no alternative.
In that issue, even with the proposed use of the git ref-source --verify exit-status to verify the reference exists, you need to read .stdout to return the reference SHA.

A very common use case for shell scripts is to pipe the stdout or stderr to files for later processing.

Can We Help You Implement This?:

Feedback on whether this would be accepted.
Feedback on the interface:

  • fstdout, fstderr (and fstdin if straight forward). When these are set stdout, stderr (and stdin)
  • Default is textmode, but support a binmode (rb and wb)
  • Default encoding is UTF-8, but support this being set.

Any other suggestion, pointers, hints or tips would be appreciated.

Allow live_stdin for interactive processes

I'd like to add support for live_stdin to allow passing an IO object as stdin to the called process (similar to live_stdout and live_stderr).

I wanted to file this an issue before doing the ground work to make sure this was a feature you guys want.
I'm happy to put in a PR for it if you think it's useful. If not, I can find a different way to solve the issue I'm having.

Details about my use-case

I have a ruby script which does some setup and tear-down for a chef-client run (things like setting environment variables, merging json attributes files, moving around logs etc.). It invokes chef-client using Mixlib::Shellout.

I'd like to modify it to be able to drop to a chef-shell instead of invoking chef-client. This way the setup/teardown is re-used and the chef-shell is invoked in the same context as a chef-client would be.
The easiest way to do this would just be to swap chef-client for chef-shell in the command (removing any arguments that are chef-client exclusive).

The issue is that there's no way for me to pass the stdin from the wrapper script to the chef-shell process, so the chef-shell process starts, hits EOF on stdin, and quits.
I can pass it individual commands by modifying the wrapper script, but what I'd really like to get an interactive chef-shell.

Sensitive attribute is not respected in shell_out when a timeout occurs

Description

The sensitive attribute does not work in a shell_out block when the command times out. This is very similar to issue chef/chef#7517

Chef Version

chef_version=14.12.3

Platform Version

platform=windows
platform_version=10.0.14393 (Windows Server 2016)

Replication Case

Create a recipe with the following code:

shell_out!('sleep 10; echo "my super secret"', timeout: 5, sensitive: true)

Client Output

Synchronizing Cookbooks:
         - sensitive_test (0.1.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       
       ================================================================================
       Recipe Compile Error in C:/Users/ADMINI~1/AppData/Local/Temp/kitchen/cache/cookbooks/sensitive_test/recipes/default.rb
       ================================================================================
       
       Mixlib::ShellOut::CommandTimeout
       --------------------------------
       command timed out:
       Command execution failed. STDOUT/STDERR suppressed for sensitive resource
       ProcessId: 2824
       app_name: C:\Windows\system32\cmd.exe
       command_line: cmd /c "sleep 10; echo "my super secret""
       timeout: 5
       
       System Info:
       ------------
       chef_version=14.12.3
       platform=windows
       platform_version=10.0.14393
       ruby=ruby 2.5.5p157 (2019-03-15 revision 67260) [x64-mingw32]
       program_name=C:/opscode/chef/bin/chef-client
       executable=C:/opscode/chef/bin/chef-client
       
       
       Running handlers:
       [2020-01-31T21:00:21+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2020-01-31T21:00:21+00:00] ERROR: Exception handlers complete
       Chef Client failed. 0 resources updated in 14 seconds
       [2020-01-31T21:00:21+00:00] FATAL: Stacktrace dumped to C:/Users/ADMINI~1/AppData/Local/Temp/kitchen/cache/chef-stacktrace.out
       [2020-01-31T21:00:21+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2020-01-31T21:00:21+00:00] FATAL: Mixlib::ShellOut::CommandTimeout: command timed out:
       Command execution failed. STDOUT/STDERR suppressed for sensitive resource
       ProcessId: 2824
       app_name: C:\Windows\system32\cmd.exe
       command_line: cmd /c "sleep 10; echo "my super secret""
       timeout: 5

Running a command with non-string array element raises "TypeError: no implicit conversion from nil to integer"

STR with chef-12.0.3:

  1. chef-apply -e 'execute "blah" do command ["/bin/true", 0] end' -l debug

Expected results:
"/bin/true 0" is executed

Actual results:

[2015-02-05T10:04:21-05:00] DEBUG: TypeError: execute[blah] ((chef-apply cookbook)::(chef-apply recipe) line 1) had an error: TypeError: no implicit conversion from nil to integer
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.0.0/lib/mixlib/shellout/unix.rb:337:in `kill'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.0.0/lib/mixlib/shellout/unix.rb:337:in `reap_errant_child'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.0.0/lib/mixlib/shellout/unix.rb:97:in `ensure in run_command'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.0.0/lib/mixlib/shellout/unix.rb:105:in `run_command'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.0.0/lib/mixlib/shellout.rb:237:in `run_command'
/opt/chef/embedded/apps/chef/lib/chef/mixin/shell_out.rb:91:in `shell_out_command'
/opt/chef/embedded/apps/chef/lib/chef/mixin/shell_out.rb:44:in `shell_out'
/opt/chef/embedded/apps/chef/lib/chef/mixin/shell_out.rb:49:in `shell_out!'
/opt/chef/embedded/apps/chef/lib/chef/provider/execute.rb:60:in `block in action_run'
/opt/chef/embedded/apps/chef/lib/chef/mixin/why_run.rb:52:in `call'
/opt/chef/embedded/apps/chef/lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/chef/embedded/apps/chef/lib/chef/provider.rb:180:in `converge_by'
/opt/chef/embedded/apps/chef/lib/chef/provider/execute.rb:59:in `action_run'
/opt/chef/embedded/apps/chef/lib/chef/provider.rb:145:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/resource.rb:582:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:49:in `run_action'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block (2 levels) in converge'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `each'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:81:in `block in converge'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:83:in `block in execute_each_resource'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/apps/chef/lib/chef/resource_collection/resource_list.rb:81:in `execute_each_resource'
/opt/chef/embedded/apps/chef/lib/chef/runner.rb:80:in `converge'
/opt/chef/embedded/apps/chef/lib/chef/application/apply.rb:150:in `run_chef_recipe'
/opt/chef/embedded/apps/chef/lib/chef/application/apply.rb:159:in `run_application'
/opt/chef/embedded/apps/chef/lib/chef/application/apply.rb:172:in `run'
/opt/chef/embedded/apps/chef/bin/chef-apply:25:in `<top (required)>'
/usr/bin/chef-apply:40:in `load'
/usr/bin/chef-apply:40:in `<main>'

Reaper doesn't like sudo'd processes

In the case where the process doing shell-out-ing is an unpriv'd user, and the command being run either directly or indirectly ends up with elevated perms, we can end up with a situation where the kill() call in reap_errant_child fails with EPERM.

We should probably just ignore it like we do with ESRCH, maybe with a warning that reaping failed due to process permissions.

shell_out without a bang can fail if there's no shell metacharacters and the file does not exist

[1] pry(main)> require 'chef/mixin/shell_out'
=> true
[2] pry(main)> include Chef::Mixin::ShellOut
=> Object
[3] pry(main)> shell_out("lskdjflskj 'foo'")
=> <Mixlib::ShellOut#70318079326640: command: 'lskdjflskj 'foo'' process_status: #<Process::Status: pid 4874 exit 127> stdout: '' stderr: 'sh: lskdjflskj: command not found' child_pid: 4874 environment: {"LC_ALL"=>"en_US.UTF-8", "LANGUAGE"=>"en_US.UTF-8", "LANG"=>"en_US.UTF-8", "PATH"=>"/Users/lamont/.rvm/gems/ruby-2.5.1/bin:/Users/lamont/.rvm/gems/ruby-2.5.1@global/bin:/Users/lamont/.rvm/rubies/ruby-2.5.1/bin:/opt/chefdk/bin:/Users/lamont/.chefdk/gem/ruby/2.5.0/bin:/opt/chefdk/embedded/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/lamont/bin:/usr/local/sbin:/opt/chefdk/gitbin:/Users/lamont/.rvm/bin"} timeout: 600 user:  group:  working_dir:  >
[4] pry(main)> shell_out("lskdjflskj")
Errno::ENOENT: No such file or directory - lskdjflskj
from /Users/lamont/.rvm/gems/ruby-2.5.1/gems/mixlib-shellout-2.3.2/lib/mixlib/shellout/unix.rb:340:in `exec'

The second case should probably catch Errno::ENOENT and remap it onto the same error message you get from calling error!:

[5] pry(main)> shell_out("lskdjflskj 'foo'").error!
Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '127'
---- Begin output of lskdjflskj 'foo' ----
STDOUT:
STDERR: sh: lskdjflskj: command not found
---- End output of lskdjflskj 'foo' ----
Ran lskdjflskj 'foo' returned 127
from /Users/lamont/.rvm/gems/ruby-2.5.1/gems/mixlib-shellout-2.3.2/lib/mixlib/shellout.rb:293:in `invalid!'

2.2.0.rc.0 seems to break dependency resolving

Since the rc release of 2.2.0 our chef installation breaks with the following:

2015-09-01 15:30:21,682 [INFO] Installing chef version 12.4.0 via gem
2015-09-01 15:30:23,667 [ERROR] Gem failed. Output: ERROR: While executing gem ... (Gem::Dependency
Error)
Unresolved dependency found during sorting - mixlib-shellout (~> 2.0) (requested by ohai-8.5.1)

Windows impersonation does not load user profile correctly

Mixlib::ShellOut does not load user profile of the impersonated user account, so the user's environment variables such as USERNAME, USERPROFILE, TEMP, TMP, APPDATA, and LOCALAPPDATA still point to the parent process user name and directories.

To reproduce the problem use Mixlib::ShellOut to run windows .bat with single command set (that prints values of the environment variables) under as a different user.

Tested with Chef Client version 13.11.3.

options with arguments are unsupported (or undocumented)

Mixlib::Config allows us to have options like "--option value" but passing this into Mixlib::ShellOut.new() doesn't work. For example:

sw = Mixlib::ShellOut.new(@binary, '-r', '--only roles', '--no-validation' , 'example.yml', environment: { 'PWD' => "#{ENV['PWD']}/test/chef-repo" })
sw.run_command

fails to recognize "--only roles" as a valid option. Multiple single options with no parameters work just fine, but no combination of quotes or escaping seem to work.

I've been using this for rspec testing Spiceweasel and this has never worked.

Add "login" setting to simulate "su -l"

Like CHEF-2288, the basic impersonation made by chef's execute is insufficient for many processes.
In my case, I'm missing the HOME environment variable.
I'm interested in helping, but first wanted to make sure this is the right place to complain :-)

I thought about adding supplementary groups here (like #68) and populate the (environment hash)[https://github.com/opscode/mixlib-shellout/blob/master/lib/mixlib/shellout/unix.rb#L137] with some predefined values

First PATH line is ignored on windows 2016 server

Version:

2.4.4

Environment:

Windows 2016 Server, Ruby 2.5.1

Scenario:

test = Mixlib::ShellOut.new("echo test |findstr "test"")
puts test.run_command.stderr

Steps to Reproduce:

Open Windows 'Environment Variables - > PATH' and make sure first line is 'C:\Windows\system32'

Expected Result:

No errors

Actual Result:

'findstr' is not recognized as an internal or external command

[What actually happens after the reproduction steps?]
'findstr' is not recognized as an internal or external command

Workaround

Add dummy PATH entry as first line

Got Resource temporarily unavailable (Errno::EAGAIN) error when performing "execute" in chef

Hi, I am new in Chef. I got following error message by calling "execute"

Ran tar xfz /tmp/logstash-1.4.0.tar.gz  returned 1
* Mixlib::ShellOut::ShellCommandFailed occurred in delayed notification: execute[extract /tmp/logstash-1.4.0.tar.gz] (/var/chef/cache/cookbooks/tar/providers/extract.rb line 81) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of tar xfz /tmp/logstash-1.4.0.tar.gz  ----
STDOUT: 
STDERR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:93:in `uid=': Resource temporarily unavailable (Errno::EAGAIN)
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:93:in `set_user'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:226:in `block in fork_subprocess'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:222:in `fork'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:222:in `fork_subprocess'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:35:in `run_command'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:222:in `run_command'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/mixin/shell_out.rb:30:in `shell_out'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/mixin/shell_out.rb:35:in `shell_out!'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/provider/execute.rb:62:in `block in action_run'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/mixin/why_run.rb:52:in `call'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/mixin/why_run.rb:52:in `add_action'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/provider.rb:151:in `converge_by'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/provider/execute.rb:61:in `action_run'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/provider.rb:114:in `run_action'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource.rb:606:in `run_action'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:50:in `run_action'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:112:in `run_delayed_notification'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:100:in `block in run_delayed_notifications'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:99:in `each'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:99:in `run_delayed_notifications'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:87:in `rescue in converge'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:76:in `converge'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/provider/lwrp_base.rb:61:in `recipe_eval_with_update_check'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/provider/lwrp_base.rb:45:in `block in action'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/provider.rb:114:in `run_action'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource.rb:606:in `run_action'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:50:in `run_action'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:82:in `block (2 levels) in converge'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:82:in `each'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:82:in `block in converge'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource_collection.rb:94:in `block in execute_each_resource'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/resource_collection.rb:92:in `execute_each_resource'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/runner.rb:81:in `converge'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/client.rb:404:in `converge'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/client.rb:469:in `do_run'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/client.rb:200:in `run'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/application.rb:190:in `run_chef_client'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/application/client.rb:297:in `block in run_application'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/application/client.rb:290:in `loop'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/application/client.rb:290:in `run_application'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/lib/chef/application.rb:73:in `run'
        from /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.4/bin/chef-client:26:in `<top (required)>'
        from /usr/bin/chef-client:23:in `load'
        from /usr/bin/chef-client:23:in `<main>'
---- End output of tar xfz /tmp/logstash-1.4.0.tar.gz  ----
Ran tar xfz /tmp/logstash-1.4.0.tar.gz  returned 1

The process number limit of myuser is unlimited, number of processes shouldn't exceed that.

limits of myuser by executing

ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 1032113
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 16000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Process number of myuser by executing

ps -eLF | grep myuser | wc -l
2201

I thought this issue may exist in mixlib-shellout

So I wrote this code

ls = Mixlib::ShellOut.new("ls /tmp", :user => 'myuser')
ls.run_command
puts ls.stdout
ls.error!

I tried it with myuser, it workd perfectly.
While I tried it with root, I got the same error of Resource temporarily unavailable (Errno::EAGAIN)

STDOUT:
STDERR: /usr/lib/ruby/gems/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:93:in `uid=': Resource temporarily unavailable (Errno::EAGAIN)
        from /usr/lib/ruby/gems/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:93:in `set_user'
        from /usr/lib/ruby/gems/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:226:in `fork_subprocess'
        from /usr/lib/ruby/gems/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:222:in `fork'
        from /usr/lib/ruby/gems/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:222:in `fork_subprocess'
        from /usr/lib/ruby/gems/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout/unix.rb:35:in `run_command'
        from /usr/lib/ruby/gems/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:222:in `run_command'
        from mixlib-ls.rb:8
---- End output of ls /tmp ----
Ran ls /tmp returned 1
        from /usr/lib/ruby/gems/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:234:in `error!'
        from mixlib-ls.rb:12

Is there any one can help me?

Thank you in advance.

Mixlib::ShellOut - timeout

I am trying to use Mixlib::ShellOut to execute commands under ruby_block inside a chef recipe. In Some situations, we cannot complete the task in 600 seconds, and I would like extend further. I have added command in below way,

vsphere_output = Mixlib::ShellOut.new(_command, :timeout => 15000)

and I suspect it is not respecting timeout value. Please advise.

Timeout while running copy cookbook

Hi! I need to write a chef recipe which copy mounted folder [source folder] to another [direct folder].
I wrote this recipe:

execute "copy" do
command ".#{node[some path in atribute]/copy.sh"
timeout 100
action :nothing
end
template "#{node[some path in atribute]/copy.sh" do
source "template.erb"
mode 0755
notifies :run, "execute[copy]", :immediately
end

The template is:

sshpass -p <%= node['pass from attribute']%> scp -r -o StrictHostKeyChecking=no <%= node[client name from attribute]%>@[host]:/[source path] <%= node[direct path from attribute]%>

When i run my cookbook I receive in console:

Mixlib::ShellOut::CommandTimeout
--------------------------------
Command timed out after 100s:
Command exceeded allowed execution time, process terminated
---- Begin output of ./[path]/copy.sh ----
STDOUT:
STDERR: Killed by signal 1.
---- End output of ./tmp/devops.sh ----
Ran ./[path]/copy.sh returned

Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/copy/recipes/default.rb

///code from recipe

Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/copy/recipes/default.rb:4:in `from_file'

execute("copy") do
  action [:nothing]
  retries 0
  retry_delay 2
  default_guard_interpreter :execute
  command "./[path]/copy.sh"
  backup 5
  returns 0
  timeout 100
  declared_type :execute
  cookbook_name "copy"
  recipe_name "default"
end

Platform:
---------
x86_64-linux

Running handlers:
[2016-07-19T14:35:17+02:00] ERROR: Running exception handlers
Running handlers complete
[2016-07-19T14:35:17+02:00] ERROR: Exception handlers complete
Chef Client failed. 3 resources updated in 02 minutes 02 seconds
[2016-07-19T14:35:17+02:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2016-07-19T14:35:17+02:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2016-07-19T14:35:17+02:00] ERROR: execute[devops](copy::default line 4) had an error: Mixlib::ShellOut::CommandTimeout: Command timed out after 100s:
Command exceeded allowed execution time, process terminated
---- Begin output of ./tmp/devops.sh ----
STDOUT:
STDERR: Killed by signal 1.
---- End output of ./tmp/devops.sh ----
Ran ./tmp/devops.sh returned
[2016-07-19T14:35:17+02:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

What is wrong?? I have no idea

Dependabot can't evaluate your Ruby dependency files

Dependabot can't evaluate your Ruby dependency files.

As a result, Dependabot couldn't check whether any of your dependencies are out-of-date.

The error Dependabot encountered was:

Bundler::Dsl::DSLError with message: 
[!] There was an error parsing `Gemfile`: 
[!] There was an error while loading `mixlib-shellout-universal-mingw32.gemspec`: (eval):1: unexpected fraction part after numeric literal
0.0.1
^~~. Bundler cannot continue.

 #  from /home/dependabot/dependabot-updater/dependabot_tmp_dir/mixlib-shellout-universal-mingw32.gemspec:1
 #  -------------------------------------------
 >  gemspec = instance_eval("0.0.1")
 #  
 #  -------------------------------------------
. Bundler cannot continue.

 #  from /home/dependabot/dependabot-updater/dependabot_tmp_dir/Gemfile:3
 #  -------------------------------------------
 #  
 >  gemspec name: "mixlib-shellout"
 #  
 #  -------------------------------------------

passing :valid_exit_codes option throws an error

I'm trying to get the output from running the git --version command. I do not want the program to exit if git is not installed on the system but simply not to set the installed_version variable. In order to do that I am trying to pass 127 as a valid exit code but i get the error in the output below.

cmd = Mixlib::ShellOut.new('git', '--version', :valid_exit_codes => [0, 127])
cmd.run_command
if cmd.stderr.empty? and !cmd.stdout.empty?
installed_version = cmd.stdout.scan(/\d+/).join('.')
end

/home/ckehoe/.gem/ruby/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout.rb:343:in block in parse_options': option ':valid_exit_codes' is not a valid option for Mixlib::ShellOut (Mixlib::ShellOut::InvalidCommandOption) from /home/ckehoe/.gem/ruby/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout.rb:301:ineach'
from /home/ckehoe/.gem/ruby/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout.rb:301:in parse_options' from /home/ckehoe/.gem/ruby/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout.rb:177:ininitialize'
from /home/ckehoe/test.rb:63:in new' from /home/ckehoe/test.rb:63:invalid_git_installation?'
from /home/ckehoe/test.rb:80:in `

'

Missing documentation on live_stdout / live_stderr functionality

We run a number of long running processes, which we'd like to livestream the output. I can see that there is functionality in mixlib-shellout for this, but I couldn't work out how to use it, nor where this live-streamed output gets put.

Could you please add some documentation about how to use this functionality?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.