alertlogic / al_agents Goto Github PK

View Code? Open in Web Editor NEW

8.0 8.0 22.0 140 KB

Alert Logic Agent Cookbook

Home Page: http://www.alertlogic.com/

License: Apache License 2.0

Ruby 98.85% Shell 0.48% HTML 0.67%

al_agents's People

Contributors

Stargazers

Watchers

al_agents's Issues

SUSE support

The agent supports SUSE but the cookbook does not.

Can this be added please?

Library helpers conflict with other cookbooks

If you use the alert logic cookbook on windows, it is easy to get conflicting variables.

For example, if I have a cookbook that uses a variable or method named "service_name", the alert logic cookbook will overwrite this setting, resulting in weird things as discussed in the mailing list:

https://discourse.chef.io/t/how-do-you-avoid-variable-conflicts-in-custom-resources/8296/5

The best solution would probably be to not send the entire helper to all chef resources, instead only send it to the resources needed.

Instead of this:

::Chef::Resource.send(:include, AlAgents::Helpers)

Do this:

::Chef::Resource::Batch.send(:include, AlAgents::Helpers)

That way other cookbooks have a much narrower risk of collisions.

Tag and release

Please lets start, at minimum, start tagging in sync with the cookbook metadata version. A github release with release notes included are then the shiny finish.

Is windows supported

The readme doesn't specify windows as being supported, however there does appear to be a windows recipe.

Is windows supported? If so, which versions are tested?

cloud-init and licence?

Out of curiosity, where all cloud-init related stuff disappeared?
And what is licence for this project right now?

git Tag and GitHub Release in sync with metadata.rb version bump

At minimum, let's please start tagging in sync with the cookbook metadata version. A github release with release notes included are then the shiny finishing touches.

This was previously issue #12. Two new releases went out without git tags or GitHub releases.

Default Egress Settings Don't Work

The default of https://vaporator.alertlogic.com:443/ does not work. If you remove the port, it will successfully provision itself.

Tries to open .pem file

Every time chef runs, the cookbook tries to open the .pem file for some reason.

This results in a pop up on my screen

PS C:\chef\cache> chef-client -l debug -L c:\foo.log
Starting Chef Client, version 12.5.1
resolving cookbooks for run list: ["al_agents::default"]
Synchronizing Cookbooks:
  - al_agents (1.0.8)
  - line (0.6.3)
  - rsyslog (2.2.0)
  - selinux_policy (0.9.2)
Compiling Cookbooks...
Converging 3 resources
Recipe: al_agents::_windows
  * remote_file[al_agent-LATEST.msi] action create_if_missing (up to date)
  * windows_package[al_agent-LATEST.msi] action install (skipped due to not_if)
Recipe: al_agents::start
  * windows_service[al_agent] action start (up to date)

Running handlers:
Running handlers complete
Chef Client finished, 0/3 resources updated in 14 seconds���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������

Issues with OpsWorks

The dependent cookbook, selinux_policy, require Chef 12 to run which is not available via OpsWorks. We needed to clone selinux_policy v1.1.1 and then make two further modifications so that the cookbook worked properly:

added if respond_to? for the source_url and issues_url lines
Removed the dependency on yum

Having issues with AWS OpsWorks installation

I'm trying to get the agents to install through AWS OpsWorks (Chef 11.10) and receiving the following:

Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of ./al-agent provision --key [KEY REDACTED] --inst-type host ----
STDOUT: 
STDERR: May 11 22:40:20 2016 al-agent[24242]: ALC00089I [alc_config_set_api_key] setting api_key
May 11 22:40:20 2016 al-agent[24242]: ALC00083I [unregister] clean up registration artefacts
May 11 22:40:20 2016 al-agent[24242]: ALC00315E [alc_socket_send_request] Response http code is 301
May 11 22:40:20 2016 al-agent[24242]: ALC00074E [alc_config_provision_host] Could not issue provisioning request
May 11 22:40:20 2016 al-agent[24242]: ALC00079E [alc_config_init] Could not provision the host
---- End output of ./al-agent provision --key [KEY REDACTED] --inst-type host ----
Ran ./al-agent provision --key [KEY REDACTED] --inst-type host returned 1

Not sure where the 301 Moved Permanently is coming from....

Any ideas?

Package Install Only Option

We use Chef to build our AMIs and only want to install the al_agent package (not to configure or provision). We will handle configuration and provisioning at boot time. We hacked this into a forked repo (al_agents::package), but it would be nice to have this option available in the main cookbook.

Log Agent attempts to add info on every run

On every run, the code appears to attempt to add info to syslog-ng, even though it is not installed.

CentOS syslog-ng misconfiguration

CentOS 6+ uses s_all not s_sys for syslog-ng global logger. This will cause syslog-ng to fail to start

The correction needed is in defaults.rb to evaluate what platform_version we are on.

Why is rsyslog set to send logs using TCP?

Since the communication is local, packet loss should not be an issue, what was the though process around the use of UDP? Or even better the use of a socket?

Remove threat_host and log_agent

Are we ok with removing deprecated recipes?
Otherwise, If we holding this for a while, all changes must be done everywhere, including #4

Namespacing issue with libraries/helpers.rb

Howdy!

TL;DR: libraries/helpers.rb needs to be namespaced.

We were attempting to use this cookbook in conjunction with others, and we noticed that helpers.rb isn't namespaced which causes issues with other cookbooks. I've made a simple example cookbook in the following repo:

https://github.com/brint/libexample

Run kitchen test to see the trace. I've included a sample trace at the bottom of this issue.

The service_name function was the one that bit us. We started noticing al-agent showing up as the service_name with the mysql cookbook (~> 5.0). In the mysql cookbook, service_name is a variable, and the helper function in this cookbook trumps the variable used in the library in their cookbook. Here's the specific line where the al_agents service_name function is causing issues within the mysql cookbook:

https://github.com/chef-cookbooks/mysql/blob/v5.6.3/libraries/provider_mysql_service_rhel.rb#L123

Example trace using the libexample cookbook whose recipe only has a line of include_recipe 'mysql::server' and dependencies of mysql and al_agents:

       Recipe: mysql::server


           ================================================================================
           Error executing action `create` on resource 'mysql_service[default]'
           ================================================================================

           Chef::Exceptions::ResourceNotFound
           ----------------------------------
           resource template[/etc/my.cnf] is configured to notify resource service[al-agent] with action restart, but service[al-agent] cannot be found in the resource collection. template[/etc/my.cnf] is defined in /tmp/kitchen/cache/cookbooks/mysql/libraries/provider_mysql_service_rhel.rb:101:in `block in <class:Rhel>'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/mysql/recipes/server.rb

            20: mysql_service node['mysql']['service_name'] do
            21:   version node['mysql']['version']
            22:   port node['mysql']['port']
            23:   data_dir node['mysql']['data_dir']
            24:   server_root_password node['mysql']['server_root_password']
            25:   server_debian_password node['mysql']['server_debian_password']
            26:   server_repl_password node['mysql']['server_repl_password']
            27:   allow_remote_root node['mysql']['allow_remote_root']
            28:   remove_anonymous_users node['mysql']['remove_anonymous_users']
            29:   remove_test_database node['mysql']['remove_test_database']
            30:   root_network_acl node['mysql']['root_network_acl']
            31:   package_version node['mysql']['server_package_version']
            32:   package_action node['mysql']['server_package_action']
            33:   enable_utf8 node['mysql']['enable_utf8']
            34:   action :create
            35: end

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/mysql/recipes/server.rb:20:in `from_file'

           mysql_service("default") do
             action [:create]
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             declared_type :mysql_service
             cookbook_name "mysql"
             recipe_name "server"
             port "3306"
             data_dir "/var/lib/mysql"
             server_root_password "mysql_test"
             server_debian_password "gnuslashlinux4ev4r"
             remove_anonymous_users true
             remove_test_database true
             package_action "install"
           end


       Running handlers:
       [2015-08-12T02:06:23+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2015-08-12T02:06:23+00:00] ERROR: Exception handlers complete
       Chef Client failed. 0 resources updated in 3.364878055 seconds
       [2015-08-12T02:06:23+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2015-08-12T02:06:23+00:00] ERROR: mysql_service[default] (mysql::server line 20) had an error: Chef::Exceptions::ResourceNotFound: resource template[/etc/my.cnf] is configured to notify resource service[al-agent] with action restart, but service[al-agent] cannot be found in the resource collection. template[/etc/my.cnf] is defined in /tmp/kitchen/cache/cookbooks/mysql/libraries/provider_mysql_service_rhel.rb:101:in `block in <class:Rhel>'

       [2015-08-12T02:06:24+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Add al_agent to supermarket

As a user of the al_agent cookbook
I want to reference al_agent in metadata.rb
And I do not want to reference the github location in Berksfile
So that it is very easy to use the al_agent cookbook

Today I must have a depends 'al_agent', '~> 0.3' in my metadata.rb
And I must have cookbook 'al_agents', git: 'https://github.com/alertlogic/al_agents.git', tag: 'v0.3.0'

If al_agent cookbook were loaded into the supermarket
Then only the entry in metadata.rb would be required.

https://supermarket.chef.io/cookbooks?q=al_agent

Allowing failures

Would it be possible to have an attribute (or some mechanism) whereby we can allow failures for various execute statements? We have a problem when the recipe runs the ./al-agent provision command we get a non-zero errorcode if our AlertLogic appliance is offline. Although AlertLogic being unavailable is a problem, we do not want it to stop the chef run and prevent our application deployments.

Under normal circumstances, I'd use a wrapper cookbook to target the particular execute statement and add the ignore_failure true property, however, since the name of the execute statement is based on a variable that's generated from a library, I'm not sure of a way of getting around this.

Any advice welcome.

RPM package for al-agent fails to install on RHEL 6x

There is an issue with the package resource as the "al-agent-LATEST-1.x86_64.rpm" installation fails during the Chef run on RHEL 6x. We need to mention the correct provider for this scenario.
The following fix helps
package basename do
source cached_package
action :install
provider Chef::Provider::Package::Dpkg if node['platform_family'] == 'debian'
provider Chef::Provider::Package::Rpm if node['platform_family'] == 'rhel' && node['platform_version'].to_i == 6
end

Support for Amazon Linux

From Direct Supply (Customer):

There seems to be explicit support for the ‘rhel’, ‘fedora’, and ‘debian’ platform families, based on how it gets the package to install for the agent, but not for ‘amazon’ which is what Amazon Linux is classified as. As such, it fails when trying to get the package to install the agent

========================================================================================
Recipe Compile Error in /tmp/kitchen/cache/cookbooks/amazonlinux-base/recipes/default.rb
========================================================================================

URI::InvalidURIError
--------------------
bad URI(is not URI?):

Cookbook Trace:
---------------
  /tmp/kitchen/cache/cookbooks/al_agents/libraries/helpers.rb:14:in `agent_file'
  /tmp/kitchen/cache/cookbooks/al_agents/libraries/helpers.rb:18:in `agent_basename'
  /tmp/kitchen/cache/cookbooks/al_agents/recipes/install.rb:11:in `from_file'
  /tmp/kitchen/cache/cookbooks/al_agents/recipes/_linux.rb:7:in `from_file'
  /tmp/kitchen/cache/cookbooks/al_agents/recipes/default.rb:11:in `from_file'
  /tmp/kitchen/cache/cookbooks/amazonlinux-base/recipes/default.rb:120:in `from_file'

Relevant File Content:
----------------------
/tmp/kitchen/cache/cookbooks/al_agents/libraries/helpers.rb:

  7:      def al_agent
  8:        node['al_agents']['package']['name']
  9:      end
 10:
 11:      def agent_file(uri)
 12:        require 'pathname'
 13:        require 'uri'
 14>>       Pathname.new(URI.parse(uri).path).basename.to_s
 15:      end
 16:
 17:      def agent_basename
 18:        agent_file(node['al_agents']['package']['url'])
 19:      end
 20:
 21:      def al_agent_service
 22:        node['al_agents']['agent']['al_agent_service']
 23:      end

Update rsyslog dependencies

Rsyslog being pinned at such an old version is causing issues in our environment and is forcing us to use 1.0.6 of this cookbook. Re-factoring to support newer versions of rsyslog would be great!

Bug - Provisioning only mode is enabled by default

I believe the logic for the for_imaging flag is wrong. According to the readme, the default value of for_imaging is false.

al_agents/libraries/helpers.rb

Line 90 in 51b7799

windows_options << "prov_only=#{inst_type_value}" unless for_imaging

Yet if you run the cookbook without specifying this attribute, then the installer installs in provisioning-only mode.

Here is the command that chef ran.

 msiexec /qn /i "c:\chef\cache\al_agent-latest.msi"  /quiet prov_key=xxx
 prov_only=host sensor_host=xxxxxxxx sensor_port=80

If you run this command without the /qn option, you see the following. Notice the "provisioning-only mode" at the bottom? That shouldn't be there if prov_only is actually false

Here is proof that the value is set to false

PS C:\chef\cache> chef-shell -z
chef (12.5.1)> puts node['al_agents']['agent']['for_imaging']
false
 => nil
chef (12.5.1)> puts node['al_agents']['agent']['for_autoscaling']
false


chef (12.5.1)> puts node['al_agents']
{"agent"=>{"registration_key"=>"xxxxxxxxx", "egress_url"=>"xxxxxxxxxx", "proxy_url"=>nil, "for_autoscaling"=>false, "for_imaging"=>false, "service_name"=>"al_agent"}, "package"=>{"name"=>"al-agent", "url"=>"https://scc.alertlogic.net/software/al_agent-LATEST.msi"}, "windows_install_guard"=>"C:\\Program Files (x86)\\Common Files\\AlertLogic\\host_key.pem"}

Work around:

You must set the value to false in the role. The msi then installs properly. I can reproduce this consistently.

    "al_agents": {
      "agent": {
        "registration_key": "xxxxxxxx",
        "egress_url": "https://vaporator.alertlogic.com:443",
        "for_imaging": false,
        "for_autoscaling": false
      }
    },

Question - Can cookbook change egress_url?

By default the msi configures to log to the following egress_url

    "egress_url": "https://vaporator.alertlogic.com:443"

Is there a way to change that later; maybe with an environment variable or config file? Or are you expected to destroy and recreate your nodes?

default recipe must be agent.rb and not log_agent.rb

https://github.com/alertlogic/al_agents/blob/master/recipes/default.rb#L20

Installation fails when both rsyslog and syslog_ng are installed

On Linux systems where rsyslog is installed but not enabled, the installation fails.

During the installation, these recipes are conditionally included:

include_recipe 'al_agents::selinux' if selinux_enabled?
include_recipe 'al_agents::rsyslog' if rsyslog_detected?
include_recipe 'al_agents::syslog_ng' if syslogng_detected?

The rsyslog_detected helper function only detects whether rsyslog is installed, not whether it is enabled. So the corresponding recipe is runs to configure for rsyslog and restart the service (which fails). It then never gets to the syslog_ng check/installation.

Recipe: al_agents::_windows
  * remote_file[al_agent-LATEST.msi] action create_if_missing
    - create new file C:\chef\cache/al_agent-LATEST.msi
    - update content in file C:\chef\cache/al_agent-LATEST.msi from none to 0d8fc0
    (new content is binary, diff output suppressed)
  * windows_package[al_agent-LATEST.msi] action install

    ================================================================================
    Error executing action `install` on resource 'windows_package[al_agent-LATEST.msi]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1603'
    ---- Begin output of msiexec /qn /i "c:\chef\cache\al_agent-latest.msi"  /quiet prov_key=xxxxxxxxxxxxx prov_only=host sensor_host=10.254.90.220 sensor_port=80 ----
    STDOUT:
    STDERR:
    ---- End output of msiexec /qn /i "c:\chef\cache\al_agent-latest.msi"  /quiet prov_key=xxxxxxxxxxxxx prov_only=host sensor_host=10.254.90.220 sensor_port=80 ----
    Ran msiexec /qn /i "c:\chef\cache\al_agent-latest.msi"  /quiet prov_key=xxxxxxxxxxxxx prov_only=host sensor_host=10.254.90.220 sensor_port=80 returned 1603

    Resource Declaration:
    ---------------------
    # In C:/chef/cache/cookbooks/al_agents/recipes/_windows.rb

     16: package basename do
     17:   source cached_package
     18:   action :install
     19:   options windows_options
     20:   not_if windows_install_guard
     21: end
     22:

    Compiled Resource:
    ------------------
    # Declared in C:/chef/cache/cookbooks/al_agents/recipes/_windows.rb:16:in `from_file'

    windows_package("al_agent-LATEST.msi") do
      action [:install]
      retries 0
      retry_delay 2
      default_guard_interpreter :default
      options "/quiet prov_key=xxxxxxxxxxxxx prov_only=host sensor_host=10.254.90.220 sensor_port=80"
      package_name "al_agent-LATEST.msi"
      source "c:\\chef\\cache\\al_agent-latest.msi"
      version "2.1.2.0"
      timeout 600
      returns [0]
      declared_type :package
      cookbook_name "al_agents"
      recipe_name "_windows"
      not_if "C:\Program Files (x86)\Common Files\AlertLogic\host_key.pem"
    end

I can't tell why the msi doesn't work. Can you provide any assistance to get this cookbook working?

Update

Looks like running the following MSI command manually works. (Removed the provision only flag)

msiexec /i "c:\chef\cache\al_agent-latest.msi" prov_key=xxxxxxx sensor_host=vaporator.alertlogic.com sensor_port=443��

alertlogic / al_agents Goto Github PK

al_agents's People

Contributors

Stargazers

Watchers

Forkers

al_agents's Issues

Work around:

Recommend Projects

Recommend Topics

Recommend Org