Giter Club home page Giter Club logo

sumologic-collector-puppet-module's Introduction

sumo-collector-puppet-module

Sumo Puppet Module Overview

  • The Sumo Puppet module downloads the sumo logic collector agent from the Internet and installs the Sumo Collector agent. The module needs to download the package from the Sumologic API's so Internet access is required on your machines.
  • The module also allows sources to be created during installation and updated afterwards.
  • Currently, the module only supports the installation of latest collector version.
  • Upgrade of the collector is not supported currently.

Note:

For Puppet 4.x and later:

  • Utilize the release v0.2.1 or later. Release v0.2.1 and subsequent v0.2.x releases will be enhanced per puppet guidelines to support latest puppet versions.

For Puppet 3.x:

  • Utilize the release v1.0.6 if you need sumo.conf compatibility.
  • Utilize the release v1.0.8 if you need latest SumoLogic collector compatibility.

Setup

To begin using this module, use the Puppet Module Tool (PMT) from the command line to install this module:

puppet module install sumologic-sumo

To install Sumo Puppet Module from GitHub, follow below steps:

  • Navigate to Puppet global modules directory or specific environment modules directory.
  • Clone the Sumo Puppet module.
  • Rename the module as sumo.

Usage

Once the Sumo puppet module is installed, you will need to supply the sumo class with the authentication information.

  • The accessid.
  • The accesskey.

OR

  • One-time token for installation.

Additional optional parameters can be passed as required. The parameters details are explained in the next section.

Examples

A basic example, using accesskey/accessid, module expects the sources.json to be present on agent nodes in the default directory:

class { 'sumo':
  accessid       => 'accessid',
  accesskey      => 'accesskey',
}

Another example, using accesskey/accessid and manage_sources as true, the module will copy the sources.json on agent nodes in the default directory:

class { 'sumo':
  accessid       => 'accessid',
  accesskey      => 'accesskey',
  manage_sources => true,
}

Note: manage_sources will be deprecated in future releases, use sources_override instead.

Below example illustrates setting up local configuration management. In this case the syncsources.json file will be used for describing Sources to configure on registration, which will be continuously monitored and synchronized with the Collector's configuration. In this case the module expects the syncsources.json to be present on agent nodes in the default directory:

class { 'sumo':
  accessid              => 'accessid',
  accesskey             => 'accesskey',
  local_config_mgmt     => true,
}

Below example illustrates local configuration management with the syncsources.json being managed by the puppet master i.e. the module will copy the syncsources.json on agent nodes in the default directory:

class { 'sumo':
  accessid              => 'accessid',
  accesskey             => 'accesskey',
  local_config_mgmt     => true,
  sync_sources_override => true,
}

All the above examples illustrate setting up the sources using a single JSON file (sources.json/syncsources.json) present under the default directory. To specify sources in multiple JSON files, specify the parameter sources_directory_or_file as 'dir'. If specified as 'dir', the collector will look for any JSON files present under the default directory:

class { 'sumo':
  accessid                  => 'accessid',
  accesskey                 => 'accesskey',
  sources_directory_or_file => 'dir',
}

With local configuration management:

class { 'sumo':
  accessid                  => 'accessid',
  accesskey                 => 'accesskey',
  local_config_mgmt         => true,
  sources_directory_or_file => 'dir',
}

Advanced example illustrating passing additional command-line parameters and using rpm or deb package for installation:

class { 'sumo':
  accessid                => 'accessid',
  accesskey               => 'accesskey',
  sync_sources_override   => true,
  use_package	          => true,
  local_config_mgmt       => true,
  clobber                 => false, 
  ephemeral               => true,
  skip_access_key_removal => true,
}

Advanced example illustrating tarball binary package for installation:

class { 'sumo':
  accessid                => 'accessid',
  accesskey               => 'accesskey',
  sync_sources_override   => true,
  use_package	          => false,
  use_tar_pkg             => true,
  local_config_mgmt       => true,
  clobber                 => false, 
  ephemeral               => true,
  skip_access_key_removal => true,
}

Parameters

This module supports all the command-line options except dir [directory] listed in SumoLogic's documentation. Head there for a full explanation of what each option does to the SumoLogic collector.

The only required parameters are a pair of authentication parameters: accessid and accesskey.

Parameter Name Description Default value (in the module, not the collector)
accessid The access id for the collector to register with undef
accesskey The access key for the collector to register with undef
token One-time token for installation undef
clobber Whether you want to clobber the collector false
collector_name Name of the collector undef
collector_secure_files To enable or disable Enhanced File System Security undef
collector_url The URL used to download and register Collector 'https://collectors.sumologic.com'
description Description for the Collector to appear in Sumo Logic undef
disable_action_source To disable the running of script-based Sources undef
disable_script_source To disable the running of script-based action Sources undef
disable_upgrade If true, the Collector rejects upgrade requests from Sumo Logic undef
ephemeral Whether to mark the collector as ephemeral false
hostname The host name of the machine on which the Collector is running Hostname
local_config_mgmt If you want this module to enable local config management false
manage_sources If you want this module to manage your sources file, will be deprecated in later releases. Use sources_override false
proxy_host When using a proxy, the hostname to connect to undef
proxy_ntlmdomain When using an NTML proxy, the URL used to connect undef
proxy_password When using a proxy, the password to use to connect undef
proxy_port When using a proxy, the port to connect to undef
proxy_user When using a proxy, the user to connect as undef
runas_username When set, the Collector will run as the specified user undef
skip_access_key_removal If true, it will skip the access key removal from the user.properties file false
skip_registration When true, the Collector will install files and create user.properties file, but not register or start the Collector 'false'
sources_override If you want this module to manage your sources file false
sources_directory_or_file 'file': Sources are listed in a single JSON file (sources.json/syncsources.json) in default dir. 'dir': Sources are listed in multiple JSON files under the default directory. (/usr/local/sumo or c:\sumo) 'file'
sumo_exec The installation executable name architecture specific
sumo_json_source_path The Puppet URL for your sources.json file puppet:///modules/sumo/sources.json
sumo_json_sync_source_path The Puppet URL for your syncsources.json file puppet:///modules/sumo/syncsources.json
sumo_package_filename Name to store the downloaded sumo package with architecture specific (if use_package == true)
sumo_package_suffix The final string on the download URL architecture specific (if use_package == true)
sumo_package_provider Puppet package provider to install the package architecture specific (if use_package == true)
sumo_short_arch The shortened architecture to download architecture specific
sumo_win_arch The shortened architecture to download architecture specific
sync_sources_override If you want this module to manage your sync sources file false
target_cpu You can choose to set a CPU target to limit the amount of CPU processing a Collector uses undef
time_zone The time zone to use when the time zone can't be extracted from the time stamp undef
use_package Install from a rpm or a deb package. This flag overrides the tarball flag: use_tar_pkg i.e. if both use_package and use_tar_pkg are true, the rpm or the deb package will be used for installation. false
use_tar_pkg Install from a tarball. false
win_run_as_password When set in conjunction with -VrunAs.username, the Collector will run as the specified user with the specified password undef

Testing / Contributing

See CONTRIBUTING.md.

sumologic-collector-puppet-module's People

Contributors

arunpatyal avatar benoitnewton avatar drewrothstein avatar duchatran avatar eugk avatar mcasper avatar mleonidas avatar noodlesnz avatar quixey-githooks-pm-deploy avatar sumosourabh avatar terribledev avatar weirded avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sumologic-collector-puppet-module's Issues

Can you give a step-by-step example for you to use this module?

I've already got a puppet module that I'm firing up using puppet-librarian.

I tried to pull this in directly from git using puppet librarian. That didn't work. Next I tried to reference it in my puppet module's manifest/site.pp using the directive:
include sumologic::sumo

after having cloned the repo to /etc/puppet/modules_dist/sumologic and copied my config files here:
/etc/puppet/files/sumo/sumo.conf
/etc/puppet/files/sumo/sumo.json

modules_dist is in the modulepath set in my modules puppet.conf.

I didn't see any complaints in the cloud-init-ouput.log. But sumologic did not get installed.

Anyway, a step by step guide to using this module would be helpful.

ubuntu 18.04 install error

The puppet module is run during server boot up which installs and configures the sumo module.

The installation is done using the 'script based installer' using these options.

  class { 'sumo':
    accessid                   => $sumo_access_id,
    accesskey                  => $sumo_access_key,
    use_package                => false,
    local_config_mgmt          => true,
    sync_sources_override      => true,
    sumo_json_sync_source_path => 'puppet:///modules/profile/sumo/usr/local/sumo/sumo.os.json'
  }

The puppet run fails

[mNotice: /Stage[main]/Sumo::Nix_install/Exec[Execute sumo]/returns: executed successfully^[[0m
^[[1;31mError: Systemd start for collector failed!
journalctl log for collector:
-- Logs begin at Thu 2020-04-30 08:21:03 UTC, end at Thu 2020-04-30 09:14:52 UTC. --
Apr 30 09:14:52 ip-masked systemd[1]: Starting LSB: @app.long.name@...
Apr 30 09:14:52 ip-masked collector[2902]: Starting SumoLogic Collector...
Apr 30 09:14:52 ip-masked collector[2902]: SumoLogic Collector is already running.
Apr 30 09:14:52 ip-masked systemd[1]: collector.service: Control process exited, code=exited status=1
Apr 30 09:14:52 ip-masked systemd[1]: collector.service: Failed with result 'exit-code'.
Apr 30 09:14:52 ip-masked systemd[1]: Failed to start LSB: @app.long.name@.
^[[0m
^[[1;31mError: /Stage[main]/Sumo::Nix_install/Service[collector]/ensure: change from 'stopped' to 'running' failed: Systemd start for collector failed!
journalctl log for collector:
-- Logs begin at Thu 2020-04-30 08:21:03 UTC, end at Thu 2020-04-30 09:14:52 UTC. --
Apr 30 09:14:52 ip-masked systemd[1]: Starting LSB: @app.long.name@...
Apr 30 09:14:52 ip-masked collector[2902]: Starting SumoLogic Collector...
Apr 30 09:14:52 ip-masked collector[2902]: SumoLogic Collector is already running.
Apr 30 09:14:52 ip-masked systemd[1]: collector.service: Control process exited, code=exited status=1
Apr 30 09:14:52 ip-masked systemd[1]: collector.service: Failed with result 'exit-code'.
Apr 30 09:14:52 ip-masked systemd[1]: Failed to start LSB: @app.long.name@.
^[[0m
^[[mNotice: /Stage[main]/Sumo::Nix_install/Service[collector]: Triggered 'refresh' from 1 event^[[0m
^[[mNotice: /Stage[main]/Os_hardening::Sysctl/Sysctl[fs.suid_dumpable]/value: changed live value from '2' to '0'^[[0m
^[[mNotice: EC2 Role: haproxy^[[0m

however the sumo installation is complete. There are sumo PIDs active and the logs are getting ingested to the cloud ok.

root@ip-masked:/opt/SumoCollector/logs# ps -eaf | grep -i sumo
root      2705     1  0 09:14 ?        00:00:00 /opt/SumoCollector/./wrapper /opt/SumoCollector/./config/wrapper.conf wrapper.syslog.ident=collector wrapper.pidfile=/opt/SumoCollector/./collector.pid wrapper.daemonize=TRUE wrapper.name=collector wrapper.displayname=SumoLogic Collector wrapper.statusfile=/opt/SumoCollector/./collector.status wrapper.java.statusfile=/opt/SumoCollector/./collector.java.status wrapper.lockfile=/var/lock/subsys/collector wrapper.script.version=3.5.33
root      2708  2705  3 09:14 ?        00:00:21 /opt/SumoCollector/jre/bin/java -XX:+UseParallelGC -server -Djava.security.egd=file:/dev/./urandom -Xms64m -Xmx128m -Djava.library.path=./19.288-3/bin/native/lib -classpath ./19.288-3/lib/HikariCP-java7-2.4.13.jar:./19.288-3/lib/aether-api-0.9.0.M3.jar:./19.288-3/lib/aether-util-0.9.0.M3.jar:./19.288-3/lib/ant-1.8.4.jar:./19.288-3/lib/ant-launcher-1.8.4.jar:./19.288-3/lib/aopalliance-repackaged-2.4.0-b34.jar:./19.288-3/lib/archaius-core-0.7.6.SUMO-4.jar:./19.288-3/lib/archaius-scala_2.12-0.7.6.SUMO-4.jar:./19.288-3/lib/avro-1.8.2.jar:./19.288-3/lib/avro-ipc-1.8.2.jar:./19.288-3/lib/avrox-19.288-2.jar:./19.288-3/lib/aws-java-sdk-core-1.11.707.jar:./19.288-3/lib/aws-java-sdk-kms-1.11.707.jar:./19.288-3/lib/aws-java-sdk-s3-1.11.707.jar:./19.288-3/lib/backport-util-concurrent-3.1.jar:./19.288-3/lib/c3p0-0.9.5.5.jar:./19.288-3/lib/classgraph-4.8.43.jar:./19.288-3/lib/classworlds-1.1.jar:./19.288-3/lib/collector-19.288-3.jar:./19.288-3/lib/collector-base-19.288-1.jar:./19.288-3/lib/collector-interchange-19.288-1.jar:./19.288-3/lib/collector-native-19.288-1.jar:./19.288-3/lib/collector-util-19.288-1.jar:./19.288-3/lib/commons-cli-1.2.jar:./19.288-3/lib/commons-codec-1.12.jar:./19.288-3/lib/commons-collections-3.2.2.jar:./19.288-3/lib/commons-compress-1.19.jar:./19.288-3/lib/commons-configuration-1.10.jar:./19.288-3/lib/commons-httpclient-3.1.jar:./19.288-3/lib/commons-io-2.4.jar:./19.288-3/lib/commons-lang-2.6.jar:./19.288-3/lib/commons-logging-1.2.jar:./19.288-3/lib/compiler-0.9.6.jar:./19.288-3/lib/docker-java-3.0.17.jar:./19.288-3/lib/doxia-logging-api-1.1.jar:./19.288-3/lib/doxia-sink-api-1.1.jar:./19.288-3/lib/ecc-25519-java-1.0.3.jar:./19.288-3/lib/eddsa-0.2.0.jar:./19.288-3/lib/guava-19.0.jar:./19.288-3/lib/hk2-api-2.4.0-b34.jar:./19.288-3/lib/hk2-locator-2.4.0-b34.jar:./19.288-3/lib/hk2-utils-2.4.0-b34.jar:./19.288-3/lib/httpclient-4.5.10.jar:./19.288-3/lib/httpcore-4.4.12.jar:./19.288-3/lib/install4j-maven-plugin-1.0.7.jar:./19.288-3/lib/ion-java-1.0.2.jar:./19.288-3/lib/jackrabbit-jcr-commons-2.1.0.jar:./19.288-3/lib/jackrabbit-webdav-2.1.0.jar:./19.288-3/lib/jackson-annotations-2.

However on the m/c, the service configured on the OS called 'collector' has a status of 'Active: failed'. 

sj@ip-masked:~$ service collector status
● collector.service - LSB: @app.long.name@
   Loaded: loaded (/etc/init.d/collector; generated)
   Active: failed (Result: exit-code) since Thu 2020-04-30 09:14:52 UTC; 8min ago
     Docs: man:systemd-sysv-generator(8)

If I try to start the service 'collector', that fails as well. 

To fix the scenario, I have to manually kill the sumo PIDs and start the collector. 

root@ip-masked:/opt/SumoCollector/logs# kill -9 2705

root@ip-masked:/opt/SumoCollector/logs# service collector restart
root@ip-masked:/opt/SumoCollector/logs# service collector status
● collector.service - LSB: @app.long.name@
   Loaded: loaded (/etc/init.d/collector; generated)
   Active: active (running) since Thu 2020-04-30 09:42:21 UTC; 3s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 21292 ExecStart=/etc/init.d/collector start (code=exited, status=0/SUCCESS)
    Tasks: 46 (limit: 4417)
   CGroup: /system.slice/collector.service
           ├─21382 /opt/SumoCollector/./wrapper /opt/SumoCollector/./config/wrapper.conf wrapper.syslog.ident=collector wrapper.pidfile=/opt/SumoCollector/./collector.pid wrapper.daemonize=TRUE wrapper.name=collector
           └─21385 /opt/SumoCollector/jre/bin/java -XX:+UseParallelGC -server -Djava.security.egd=file:/dev/./urandom -Xms64m -Xmx128m -Djava.library.path=./19.288-3/bin/native/lib -classpath ./19.288-3/lib/HikariCP-

Apr 30 09:42:18 ip-masked systemd[1]: Starting LSB: @app.long.name@...
Apr 30 09:42:18 ip-masked collector[21292]: Starting SumoLogic Collector...
Apr 30 09:42:18 ip-masked collector[21292]: Removed stale pid file: /opt/SumoCollector/./collector.pid
Apr 30 09:42:21 ip-masked collector[21292]: Waiting for SumoLogic Collector......
Apr 30 09:42:21 ip-masked collector[21292]: running: PID:21382
Apr 30 09:42:21 ip-masked systemd[1]: Started LSB: @app.long.name@.

Most likely the install command (

command => "/bin/sh /usr/local/sumo/${sumo_exec} -q -varfile /etc/sumo/sumoVarFile.txt",
) also kicks off the sumo processes before the service is configured.

PR 32 breaks compatibility with older versions of puppet

Pull request 32 merges in data type declarations that are incompatible with old versions of puppet. Some environments are unable to use the latest and greatest versions of puppet, rendering the master branch broken for them (us).

sumo_json_source_path not defined in nix_config.pp

Hi There,

From what i can tell, nix_config.pp is missing a value defined for sumo_json_source_path variable.

To fix this in my local i have included inherits sumo::params in nix_config.pp

Is this a bug or am i missing something?
Cheers
Adam

Support for windows

I'd like to spend some time adding proper support for windows. I can do this myself. Any objections?

support /usr/local/SumoCollector installs

Sometimes the collector seems to install itself in /usr/local instead of /opt. This means the puppet manifest does not recognize that the collector has been installed and thus will run the executable for each puppet run.

Using one's own CA certs requires a tortuous workaround

Using one's own CA certs, a pretty common practice in many larger environments, causes PKI errors (the infamous "unable to find valid certification path to requested target"). This is fixable manually, but the puppet solution is not obvious. sumo::use_package must be set true and one must make a soft link to one's own cacerts:

  file { '/opt/SumoCollector/jre/lib/security/cacerts':
    ensure  => link,
    target  => '/etc/pki/ca-trust/extracted/java/cacerts',
    require => Package['collector'],
    notify  => Service['collector']
  }

I humbly suggest creating a configuration item for a cacert file location to avoid this workaround.

Windows manifest doesn't match Sumo Logic documentation

In the windows manifest in this module, the installation dir appears to be C:\sumo\

However the docs state that the installer ends up in C:\Program Files\Sumo Logic Collector\ (or in the (x86) one

Questions

  • Is C:\sumo\ just for downloading the installer?
  • The docs state that user.properties is supposed to go into the installed dir. But this module has sumo.conf going into c:\sumo. Please clarify what's going on here.

Thanks

Release?

It looks like there are release tags, the latest of which was created about a year ago. The latest changes committed to the master branch are 5 months old and are not in any release. Is this puppet module still being maintained and supported by SumoLogic?

sumologic-collector-puppet-module added to puppet forge ?

Are there any plans to add the sumologic-collector-puppet-module to puppet forge.

I would like to start working with this module, but my company has systems to pull from the Forge.

Any chance we could get the last release added to puppet forge? Or does anyone mind if I fork and add it to puppet forge?

Thanks in advance.

Does not appear to support later versions of Sum collector

There were some changes after version 19.137 of the Sumo Logic collection agent that seem to be unsupported by this Puppet module. Most notably, the user.properties file replaces sumo.conf

Is this Puppet module still actively developed? will it work for the newest Sumo Logic collector?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.