Giter Club home page Giter Club logo

sa-cim_vladiator's Introduction

Welcome

This Splunk app was developed with one goal in mind, reduce amount of time spent validating Splunk Common Information Model (CIM) compliance of technology add-ons (TA's). Use of this app simplifies validation process in several ways:

  • Identifies fields that are required, but missing
  • Validates data confirms to expected CIM values
  • Rapid prototyping and validation

This project is hosted on GitHub, https://github.com/hire-vladimir/SA-cim_vladiator

Install

BYOL (on-prem) install

App installation is simple, and only needs to be present on the search head. Documentation around app installation can be found at http://docs.splunk.com/Documentation/AddOns/released/Overview/Singleserverinstall

Splunk Cloud install

App installation can be completed using the self-service capabilities. Documentation around app installation can be found at https://docs.splunk.com/Documentation/SplunkCloud/latest/User/PrivateApps

Getting Started

Screenshot

CIM validator

System requirements

App was developed for use with Splunk Enterprise and Splunk Cloud 8.x+, 9.x+

Special Thanks

Thank you to Lowell Alleman for python3 support, Annette Quach for UBA support.

Legal

  • Splunk is a registered trademark of Splunk, Inc.

sa-cim_vladiator's People

Contributors

annettefo avatar hire-vladimir avatar lowell80 avatar mcm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sa-cim_vladiator's Issues

eventtype / tag analysis

a summary view of all the eventtypes that make up a tag would be useful such that it could be tackled by individual ETs

Speaking at a Splunk User Group

I run a Splunk User Group and the SA-cim-vladiator / SA-cim_validator was requested for a topic of a meeting by a user. Would you be interested in speaking at one of our virtual meeting?

datamodel mode validation issue

The DM search seems to now call fields as DM.* type format, causing no values to be returned. Need to enhance the logic to perform | rename <<DM_NAME>>.* AS * in the pipeline to work around this.

Re-checking

I would love to be able to be notified if my data has changed. The app is great, and it would be great to be able to specify a list of monitored sourcetypes that can be rechecked periodically (nightly?) so that I can get a notification of changes. Use case is that some log sources change periodically; I might have an upstream log source updated, and all of a sudden I would end up with new values for action that are not desired.

mvmath incorrect %

when field has only 1 value with 100% coverage, mvmath_result shows 0.01%

Upgrade Readiness App fail

The Upgrade Readiness App reports "[t]his app is [in]compatible with Python 3."

Issue:
File path designates Python 2 library.
Incompatible File Paths
File Location

  1. .../bin/mvmath.py

App:SA-cim_vladiator
File Path:.../bin/mvmath.py
Issue No.
Issues
1.
@@ -14,8 +14,9 @@

 # Mini 'six'-like compat layer
 import sys
+import six
 if sys.version_info[0] == 2:
-    string_types = (basestring,)
+    string_types = (six.string_types,)
 else:
     string_types = (str,)

validation logic result

Presently, logic is controlled by a case() statement, meaning only one "outcome" can be shown. This is fine, however, should the validatator show multiple outcomes?

present:
< 90% coverage

possible:
`< 90% coverage AND your data is junk``

Trying to get Splunk Cloud Approval

So I wanted to see if you can help me make these updates by suggestions etc.

I would need to fix the items below in order to use your app in the cloud.

here is more info on the results below : http://dev.splunk.com/view/appinspect/SP-CAAAE9U

Check Message
Check that apps with app.manifest are valid or apps without an app.manifest can generate one. App folder name SA-cim_vladiator-master does not match App ID SA-cim_vladiator

Check that when decompressed the Splunk App directory name matches the app.conf [package] stanza's id property. | The app.conf [package] stanza has an id property that does not match the uncompressed directory's name. app.conf [package] id: SA-cim_vladiator uncompressed directory name: SA-cim_vladiator-master

IPv6 addresses flagged as "unexpected values"

When valid IPv6 addresses are present in the dest_ip, dest, src_ip, or src fields, the CIM Validator flags all of the IPv6 addresses as "unexpected values" and marks that field as having elevated issues.
Please extend the validation for those fields to include IPv6 support.

XML version issue

I am able to install this successfully in on-prem, but when installing in Splunk cloud im able to see multiple errors attached below.
image
image

Failures will block the Cloud Vetting. They must be fixed.
check_that_extracted_splunk_app_does_not_contain_prohibited_directories_or_files

A prohibited file or directory was found in the extracted Splunk App: .gitignore

check_that_splunk_app_package_does_not_contain_files_outside_of_app

A file or folder was found outside of the app within the overall package. OR the file or folder does not have expected permission. Please remove this file or folder OR modify the permission : SA-cim_vladiator-1.8.0

check_simplexml_standards_version

Change the version attribute in the root node of your Simple XML dashboard default/data/ui/views/cim_dictionary.xml to `<version=1.1>`. Earlier dashboard versions introduce security vulnerabilities into your apps and are not permitted in Splunk Cloud File: default/data/ui/views/cim_dictionary.xml

Change the version attribute in the root node of your Simple XML dashboard default/data/ui/views/cim_validator.xml to `<version=1.1>`. Earlier dashboard versions introduce security vulnerabilities into your apps and are not permitted in Splunk Cloud File: default/data/ui/views/cim_validator.xml

Multiple data model names conflicting with Splunk CIM listing

While using the app, came across the following:

Domain Analysis - not official data model
Compute Inventory - that's the name of the .json file, can it be named "Inventory" like official documentation?
Identity Management - not official data model
Incident Management - not official data model, changed to Ticket Management?
Risk - not official data model
Threat Intelligence - not official data model

Not compatible with jQuery 3.5

Any chance on an upgrade?
Splunk cloud's upgrade readiness app is reporting:

Details
This app is not compatible with jQuery 3.5.
Version
1.8.0
Remote Version
1.8.0
Application Path
/opt/splunk/etc/apps/SA-cim_vladiator
Required Action
Update this app or request to uninstall it. If you do nothing, the app will fail in future Splunk upgrades that use jQuery 3.5.

Dismiss App

Email Result

Export Result
Issue:
Splunk dashboard jQuery version check
Incompatible File Paths
.../default/data/ui/views/cim_validator.xml
.../default/data/ui/views/cim_dictionary.xml

CIM Dictionary is outdated.

CIM Dictionary is outdated. Splunk's CIM version is at 4.18.0, but the CIM validator's dictionary is at 4.3.1.

How can we update the dictionary (build the csv)?

Splunk Cloud / Splunk Enterprise 8.0 / Python 3 Compatibility

Splunk Cloud will soon require Python 3 compatibility. This is also required for Splunk Enterprise 8.0
From an e-mail from Splunk regarding Splunk Cloud:
"You are required to make all 3rd party and private apps in your Splunk Cloud instances compatible with Python 3 by August 1, 2020."

Python 3 Migration

I ran 2to3 against mvmath.py and mvrex.py, and it said changes needed to be done. However, since I'm not as familiar with python as I want to be, I don't want to blindy apply those changes and call it ok.

Attached is the output of 2to3
2to3-CIM-validator.txt

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.