splunkconsumer's Introduction

SplunkConsumer

This project is a Proof of Concept (POC) and mentorship opportunity designed to demonstrate the use of Python packages for consuming data from external APIs. By leveraging Python's powerful libraries and best practices, this project serves as an educational resource for those looking to learn about API integration with Python.

Key Features:

API Integration: Explore how to connect and retrieve data from external APIs using Python packages.
Package Showcase: Demonstrate the usage of popular Python packages like requests or http.client for making API requests.
Documentation: Provide clear documentation and code examples to help learners understand the process.
Mentorship: Offer mentorship and guidance to those who want to expand their skills in API integration and Python programming.

The main goal with this project it's to implement a Prof of Concept with the following items:

First Step
- HTTP requests using python package requests to collect data from Splunk.
- Use of the python JSON package.
- Unit Coverage test using pytest
- Use of external tools like Postman / Insomnia
- Documentation following Docstring pattern
- Focus in a more python code following list and dictionary comprehension.
Second Step
- Data persistence using PyMongo.
- Schema Validation
- Integration and E2E test using RobotFramework or Behave
Third Step
- All the CI infra needs to be inside a container Dockerfile Doc.
  - Create a CI environmentto run the unit tests
  - Create a CI environmentto run the integration tests
  - Create a CI environmentto run the E2E test
Last Step
- Configure Grafana to consume data from MongoDB.

splunkconsumer's People

Contributors

Watchers

splunkconsumer's Issues

Query Builder

Develop a class builder to help create queries. This class will improve the lack of documentation related to Splunk queries and will introduce one of the most famous design patterns, the builder.

class SplunkQueryBuilder:
    def __init__(self):
        self.query = ''

    def select(self, fields):
        self.query += f'select {fields} '

    def from_table(self, table):
        self.query += f'from {table} '

    def where(self, condition):
        self.query += f'where {condition} '

    def group_by(self, fields):
        self.query += f'group by {fields} '

    def order_by(self, fields):
        self.query += f'order by {fields} '

    def build(self):
        return self.query


# Example usage of the SplunkQueryBuilder
query_builder = SplunkQueryBuilder()
query_builder.select('field1, field2') \
    .from_table('example_table') \
    .where('field3="value"') \
    .group_by('field1') \
    .order_by('field2')

splunk_query = query_builder.build()
print("Generated Splunk Query:", splunk_query)

It's very important to document every method used in this builder, as this will serve as a non-official documentation for Splunk queries used in the project.

Recommend Projects

lucasdfranchi / splunkconsumer Goto Github PK

splunkconsumer's Introduction

SplunkConsumer

splunkconsumer's People

Contributors

Watchers

Forkers

splunkconsumer's Issues

Query Builder

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent