Giter Club home page Giter Club logo

log-analytics-starter-kit's Introduction

Slack Status

Tinybird Log Analytics Starter Kit

Data can help developers build better software, but it can be hard to find data tools that are easy to integrate, scale with you as you grow, and are flexible enough to meet different demands. This Stater Kit provides an example for you can build your own logging & telemetry solution powered by Tinybird.

This is an opinionated Starter Kit that uses Web Applications as the example, but you can use this kit to bootstrap logging for any software project. Take this kit and make it your own!

Tinybird Functions Analytics Dashboard

What's in the box?

This Starter Kit includes:

  • Tinybird data project
  • Analytics Dashboard built with Next.js & Tremor
  • Example Log Capture implementations
    • TypeScript & Next.js Logger + example Vercel functions
    • Python & FastAPI Logger

Tinybird Data Project

The Tinybird data project includes all of the Tinybird resources you need to ingest, analyze and publish your logs. This includes Data Sources, Pipes, Materialized Views, and APIs.

Tinybird Functions Analytics DataFlow

Analytics Dashboard

The Analytics Dashboard is an example of how of you can build your own frontend to visualize the data from Tinybird.

The dashboard is implemented using Next.js & Tremor. All of the charts on the dashboard consume data over the HTTP API Endpoints that are publish from Tinybird.

There is no caching or frontend magic, every chart is consuming live data in real time.

Tinybird Functions Analytics Dashboard

Example Log Capture

All examples of capturing logs use the Tinybird's Events API, sending logs as JSON via a standard HTTP POST request. Because this is simply JSON over HTTP, you can use this approach to integrate any framework, language or application to send data to Tinybird.

Each logger exposes 3 methods info, warn and error that can be used to capture log messages with different log levels. Each one in turn calls the log method, which captures additional information about the incoming request.

Note that each logger expects an Environment Variable TINYBIRD_TOKEN to authenticate the request to Tinybird. You can copy this token from your Tinybird account by logging into the Tinybird UI.

TypeScript & Next.js

In logger.ts you'll find a simple reference implementation for a log capture class in TypeScript. This is built with Next.js in mind, but could be adapted to any other framework.

Vercel Functions

An example Vercel Function is included at getProductExample.ts. This function simply demonstrates how to use the logger in a function.

Python & FastAPI

In logger.py you'll find a simple reference implementation for a log capture class in Python. This is built with FastAPI in mind, but could be adapted to any other framework.

FastAPI Endpoint

An example FastAPI Endpoint is included at main.py. This endpoint simply demonstrates how to use the logger in a function.

Deploy

There are two components to this Starter Kit that you need to deploy.

1. Tinybird Data Project

Use the button below to deploy this Starter Kit to Tinybird.

Deploy to Tinybird

2. Analytics Dashboard

Use the button below to deploy this Starter Kit to Vercel.

Deploy with Vercel

log-analytics-starter-kit's People

Contributors

chronark avatar dmytro-tinybird avatar mitrotasios avatar radchukd avatar rmorehig avatar sdairs avatar severinlandolt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

log-analytics-starter-kit's Issues

[Feature Request] Less annoying issue templates

Describe the feature you'd like to request

These issue templates are annoying.

Describe the solution you'd like

Let's bin them and have something much lighter touch.

Describe alternatives you've considered

No response

[Bug] Mockingbird generator isn't working

Module

Dashboard

Describe the Bug

We ran into this issue on the LASK live stream & seen again here #9 (comment)

I think there is an error in the Mockingbird setup, it wasn't generating data and it seemed to be using ui.tinybird.co rather than api.tinybird.co ? Need to investigate

Expected Behavior

.

To Reproduce

.

Details

No response

[Feature Request] Update Mockingbird

Describe the feature you'd like to request

We should update the embedded mockingbird widget to use the latest Mockingbird version

Describe the solution you'd like

.

Describe alternatives you've considered

.

[Feature Request] Readme is confusing

Describe the feature you'd like to request

We need to improve the README to line up with the starter kit landing page and deploy flow inside the product.

Describe the solution you'd like

The README should start with deploying the Tinybird data project, then Vercel, then discuss the optional steps of using the Loggers.

We should discuss generating fake data with Mockingbird.

We should discuss the Next.js logger and other loggers, to make it clear that the project is language agnostic.

Describe alternatives you've considered

No response

[Bug] The app is not building properly in Vercel

Module

Dashboard

Describe the Bug

The deploy button has a missing argument (root-directory=dashboard). Vercel needs this data to know where to look for the app code.

Expected Behavior

The app should build properly in Vercel when using the Deploy button.

To Reproduce

Click the Deploy button and follow the process.

Details

No response

[Feature Request] Rename the ingest DS to 'logs'

Describe the feature you'd like to request

The ingest DS is current function_logs which is a hang over from the old theme; we should rename this to just logs for clarity.

Describe the solution you'd like

...

Describe alternatives you've considered

No response

[Feature Request] Python Logger

Describe the feature you'd like to request

We should demonstrate how to ingest logs via Python

Describe the solution you'd like

Let's build a super-simple logger that could be dropped into a web-project built with Python, using e.g. Django or FastAPI.

If you look at the Next.js logger, we expect to have e.g. NextRequest. So we can do something similar for whatever Request objects we would get from using a Python framework.

Describe alternatives you've considered

No response

[Feature Request] Upgrade Tremor to 1.5.0

Describe the feature you'd like to request

Tremor will reach 1.5.0 soon & I believe we are on 1.2.0.

IIRC The updates include the re-worked DatePicker. Let's get ourselves up the latest release so we don't fall too far behind, and test if the new DatePicker fixes the issues we had.

Let's start staging on the work with 1.5.0beta, but wait for the full 1.5.0 to merge

Describe the solution you'd like

.

Describe alternatives you've considered

No response

[Feature Request] Add GitHub link to credential modal & dashboard

Describe the feature you'd like to request

Can we add a GitHub link/button both to the Credentials modal & somewhere on the dashboard?

In the Credentials modal there should be a button to get to the GitHub repo, but also an area some text to send people to the README. Right now, if you land on the page & you don't already know what you're doing, you get stuck in this modal being asked for a token, but there's nothing saying what the token is, how to get it, why you should care, etc. The README for this repo is the best place to find that information atm, so we should try to guide a user to this repo ("Not sure what to do next? The README for this Starter Kit explains how to get started" or something like this)

On the Dashboard, we should have a button that takes the user to the repo. We will have a live demo running, so we want users to be able to easily find the repo from the live demo.

Describe the solution you'd like

GitHub button

Describe alternatives you've considered

No response

[Feature Request] Upgrade tremor to 1.7.0

Describe the feature you'd like to request

Upgrade tremor to 1.7.0

Describe the solution you'd like

Upgrade tremor to 1.7.0

Describe alternatives you've considered

No response

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.