Giter Club home page Giter Club logo

duck's Introduction

Duck

Duck is a build system agnostic build monitor written in Rust (backend server) and Vue (frontend). It also support sending messages to Slack and controlling Philips Hue lights.

The frontend

Table of Contents

  1. Getting started
  2. How it works
  3. Supported providers
  4. Configuration
  5. License

Getting started

The absolute easiest way of getting started with Duck is to use the docker image. You will have to provide a configuration file for it to work.

> docker run --rm -it -v ~/.config/duck:/data -p 8080:15825 duckhq/duck:latest start --config /data/duck.json

If you're using WSL, you will need to specify the absolute path to your data directory.

Development

There is also a Docker Compose file and separate dockerfiles for running Duck in docker for development. This runs two different containers: one for the rust app and one for the Vue UI. Both containers are set up to watch for changes to their respective source code.

> docker-compose up

Note: Although the ui container is set up to depend on the app container (to ensure that running the ui will also run the app) it is your browser which has the dependency on the app, not the ui.

How it works

The server part is responsible for collecting information from different collectors and forward interesting events to observers. The aggregated information is available for other services (such as the Vue frontend) via a HTTP API.

Observers can either be dependent on events from all collectors, or opt in to one or more collectors. This makes it easy to setup team specific build lights or Slack integration that's only dependent on specific collectors.

Overview

Supported providers

Collectors

Collectors are responsible for collecting builds and deployments from other systems such as build servers.

Observers

Observers monitor builds and deployments and react to any changes that is made to their state.

Configuration

Example

Below is an example configuration that specifies multiple collectors and observers. By specifying the schema you will get autocomplete support in editors that support it like Visual Studio Code.

{
    "$schema": "https://raw.githubusercontent.com/duckhq/duck/master/schemas/v0.14.json",
    "interval": 30,
    "views": [
        {
            "id": "devs",
            "name": "Developers",
            "collectors": [
                "github_pullrequests",
                "teamcity_internal",
                "teamcity_local",
                "octopus_local"
            ]
        },
        {
            "id": "ops",
            "name": "Operations",
            "collectors": [
                "octopus_local"
            ]
        }
    ],
    "collectors": [
        {
            "teamcity": {
                "id": "teamcity_internal",
                "serverUrl": "https://${TEAMCITY_HOST}:${TEAMCITY_PORT}/",
                "credentials": "guest",
                "builds": [
                    "My_Project_Definition",
                    "My_Other_Build_Definition"
                ]
            }
        },
        {
            "github": {
                "id": "github_pullrequests",
                "owner": "duckhq",
                "repository": "duck",
                "workflow": "pull_request.yml",
                "credentials": {
                    "basic": {
                        "username": "patriksvensson",
                        "password": "hunter1!"
                    }
                }
            }
        },
        {
            "teamcity": {
                "id": "teamcity_local",
                "enabled": false,
                "serverUrl": "192.168.0.1:8111",
                "credentials": {
                    "basic": {
                        "username": "admin",
                        "password": "hunter1!"
                    }
                },
                "builds": [
                    "My_Third_Build_Definition"
                ]
            }
        },
        {
            "azure": {
                "id": "azure_cake",
                "enabled": false,
                "organization": "cake-build",
                "project": "Cake",
                "credentials": "anonymous",
                "definitions": [ "1", "3", "5" ],
                "branches": [
                    "refs/heads/develop",
                    "refs/heads/main"
                ]
            }
        },
        {
            "azure": {
                "id": "azure_private",
                "enabled": false,
                "serverUrl": "https://example.com",
                "organization": "some-organization",
                "project": "some-project",
                "credentials": {
                    "pat": "${AZURE_PAT}"
                },
                "definitions": [ "1" ],
                "branches": [
                    "refs/heads/master"
                ]
            }
        },
        {
            "octopus": {
                "id": "octopus_local",
                "serverUrl": "https://localhost:9000",
                "credentials": {
                    "apiKey": "${OCTOPUS_API_KEY}"
                },
                "projects": [
                    {
                        "projectId": "Projects-1",
                        "environments": [
                            "Environments-1",
                            "Environments-2"
                        ]
                    }
                ]
            }
        },
        {
            "appveyor": {
                "id": "appveyor",
                "credentials": {
                    "bearer": "${APPVEYOR_BEARER_TOKEN}"
                },
                "account": "myaccount",
                "project": "myproject-slug"
            }
        }
    ],
    "observers": [
        {
            "hue": {
                "id": "hue_team_1_builds",
                "collectors": [
                    "teamcity_local"
                ],
                "hubUrl": "http://192.168.1.99",
                "username": "THE-HUE-USERNAME",
                "lights": [ "1" ]
            }
        },
        {
            "hue": {
                "id": "hue_all_builds_dimmed",
                "hubUrl": "http://192.168.1.99",
                "username": "THE-HUE-USERNAME",
                "brightness": 50,
                "lights": [ "2", "3" ]
            }
        },
        {
            "slack": {
                "id": "slack_team1",
                "collectors": [ "teamcity_local" ],
                "credentials": {
                    "webhook": {
                        "url": "https://hooks.slack.com/services/MY-WEBHOOK-URL"
                    }
                }
            }
        },
        {
            "mattermost": {
                "id": "mattermost",
                "channel": "build-status",
                "credentials": {
                    "webhook": {
                        "url": "https://mattermost.example.com"
                    }
                }
            }
        }
    ]
}

License

Copyright © Patrik Svensson and Gary McLean Hall.

Duck is provided as-is under the MIT license. For more information see LICENSE.

duck's People

Contributors

dependabot[bot] avatar garymcleanhall avatar patriksvensson avatar ryanrousseau avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

duck's Issues

Get UI title from Duck server

We should be able to provide the title as an argument (or configuration setting) to the Duck server and let the Duck UI retrieve it together with other data.

View collectors are not verified at startup

Describe the bug
It's possible to have a typo in a view collector id without Duck complaining about it.

Expected behavior
Duck should show an error at startup:

Error: The view 'foo' depends on collector 'bar' which does not exist

Create default configuration if missing

Is your feature request related to a problem? Please describe.
If a configuration is missing, it would be nice if we could create a configuration for the user. Of course, this would have to be opt in, and non-destructive.

Describe the solution you'd like
If the user pass the flag -c|--create-config and the configuration is missing, an empty configuration should be created for the user. If there already is a configuration in place, the configuration should not be created but use the existing one. If the creation returns an error and it's running in a Docker container, it should inform the user about a potential problem with volume mapping.

UI: Add way of switching views

Right now we can switch the current "view" by adding ?view=foo to the Duck url. Would be nice with some (unintrusive) way of doing this directly in the UI.

Add GitHub actions collector

Is your feature request related to a problem? Please describe.
Would be nice with support for GitHub actions.

Support Azure DevOps Server (on-prem)

Is your feature request related to a problem? Please describe.
Add support for collecting builds from an on-prem Azure DevOps Server 2019.

Configuration example:

"azure":{
    "id": "azure_cake",
    "serverUrl": "https://example.com/",
    "organization": "cake-build",
    "project": "cake",
    "definitions": [ "5", "6", "7" ],
    "branches": [
        "refs/heads/main",
        "refs/heads/develop"
    ],
    "credentials": "anonymous"
}

Publish to Chocolatey

Is your feature request related to a problem? Please describe.
When we're doing a release of Duck, we should publish Duck to Chocolatey.

Add query language to filter builds per observer

Add a query language to filter builds per observer. This makes it possible to filter specific builds for a specific observer (i.e. if you don't want an update for a specific build definition to be sent to Slack).

Properties that should be supported:

Property Type Description
branch String The branch
status BuildStatus success, canceled, skipped, failed, running, queued The build status
project String The project ID
build String The build definition ID
buildId String The build ID
buildNumber String The build number
collector String The collector id
provider String The build provider
finished boolean Whether or not the build has finished running

An example of this could be:

{
    "collectors": [
        {
            "github": {
                "id": "github_duck",
                "owner": "duckhq",
                "repository": "duck",
                "workflow": "ci.yaml",
                "filter": "branch != 'master' and status == skipped",
                "credentials": {
                    "basic": {
                        "username": "patriksvensson",
                        "password": "SUPER-SECRET-API-KEY"
                    }
                }
            }
        }
    ]
}

Add webhook observer

Is your feature request related to a problem? Please describe.
It would be useful to be able to add a webhook observer that takes events generated by Duck and posts them to a webhook.

Describe the solution you'd like
Would be best implemented as an observer. It should be configurable what events to publish.

Dark Theme

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
It would be awesome if we could set a "DarkTheme": "true" property somewhere in the config file, which would result in a black background instead of the white we have today.

Run tests on other platforms than Linux

Is your feature request related to a problem? Please describe.
At the moment we only run tests on Linux. We should run tests on all available platforms that Duck runs on.

Same build present in two collectors appear twice

Describe the bug
If there are two collectors which contains the same build definition, the build will appear twice.

To Reproduce
Steps to reproduce the behavior:

  1. Create two identical collectors
  2. Trigger a build or deploy
  3. Check UI

Expected behavior
There should only be one representation of the build in the UI and the API response.

Screenshots
image

Additional context
Perhaps we should not take collector into account when determining whether or not a build is unique.

Add support for custom views

Is your feature request related to a problem? Please describe.
Having two different teams share the same dashboard doesn't always make sense. It would be nice if we could define views that could be switched between in the UI.

Describe the solution you'd like
There could be a new section in the configuraion that defines the views.

{
    "$schema": "https://raw.githubusercontent.com/spectresystems/duck/master/schemas/v0.4.json",
    "collectors": [ ],
    "observers": [ ],
    "views": [
        {
            "name": "team_a",
            "title": "A-Team",
            "collectors": [
                "collector1", "collector2"
            ]
        },
        {
            "name": "team_b",
            "title": "B-Team",
            "collectors": [
                "collector1", "collector3"
            ]
        }
    ]
}

Add tests for observers

We need to add tests for observers. To do this we need to abstract away the HTTP calls made to the observers.

Allow expansion of environment variables in configuration

Is your feature request related to a problem? Please describe.
When a Duck configuration is persisted to source control, it would be nice not having API keys and similar in plain text.

Describe the solution you'd like
Would be nice if configuration fields could contain some kind of templating language that is expanded when the provider is created.

"octopus": {
    "id": "octopus",
    "serverUrl": "http://localhost:9000",
    "credentials": {
        "apiKey": "${DUCK_OCTOPUS_KEY}"
    },
    "projects": [
        {
            "projectId": "Projects-1",
            "environments": [
                "Environments-1",
                "Environments-2",
            ]
        }
    ]
}

Run Duck as a Windows Service

Is your feature request related to a problem? Please describe.
Would be nice to be able to run Duck as a Windows Service

To install:

> duck install

To uninstall:

> duck uninstall

Add command to validate configuration

Is your feature request related to a problem? Please describe.
Would be nice if there was a command to validate the configuration without starting Duck.

Describe the solution you'd like
Add a new command validate that only runs the validation of the specified configuration.

Add skeleton loading cards to the UI

Describe the solution you'd like
To avoid a perceived unresponsiveness when pulling the build data from the API, we can display skeleton loading cards in the interim.

Add debug collector

Is your feature request related to a problem? Please describe.
Add a collector that gets builds from the Ducktor

Duck collector: Add support for views

Is your feature request related to a problem? Please describe.
When collecting builds from another Duck instance, it would be nice if I could specify which view to use.

Describe the solution you'd like

{
    "collectors": [
        {
            "duck": {
                "id": "duck_other",
                "serverUrl": "http://localhost:8081",
                "view": "the_view_id"
            }
        }
    ]
}

Mattermost observer reports build status twice

Describe the bug
When a build status change, the Mattermost observer reports the change twice.

To Reproduce
Steps to reproduce the behavior:

  1. Make sure there's a Mattermost observer configured.
  2. Trigger a build status change.

Expected behavior
The build status should only be reported once.

Add macOS artifacts

Is your feature request related to a problem? Please describe.
When we create the release artifacts we should compile Duck for macOS as well (not just Linux and Windows)

Strange message if running in Docker when Duck server can't be reached

Describe the bug
When running in a docker container, the Duck server endpoint is not written to the output if the Duck server can't be reached.

The Duck server could not be reached at . Retrying...

Expected behavior
It should say something like The Duck server could not be reached. Retrying... if there is no explicit Duck server endpoint.

Screenshots

image

Octopus Deploy: Canceled status is treated as "Running"

Describe the bug
If a deployment is canceled, it's being treated as it's still running.

To Reproduce
Steps to reproduce the behavior:

  1. Cancel a Octopus Deploy deployment
  2. See deployment still running in Duck dashboard

Expected behavior
It should be treated as "Canceled", but since there isn't a status like that yet in Duck, we should (temporarily) treat it as "Failed".

Brain dump on frontend per twitter convo

-Not a major deal, but you might want to be consistent with capatilization
https://github.com/spectresystems/duck/blob/f5fe6b9df0fa722f88db3195c29d837c16b3c897/web/src/App.vue#L41

  • General point -> == vs ===, ect.

  • For typescript export actual Vue.extend, and you can then use interface for methods, props, ect.

interface Props {
	name: string;
}
interface Methods {
	sayHello: () => void;
}
// Here to illustrate placement of arguments, if you don't need them, you can pass {} in the generics
interface Data {}
interface Computed {}

export default Vue.extend<Data, Methods, Computed, Props>({
	props: {
		name: {}
	},
	methods: {
		sayHello(){
			// get intellisense and type checking for `this`
			return `Hello ${this.name}`;
		}
	}
});

Add hot reloading of configuration

Is your feature request related to a problem? Please describe.
If a configuration is changed while Duck is running, the configuration should be reloaded.

Describe the solution you'd like
Reload the configuration if it's changed. If there is an error, do not exit duck but show the error in the log output and a generic error in the UI.

Rename release artifacts

While the name of the release artifacts are super accurate, they’re not very user friendly. We should consider renaming them to something like:

  • duck-0.12.0-linux_x64.tar.gz
  • duck-0.12.0-linux_arm.tar.gz
  • duck-0.12.0-windows_x64.zip

Add Duck collector

Is your feature request related to a problem? Please describe.
Add a collector that gets builds from another Duck instance. This will make it possible (i.e. when you're working from home) to use an existing configuration, but use your own observers, perhaps a build lamp on your desk.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.