oliviabarrick / fluxcloud Goto Github PK
View Code? Open in Web Editor NEWSlack notifications for Weave Flux without Weave Cloud
License: Apache License 2.0
Slack notifications for Weave Flux without Weave Cloud
License: Apache License 2.0
I was wondering if it would be desirable for fluxcloud to expose a HTTP endpoint which would return a list of the events it's received (limited to some user configurable value for memory reasons) that could be polled by an external service?
We have a use case where we'd like to receive flux events in a system where we can't accept inbound connections (so webhooks don't work) but we can make outbound connections to other services.
I'm not sure if this even makes sense for fluxcloud or should be its own service that fluxcloud can talk to via webhooks. Maybe even something like a redis/kafka/whatever exporter would work (fluxcloud writes events to redis, we poll redis for new events ๐คทโโ๏ธ ).
Wanted to find out what you think before trying to implement this.
I would like to have the information from helm-operator sent via an exporter as well.
In case there is a failed helm deployment, such an information would help us really much.
Are you thinking about implementing such behaviour?
##flux log
ts=2019-06-04T11:26:46.976670041Z caller=sync.go:470 component=cluster method=Sync cmd=apply args= count=9
ts=2019-06-04T11:26:47.299043655Z caller=sync.go:536 component=cluster method=Sync cmd="kubectl apply -f -" took=322.205024ms err=null output="helmrelease.flux.weave.works/deployment-test-container-crash configured"
ts=2019-06-04T11:26:47.314261782Z caller=daemon.go:627 component=daemon event="Sync: b182e3d, default:helmrelease/deployment-test-container-crash" logupstream=true
ts=2019-06-04T11:26:47.747839441Z caller=loop.go:441 component=sync-loop tag=flux-sync old=ea0fd97fb37a55f3da6ccdac6adc22c8fbf81eb7 new=b182e3dc8f2b9afb1d78a75b008265a1207de239
ts=2019-06-04T11:26:47.923236797Z caller=loop.go:103 component=sync-loop event=refreshed url=###HIDDEN_URL### branch=master HEAD=b182e3dc8f2b9afb1d78a75b008265a1207de239
##fluxcloud registered events
`Request for:/v6/events
{"id":0,"serviceIDs":["default:helmrelease/deployment-test-container-crash"],"type":"sync","startedAt":"2019-06-04T11:26:28.8560952Z","endedAt":"2019-06-04T11:26:28.8560952Z","logLevel":"info","metadata":{"commits":[{"revision":"b182e3dc8f2b9afb1d78a75b008265a1207de239","message":"test failed deployment event"}],"includes":{"other":true}}}
{"channel":"#weave-flux-notif","icon_emoji":":ghost:","username":"Flux Operator","attachments":[{"color":"#4286f4","title":"Flux deployment:","title_link":"###HIDDEN_URL###","text":"Event: Sync: b182e3d, default:helmrelease/deployment-test-container-crash\nCommits:\n\n* \u003c###HIDDEN_URL###|b182e3\u003e: test failed deployment event\n\nResources updated:\n\n* default:helmrelease/deployment-test-container-crash"}]}
{#weave-flux-notif ๐ป Flux Operator [{#4286f4 ###HIDDEN_MSG&URL### Event: Sync: b182e3d, default:helmrelease/deployment-test-container-crash
Commits:
Resources updated:
##helm-operator logs
ts=2019-06-04T11:27:32.528227847Z caller=chartsync.go:300 component=chartsync warning="unable to proceed with release" resource=default:helmrelease/deployment-test-container-crash release=deployment-test-container-crash err="release requires a rollback before it can be upgraded (FAILED)"
Thanks for fluxcloud!
Is it possible to have a new tagged release - looks like the multiple-exporter option is not available in a release (and it's a really cool feature!).
Thanks again.
Hi.
I've deployed both deployment and sidecar.. tried both.
It does not work. I've created a webhook and I see nothing in slack.
don't see any errors from fluxcloud or anything.
just nothing is happening.
tried this on two different clusters.
EKS cluster.
Kubernetes 1.13
flux is working.
I'm using MicrosoftTeams and unless I create a different connector with different names for each of my clusters, I have no way to identify which cluster produced the message.
My suggestion is add the ability to append custom prefix to the title for such use cases. WDYT?
I deployed bitnami's sealed secrets (https://github.com/bitnami-labs/sealed-secrets) into my cluster and now I get the following error in fluxcloud.
Request for:/v6/events
parsing default:customresourcedefinition/sealedsecrets.bitnami.com: invalid service ID
Request for:/v6/events
parsing default:customresourcedefinition/sealedsecrets.bitnami.com: invalid service ID
Request for:/v6/events
parsing default:customresourcedefinition/sealedsecrets.bitnami.com: invalid service ID
Request for:/v6/events
parsing default:customresourcedefinition/sealedsecrets.bitnami.com: invalid service ID
Hello,
I was pointed to this project when I asked if flux
would have alerts for deploys etc at OSS.
So i have a few questions!
Is there any reason why not to implement that direct at flux oss?
This works with helm-op
? We use flux
with helm-op
for everything.
If yes, does it get git url from the release.
Are there other goals? (cause of the name)
Is there an example msg on how it would appears?
Sorry I didn't look at the code yet, so I am asking this questions!
Thanks!
The new default install of flux recommends this line:
# serve /metrics endpoint at different port.
# make sure to set prometheus' annotation to scrape the port value.
- --listen-metrics=:3031
In order to get fluxcloud to start I needed to change their prometheus port, since I didn't see any way to specify a different port in the fluxcloud env vars.
Not sure what the best answer is here, at least mentioning it in the deployment file would be helpful. If you could change to an unused port that would be great.
Hi,
I've installed flux using helm and then fluxcloud using examples/fluxcloud.yaml. Both are on the same namespace:
I added
- --connect=ws://fluxcloud
I can see the two are connected from fluxcloud logs:
Using Slack exporter
Request for:/v10/daemon
client connected!
However I'm not receiving any notifications to Slack. Any ideas how I can debug this further?
First thing, thanks for this module!
We would be interested in receiving notifications only when there is an error during the flux sync (to reduce notification noise - if there is no error, then we can be fairly sure that eventually changes will be applied).
I'm aware of #12 but I don't think we can use it for this purpose.
Would you be interested in supporting such a feature? I make no promises that I'll have the time to provide a PR, but maybe one day if nobody beats me to it...
Getting simple feedback whether or not a commit has been applied by Flux is great for developers. Slack notifications help but can be a bit much. A minimalist option would be to update the status of the given commit.
Many git providers allow external sources to set the status of a commit.
This feature has been suggested to be added to Flux in fluxcd/flux#2899 but is considered a bit to vendor specific to be added to the project.
Is this something that could be added to fluxcloud instead? I haven't looked to much at the codebase but I don't think I would have any problems implementing the feature if it would be accepted.
If I delete an helmrelease object manualy and it's get redeployed automatically,
I won't get any fluxcloud notification.
I would expect a notification anytime a deployment action was done.
Thanks for this module. When trying it, we saw that there are 3 events reported when flux does a deployment. Is there a way to specify for which of these events a message should be send?
I have looked through the code and found that it would probably not be hard to implement as an additional step in the v6 route. Would you accept such PR?
I've been seeing this issue a lot lately:
ts=2019-03-19T18:15:52.144930335Z caller=loop.go:412 component=sync-loop err="executing HTTP request: 400 Bad Request parsing <cluster>:clusterrole/kube2iam: invalid service ID\n"
It causes fluxctl sync
to return Error: timeout
, but no other discernible issues.
See fluxcd/flux#1843
I installed fluxcloud as sidecar and only enable msteam exporter.
For the first time the pod is created, it always failed to send message, the error message is:
"Could not post to MS Teams: context canceled."
However, when I delete the pod and re-created by replicaset, the problem is gone and the message can be sent successfully.
Currently I have created following manifest -
apiVersion: v1
kind: Namespace
metadata:
annotations:
iam.amazonaws.com/permitted: ".*"
labels:
name: rbac-testing
name: rbac-testing
---
apiVersion: v1
kind: Namespace
metadata:
annotations:
iam.amazonaws.com/permitted: ".*"
labels:
name: rbac-testing-ro
name: rbac-testing-ro
---
apiVersion: v1
kind: Namespace
metadata:
annotations:
iam.amazonaws.com/permitted: ".*"
labels:
name: rbac-testing-misc
name: rbac-testing-misc
the problem is, however, that in the Slack group I can only see the latest resource whereas all the other resources were added as well.
Resources updated:
* <cluster>:namespace/rbac-testing-misc
This could be either a feature or a bug, not sure.
Thanks!
I'll submit a PR soon (it's nearly working just need to write tests).
What this effectively means is that we can dump the flux-events via flux-cloud to something like filebeat, and then for example, onto ElasticSearch.
Installed Flux and deployed examples/fluxcloud.yaml as separate deployment. Communication not happened and I didnt see any logs as well.
Logs:
[{#kubernates *}]
Using Slack exporter
I am trying to use it with bitbucket and the path in bitbucket it is commits. Would it be good to take the whole commit link in param and append commmit id or template to replace the commit number?
Hey there, we're interested in using and contributing to fluxcloud a bit but before that we'd really like to know what the license of it looks like. Thank you :)
I love seeing things get deployed, etc. on the slack channel. Would be great if I could get error messages from the flux logs as well, to know when I've committed something wrong.
https://github.com/containrrr/shoutrrr currently supports:
This could offload a lot code to this dependency
It is great to be able to track flux events as they happen, however the really useful side of notifications are those that bring ones attention to when things don't go according to plan.
Positive notifications create a lot of chatter and are quickly being ignored by developer teams.
In most real-life scenarios it is typical to only enable notifications for when exceptions, such as failed deployments, happen.
Hey ! ๐
Describe the bug
To Reproduce
--connect: ws
examples/fluxcloud.yaml
As the connection from Weave Flux to Fluxcloud seems buggy, there is no logs posted anywhere.
Expected behavior
Connection from Weave Flux to Fluxcloud should be working and logs should be posted to the webhook.
Logs
Logs of fluxcloud:
Using Webhook exporter
Request for:/v11/daemon
client connected!
Request for:/
upgrade:websocket: not a websocket handshake: 'upgrade' token not found in 'Connection' header
Request for:/
upgrade:websocket: not a websocket handshake: 'upgrade' token not found in 'Connection' header
Request for:/
upgrade:websocket: not a websocket handshake: 'upgrade' token not found in 'Connection' header
Request for:/
upgrade:websocket: not a websocket handshake: 'upgrade' token not found in 'Connection' header
Logs of Weave Flux:
...
caller=upstream.go:118 component=upstream err="executing websocket ws://fluxcloud/v11/daemon: dial tcp 10.233.55.12:80: connect: connection refused"
...
examples/fluxcloud.yaml
spec:
securityContext:
runAsUser: 999
containers:
- name: fluxcloud
image: justinbarrick/fluxcloud:v0.3.9
imagePullPolicy: Always
ports:
- containerPort: 3032
env:
- name: EXPORTER_TYPE
value: webhook
- name: WEBHOOK_URL
value: http://10.10.2.100:8080/hooks/cbzjhjthxbyzzqmq1bgj1i5f8o
- name: GITHUB_URL
value: http://10.10.2.100:10080/INFRA/projects.git
- name: LISTEN_ADDRESS
value: ":3032"
Additional context
With default config of fluxcloud, and the username env var set to something other than Flux
, I can't see the username being changed in Slack channel. My env config:
- name: SLACK_USERNAME
value: "Cool cluster"
- name: SLACK_URL
value: "https://hooks.slack.com/services/XXXXXXXXX"
- name: SLACK_CHANNEL
value: "#my-channel"
- name: SLACK_ICON_EMOJI
value: ":sync:"
- name: GITHUB_URL
value: "https://github.com/my-repo"
- name: LISTEN_ADDRESS
value: ":3032"
It would be useful to forward events to multiple upstreams, for example, you might want to get notifications on slack, send them to the fluxcloud-webhook-monitor, and Sentry.
"Please see the Weave Flux setup documentation for setting up Flux."
Tha page shows HTTP 404 Page not found.
Hi everyone,
Since we've been working on this project a lot recently and after not receiving response from the maintainer (not even through email) for a long time, we've decided to continue the development in our fork and allow external contributors to submit their PR's.
We're more than happy to review your submissions and continue to grow this project there.
The new repo is https://github.com/topfreegames/fluxcloud
Cheers!
When trying to use fluxcloud notifications with a private bitbucket-based repository, the COMMIT_TEMPLATE does not seem to do anything.
From the kubectl describe pod
command:
Environment:
...
COMMIT_TEMPLATE: {{ .VCSLink }}/commits/{{ .Commit }}
Resulting link in the notification (with placeholder words for the domain and project/repo): http://git.private.com/projects/myproject/repos/my-flux-repo/commit/sha
Edit: not sure if I should create a separate issue for that, but when I change the chart in my flux repo, no notification is sent, though the event can be seen in the fluxcloud logs, is that an intentional behaviour?
Well, duh, but there's one simple difference:
GitHub has /commit/<sha>
BitBucket has /commits/<sha>
.
There's only a single-character between the two URL schemes.
Could URL support please be extended to include BitBucket as well?
Hey, we'd like to add support for an additional exporter (Kafka or Redis, we're not sure yet honestly) to support some different use cases we have at my workplace. We're happy to do the work to add this functionality but I wanted to check if that was a desirable feature for this project before starting to write any code?
Thanks.
We would like to both trigger a webhook and a slack message. Is it currently possible to configure and use two different exporters at the same time?
If not, could we get this in as a feature request?
Build status badge in readme does not show anything.
Would you please consider adding proper support for discord webhooks to fluxcloud?
Although you can configure fluxcloud to use a discord webhook using the /slack emulation trick (eg, https://discordapp.com/api/webhooks/1234567890/abc123efg456/slack) and it works for smaller flux repo changes, fluxcloud will choke sending very large payloads.
Here's the error I get when fluxcloud is sent a giant payload from flux:
Could not post to slack, status: 400 Bad Request
Exporter Slack got an error: Could not post to slack, status: 400
I assume that discord limits webhook payload sizes to a smaller size than slack or something like that. Either way, the /slack endpoint is not a reliable way to receive fluxcloud messages on discord at the moment.
Thanks!
@justinbarrick are you interested in receiving a PR for a Google Chat exporter?
Messages can be formatted as a card or simple text.
Seems like it will work similar to others as all it needs is the right message format and a URL. Creating Google Chat webhooks is relatively simple.
My connection between flux and fluxcloud works fine but I am experiencing something similar to this issue #26
I'm using flux 1.19.0 and fluxcloud 0.3.9
here is the log from fluxcloud:
Request for:/v6/events
{"id":0,"serviceIDs":["\u003ccluster\u003e:namespace/"],"type":"sync","startedAt":"2020-04-15T10:34:18.964351567Z","endedAt":"2020-04-15T10:34:18.964351567Z","logLevel":"info","metadata":{"commits":[{"revision":"7eda317cbeaa70fb1b76f73f977a71725e7c2f21","message":"test namespace"}],"includes":{"other":true},"errors":[{"ID":"\u003ccluster\u003e:namespace/","Path":"namespaces/test.yaml","Error":"running kubectl: error: error validating \"STDIN\": error validating data: ValidationError(Namespace.metadata): unknown field \"names\" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta; if you choose to ignore these errors, turn validation off with --validate=false"}]}}
parsing <cluster>:namespace/: invalid service ID
Request for:/v6/events
parsing <cluster>:namespace/: invalid service ID
- everytime this occurs in the log, message is not sent. It only happens when there is some kind of error on flux side, e.g. yaml resource validation. So for now we are not notified about errors which is not good.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.