fluxcd / flux-recv Goto Github PK

Hey there 👋

We're using Bitbucket Server and we'd like flux-recv to support it. Bitbucket Server (and probably Cloud too) supports push events webhooks (see here).
I could implement the handler myself once I have time but if anyone wants to jump in, feel free.

Hey,

We have flux deployed on all of our clusters and have been looking for a way to get the flux git source synced instantly instead of waiting for the 5m interval auto-sync method.

The issue is we have deployed our Flux deployments based on the Helm Chart, we're trying to figure out how to get this to work, without manually modifying the deployment created by Helm.

I was thinking of 2 solutions and would like to know if they are possible:

Have the flux-recv added as an optional config to the main Flux helm chart

I hope that's on the future plans, but I'm sure it wont be immediate

Deploying the flux-recv as an individual deployment, on the same namespace on the flux deployment, notifying it on changes

This seems to be the faster solution, although we couldn't find a parameter or a way to tell the flux-recv that flux isn't available in localhost:3030 and is available via its service at http://flux:3030

Thanks! Shahar

It seems that when I receive a webhook from github it throws the following error.

{"level":"error","ts":"2022-01-05T07:01:11.885Z","logger":"receiver-server","msg":"unable to validate payload","reconciler kind":"Receiver","name":"github-receiver","namespace":"flux-system","error":"the GitHub signature header is invalid, err: payload signature check failed"}

Add support for git repository synchronization from Azure DevOps webhook:

https://docs.microsoft.com/en-us/azure/devops/service-hooks/services/webhooks?view=azure-devops

Should support the following events with payload set to none or minimal:

• git.push

Happy to implement but wanted to reach out to the flux community first.

Hi I'm wondering if I could create a helm chart to install this into a cluster that already has flux installed (or should I go and update the helm chart in flux to accommodate this?)

This could be supported in 1 of 2 ways:

• Pub/sub integration (listen for GCB events)
  or
• wire up a google cloud function with listens for the pub/sub event and pushes to flux-recv via http (more natural in keeping with the webhook idea)

I was wondering if there is a way currently to disable scheduled syncing in flux and only use webhooks to trigger the sync? If not seems like a good feature to have for doing things like gating the manifest application by waiting for a webhook.

Would be great to have support for harbor webhook format:

https://github.com/goharbor/harbor/blob/master/docs/1.10/working-with-projects/project-configuration/configure-webhooks.md#json-payload-format

Example:

{
  "event_type": "pushImage",
  "events": [
    {
      "project": "prj",
      "repo_name": "repo1",
      "tag": "latest",
      "full_name": "prj/repo1",
      "trigger_time": 158322233213,
      "image_id": "9e2c9d5f44efbb6ee83aecd17a120c513047d289d142ec5738c9f02f9b24ad07",
      "project_type": "Private"
    }
  ]
}

It should be possible to set Authorization header: https://github.com/goharbor/harbor/blob/master/src/jobservice/job/impl/notification/webhook_job.go#L94

GKE ingress configures a load balancer with a health check to the flux-recv port. The requirements for the health check are that a 200 response is returned: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress#health_checks

A Service exposed through an Ingress must respond to health checks from the load balancer. Any container that is the final destination of load-balanced traffic must do one of the following to indicate that it is healthy:

• Serve a response with an HTTP 200 status to GET requests on the / path.
• Configure an HTTP readiness probe. Serve a response with an HTTP 200 status to GET requests on the path specified by the readiness probe. The Service exposed through an Ingress must point to the same container port on which the readiness probe is enabled.

Since the load balancer does not have the webhook secret all response are either 401 or 404s. Can a health check endpoint be added which returns a 200 response.

With DockerHub and Harbor added we already have a pretty broad list of supported registry vendors. Support for Quay.io is however still missing, and they are known for having a pretty aggressive rate limit in place.

Quay.io sends out a repository push payload in the following format:

{
  "name": "repository",
  "repository": "mynamespace/repository",
  "namespace": "mynamespace",
  "docker_url": "quay.io/mynamespace/repository",
  "homepage": "https://quay.io/repository/mynamespace/repository",
  "updated_tags": [
    "latest"
  ]
}

disableNameSuffixHash is on the same indent as generatorOptions

Current:

secretGenerator:
- name: fluxrecv-config
  files:
  - github.key
  - fluxrecv.yaml
generatorOptions:
disableNameSuffixHash: true

Correct:

secretGenerator:
- name: fluxrecv-config
  files:
  - github.key
  - fluxrecv.yaml
generatorOptions:
  disableNameSuffixHash: true

Would like to add support for handling GCR pubsub notifications for image repository updates based on:

https://cloud.google.com/container-registry/docs/configuring-notifications

and:

https://cloud.google.com/pubsub/docs/push

GCR has some pretty tight limits around API requests and some setups where a single GCR registry used by many clusters is hitting the limits pretty hard. Using this webhook receiver, GCR users could disable long polling entirely using --exclude-images and rely on the webhook receiver to update the image cache.

The implementation should support the following payloads:

• add image (new)
• add tag

It should also support authentication token validation from google if configured to do so.

I'm happy to implement this, but wanted to reach out to the community for thoughts before doing so.

My team uses Nexus for our Docker Registry. Support for Nexus Repository Manager 3 webhooks would be great!

It appears that one cannot run flux-recv unless it is a sidecar because it connects to localhost:3030.

I see how to overwrite it. However, this is a bit indirect. It would be better to expose the API as an env variable don't you think?