fluxcd / multi-tenancy Goto Github PK

Flux v1: Manage a multi-tenant cluster with Flux and Kustomize

License: Apache License 2.0

Shell 24.31% Open Policy Agent 75.69%

gitops multi-tenancy kubernetes kustomize flux flagger open-policy-agent

multi-tenancy's Issues

fluxctl list-images --k8s-fwd-ns=xxx --workload=deployment/xxx returns empty

I have mostly successfully set up a multi tenant flux cluster with currently two namespaces managed by flux.
It basically works in the sense that a new tag in the registry triggers an update in the namespace repository and then in the namespace.

The remaining problem is, that

fluxctl list-images --k8s-fwd-ns=xxx --workload=deployment/xxx

returns nothing.

What might be the cause?

The team flux causes lots of 403 error events in the audit log

Each time a namespaced team flux is running its sync it gets a bunch of 403 Forbidden from the API, cluttering the the audit log with

{
    "kind": "Event",
    "apiVersion": "audit.k8s.io/v1",
    "level": "Metadata",
    "auditID": "20162fc3-bb05-458f-906e-8c3eb60f04a1",
    "stage": "ResponseComplete",
    "requestURI": "/apis/crd.k8s.amazonaws.com/v1alpha1/eniconfigs?labelSelector=fluxcd.io%2Fsync-gc-mark",
    "verb": "list",
    "user": {
        "username": "system:serviceaccount:team1:flux",
        "uid": "9b41e074-5dec-11ea-a627-06ab94fdafa0",
        "groups": [
            "system:serviceaccounts",
            "system:serviceaccounts:team1",
            "system:authenticated"
        ]
    },
    "sourceIPs": [
        "10.41.72.187"
    ],
    "userAgent": "fluxd/v0.0.0 (linux/amd64) kubernetes/$Format",
    "objectRef": {
        "resource": "eniconfigs",
        "apiGroup": "crd.k8s.amazonaws.com",
        "apiVersion": "v1alpha1"
    },
    "responseStatus": {
        "metadata": {},
        "status": "Failure",
        "reason": "Forbidden",
        "code": 403
    },
    "requestReceivedTimestamp": "2020-06-17T13:36:10.116307Z",
    "stageTimestamp": "2020-06-17T13:36:10.116387Z",
    "annotations": {
        "authorization.k8s.io/decision": "forbid",
        "authorization.k8s.io/reason": ""
    }
}

I guess its rooted in the cluster role flux-readonly. Is there anything we can do to improve the situation, or event have flux to not check stuff without having permission?

Flux Support Multi Environments?

Hi All,
Please help me on the below scenario:
I have 1 single AKS cluster, on this cluster I have created 3 different environments (Dev, QA, and Prod), for each environment 1 name spaces with 1 Nginx ingress controllers, Now I want to use GitOps with Flux for the application deployment,

Do I need to install 3 flux environments for dev, QA and Prod?
Do I need to add each environment public SSH key on GitHub? Is there any other best practices we can follow?
Currently I am using 1 single GitHub repository with set of Yaml files, with 3 different branches (QA, Dev and Prod)? Flux will support for 3 branches?

rbac issue on team namespace

I've followed the guide and I'm getting this issue;

ts=2019-10-09T11:14:19.220962472Z caller=main.go:243 version=1.14.2
ts=2019-10-09T11:14:19.220997437Z caller=main.go:372 msg="using in cluster config to connect to the cluster"
ts=2019-10-09T11:14:19.292022192Z caller=main.go:450 err="secrets \"flux-git-deploy\" is forbidden: User \"system:serviceaccount:adam:flux\" cannot patch resource \"secrets\" in API group \"\" in the namespace \"adam\": RBAC: [clusterrole.rbac.authorization.k8s.io \"flux-readonly\" not found, clusterrole.rbac.authorization.k8s.io \"flux-psp\" not found]"

adam is the name of the team. any ideas?
the flux-git-deploy secret get's created but is empty.

Can Flux support multiple versions of the same service in a cluster?

What I would like to do is test multiple versions of a service in the same cluster. I can see how to do at the k8s level. What I keep getting caught up on is how would this work at the git level. I have not seen (might simply be my lack of knowledge) something as simple as getting a service from a branch. Or is this simply multiple giturl settings?

flux not being able to locate flux-git-deploy secret

[takwo@master1 clusterflux]$ kubectl get secrets -n flux-system
NAME                  TYPE                                  DATA   AGE
default-token-52v2g   kubernetes.io/service-account-token   3      13m
flux-git-deploy       Opaque                                0      8m18s
flux-token-24rdc      kubernetes.io/service-account-token   3      13m

I see that flux-git-deploy is created by kustomize and is there when i use kubectl get secrets -n flux-system
however I get the error

ts=2020-01-16T23:05:52.220272492Z caller=main.go:250 version=1.17.0
ts=2020-01-16T23:05:52.220373689Z caller=main.go:389 msg="using in cluster config to connect to the cluster"
ts=2020-01-16T23:05:52.811797455Z caller=main.go:467 err="secrets \" flux-git-deploy\" not found"

Flux dose not create clusterrole and service account in team namespace

Using fluxcd multi-tenancy , in the flux namespace i've cluster role and service account manifests my customization is like so

workloads/controller/kustomization.yaml
resources:

clusterrole.yaml
clusterrolebinding.yaml
deployment.yaml
serviceaccount.yaml

Flux only create the deployment but not clusterrole and sa

.flux.yaml

version: 1
commandUpdated:
generators:
- command: kustomize build .

This is the log

flux" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope; :clusterrole/goldilocks-dashboard: running kubectl: Error from server (Forbidden): error when retrieving current configuration of:\nResource: "rbac.authorization.k8s.io/v1beta1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1beta1,

[Question] Can multi weave flux shares the same memcached services

Hi,

I am implementing the multi-tenancy via weave flux operators. My approach is the same, a cluster repo--> a cluster flux operator, and a namespace -> a namespace flux operator.

However, given that I do not want the flux operator be able to deleted or modified by mistakes by the namespace users, i decided to put all the namespace flux operators in the namesapce flux as below:

$ kubectl get pods -n flux
NAME                          READY   STATUS    RESTARTS   AGE
demo2-flux-5c5f58f547-zvjb5   1/1     Running   0          5m23s
flux-6f6d459df5-jsqld         1/1     Running   0          9h
memcached-7b4c8bd545-5ks9g    1/1     Running   0          2d8h

demo2 is the namespace name, which I use the below to get it working

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  labels:
    name: demo2-flux
  name: demo2-flux
  namespace: demo2
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flux
subjects:
  - kind: ServiceAccount
    name: demo2-flux
    namespace: flux
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  labels:
    name: demo2-flux-secrets
  name: demo2-flux-secrets
  namespace: flux
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  labels:
    name: demo2-flux-secrets
  name: demo2-flux-secrets
  namespace: flux
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flux
subjects:
  - kind: ServiceAccount
    name: demo2-flux
    namespace: flux

Obviously, in this case all the flux operators are sharing with one memcached services (pod).
My question is, what will be some of the considerations for working with one memcached services ?

Kustomize error - SA

Tried deploying multiple teams and getting this error;

caller=images.go:17 component=sync-loop msg="polling for new images for automated workloads"
ts=2019-10-21T12:41:32.048856366Z caller=images.go:23 component=sync-loop error="getting unlocked automated resources: error executing generator command \"kubectl kustomize .\" from file \".flux.yaml\": exit status 1\nerror output:\nError: Multiple matches for name ~G_v1_ServiceAccount|ruben|~P|flux|~S:\n  [~G_v1_ServiceAccount|ruben|~P|flux|~S ~G_v1_ServiceAccount|adam|~P|flux|~S]\n\n\nExamples:\n  # Use the current working directory\n  kubectl kustomize .\n  \n  # Use some shared configuration directory\n  kubectl kustomize /home/configuration/production\n  \n  # Use a URL\n  kubectl kustomize github.com/kubernetes-sigs/kustomize.git/examples/helloWorld?ref=v1.0.6\n\nUsage:\n  kubectl kustomize <dir> [flags] [options]\n\nUse \"kubectl options\" for a list of global command-line options (applies to all commands).\n\nMultiple matches for name ~G_v1_ServiceAccount|ruben|~P|flux|~S:\n  [~G_v1_ServiceAccount|ruben|~P|flux|~S ~G_v1_ServiceAccount|adam|~P|flux|~S]\n\ngenerated output:\nError: Multiple matches for name ~G_v1_ServiceAccount|ruben|~P|flux|~S:\n  [~G_v1_ServiceAccount|ruben|~P|flux|~S ~G_v1_ServiceAccount|adam|~P|flux|~S]\n\n\nExamples:\n  # Use the current working directory\n  kubectl kustomize .\n  \n  # Use some shared configuration directory\n  kubectl kustomize /home/configuration/production\n  \n  # Use a URL\n  kubectl kustomize github.com/kubernetes-sigs/kustomize.git/examples/helloWorld?ref=v1.0.6\n\nUsage:\n  kubectl kustomize

where adam is already a name of a team deployed to the cluster.

Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"d647ddbd755faf07169599a625faf302ffc34458", GitTreeState:"clean", BuildDate:"2019-10-02T23:49:07Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.6-eks-5047ed", GitCommit:"5047edce664593832e9b889e447ac75ab104f527", GitTreeState:"clean", BuildDate:"2019-08-21T22:32:40Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

[Question] Can flux support multi-tenancy within a monorepo?

Can we house multiple namespace specific directories within a monorepo and have a single privileged Flux Operator deploy these applications to their corresponding namespaces?

fluxcd / multi-tenancy Goto Github PK

multi-tenancy's Issues

fluxctl list-images --k8s-fwd-ns=xxx --workload=deployment/xxx returns empty

The team flux causes lots of 403 error events in the audit log

Flux Support Multi Environments?

rbac issue on team namespace

Can Flux support multiple versions of the same service in a cluster?

flux not being able to locate flux-git-deploy secret

Flux dose not create clusterrole and service account in team namespace

[Question] Can multi weave flux shares the same memcached services

Kustomize error - SA

[Question] Can flux support multi-tenancy within a monorepo?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent