<a href="https://github.com/szinn/k8s-homelab/actions/runs/5206160235/jobs/9392423235"

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

(venv) bash-3.2$ ~/Development/public-repos/flux-local//run-in-env.sh flux-local

<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clip

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Couldn't find helm repository? about flux-local HOT 20 CLOSED

szinn commented on August 30, 2024

Couldn't find helm repository?

from flux-local.

Comments (20)

allenporter commented on August 30, 2024 1

@szinn I think #231 fixed this if you want to try running at main / head. Otherwise, I will do a release after a couple more fixes.

from flux-local.

szinn commented on August 30, 2024 1

(venv) bash-3.2$ ~/Development/public-repos/flux-local/script/run-in-env.sh flux-local get cluster --path ./kubernetes/cluster/config
NAME PATH KUSTOMIZATIONS
cluster ./kubernetes/cluster 87

Looks like it worked

from flux-local.

allenporter commented on August 30, 2024

It looks like it is having a problem finding the HelmRepository:

flux-local error:  Unable to find HelmRepository for flux-system-bjw-s-charts/app-template for HelmRelease lidarr-exporter

from flux-local.

allenporter commented on August 30, 2024

$ git clone https://github.com/szinn/k8s-homelab.git
$ cd k8s-homelab/
$ python3 -m venv venv
$ source venv/bin/activate
$ pip3 install flux-local
$ flux-local get cluster --path kubernetes/ 
NAME                            PATH                                              KUSTOMIZATIONS    
flux-system-add-ons             ./kubernetes/apps/flux-system/add-ons             1                 
media-lidarr                    ./kubernetes/apps/media/lidarr                    1                 
media-radarr                    ./kubernetes/apps/media/radarr                    1                 
media-radarr-4k                 ./kubernetes/apps/media/radarr-4k                 1                 
media-sonarr                    ./kubernetes/apps/media/sonarr                    1                 
monitoring-vector               ./kubernetes/apps/monitoring/vector               1                 
networking-external-services    ./kubernetes/apps/networking/external-services    1                 
self-hosted-shlink              ./kubernetes/apps/self-hosted/shlink              1                 
system-intel-device-plugin      ./kubernetes/apps/system/intel-device-plugin      1                 
cluster                         ./kubernetes/cluster                              78

This is showing each of these objects above as a separate cluster. For some reason it can't make the linkage between kubernetes/cluster and those other applications. (e.g. cluster has 78 kustomizations, but i'm assuming all those other ones should be in the same cluster too.

from flux-local.

szinn commented on August 30, 2024

Very interesting - so I should be able to play with it locally as well?

Will tinker a bit this week

from flux-local.

allenporter commented on August 30, 2024

@szinn Yeah you can, its all based on a command line tool in this repo. You can set --log-level=DEBUG to see a spew of output about how its discovering the repository. Basically, what it does is tries to find all the Kustomization objects and link them together.

Which Fluxtomization is supposed to include the Kustomization ./kubernetes/apps/media/lidarr? I'm having trouble finding this.

from flux-local.

szinn commented on August 30, 2024

./kubernetes/cluster/config/cluster.yaml pulls in ./kubernetes/cluster/apps.yaml which loads ./kubernetes/apps which is processed by directory traversal rather than an explicit kustomization.

from flux-local.

allenporter commented on August 30, 2024

OK thanks for confirming that. I suspected that was the case, but wanted to be sure.

As its walking through the paths it seems to have trouble with lidar's directory structure:

DEBUG:flux_local.git_repo:---
DEBUG:flux_local.git_repo:Source path: kubernetes/apps/media/calibre-web/install.yaml
DEBUG:flux_local.git_repo:Path       : kubernetes/apps/media/calibre-web/app
DEBUG:flux_local.git_repo:Parent path: kubernetes/apps/media/calibre-web/app
DEBUG:flux_local.git_repo:Found parent kubernetes/apps/media/calibre-web/app => kubernetes/apps (kubernetes/apps/media/calibre-web/install.yaml)
DEBUG:flux_local.git_repo:---
DEBUG:flux_local.git_repo:Source path: kubernetes/apps/media/immich/install.yaml
DEBUG:flux_local.git_repo:Path       : kubernetes/apps/media/immich/app
DEBUG:flux_local.git_repo:Parent path: kubernetes/apps/media/immich/app
DEBUG:flux_local.git_repo:Found parent kubernetes/apps/media/immich/app => kubernetes/apps (kubernetes/apps/media/immich/install.yaml)
DEBUG:flux_local.git_repo:---
DEBUG:flux_local.git_repo:Source path: kubernetes/apps/media/lidarr/install.yaml
DEBUG:flux_local.git_repo:Path       : kubernetes/apps/media/lidarr
DEBUG:flux_local.git_repo:Parent path: kubernetes/apps/media/lidarr
DEBUG:flux_local.git_repo:No parent for kubernetes/apps/media/lidarr (kubernetes/apps/media/lidarr/install.yaml) (parent_path=kubernetes/apps/media/lidarr)

I'm taking a closer look. I think it may be getting confused because of the explicit install.yaml used here and it happens to work on the others since they aren't in the same directory.

from flux-local.

allenporter commented on August 30, 2024

I think this is getting confused because there are two types of kustomizations in a single directory. A Kustomization in kustomization.yaml and a fluxtomization in install.yaml.

The tool assumes that the directory structure has meaning with respect to representing dependencies. Yikes.

from flux-local.

szinn commented on August 30, 2024

Ah, with my layout (and others that follow / riff on onedr0p's) the directory structure has meaning but not necessarily for dependencies.

./kubernetes/cluster/config/cluster.yaml is the root entity and it brings in other fluxtomizations and kustomization entities and those alternate back and forth.

For the directory you mention with both types, the parent directory will bring in the install.yaml fluxtomization and that will bring in the kustomization in the same directory which loads the app.

from flux-local.

allenporter commented on August 30, 2024

Yes, though I haven't seen this in onedr0p's repo fwiw:

$ cd home-ops
$ flux-local get cluster --path kubernetes/
NAME       PATH                 KUSTOMIZATIONS    
cluster    ./kubernetes/flux    86

from flux-local.

allenporter commented on August 30, 2024

I think it works in his repo because he uses a separate app subdirectory of the fluxtomization directory that contains the kustomization. It seems like flux-local fails when they're in the same directory. I can see if its possible to adjust, but in the mean time moving to an app subdir can work as a workaround.

from flux-local.

szinn commented on August 30, 2024

I think Devin has it in apps/monitoring/vector

from flux-local.

szinn commented on August 30, 2024

How do I run flux-local from a clone of the repo?

from flux-local.

szinn commented on August 30, 2024

Ah yes, I see what he has - I didn't think the extra sub-directory to hold the two packages was necessary :-) Same pattern for lidarr

from flux-local.

allenporter commented on August 30, 2024

I have a test repo where i have been able to reproduce this and can see if its straight forward to fix:
#231

from flux-local.

allenporter commented on August 30, 2024

Great, this has been released in 1.3.1
https://github.com/allenporter/flux-local/releases/tag/1.3.1

Thanks for the report 👍🏼 and let me know if anything else acts up.

from flux-local.

chrisbsmith commented on August 30, 2024

Sorry to revisit an old issue, but I'm having a similar issue with my PR flux diff checks.

Run docker://ghcr.io/allenporter/flux-local:main
/usr/bin/docker run --name ghcrioallenporterfluxlocalmain_ed81d7 --label 6ed130 --workdir /github/workspace --rm -e "INPUT_ARGS" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/homelab/homelab":"/github/workspace" ghcr.io/allenporter/flux-local:main diff helmrelease --unified 6 --path /github/workspace/pull/kubernetes/apps --path-orig /github/workspace/default/kubernetes/apps --strip-attrs "helm.sh/chart,checksum/config,app.kubernetes.io/version,chart" --limit-bytes 10000 --all-namespaces --sources "home-kubernetes" --output-file diff.patch
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
flux-local error:  Unable to find HelmRepository for flux-system-prometheus-community/kube-prometheus-stack for HelmRelease kube-prometheus-stack

I've run your local debugging testing and see results that I believe look correct

/ # flux-local get cluster --path /homelab/kubernetes/
PATH          KUSTOMIZATIONS
kubernetes    38

/ # flux-local get helmreleases --path /homelab/kubernetes --all-namespaces
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
NAMESPACE       NAME                       REVISION    CHART                                                SOURCE
services        atuin                      2.4.0       services-app-template                                bjw-s
auth            authelia                   2.4.0       auth-app-template                                    bjw-s
cert-manager    cert-manager               v1.13.3     cert-manager-cert-manager                            jetstack
kube-system     cilium                     1.14.5      kube-system-cilium                                   cilium
networking      cloudflared                2.4.0       networking-app-template                              bjw-s
database        cloudnative-pg             0.20.0      database-cloudnative-pg                              cloudnative-pg
kube-system     coredns                    1.29.0      kube-system-coredns                                  coredns
networking      echo-server                2.4.0       networking-app-template                              bjw-s
networking      external-dns-cloudflare    1.13.1      networking-external-dns                              external-dns
networking      external-dns-pihole        1.13.1      networking-external-dns                              external-dns
kube-system     external-secrets           0.9.11      kube-system-external-secrets                         external-secrets
kube-system     onepassword-connect        2.4.0       kube-system-app-template                             bjw-s
monitoring      gatus                      2.4.0       monitoring-app-template                              bjw-s
monitoring      grafana                    7.0.19      monitoring-grafana                                   grafana
monitoring      kube-prometheus-stack      55.5.1      monitoring-kube-prometheus-stack                     prometheus-community
monitoring      kubernetes-dashboard       6.0.8       monitoring-kubernetes-dashboard                      kubernetes-dashboard
auth            lldap                      2.4.0       auth-app-template                                    bjw-s
kube-system     local-path-provisioner     None        kube-system-./deploy/chart/local-path-provisioner    local-path-provisioner
kube-system     metrics-server             3.11.0      kube-system-metrics-server                           metrics-server
storage         minio                      2.4.0       storage-app-template                                 bjw-s
networking      nginx-external             4.9.0       networking-ingress-nginx                             ingress-nginx
networking      nginx-internal             4.9.0       networking-ingress-nginx                             ingress-nginx
database        redis                      18.6.2      database-redis                                       bitnami
kube-system     reloader                   1.0.60      kube-system-reloader                                 stakater
default         smtp-relay                 2.4.0       default-app-template                                 bjw-s
kube-system     snapshot-controller        2.0.4       kube-system-snapshot-controller                      piraeus
monitoring      snmp-exporter              2.4.0       monitoring-app-template                              bjw-s
kube-system     synology-csi               0.9.7       kube-system-synology-csi                             synology-csi
monitoring      unpoller                   2.4.0       monitoring-app-template                              bjw-s
volsync         volsync                    0.8.0       volsync-volsync                                      backube
flux-system     weave-gitops               4.0.36      flux-system-weave-gitops                             weave-gitops

The flux-local get helmreleases even shows the kube-prometheus-stack helmrelease that the PR check is reporting missing.

FWIW, I pulled the latest main docker image and ran this test there.

What is maybe interesting is that I get a missing HelmRepository that seems to relate to the changing package in the PR. In another one of my PRs it was a missing redis HelmRepository

Any ideas?

from flux-local.

allenporter commented on August 30, 2024

Would you be up for filing a new separate issue and we can continue discussing there?

One thought I have is that the root cause is #483 but we can dig in more and see.

One thing we could do is have you enable debug: true when running the actions and we can see if something fishy is happening in the intermediate steps that drops the HelmRepository

from flux-local.

chrisbsmith commented on August 30, 2024

Sure thing!

I'll go ahead and add the debug flag now and trigger the build again.

from flux-local.

Couldn't find helm repository? about flux-local HOT 20 CLOSED

Comments (20)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent