Similar to what grafana does here:
https://github.com/grafana/helm-charts/tree/main/charts/grafana#how-to-serve-grafana-with-a-path-prefix-grafana

mb I miss something, but right now we forced to use configmap provided by helm chart itself to store data we need to work with oauth. MB better to at least allow to provide it from secret outside of helm chart or by file specified in container, so we could mount them there.

To provide security of load balancer add loadBalancerSourceRanges into service

Prometheus operator doesn't use annotations, but uses ServiceMonitors/PodMonitors instead. It would be nice to have first class support for it.

Though I agree that this should be a conscious choice by the user, some enterprise environments do have strict policies on setting resource limits for all deployed services in areas where sensitive processes may be happening.

I noticed in my own use, 100Mi was too small and resulted in the container throwing an OOM error.

Some documentation on minimum memory requirements and CPU recommendations for specific loads should be linked here.

An open question I have is how many services/pods can the collector handle at

limits:
  cpu: 100m
  memory: 128Mi

if I pass helm --namespace pyroscope it generates proper template with pyroscope namespcae, the only thing is that it does not create the namespace itself.
And I get Error: INSTALLATION FAILED: create: failed to create: namespaces "pyroscope" not found

Should we create namespace separately?

Currently, liveness and readiness probes are configured with initialDelaySeconds set to 30s which is fairly high value. However, in case of the container crash, Pyroscope server may need even longer time to recover the storage (it is hard to estimate the procedure duration, but a minute or two is what we may expect).

A proper solution would be to separate implementations of the readiness and liveness checks:

• liveness probe starts serving requests in the very beginning of the server initialisation (before any other component)
• readiness probe starts serving requests only when all the components finished the initialisation

Increasing initialDelaySeconds further by default for readiness probe might be undesirable because it will introduce noticeable unconditional delay between the server start and the moment when it actually starts serving requests.

As a workaround, I think we may adjust the default settings so that the pod has at least 90s to finish initialisation, but does not prevent server from handling requests if it managed to complete initialisation sooner:

readinessProbe:
  enabled: true
  httpGet:
    path: /healthz
    port: 4040
  initialDelaySeconds: 5
  periodSeconds: 10
  timeoutSeconds: 30
  failureThreshold: 10
  successThreshold: 1

# Despite the fact that the initial delay is 60 seconds, if the pod crashes
# after initialisation (this is the only realistic reason why the probe may fail),
# it will be restarted.
#
# Note that livenessProbe does not wait for readinessProbe to succeed. 
livenessProbe:
  enabled: true
  httpGet:
    path: /healthz
    port: 4040
  initialDelaySeconds: 60
  periodSeconds: 10
  timeoutSeconds: 30
  failureThreshold: 3
  successThreshold: 1

The current config:

readinessProbe:
  # -- Enable Pyroscope server readiness
  enabled: true
  httpGet:
    # -- Pyroscope server readiness check path
    path: /healthz
    # -- Pyroscope server readiness check port
    port: 4040
  # -- Pyroscope server readiness initial delay in seconds
  initialDelaySeconds: 30
  # -- Pyroscope server readiness check frequency in seconds
  periodSeconds: 5
  # -- Pyroscope server readiness check request timeout
  timeoutSeconds: 30
  # -- Pyroscope server readiness check failure threshold count
  failureThreshold: 3
  # -- Pyroscope server readiness check success threshold count
  successThreshold: 1

livenessProbe:
  # -- Enable Pyroscope server liveness
  enabled: true
  httpGet:
    # -- Pyroscope server liveness check path
    path: /healthz
    # -- Pyroscope server liveness check port
    port: 4040
  # -- Pyroscope server liveness check intial delay in seconds
  initialDelaySeconds: 30
  # -- Pyroscope server liveness check frequency in seconds
  periodSeconds: 15
  # -- Pyroscope server liveness check request timeout
  timeoutSeconds: 30
  # -- Pyroscope server liveness check failure threshold
  failureThreshold: 3
  # -- Pyroscope server liveness check success threshold
  successThreshold: 1

I have been trying to deploy pyroscope with persistence but as the container is running as non-root (101), its not able to create folder in the mounted volume. This even with the securityContext settings.

persistence:
  enabled: true
  size: 16Gi
  storageClassName: ssd
  accessModes:
    - ReadWriteOnce
  finalizers:
    - kubernetes.io/pvc-protection

I have managed to resolved the issue by adding the below initContainer to the deployment

      initContainers:
      - name: volume-mount-fix
        command:
        - sh
        - -c
        - chmod 777 /var/lib/pyroscope && chown -R 101:101 /var/lib/pyroscope
        image: busybox
        volumeMounts:
        - name: storage
          mountPath: /var/lib/pyroscope

Hope this can be added in the official chart or at least the option of customizing the initContainers section.

Step to reproduce

1. Install Pyroscope using Helm

helm repo add pyroscope-io https://pyroscope-io.github.io/helm-chart
helm pull pyroscope-io/pyroscope 
tar -zxvf pyroscope-*

2. Edit values.yaml file
   Set persistent.enabled to true

Problem

$ kubectl get po -n monitoring
NAME                              READY   STATUS             RESTARTS   AGE
pyro-pyroscope-5f76c87778-zhmx6   0/1     CrashLoopBackOff   6          7m42s
$ kubectl logs -n monitoring pyro-pyroscope-5f76c87778-zhmx6
Error: could not initialize server: new storage: mkdir /var/lib/pyroscope/main: permission denied

I think this is because pyroscope pod run as non-root ?

There is room for improvement in the release pipeline at this point:

• Eliminate manual version updates (Chart.yaml, docs, badges, etc).
• Trigger release pipeline on tags.

Would be nice to have renovate running against this repository, so that it updates the image automatically.

There are 2 levels for this implementation in my mind

• Open up the PR, but we still merge manually
• Open the PR and merge automatically

During helm upgrades of application, pyroscope stops sending data

After restart of the application , the data starts to flow in pyroscope server

the version is latest

Any ideas on how we can fix this issue

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: None
Error type: An existing branch is blocking Renovate
Message: Renovate needs to create the branch "renovate/pyroscope-pyroscope-0.x" but is blocked from doing so because of an existing branch called "renovate". Please remove it so that Renovate can proceed.