Sidecar connection issues with placement service with on-disk raft logs about dapr HOT 5 OPEN

/assign

from dapr.

I am preparing to use dapr_placement.cluster.forceInMemoryLog=true in a production environment. I haven't enabled high availability (HA) before. This makes me very concerned.

from dapr.

I updated the title and description to reflect that we actually use on-disk raft logs. I saw the dapr flag default being in memory so assumed that's what we were using (see here); however, we actually use the helm value default which sets placement to on-disk for raft logs. So, the issue in my case is specific to on-disk raft logs.

ping @Cherrs - just wanted to let you know the update :)

from dapr.

I updated the title and description to reflect that we actually use on-disk raft logs. I saw the dapr flag default being in memory so assumed that's what we were using (see here); however, we actually use the helm value default which sets placement to on-disk for raft logs. So, the issue in my case is specific to on-disk raft logs.

ping @Cherrs - just wanted to let you know the update :)

Isn't this Helm default (forceInMemoryLog=false) only actually used when the HA value is also set to true? Check the statefulset manifest

from dapr.

Yeah, that's correct, we can only have on disk log when we have a placement cluster:

{{- if or (eq .Values.global.ha.enabled true) (eq .Values.ha true) }}
        - "--id"
        - "$(PLACEMENT_ID)"
        - "--initial-cluster"
        - {{ template "dapr_placement.initialcluster" . }}
  {{- if eq .Values.cluster.forceInMemoryLog false }}
        - "--raft-logstore-path"
    {{- if eq .Values.global.daprControlPlaneOs "windows" }}
        - "{{ .Values.cluster.logStoreWinPath }}\\cluster-v2-$(PLACEMENT_ID)"
    {{- else }}
        - "{{ .Values.cluster.logStorePath }}/cluster-v2-$(PLACEMENT_ID)"
    {{- end }}
  {{- end }}
{{- end }}

from dapr.