Provider is removing roles from Pulumi and non-Pulumi-managed accounts about pulumi-gcp HOT 5 CLOSED

Observations:

• it always occurs with a service account that targets a single role roles/storage.admin
• it affects pulumi, or non-pulumi service accounts without an identified pattern
• it does appear correlated to my prototyping in a separate cluster but same gcp project
• all service accounts for pulumi are independent of the current production service accounts

From a set of stacks (identity, infrastructure, app) that were up with verified roles, I destroyed each and rechecked roles, still there. Left the identity stack in place.

From existing identity stack, upped identity and app stacks, roles still good.

Continued prototyping more deployment/service/ingress (which had failures) and eventually found that a non-pulumi sa had the role missing again.

Unfortunately I could never narrow this further and cause it to happen directly.

from pulumi-gcp.

Another observation. The following was created with two roles:

export const ciApp = new Identity(
  'ciApp',
  {
    project,
    iamRoles: [
      // for deploying new services/deployments
      'roles/container.developer',
      // for pushing new images to gcr
      'roles/storage.admin',
    ],
  },
  { protect },
)

We found non-pulumi iam had lost roles/storage.admin and found a pulumi managed iam had lost only roles/storage.admin:

So it is something specific to roles/storage.admin or so it seems.

from pulumi-gcp.

Title should change

- Provider is removing roles from non-Pulumi-managed accounts
+ Provider is removing roles from Pulumi and non-Pulumi-managed accounts

from pulumi-gcp.

@jen20 were you able to reproduce this? I don’t believe we’ve seen this again since the initial reports, and I’m not exactly sure how to reproduce or diagnose further.

from pulumi-gcp.

We haven't been able to reproduce this and haven't seen any other reports of this. If it is possible for this to happen, it is almost certainly an issue in the upstream provider, and we're love to try and pinpoint that issue and report/fix it upstream, but have not been able to due to the inability to reproduce. Closing out - but if anyone sees this and can share a reproduction - happy to re-open.

from pulumi-gcp.