[BUG]: Azure Functions Deploy task failing repeatedly about azure-pipelines-tasks HOT 6 OPEN

@abagonhishead Thanks for the information on the workaround, I'll take that back to our delivery infrastructure team and see if they can help me with that.

But yes, still hoping for a fix here :(

from azure-pipelines-tasks.

I'm also having the same issue recently. I managed to deploy application by switching from AzureRmWebAppDeployment@4 tasks for (Web App & Functions) to AzureFunctionApp@1 & AzureWebApp@1.
However, it's failing now on AzureAppServiceManage@0 when starting the App Service.
It used to work perfectly, I suspect a regression in these tasks with latest releases.

from azure-pipelines-tasks.

Is there any documentation about what permissions are needed by the Service Connection? At the moment my service connection has a Contributor role at the resource group level, the same resource group under which the function app resides. This is the same as the service connections for my other deployments

from azure-pipelines-tasks.

Also having this issue, but rather than Azure Functions, we're trying to deploy an app service to Azure China. Specifically it's the 'Azure App Service deploy' task v4 that we're using, although I also tried v3 and had the same problem.
We are getting this on both Azure-hosted agents and self-hosted agents.

I am fairly certain this isn't a permissions issue. I tested this with a manually configured ARM service principal service connection, and also set up a new ARM identity federation service connection according to this guide. Both service connections exhibit exactly the same issue with app service deployments, but work fine with everything else -- we have multiple Azure Powershell release tasks using the same service connections, some of them doing very privileged things like deploying container apps, and they are all working fine.
Our two Azure App Service deploy tasks fail completely, however, with the following:

2024-04-23T13:25:40.3283519Z ##[section]Starting: Deploy: set app service to deployed image
2024-04-23T13:25:40.3293014Z ==============================================================================
2024-04-23T13:25:40.3293173Z Task         : Azure App Service deploy
2024-04-23T13:25:40.3293283Z Description  : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
2024-04-23T13:25:40.3293471Z Version      : 4.238.1
2024-04-23T13:25:40.3293562Z Author       : Microsoft Corporation
2024-04-23T13:25:40.3293656Z Help         : https://aka.ms/azureappservicetroubleshooting
2024-04-23T13:25:40.3293786Z ==============================================================================
2024-04-23T13:25:44.0584652Z Got service connection details for Azure App Service:'***'
2024-04-23T13:25:53.2883895Z ##[error]Error: Failed to get resource ID for resource type 'Microsoft.Web/Sites' and resource name '***'. Error: Could not fetch access token for Azure. Status code: endpoints_resolution_error, status message: Error: could not resolve endpoints. Please check network and try again. Detail: ClientAuthError: openid_config_error: Could not retrieve endpoints. Check your authority and verify the .well-known/openid-configuration endpoint returns the required endpoints. Attempted to retrieve endpoints from: https://login.partner.microsoftonline.cn/***/v2.0/.well-known/openid-configuration
2024-04-23T13:25:53.2919412Z ##[section]Finishing: Deploy: set app service to deployed image

The important bit is:
Status code: endpoints_resolution_error, status message: Error: could not resolve endpoints. Please check network and try again. Detail: ClientAuthError: openid_config_error: Could not retrieve endpoints. Check your authority and verify the .well-known/openid-configuration endpoint returns the required endpoints. Attempted to retrieve endpoints from: https://login.partner.microsoftonline.cn/***/v2.0/.well-known/openid-configuration

Based on the error message, and the fact that routing the request through a proxy server first resolves the issue (see my workaround below,) I think it might be related to this issue in azure/msal-node. Maybe there's a transparent proxy server somewhere along the route to China that the library doesn't like?

This is really frustrating and has taken me almost all day to work around. Not only do we now have to maintain a self-hosted agent purely for app service deployments into China, we're also unable to make use of our parallel jobs on our China deployment pipelines.

Any ideas on a fix, please?

EDIT: Not sure if this is important, but we're based in the UK, which may affect the location of Azure-hosted agents that are assigned to our organisation (and therefore the routing to Azure China.)

Workaround

If you're able to set up your own self-hosted agent and use that, then there is a workaround. It worked for us, at least!

• Set up and run a Linux self-hosted agent in the usual way (I haven't tested this with Windows agents)
• On the same machine, run a Squid forward proxy server in a docker container
  • Make sure you bind the listen port (3128) to 127.0.0.1, e.g. ... -p 127.0.0.1:1234:3128 ...
    • This is important! If you don't bind to 127.0.0.1 then Squid may be open to the Internet
• Before starting the agent, set the environment variable VSTS_HTTP_PROXY to point at the Squid container, e.g. $ export VSTS_HTTP_PROXY=http://127.0.0.1:1234/
  • You may want to put this in the .env file in the agent root directory, since it looks like it's sourced when the agent starts

from azure-pipelines-tasks.

It looks like more and more people face this issue as seen at the link below. Still waiting for a response here.
https://developercommunity.visualstudio.com/t/Deploying-to-Azure-China:-Could-not-fetc/10652428?viewtype=all

from azure-pipelines-tasks.

@abagonhishead I think I've found another workaround. I switched the agent from a linux agent to a windows agent. The deployment worked fine on that agent. Did you already try this? It looks like this issue only affects linux agents. Still, only a workaround.

from azure-pipelines-tasks.