logicappunit / testingframework Goto Github PK

Unit testing framework for Azure Logic Apps (Standard)

License: MIT License

C# 100.00%

azure logic-apps unit-testing integration-testing logicappunit

testingframework's Introduction

LogicAppUnit Testing Framework

LogicAppUnit is a testing framework that simplifies the creation of automated unit tests for Standard Logic Apps running in a local development environment, or in a build server as part of a DevOps pipeline. Standard Logic Apps do not include an out-of-the-box testing capability and this framework has been designed to fill this gap. The framework is based on the Logic Apps Sample Test Framework that was developed by Henry Liu, and includes additional functionality to make it easier to author and run tests and validate (assert) the results.

The framework does not support the testing of:

Consumption Logic App workflows.
Standard Logic App workflows that have been deployed to Azure.

The testing framework has been designed to make it easier to perform isolated unit testing of a workflow. The framework does this by modifying a copy of the workflow definition to remove the dependencies on external services and APIs, without affecting the functionality or behaviour of the workflow. This means that workflows can be easily tested in a developer's local environment, and by a DevOps pipeline running on a build server, where there is no access to Azure services or any other workflow dependencies.

Key Features

Replace non-HTTP triggers with HTTP triggers to enable automated testing of every workflow, irrespective of the trigger type.
Remove external service dependencies for built-in service provider connectors by replacing these actions with HTTP actions and a mock HTTP server that is managed by the framework.
Remove external service dependencies for managed API connectors by automatically re-configuring managed API connections to use a mock HTTP server that is managed by the framework.
Remove dependencies on invoked workflows and called local functions by replacing the Invoke Workflow and Call Local Function actions with HTTP actions and a mock HTTP server that is managed by the framework.
Remove all retry policies to ensure that tests exercising failure scenarios do not take a long time to execute.
A fluent API to configure request matching and the creation of responses for the mock HTTP server.
Detailed test execution logging to help with workflow test authoring and debugging.
Programmatic access to the workflow run history to enable assertion of workflow run status, response status, action status, input and output messages and more. This includes support for action repetitions inside a loop.
Programmatic access to the requests sent to the mock HTTP server to enable assertion of the data sent from the workflow to external services and APIs.
Override specific local settings for a test case to enable more testing scenarios (e.g. feature flags).

Projects

This code repository includes four projects:

Name	Description
LogicAppUnit	The testing framework.
LogicAppUnit.Samples.LogicApps.Tests	Test project that demonstrates the features of the testing framework.
LogicAppUnit.Samples.LogicApps	Workflows that are tested by the sample test project.
LogicAppUnit.Samples.Functions	Local .NET Framework functions that are called by workflows.

Packages

Download the LogicAppUnit testing framework package from nuget: https://www.nuget.org/packages/LogicAppUnit/

Compatibility

The framework has been tested with these environments:

Windows
Linux (Ubuntu)
MacOS

Give a Star ⭐

If you like or are using this project please give it a star. Thanks.

Main Contributors

Documentation

The best way to understand how the framework works and how to write tests using it is to read the wiki and look at the example tests in the LogicAppUnit.Samples.LogicApps.Tests project.

Future Improvements and Changes

This is a list of possible future improvements and changes for the framework. Please create a new issue if there are other features that you would like to see.

Add more features to the fluent API for request matching and the creation of mock responses.
Add a Verifiable() feature to the fluent API so that a test case can assert that a test execution did send a request to the mock HTTP server that was successfully matched. This would work in a simialar way to the Verifiable() feature in the moq unit testing framework.
Auto-generate C# test cases based on a workflow's run history in the local development environment.

testingframework's People

Contributors

Stargazers

Watchers

Forkers

atthevergeof eashi alexanderdobrescu danielduartemindera xernaj ricky-g jeanpaulsmit nunoviaes jbray1982 anushahn10 sschutten

testingframework's Issues

Mock large message (>1kb) handling in create blob v2 action

Hi,

Has anyone run into an issue with mocking large messages (>1kb) in a create blob v2 action? (macos)

status='Failed', statusCode='InvalidProtocolResponse', error='{"code":"InvalidProtocolResponse","message":"The response to partial content upload initiating request is not valid. The response to initiating partial content upload request must contain a valid location header."}

If I mock the "Location" header then this error appears:

{"code":"InvalidProtocolResponse","message":"The response to partial content upload request is not valid. The location header value returned in the response 'https%3A%2F%2Flocalhost%3A7075%2Fchunk' must be a well formed absolute URI not referencing local host or UNC path."}

It appears that chunking can't be mocked so I've started on something that will disable chunking. Hopefully this is the right path?

CallAzureFunctionEndpoint returns 'No such host is known.'

Hi and thanks for the awesome library.
I am trying to test a workflow that calls a function app:

"Update_Clients": {
  "type": "Function",
  "inputs": {
    "method": "POST",
    "body": {
      "apiPath": "@parameters(endpoint')",
      "entity": "@body('Transform_JSON_To_JSON')"
    },
    "function": {
      "connectionName": "postEntity"
    }
  },
  "runAfter": {
    "Transform_JSON_To_JSON": ["SUCCEEDED"]
  }
},

I am mocking the external url like so:

{
"workflow": {
  "externalApiUrlsToMock": [
    "https://functionappname.azurewebsites.net"
  ]
}

But I am unsuccessful.

[2024-04-16T05:46:47.692Z] Executing 'Functions.WorkflowDispatcher' (Reason='(null)', Id=8fb43413-7e49-412c-bcb8-d51ef980954a)
[2024-04-16T05:46:47.784Z] Outgoing HTTP request ends with server failure: correlationId='8e89da72-994d-40c4-8449-48dc4dae33a5', operationName='CallAzureFunctionEndpoint', httpMethod='POST', hostName='functionappname.azurewebsites.net', targetUri='https://functionappname.azurewebsites.net:443/api/Post?Code=*sanitized*', clientRequestId='32a8029e-41a9-4442-8b6b-660eaa383935', clientSessionId='', clientApplicationId='', apiVersion='', contentLength='-1', serviceRequestId='', durationInMilliseconds='68', httpStatusCode='0', exceptionMessage='System.Net.Sockets.SocketException (11001): No such host is known.
[2024-04-16T05:46:47.784Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
[2024-04-16T05:46:47.785Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
[2024-04-16T05:46:47.785Z]    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
[2024-04-16T05:46:47.785Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)', errorCode='', errorMessage='', referer='<null>', failureCause='', contentType='', contentEncoding='', armServiceRequestId='', organizationId='', activityVector='IN.04', locale='en-US', additionalProperties='{"ParentActivityId":"00-075c8d37f7630662d86b485b2b6443ca-6f4587e233d41965-00"}', targetResourceProvider='', targetResourceType='', extensionVersion='1.62.30.0', siteName='UNDEFINED_SITE_NAME', slotName='', activityId='32a8029e-41a9-4442-8b6b-660eaa383935'.
[2024-04-16T05:46:47.839Z] Executed 'Functions.WorkflowDispatcher' (Succeeded, Id=8fb43413-7e49-412c-bcb8-d51ef980954a, Duration=147ms)

testConfiguration.json should not be empty?

I'm trying the framework for use in our Logic Apps Standards solution and according to the wiki the testConfiguration.json can be empty.
However, while testing I got the "The test configuration could not be loaded" exception, although the file was there and I copied the {} value as mentioned in the wiki.

Copying over the contents from the repo it started to work right away.
The same result when I empty the testConfiguration.json in the example from the repo.

Thanks for building and maintaining this framework!

Conditional mock of "external service"

Service Bus has been "mocked" by a Docker image (in this case Apache ActiveMQ Artemis).

It would be advisable to be able to turn the internal mocking behaviour on and off, so that potential standins (external to the Logic App, but internal from an inward looking perspective) could be used.

Turning internal mocking off would in this case mean that AMQP 1.0 protocol behaviour, when sending Service Bus messages, could be tested.

connection.json file has @appsetting, which is not recognized

I have the following connection.json file which has a managed api connection

{
"managedApiConnections": {

    "products-evgt-apiConnection": {
        "api": {
          "id": "/subscriptions/@appsetting('WORKFLOWS_SUBSCRIPTION_ID')/providers/Microsoft.Web/locations/westeurope/managedApis/azureeventgrid"
        },
        "kind": "V2",
        "authentication": {
          "type": "ManagedServiceIdentity"
        },
        "connection": {
          "id": "/subscriptions/@appsetting('WORKFLOWS_SUBSCRIPTION_ID')/resourceGroups/@appsetting('WORKFLOWS_RESOURCE_GROUP_NAME')/providers/Microsoft.Web/connections/int0313-iver-products-evgt-apiConnection"
        },
        "connectionProperties": {
          "authentication": {
            "audience": "https://management.core.windows.net/",
            "type": "ManagedServiceIdentity"
          }
        },
        "connectionRuntimeUrl": "@appsetting('EVENTGRID_CONNECTION_RUNTIMEURL')"
      }
},

As i understand, the testing framework looks at all built in and managed connections and replaces them with a localhost url. When i use the appsetting in the connectionRuntimeurl of the eventgrid api connection, it tells its an invalid url. I get the following error:
Error Message:
Initialization method INT0313.Iver.Products.Tests.Products.Products.TestInitialize threw exception. System.UriFormatException: Invalid URI: The format of the URI could not be determined..
Stack Trace:
at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind, UriCreationOptions& creationOptions)
at System.Uri..ctor(String uriString)
at LogicAppUnit.InternalHelper.ConnectionHelper.<>c.b__3_0(JProperty connection)
at System.Collections.Generic.List1.ForEach(Action1 action)
at LogicAppUnit.InternalHelper.ConnectionHelper.ReplaceManagedApiConnectionUrlsWithMockServer()
at LogicAppUnit.WorkflowTestBase.Initialize(String logicAppBasePath, String workflowName, String localSettingsFilename)
at LogicAppUnit.WorkflowTestBase.Initialize(String logicAppBasePath, String workflowName)
at INT0313.Iver.Products.Tests.Products.Products.TestInitialize() in C:\Users\PSRAUNAR\OneDrive - Peab\Dokument\INT0313.Iver.Products\INT0313.Iver.Products.Tests\Products\Products.cs:line 21

And the output stops here:
Standard Output Messages:

                         INITIALIZING TEST                               -

Azurite is listening on ports 10000 (Blob service), 10001 (Queue service) and 10002 (Table service).
Workflow 'Products_Source' is Stateful
Replacing workflow trigger 'When_a_file_is_added_or_updated' with a HTTP Request trigger.
Replacing workflow actions using a built-in connector with a HTTP action for the mock test server:
Copy_file_to_error_folder:
Connector Type: /serviceProviders/Sftp/copyFile
Mocked URL: http://pw02cvvv:7075/Copy_file_to_error_folder
Delete_file_from_source_folder:
Connector Type: /serviceProviders/Sftp/deleteFile
Mocked URL: http://pw02cvvv:7075/Delete_file_from_source_folder
Copy_file_to_processed_folder:
Connector Type: /serviceProviders/Sftp/copyFile
Mocked URL: http://pw02cvvv:7075/Copy_file_to_processed_folder
Delete_file:
Connector Type: /serviceProviders/Sftp/deleteFile
Mocked URL: http://pw02cvvv:7075/Delete_file
Upload_blob_to_storage_container-Logging:
Connector Type: /serviceProviders/AzureBlob/uploadBlob
Mocked URL: http://pw02cvvv:7075/Upload_blob_to_storage_container-Logging
Updating connections file for managed API connectors:

when i put a valid dummy url in connectionruntimeurl it works. But this is not what i want. The connections are deployed in the azure devops pipeline and i need this to be an app setting.

Performance of Unittests if you have plenty

Hello,

First and foremost, I'd like to express my gratitude for your work on this framework.

I have a concern regarding the performance of unit tests, particularly when dealing with a large number of them. Presently, I have 90 unit tests, and the runtime for executing them is approximately 30 minutes.

As I intend to expand the test suite with more tests, it's becoming crucial to enhance the performance. One potential approach I'm considering is the ability to run test classes simultaneously.

Currently, I'm utilizing Xunit with the collection feature to execute all tests(one-by-one). However, I'm wondering if there are any additional optimizations I might have overlooked in this regard. Alternatively, are there any plans to implement features aimed at improving test execution performance?

Thank you for your attention to this matter.

Getting an error when authentication type is ManagedServiceIdentity

I'm getting an error:

 "The API connection reference name salesforce-2 has invalid authentication type ManagedServiceIdentity. Only 'Raw' or 'ActiveDirectoryOAuth' authentication type is allowed in local developer environment".

My connections.json file looks like:

  "managedApiConnections": {
    "salesforce-2": {
      "api": {
        "id": "@appsetting('sfdc_api_id')"
      },
      "authentication": {
        "type": "ManagedServiceIdentity"
      },
      "connection": {
        "id": "@appsetting('sfdc_connection_id')"
      },
      "connectionRuntimeUrl": "@appsetting('sfdc_connection_runtime_url')"
    }
  }

I believe in the LogicAppUnit framework the method ReplaceManagedApiConnectionUrlsWithMockServer in the class ConnectionsWrapper.cs should also change the field authentication type to 'Raw' if it says 'ManagedServiceIdentity'. Or is there another way to resolve the error I am seeing?