jackcoded / newman-reporter-slackmsg Goto Github PK

View Code? Open in Web Editor NEW

15.0 2.0 25.0 858 KB

Custom Newman reporter for slack

JavaScript 100.00%

newman-reporter-slackmsg newman slack slack-bots webhook-url

newman-reporter-slackmsg's People

Contributors

Stargazers

Watchers

newman-reporter-slackmsg's Issues

Overide default webhook channel

The Slack webhook provides the ability to override the default channel. We are setting up newman to run a dynamic set of folders based on the service running the tests and we want to alert different team channels with the results. It would be great to be able to override the default.

https://getweave.slack.com/services/B01C28M8EKA?added=1

Channel Override
Incoming webhooks have a default channel, but it can be overridden in your JSON payload. A public channel can be specified with "channel": "#other-channel", and a Direct Message with "channel": "@username".

Specifying channel not working

I tried using the new channel option added but can't seem to get it working. Nothing is posting to the channel. The app is installed in our workspace and added to the channel I'm trying to post to. This is the command that I am running:

newman run https://api.getpostman.com/collections/10839446-8485b6f7-f984-4889-bc27-cc60eb218c0b\?apikey=<postman-apikey> --environment https://api.getpostman.com/environments/10839446-3cf10fb8-b07b-4703-ad19-1a07069aaa45\?apikey=<postman-apikey> --folder <foldername> --suppress-exit-code -r slackmsg --reporter-slackmsg-webhookurl https://slack.com/api/chat.postMessage --reporter-slackmsg-token <oAuth token> --reporter-slackmsg-channel #postman-workflow-notifications

I've tried variations on the channel argument - with and without the # sign, wrapping in single quotes and double quotes. Nothing is appearing in slack.

Here are the permissions given to the slackbot:

[Question] Is it possible to add reporting url?

Hi @jackcoded

Is it possible to add to add below information to the slack channel when newman test completed?

build URL/reporting URL (if we integrate with htmlextra)

Thanks

creating PR #14

meta: License on the project

Thanks @jackcoded for creating this reporter. It's been super helpful for us at @moov-io to include Newman test results in our workflows.

I was wondering what the license is on this code? It doesn't seem to be apparent to us.

Edit: Does the ISC declaration cover the source code and released artifacts?

Error in sending message to slack Error: Request failed with status code 400

When I sending this command:
newman run /Users/young/Desktop/TestMock.postman_collection.json --suppress-exit-code -r slackmsg --reporter-slackmsg-webhookurl 'https://hooks.slack.com/services/xxxxxxxx/B01HU2PQKCK/ayUEz4pumSbXuilJfdsfdsg2z1fX'
The error message will show:
Error in sending message to slack Error: Request failed with status code 400

telegram not working

There are 2 issues in telegram, the first is that it's no longer sending, and the next one is that when your chat_id is negative, most of the newly created group is a negative value.

error: unknown option '-6250XXXXX'

Improve error debugging with Slack

I am having a problem with connecting to Slack. The same issue as #7

I would like to be able to debug this, but there's nothing more that I can see other than there was a 400. It would be great if I got any messages from Slack and any information from the reporter on how to fix that.

Question: Is it possible to use with Github Actions?

Hi,
I have a Github Actions workflow using a newman action. I would like to use this reporter to provide automated feedback. I realise I need to change the reporters to include slackmsg, but am not sure how to include the relevant options for token, channel, etc. Any help is very much appreciated!

      - name: Run API Tests
        id: run-newman
        uses: anthonyvscode/newman-action@v1
        with:
          delayRequest: 100
          collection: ./test/newman/postman_collection.json
          environment: ./test/newman/postman_environment.json
          reporters: cli
          folder: specifc tests

Request failed with status code 400

@jackcoded following on from #7 (comment) I modified slackUtils to print out the payload object as follows.

This produced the output below, which when validated shows that the only validation errors are trailing commas.

{
  method: 'POST',
  url: '<SNIP>',
  headers: { 'content-type': 'application/json', Authorization: 'Bearer ' },
  data: '{"channel":"","blocks":[{"type":"divider"},{"type":"section","text":{"type":"mrkdwn","text":"*Test Summary*"}},{"type":"section","fields":[{"type":"mrkdwn","text":"Total Tests:"},{"type":"mrkdwn","text":"116"},{"type":"mrkdwn","text":"Test Passed:"},{"type":"mrkdwn","text":"96"},{"type":"mrkdwn","text":"Test Failed:"},{"type":"mrkdwn","text":"20"},{"type":"mrkdwn","text":"Test Skipped:"},{"type":"mrkdwn","text":"0"},{"type":"mrkdwn","text":"Test Duration:"},{"type":"mrkdwn","text":"1m 35.9s"},],},{"type":"section","fields":[{"type":"mrkdwn","text":"Assertions:"},{"type":"mrkdwn","text":"Total: 397  Failed: 40"},]},{"type":"divider"},],"attachments":[{"mrkdwn_in":["text"],"color":"#FF0000","author_name":"Newman Tests","title":":fire: Failures :fire:","fields":[{"title":"Create RealCard","short":false},{"value":"*`1. AssertionError - Status code is 201`*","short":false},{"value":"• Expected - response to have status code 201 but got 400","short":false,},{"title":"Update RealCard","short":false},{"value":"*`1. AssertionError - Status code is 200`*","short":false},{"value":"• Expected - response to have status code 200 but got 400","short":false,},{"title":"Query RealCard","short":false},{"value":"*`1. AssertionError - Status code is 200`*","short":false},{"value":"• Expected - response to have status code 200 but got 400","short":false,},{"title":"Create RealCard","short":false},{"value":"*`1. AssertionError - Status code is 201`*","short":false},{"value":"• Expected - response to have status code 201 but got 400","short":false,},{"title":"WIP - Update Account - null Id","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 1002","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - The value \\null\\ is not valid. to deeply equal Account \\Id\\ is required.","short":false,},{"title":"Delete Account - Null Id","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 1002","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - The value \\null\\ is not valid. to deeply equal Account \\Id\\ required.","short":false,},{"title":"Query Accounts - offset is -1","short":false},{"value":"*`1. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - PagerInfo \\Offset\\ must be greater than or equal to 0. to deeply equal ","short":false,},{"title":"WIP - Query Accounts - parentAccountId is non matching","short":false},{"value":"*`1. AssertionError - Status code is 404`*","short":false},{"value":"• Expected - response to have status code 404 but got 200","short":false,},{"value":"*`2. TypeError - Error Code Check`*","short":false},{"value":"• Cannot read property 0 of undefined","short":false,},{"value":"*`3. TypeError - Error Description Check`*","short":false},{"value":"• Cannot read property 0 of undefined","short":false,},{"title":"Create Real Card Account - accountId does not match","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 4001 to deeply equal 2003","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - AccountRealCardAccount \\Account. Id\\ is not an Account that exists, is in scope, or is an...","short":false,},{"title":"Enable Real Card Account - Null Id","short":false},{"value":"*`1. AssertionError - Status code is 404`*","short":false},{"value":"• Expected - response to have status code 404 but got 400","short":false,},{"title":"Query Real Card Accounts - offset Value is -1","short":false},{"value":"*`1. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - PagerInfo \\Offset\\ must be greater than or equal to 0. to deeply equal ","short":false,},{"title":"API Bug - Create Real Card - No Required Values Provided","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3004","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - Error converting value {null} to type \\System.Boolean\\. Path \\current\\, line 15, position...","short":false,},{"value":"*`3. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3029","short":false,},{"value":"*`4. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - The RealCard field is required. to deeply equal RealCard \\Name\\ is required.","short":false,},{"value":"*`5. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3008","short":false,},{"value":"*`6. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - Error converting value {null} to type \\System.Int32\\. Path \\validFromYear\\, line 11, posi...","short":false,},{"value":"*`7. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3031","short":false,},{"value":"*`8. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - Error converting value {null} to type \\System.Int32\\. Path \\expiresEndYear\\, line 9, posi...","short":false,},{"value":"*`9. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3013","short":false,},{"value":"*`10. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - Error converting value {null} to type \\System.Int32\\. Path \\validFromMonth\\, line 10, pos...","short":false,},{"title":"WIP - Update Real Card - Id Null","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3025","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - The value \\null\\ is not valid. to deeply equal \\Id\\ is required.","short":false,},{"title":"Update Real Card - Id is non matching","short":false},{"value":"*`1. JSONError - Error Code Check`*","short":false},{"value":"• UnExpected - token Q at 1:1Query returned zero rows for model RealCard.^","short":false,},{"value":"*`2. JSONError - Error Description Check`*","short":false},{"value":"• UnExpected - token Q at 1:1Query returned zero rows for model RealCard.^","short":false,},{"title":"Update Real Card - Card holder name is over 128 characters","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3003","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - The value \\null\\ is not valid. to deeply equal RealCard \\Card Holder. Name\\ must be 128 c...","short":false,},{"title":"Update Real Card - Card holder name is null","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3004","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - The value \\null\\ is not valid. to deeply equal RealCard \\Card Holder. Name\\ is required.","short":false,},{"title":"Update Real Card - Name is over 128 characters","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3028","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - The value \\null\\ is not valid. to deeply equal RealCard \\Name\\ must be 128 characters or ...","short":false,},{"title":"Update Real Card - Name is null","short":false},{"value":"*`1. AssertionError - Error Code Check`*","short":false},{"value":"• Expected - 0000 to deeply equal 3029","short":false,},{"value":"*`2. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - The value \\null\\ is not valid. to deeply equal RealCard \\Name\\ is required.","short":false,},{"title":"Query Real Cards - offset value is -1","short":false},{"value":"*`1. AssertionError - Error Description Check`*","short":false},{"value":"• Expected - PagerInfo \\Offset\\ must be greater than or equal to 0. to deeply equal RealCard \\Name\\ is...","short":false,},{"title":"WIP - Query Real Cards - real card account id value is non matching","short":false},{"value":"*`1. AssertionError - Status code is 404`*","short":false},{"value":"• Expected - response to have status code 404 but got 200","short":false,},],"footer":"Newman Test","footer_icon":"https://platform.slack-edge.com/img/default_application_icon.png",}]       }'
}

Show name of collection and environmenst

Hi there!
I really like this slack-reporter to newman.
So much better than the others.

But there is one thing I'm missing.
And the thing that I'm missing is that you cant see what collection or environment file that's been used.

It would be really nice if you could add this feature.
Maybe place this in top of everything.
Like...

Collection:
Environment:

Test Summary

Totalt Tests
etc
etc

And maybe also add number of assertions to the test summary?

Best regards Niclas

[Feature Request] Limit Failure Messages

First of all, thanks for providing such a great feature! It is really useful.

I have an issue when testing a large collection of tests where they occasionally fail due to network issues / misconfigured servers. This causes a lot of assertion failures, which completely spams slack with the failure messages.

Endpoint 1
1. AssertionError - Status code return 200
• Expected - response to have status code 200 but got 404
2. AssertionError - Status response is JSON
• Expected - response body to be a valid json but got error Unexpected token < at 1:1
3. JSONError - connection error
• UnExpected - token < at 1:1<!DOCTYPE html>^

Could you please implement an optional parameter that would limit failure message lines to any desired number. i.e display only 10 assertion failures.

--reporter-slackmsg-limitFailures '<50>'

This would help us a lot!

Thanks.

Add Pass percentage to Slack results

The pass % would be nice at a glance to see if we are getting better or worse with our tests

Customize "Newman Tests" on messages

Hello,

Is it possible to have an improvement to customize the "Newman Tests" on slack messages ?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.