callawaycloud / scheduledtestrunner Goto Github PK

Some orgs have massive test suites that are VERY slow. If I'd like to deploy without running the full suite I can specify which tests classes to run, but figure out which ones will get me to 75% code coverage for everything in my package is nearly impossible without first running the full test suite.

Perhaps the test runner could store data on which tests classes touch which apex classes and triggers along with the percent of the code they cover. With that data we could expose an endpoint we could use with an sfdx plugin to make a best guess at what test classes to use to deploy a specific piece of code.

Often enough there is a failing test method that isn't urgent to fix and I'll leave it failing for a while. But then I'm getting notifications every day about the test failure which can drown out when another test method fails that is important. It'd be nice to have an easy way to reduce that noise.

Perhaps there could be an application level notification setting to only send for new test failures. Or maybe the ability to specify that a specify test method failure can be ignored. For that route it'd be good to have that skip setting get cleared once the test starts passing again.

When a test run fails we could check to see when the last successful run was, and pull audit logs since that time. This could be really helpful in quickly determining what broke a test

https://andyinthecloud.com/2015/10/18/setup-audit-trail-api-in-winter16/

The current test failure notification just looks like this:

There was
1 unit test failure(s) in the recent test run.

To view all errors, view: https://realself.my.salesforce.com/a3E4V000001EN7a

Would be nice to list out the test method which have failed in a way that you can tell if there are new failures or not.

The easiest way to do this might be to add a summary on the test run where we can aggregate the results. Because we are also tracking "first failure" on the tests, maybe we should roll this up to determine if the test run itself have any "novel" failures. That would allow us to:

• have better messaging EG: "There were X new test failures" . "X tests have been failing for multiple days".
• potentially configure notifications to only send when there are new failures

Information to include:

• test class
• test name
• stack trace (truncated?)