home's Introduction

TestCentric Home

This repo is the home base for the TestCentric organization. It contains general info about all the TestCentric projects. We also use it for plans and discussions that go across individual projects and repositories. This document gives a general description of what the TestCentric group is all about.

Test-Centric Development

The name of the group comes from the phrase "Test-Centric Development." I use the term when conducting training or coaching teams to describe an approach where pretty much everything a team does has a test of some kind behind it. The TestCentric group is focused on developing tools and techniques that support that approach to software development.

Overview of Software Projects

Here on GitHub, the TestCentric organization aims to produce tools. Everything at this point is Open Source. There may be commercial versions of the same tools as well as new commercial tools at some time in the future.

Much of the tooling that is planned will leverage my own work on NUnit in the past but will hopefully go beyond what has been done before. The following are the projects now underway or being actively considered.

TestCentric GUI

The initial release of the GUI will look and work very much like the NUnit V2 GUI. It will support NUnit 3 tests and - using the standard NUnit extension - NUnit V2 tests as well. This project is under way and version 1 is currently in alpha.

The separate Experimental GUI project has now been terminated but the code for it has been added to the TestCentric GUI. The TestCentric GUI will be released with both the standard GUI and the experimental version. The experimental GUI is considered pre-alpha software at this point but many or most of its features will be moved into the standard GUI and it is likely to become the primary GUI when we get to version 2.

For version 2, I also envision supporting multiple test frameworks, including...

TestCentric Framework

This is still on the drawing board, so I won't say much about it.

Generally, I'm thinking of carrying forward some of NUnit's syntactic innovations in a slightly expanded form and separating the assertion facility from the test framework itself. I would also like to allow users to choose among alternate sets of features for different kinds of testing - micro-tests versus functional testing, for example.

Many of the ideas that have been germinating around this new framework were nurtured by folks in the Lonely Coaches Sodality group.

TestCentric Extension for Visual Studio

This is a bit more futuristic and will build on the experimental GUI to create a true VS extension (not an adapter) for running tests. This will allow the user to have the same views of tests either within VS or in a standalone GUI.

I have not yet decided whether this will be Open Source, commercial or dual-licensed using two different versions.

TestCentric Test Engine

I'll be evaluating whether the NUnit engine can be used to carry out all the goals of the other projects. If necessary, a separate engine will be created.

home's People

Contributors

Watchers

home's Issues

Test Engine - Definitions

I developed the idea of a TestEngine for NUnit beginning in 2007, when it was first presented to the Mono Gathering in Boston. It evolved over time and an implementation was created for NUnit 3.0. Some parts of the design were never implemented and eventually the team taking over development of the engine decided to go in a slightly different direction.

In this issue, I want to begin to lay out the needs I see for further engine development. This is the first step toward deciding whether to create a new TestCentric engine project or to continue working with the NUnit engine. It seems clear that we have to start with some notion of what is needed from the engine, so that's what I will attempt to do.

There has been some debate about the proper use of certain terms, including the term "engine" itself. For that reason, I'm starting out by defining terminology as I will use it. These definitions are not intended to be "better" than anybody else's definitions in any sense. They are simply descriptions of how I am using the words.

Platforms

A platform is a combination of hardware, operating system and runtime framework on which our software needs to run. Each individual piece of software (runners, engine, framework, etc.) may need to run on a different set of platforms. The platforms we need to support tends to drive the design to a much greater extent than any other factors.

Spaces

When talking about platfors, I think it helps to think about the two "spaces" in which test execution software has to operate. (This particular use of the word is my own, by the way.) Before discussing what platforms fall into each space, let's attempt to define them.

Development Space

As I intend the terms, Development Space is the platform on which a developer writes code, compiles it and launches tests. With NUnit, that space is generally limited to the Windows, Linux and IoS operating systems running on the desktop. (A developer writing a mobile application, for example, does not generally develop on a mobile system.)

Test Execution Space

This space encompasses all the platforms and environments under which the software being developed will eventually run and therefore where the tests must be executed. This space is a superset of the development space, since we may need to run tests on the development platforms as well. I'll try to break this down further in the next section.

Runners

Runner is a great example of a term that has been overloaded many times in many different ways. In this essay, I am using the term to mean a piece of software that interacts with the developer to allow selection of what tests to run and where and how to run them. (This definition, for example, excludes the various internal "XxxxxxxRunner" classes within the NUnit engine.)

By definition, a Runner needs to exist in the Development Space only.

Test Engine

As used here, the term Test Engine, refers to common functionality needed by all or most Runners, extracted into a library so that test discovery and execution may be done in a standard way without duplication of effort.

Like the Runner, a Test Engine does not need to exist outside of Development Space. Any functionality needed in both spaces (Development and Test Execution) needs to be extracted from the engine.

Test Agents

A test Agent is responsible for actually discovering tests and executing them. As such, it needs to run in Test Execution Space. Since that space is a superset of Development Space a version or versions need to exist that will run in Development Space as well.

Test Framework

A Test Framework is able to discover and execute tests written against a specific API on a specific platform. For example, the NUnit 3 framework allows running tests against the NUnit 3 API.

A Test Agent may be designed to deal with a particular framework or may support multiple frameworks through an API.

Conclusions?

I could draw some conclusions here WRT how these definitions might impact the design of a future Test Engine and, in fact, I have some ideas of my own already. But this seems like a good place to pause for comments and to see if anyone else is ready to offer ideas or conclusions. 😄

Create standard labels for all projects

We don't have many issues here but by creating the set of standard labels, we can use the GitHub CLI to clone them into any other projects. The set here should probably be the union of all the labels we use in any of our repos. That way we can just delete anything that's not appropriate for a given project.

Recommend Projects

testcentric / home Goto Github PK