thiagodp / concordialang Goto Github PK
View Code? Open in Web Editor NEW✅ Generate functional tests automatically from your Agile specification
Home Page: https://concordialang.org
License: Other
✅ Generate functional tests automatically from your Agile specification
Home Page: https://concordialang.org
License: Other
MacacaJS supports a wide range of platforms including native apps, web apps, and desktop apps. It has implementations for NodeJS, Java and Python.
Example:
const wd = require('macaca-wd');
describe('my site test', function() {
var driver = wd.promiseChainRemote( { host: 'localhost', port: 3456 } );
driver.configureHttp( { timeout: 600 * 1000 } );
it('has title', function() {
return driver
.get('https://www.mysite.com')
.sleep( 2 * 1000 )
.title()
.then( title => {
title.eql( 'My Site' )
});
} );
it('is searchable', function() {
return driver
.get('https://www.mysite.com')
.waitForElementById('search')
.sendKeys('Soda')
.elementById('seachButton')
.click()
.waitForElementById('searchResults')
.source()
.then(html => {
html.should.containEql('Soda');
});
} );
} );
Currently, when a constraint of a UI Element does not specify Otherwise
steps, a Test Case that explores that constraint (with an invalid input value) receives a tag @fail
, since the Test Case should not pass the tests. After executing the corresponding test scripts, Concordia evaluates expected results and considers expected failures as successful results - labelled as adjusted
instead of passed
in test reports:
An alternative would be using NLP to negate postconditions (Then
steps) and, therefore, to invert their expectations. In this case, the tag @fail
would be no longer needed and test reports would not needed to analyze results before presenting them.
...
Variant: To add an item updates the total
...
When I fill {Product Code}
and fill {Quantity}
and I click on {Add Item}
Then I do not see {Total} with "0,00"
...
UI Element: Quantity
- data type is integer
- minimum value is 1
Concordia will produce a test case in which Quantity
will receive the value 0
, in order to explore the constraint minimum value is 1
. Thus, the postcondition Then I do not see {Total} with "0,00"
should fail, since the input value is considered incorrect. However, Quantity
does not declare an Otherwise
sentence for the corresponding constraint, aiming to describe the expected behavior. Concordia will then simply add a flag @fail
to the Test Case to indicate the expectation of failure.
Negating the original post-conditions may give the same effect, without having to analyze test cases' expectations:
Then I do not see {Total} with "0,00"
would become
Then I see {Total} with "0,00"
That is, when using an invalid/icorrect input value for Quantity - such as 0
-, the feature should not be able to produce the original post-conditions. So negate them is a way to assert that.
Natural Language Processing is removing diacritics because of the Bravey library. Changes on this library may be necessary as a workaround.
Then I see that {Age} is {Age|minvalue}
The |
operator separates a UI Element from one of its properties.
Whether the property id
is #age
and the property minvalue
is 21
, the above sentence will be converted to:
Then I see that <#age> is 21
Allow the following properties:
id
type
datatype
value
🠖 Implemented in v1.2.0minlength
maxlength
minvalue
maxvalue
format
required
Example:
Then I see {Age|value}
Allow properties to be written in the document's language. Examples:
Then I see {Age|minimum value} # english
Then I see {Age|valor mínimo} # portuguese
Allow to reference UI Elements from other Features involved in a Variant. Example:
# login.feature
Feature: Login
Scenario: Successful login
Variant: Login by entering credentials
Given that I am on the [Login Screen]
When I type {Username}
and I type {Password}
and I click on {OK}
Then I have ~user logged in~
...
# logout.feature
import "login.feature"
Feature: Logout
Scenario: Successful logout
Variant: Logout by selecting the menu bar option
Given that I have ~user logged in~
and I see {Login:Username|value}
When I click on {Logout}
Then I have ~user logged out~
...
When the sentence and I see {Login:Username|value}
is converted to a Test Case sentence, it will receive the corresponding value of Login's Username.
Plugin for CodeceptJS is currently ignoring codecept.json
. Whether the file exist, it must be considered. Otherwise, it must be generated.
Some valid imports are being reported in the console as invalid imports. Although, everything is working as expected.
Currently, UI literals do not support escaped CSS selectors. For example:
Then I see <div \> a>
to locate div > a
.
These cases are currently supported as id
s of UI Elements. For example:
Then I see {Foo}
UI Element: Foo
- id is "div > a"
It is not generating the Test Case tag @fail
when two or more UI Elements receive invalid values in a Test Case, but one of them does not have Otherwise
steps defined for the affected property.
This problem could be circumvented by using --comb-invalid=1
. After the fix, this parameter is no longer needed.
To add a parameter --init
that guides the user to configure some parameters of the Concordia Compiler, such as:
./features
)test
)output
)The configuration must be stored in the current configuration file, .concordiarc
.
👉 Please read Issue #35 before continuing.
Then I see "Welcome, {Username|value}"
Whether {Username|value}
is bob
, the corresponding Test Case must be transformed into
Then I see "Welcome, bob"
id
type
datatype
value
🠖 Implemented in v1.3.0minlength
maxlength
minvalue
maxvalue
format
required
When a Test Case
has the tag @fail
, it indicates that the test script will fail, but the failure is expected as a normal behavior. In this case, any plugin must count the FAILURE as a PASS. The current plugin for CodeceptJS must implement that behavior.
Allow Test Events in the CodeceptJS plugin
First version could allow database scripts.
Second version could allow console commands.
Third version could allow file operations.
If you can create any documentation, this project needs your help!
1- Creating tutorials.
2- Creating videos to explain some part of the language or some interesting feature or trick.
3- Adding examples.
4- Improving the current language documentation.
Stuff in English or Portuguese is welcome!
Implement Test Events
First version could allow database scripts.
Second version could allow console commands.
Third version could allow file operations.
Columns and lines of Abstracts Tests Scripts are being produced incorrectly. Thus, test scripts keep a wrong reference to them.
Concordia Compiler notifies when a new version is available (since May 9th, 2018, see #1).
It would be useful for users to display an additional information when an update has a breaking change. A breaking change occurs when the MAJOR version changes. For example, an update from version 0.99.2
to version 1.0.0
is a breaking change, since the major version upgraded (from 0
to 1
).
Proposal
Display a message asking the user to read the release notes before updating. The message should include the URL of the Release Notes page. For example:
PLEASE READ THE RELEASE NOTES BEFORE UPDATING
https://github.com/thiagodp/concordialang/releases
Configuration file .concordiarc
, which is produced when the parameter --save-config
is used, is not being saved in the execution path, but in the path of the Concordia "executable".
Currently, plug-ins are part of the tool.
PROS :
npm
installsCONS :
Thus, to separate them is a clear need.
SOLUTION DETAILS
npm
package.json
files.COMPATIBILITY BREAK :
.concordiarc
, if used (e.g., codeceptjs
→ codeceptjs-webdriverio
)--plugin-list
will list installed plug-ins, instead of available ones.SOME OPTIONS BEING CONSIDERED :
--plugin-install <plugin-name>
could execute npm install --save-dev <plugin-name>
(where <plugin-name>
can be prefixed with concordialang-
if needed) and then run plugin's install
commands.--plugin-uninstall <plugin-name>
could execute npm uninstall --save-dev <plugin-name>
(where <plugin-name>
can be prefixed with concordialang-
if needed) and then run plugin's uninstall
commands.install
and uninstall
may be removed and needed applications installed via package.json
scripts like postinstall
and postuninstall
. Example: "postinstall": "((npm list -g appium) || (npm i -g appium)) && ((npm list -g codeceptjs) || (npm i -g codeceptjs))"
.install
and uninstall
may be removed and needed applications installed as plugin's dependencies and executed with NPX. Concordia is currently compatible with Node 8
or above. NodeJS 8.2.0
comes with NPX. The postinstall
script of package.json
could install NPX whether not installed, i.e., (npm list -g npx) || (npm i -g npx)
. By using NPX, global dependencies will not be needed anymore. However, users shall need to use it to execute any third-party command-line applications adopted by a plug-in, e.g., npx selenium-standalone start
, npx codeceptjs run
, npx appium
. Note: A plug-in is installed by the compiler as a development dependency so that the target application's dependencies are not affected. Every plug-in may declare external applications as its dependencies
. All of them will be removed automatically when a plug-in is uninstalled.install
and uninstall
may be removed and the plugin perform the expected behavior programmatically, instead of using external applications.--init
may try to install the chosen plugin.Auto-detect the language of the current document, when #language
is not declared.
This is related to #15
They stopped working after updating meow
Generated lines of code are referencing a same line of a test case. It is probable that more than one Test Case are sharing a same line in memory.
TestCafe supports desktop, mobile, remote and cloud browsers (UI or headless).
Example:
import { Selector } from 'testcafe'; // first import testcafe selectors
fixture `Getting Started`// declare the fixture
.page `https://devexpress.github.io/testcafe/example`; // specify the start page
//then create a test and place your code there
test('My first test', async t => {
await t
.typeText('#developer-name', 'John Smith')
.click('#submit-button')
// Use the assertion to check if the actual header text is equal to the expected one
.expect(Selector('#article-header').innerText).eql('Thank you, John Smith!');
});
Visual testing helps to verify if:
Currently a lof of tools do visual regression testing. The same applies to CodeceptJS, which now integrates with Resemble.js. So, it's an interesting improvement for the Concordia language to support visual testing comparison.
Here it is a proposal for such support:
Currently, the following sentence is already supported:
I save a screenshot "MyFeature.png"
This sentence makes a testing framework to take a picture of the screen.
So the new feature would be to introduce a sentence like
I see a visual difference considering "MyFeature.png"
Let's see some examples.
I save a screenshot "MyFeature.png"
and I see a visual difference considering "MyFeature.png"
or
I save a screenshot "MyFeature.png"
and I see a visual difference of 20% considering "MyFeature.png"
When no percentage is given, it is assumed 10%
.
I save a screenshot "MyFeature.png"
and I see a visual difference for the element <#someElement> considering "MyFeature.png"
or
I save a screenshot "MyFeature.png"
and I see a visual difference of 20% for the element <#someElement> on "MyFeature.png"
When no percentage is given, it is assumed 10%
.
A plug-in must compare the visual difference between a previously-save image with the current screen or part of the screen. This difference must consider some tolerance value (e.g., misMatchPercentage
in Resemble.js).
The plugin for CodeceptJS with WebDriverIO can support this feature using the Resemble Helper. The is no support yet for CodeceptJS with Appium.
Example 1:
Then I see "P" in the element <foo>
Example 2:
Then I see "P" in <foo>
Example 3:
Then I see "P" inside <foo>
Example 4:
Then I see <foo> with "P"
Currently, Concordia relies on queries to get the current date, time or datetime. Examples:
- value comes from "SELECT CURRENT_DATE()"
- value comes from "SELECT CURRENT_TIME()"
- value comes from "SELECT NOW()"
Formatted values are also accepted:
# dd/mm/yyyy
- value comes from 'SELECT DATE_FORMAT( CURRENT_DATE(), "%d/%m/%Y")"
# hh:mm
- value comes from 'SELECT TIME_FORMAT( CURRENT_TIME(), "%h:%i")'
# dd/mm/yyyy hh:mm:ss
- value comes from 'SELECT DATE_FORMAT( NOW(), "%d/%m/%Y %h:%i:%s")'
Instead, it should support a declaration that produces the value without having to deal with queries.
The proposed constructions should consider the property data type
.
For example, now
for a date
must return the current date, while now
for a time
must return the current time.
now
current date
today
yesterday
tomorrow
next
+ ( week
| month
| year
)next week
last
+ ( week
| month
| year
)last month
day
| days
| week
| weeks
| year
| years
) + ( ago
| ahead
| later
)2 days ago
now
current time
current
+ ( second
| minute
| hour
)second
| seconds
| minute
| minutes
| hour
| hours
) + ( ago
| ahead
| later
)1 second ago
current datetime
Example:
UI Element: Sales Last Year
- data type is integer
- max value is the current year # current year value, like 2019
UI Element: Meeting Day
- data type is integer
- max value is today # current day value, like 25
UI Element: Meeting Date
- data type is date
- min value is today
UI Element: Sales Last Year
- data type is integer
- max value is this year # current year value
UI Element: Birth Date
- data type is date
- max value is 18 years ago # the current date, but for 18 years ago
Otherwise I see "You must be at least 18 years old."
👉 New Issues may address them.
date()
, time()
, and datetime()
Both can receive an optional parameter format
. Its value should default to the UI Element locale's date/time/datetime format, if available in the language (see #30). Otherwise, the locale defined by the document's language.
The parameter format
should use the PHP format for date and time values.
By default, date()
should return day, month and year. Example for the English locale: 12/31/2018
.
By default, time()
should return hour and minute (without second). Example for the Portuguese locale: 13:00
.
By default, datetime()
should return day, month, year, hour, minute and second. Example for the English locale: 12/31/2018 10:20:30
.
When more than one precondition state is declared, the first one is combined correctly but the other ones are not. They are being inserted in the old step index (the index before combination).
Currently, when we add the tag @ignore
to a Variant (for instance), its Test Cases are not generated and the corresponding Test Cases are pruned from the .testcase
file. However, the corresponding test script files (ex. .js
files) are kept untouched.
Thus, since it's annoying to have to delete these test scripts before running the tests, I'm proposing to change the behavior of @ignore
to propagate the change to test script files.
Thanks to Rebeca Costa for suggesting it 👍.
Olá, gostaríamos de saber se tem como colocar um tempo de espera entre passos ou esperar até que um elemento apareça na tela.
Parameter --language
in the CLI is being ignored.
Executing commands in the console/terminal can be useful during Test Events.
Example:
Before Feature:
When I run the command 'mkdir foo'
and I run the command './script.sh'
Since a complete constraint solver is not available, the test case FORMAT_INVALID
should be avoided when value or length constraints are defined, because it is not able to consider them.
A sentence like
When I wait for "/foo" during 3 seconds
is currently not mapped.
e.g., some file.feature
is generating some%20file.testcase
and some%20file.js
Currently we can define values for ui literals and ui elements like this:
When I fill <foo> with "a"
and I fill {bar} with "b"
The following syntax can be also useful:
When I fill <foo> and {bar} with "x"
to make both <foo>
and {bar}
to receive the value x
No additional test cases must be produced when all the UI Elements receive (fixed) values
When Variants declare input steps with UI Elements and fixed values, they have being used to combine test cases - producing more test cases than needed.
👉 Please read Issue #44
UI Element: Name
- minimum length is 2
Otherwise I see "Please inform at least {Name|minimum length} characters."
The value in the Otherwise sentence should be transformed into "Please inform at least 2 characters.".
id
type
datatype
value
🠖 Implemented in v1.3.0minlength
maxlength
minvalue
maxvalue
format
required
locale
The property locale
would assume the document's language by default.
Usage example:
UI Element: Birth Date
- data type is date
- locale is "pt"
Every supported language should define the default formats for:
dtf
NOTE: I'm still not satisfied about using a function. Maybe special characters to denote a different kind of string could fit better.
The function means "date time format".
Currently there is the property format
, which defines a UI Element's format using a regular expression. The proposed function aims to facilitate that definition by adopting a simpler format, like the PHP date and time format.
Examples:
UI Element: Birth Date
- data type is date
- format is dtf( "d/m/Y" )
UI Element: Meeting Hour
- data type is time
- format is dtf( "h:i" )
UI Element: Last Update
- data type is datetime
- format is dtf( "d/m/Y h:i:s" )
With the proposal, format
should assume the locale's format by default. Example:
UI Element: Birth Date
- data type is date
# format is dtf( `m/d/Y` ) since the language is English and so is the locale.
A masked input field usually does not accept invalid input data. Usually such components are well tested and we can assume that they work as intended. Therefore, Concordia does not need to generate invalid input data for them. To disable that generation, a new tag could be defined to be used in UI Element's properties. For example:
UI Element: Year
- data type is integer
@generate-only-valid-values
- format is "/^[0-9]{1-3}$/"
Otherwise I must see "Year must be a number."
In the above example, a tag (@generate-only-valid-values
) is used to denote that the property that follows it (format
) should never be used to generate invalid input values.
Candidate names for the tag:
@generate-only-valid-values
@generate-no-invalid-values
@valid-values-only
@no-invalid-values
Any suggestions?
Display the current version, the newer version, and the command to install it.
Currently, the term field
(or similar) should be added to sentences, in order to make Concordia aware that it is an input field - see #37.
For example,
Given that I see the field <#name> with "Bob"
will generate
I.seeInField('#name', 'Bob');
For a sentence that refers to a UI Element that has an input-like type, it would be nice if Concordia could complete the sentence with field
automatically. For example:
Variant: ...
...
Given that I see {Name} with "Bob" # sentence without `field`
...
UI Element: Name
- id is '#name'
# - type is textbox
It could produce:
Test Case: ...
...
Given that I see the field <#name> with "Bob" # sentence with `field`
Create a plug-in for CodeceptJS with Appium.
Cypress is a promising opensource testing framework for Chrome and cross browser support is in the roadmap.
Example:
describe('My First Test', function() {
it('Gets, types and asserts', function() {
cy.visit('https://example.cypress.io')
cy.contains('type').click()
// Should be on a new URL which includes '/commands/actions'
cy.url().should('include', '/commands/actions')
// Get an input, type into it and verify that the value has been updated
cy.get('.action-email')
.type('[email protected]')
.should('have.value', '[email protected]')
})
})
Sometimes a report generated with CodeceptJS and Mocha Multi Report lacks some properties, such as stats
. The plugin for CodeceptJS must be able to handle this problem appropriately.
This Issue is centralizing the desire to have new plug-ins for different testing frameworks or programming languages.
Issues #28 , #32, and #33 described plugins for MacacaJS, Cypress, and TestCafe. These are desirable platforms for JavaScript.
However, if you would like to contribute with a plug-in for any other programming language (say PHP, Python, Java, Ruby, Delphi, C#, Go, whatever), that would be awesome!
A plug-in must be coded in JavaScript or any language that "transpiles" to JavaScript, such as TypeScript (recommended), CoffeeScript, and Dart.
A plugin can generate source code for any programming language or testing framework able to perform functional tests.
Other suggestions? Please comment.
Example:
Given that I am in the [Lista de tarefas]
When I fill <#inputtask2> with "test"
And I click on <#btnAdicionar>
Then I see "test"
Then I click on <#btnEdit>
Then I see <#btnCancelarEdicao>
And I see <#inputtask2> with "test"
Console output:
expected element #inputtask2 to include "test"
+ expected - actual
+test
Scenario Steps:
- I.see("test", "#inputtask2") at Test.Scenario (src\__tests__\e2e\tests\task.js:32:7)
- I.seeElement("#btnCancelarEdicao") at Test.Scenario (src\__tests__\e2e\tests\task.js:31:7)
- I.click("#btnEdit") at Test.Scenario (src\__tests__\e2e\tests\task.js:30:7)
- I.see("test") at Test.Scenario (src\__tests__\e2e\tests\task.js:29:7)
- I.click("#btnAdicionar") at Test.Scenario (src\__tests__\e2e\tests\task.js:28:7)
- I.fillField("#inputtask2", "test") at Test.Scenario (src\__tests__\e2e\tests\task.js:27:7)
- I.amOnPage("http://localhost:4200/") at Test.Scenario (src\__tests__\e2e\tests\task.js:26:7)
When a test executes, it produces an output file, e.g., output.json
, junit.xml
. Every plug-in should now return the name of the file produced by default, in order to allow the parameter --just-result
reading it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.