editorconfig / editorconfig-core-test Goto Github PK
View Code? Open in Web Editor NEWTestings for EditorConfig Core
Home Page: http://editorconfig.org
License: BSD 2-Clause "Simplified" License
Testings for EditorConfig Core
Home Page: http://editorconfig.org
License: BSD 2-Clause "Simplified" License
After 3d439da, multiline tests start to fail.
Example:
118/190 Testing: tab_width_default_ML
118/190 Test: tab_width_default_ML
Command: "C:/Program Files (x86)/CMake/bin/cmake.exe" "-D" "EDITORCONFIG_CMD=C:/projects/editorconfig-core-c/bin/x64-static/core/bin/editorconfig" "-D" "ECARGS:LIST=-f;tab_width_default.in;C:/projects/editorconfig-core-c/tests/properties/test.c" "-P" "C:/projects/editorconfig-core-c/tests/cmake/ec_sort.cmake"
Directory: C:/projects/editorconfig-core-c/bin/x64-static/core/tests/properties
"tab_width_default_ML" start time: Jul 01 01:11 Coordinated Universal Time
Output:
----------------------------------------------------------
CMake Error at C:/projects/editorconfig-core-c/tests/cmake/ec_sort.cmake:55 (message):
C:/projects/editorconfig-core-c/bin/x64-static/core/bin/editorconfig
-f;tab_width_default.in;C:/projects/editorconfig-core-c/tests/properties/test.c
returned a nonzero exit code
<end of output>
Test time = 0.02 sec
----------------------------------------------------------
Test Fail Reason:
Required regular expression not found.Regex=[^[
]*indent_size=4[ ]*[
]+indent_style=space[ ]*[
]+tab_width=4[
]*$
]
"tab_width_default_ML" end time: Jul 01 01:11 Coordinated Universal Time
"tab_width_default_ML" time elapsed: 00:00:00
----------------------------------------------------------
Here shows two Windows CI tests on editorconfig-core-c, one with the test submodule right before the commit, and one right after the commit.
@cxw42 Could you look into this, please?
This is a long-standing issue (years) that continues to plague me when working on the editorconfig-core-js
project. These tests run fine in the Travis-CI environment, but fail on Windows in my local dev environment.
@xuhdev do you know if there's something special about these particular tests that would make them fail on Windows, specifically? It makes it pretty difficult to do development when I have to wait for the CI environment to finish each run.
The following tests FAILED:
9 - brackets_close_inside (Failed)
11 - brackets_nclose_inside (Failed)
55 - braces_escaped_comma1 (Failed)
58 - braces_escaped_brace1 (Failed)
59 - braces_escaped_brace2 (Failed)
60 - braces_escaped_brace3 (Failed)
146 - escaped_semicolon_in_section (Failed)
153 - escaped_octothorpe_in_section (Failed)
168 - windows_separator (Failed)
169 - windows_separator2 (Failed)
Related to editorconfig/specification#11, I'd like to break this project into two pieces:
This repo can continue to be used as it is today, but it would need to reference at least the first one above as a subproject.
Seems like no version number can be found within the source code. Although git tags have these information available, this information would be missing if someone downloads a snapshot.
According to editorconfig/editorconfig-vote#11 , the version number should match that of the spec, which should also be noted somewhere in this repo.
The CLI tests only validate that the executable returned a 0 exit code, it does not validate the textual output of the command.
My app output something such as "dummy output" and it still passes the two version switch tests. I do not know much about CMake, however it looks like this function needs to accept parameters.
editorconfig-core-test/CMakeLists.txt
Lines 105 to 116 in 48610d4
https://github.com/editorconfig/editorconfig-core-c being the canonical implementation, one expects all core tests to pass there. However, this is not the case. "85 - braces_alpha_range1" as present in rev 172eb83 of core-tests is failing on core-c rev 6f3c40d510068935322e5e25c2196e12584cca04
Steps to reproduce:
# cd where editorconfig-core-c was cloned
cd editorconfig-core-c
git submodule init
git submodule update
cd tests
git status
HEAD detached at efc9b44 # quite an old revision
git checkout 172eb8324225f09f3e331af26454a5b45e314edb # currently the master's HEAD
cd ..
cmake .
ctest .
...
# Actual:
The following tests FAILED:
85 - braces_alpha_range1 (Failed)
# Expected all tests should pass
The test is defined here:
editorconfig-core-test/glob/CMakeLists.txt
Line 204 in 172eb83
{aardvark..antimater}
and we expect it to match glob [{aardvark..antelope}]
and return the proprety words=a
.
Well, given that core-c is has no support for alphanumeric ranges, the test should either
(a) expect no property to get returned
or
(b) it should pass {aardvark..antelope}
as a file name so that the [{aardvark..antelope}]
glob matches.
Or perhaps
(c) if core-c had a support for alpha ranges, the file name should be something from the range, e.g. aardvark
.
I am ready to submit a RP once somebody decides which of the options is favorable.
Citing from [1]:
For any property, a value of
unset
is to remove the effect of that property, even if it has been set before.
Searching for unset
in the core-test project returns zero matches.
Expected: there should be at least one test for each of the widely accepted properties that unsets a previously set value. This is i.a. important to make sure that handlers for properties that look like boolean or integer at the first sight can handle unset
too.
The editorconfig home page, section File Format Details says:
Special characters can be escaped with a backslash so they won't be interpreted as wildcard patterns.
So from my understanding, the following glob expressions are valid and should be tested.
[a\*.abc]
tab_width=19
[a\?.abc]
tab_width=20
[a\[.abc]
tab_width=21
[a\{.abc]
tab_width=22
I could provide a pull request if you are interested.
PS: The escaped '[' and '{' result in invalid regular expressions in editorconfig-core-py.
On the one hand, an inline sequence with a leading #
and ;
“shall neither be parsed as a comment nor as part of the section name, pair (defined below) key or value in which it was inserted” (from the spec).
On the other hand, this and this normative tests allow #
and ;
inside values.
Could you please clarify which of the statements is correct?
I have setup my own editorconfig plugin for Vim. For testing I took the CMakeList.txt
from this project and translated them to vimscript.
My project is released under the the Vim License.
At the beginning of the test files I added the following comment:
" This code is based on test code from the editorconfig-core-test project
" which is Copyright (c) 2011-2018 EditorConfig Team
" See https://github.com/editorconfig/editorconfig-core-test/blob/master/LICENSE.txt
Real example here
Is this comment sufficient, or do you want me to add something?
Maybe other requirements?
Test globs with braces that are back to back (}...{
). Inspired by editorconfig/editorconfig-core-c#50
I'm confused by the two tests named in the title.
octothorpe_in_property
The test is defined in CMakeLists.txt
(link) like this:
# test octothorpe at end of property value are included in value
new_ec_test(octothorpe_in_property comments.in test11.c
"^key=value; not comment[ \t\n\r]*$")
The matching editorconfig properties in comments.in
(link) is:
# Semicolon at end of value read as part of value
[test11.c]
key=value# not comment
escaped_octothorpe_in_property
The test is defined in CMakeList.txt
(link):
# test escaped octothorpes are included in property value
new_ec_test(escaped_octothorpe_in_property comments.in test12.c
"^key=value ; not comment[ \t\n\r]*$")
The matching editorconfig properties in comments.in
(link) is:
# Escaped semicolon in value
[test12.c]
key=value \# not comment
Question: In both tests, the regex in the assertion contains a semicolon, but the value contains a hash sign. Is this a bug or did I get this wrong?
PR follows
Been fighting with these two tests.
editorconfig-core-test/filetree/CMakeLists.txt
Lines 70 to 74 in 6c8fe68
They test these two conditions...
editorconfig-core-test/filetree/path_separator.in
Lines 5 to 9 in 6c8fe68
At this point I'm assuming the first test is incorrect...or else I am completely not understanding how to implement them.
My assumption is that path/separator
would match any of the following (especially since the comment for the first test says "Tests path separator match below top of path"
:
<root_dir>/path/separator
<root_dir>/a/path/separator
<root_dir>/b/c/path/separator
<root_dir>/nested/path/separator
However the test is expecting it to not have key1=value1
My other assumption is that /top/of/path
would only match:
<root_dir>/top/of/path
And not match any others such as:
<root_dir>/a/top/of/path
<root_dir>/b/c/top/of/path
Which seems to be correct according to the test.
Given that the tests in editorconfig-core-test git repository are the only formalish specification of the EditorConfig format, it would be handy to see which test set belongs to which version of the EditorConfig spec.
E.g. the current version of the spec is 0.12.0-final
and there should definitely exist 0.12.0-final
tag in the editorconfig-core-test repo, so that implementors can see clearly if they comply or not.
I'm currently implementing a rust version of the core and when running and digging into the test suite I see that
mostly key=value
is used.
Whereas editorconfig specification specifies which key-value pairs are allowed.
Is there some reason for this?
Why don't we use options from the specification? That means every core needs to handle key-value pairs (key=value
) which are not really defined in the spec.
The cmake files contain a BSD file but the *.in files do not.
The emacs editorconfig plugin uses this test files and I would like to include them in the Debian packaging. Thus I need to know unambiguously the licenses and copyright holders of all files.
Thank you!
(I also hate dealing with license stuff...)
This is primarily about the spaces_in_section_name
test that passes a file name that starts and ends with a space to the tested EditorConfig implementation.
File names ending with space are illegal on Windows. Citing from https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx :
Do not end a file or directory name with a space or a period. Although the underlying file system may support such names, the Windows shell and user interface does not. However, it is acceptable to specify a period as the first character of a name. For example, ".temp".
Given the above and because we handle the file paths passed to our EditorCongig binary via -f
as real file system paths, we get an exception from the underlying NTFS file system implementation saying that the path is not legal.
Clearly, this is a consequence of our decision to handle the paths as real paths and to use java.nio.file.Path
to split them into segments. To comply with the test, we could actually avoid using java.nio.file.Path
and do the path segment splitting in some other way so that paths ending with spaces are considered legal. However, I do not believe this would be the right thing to do. I do not think we should accept illegal paths just for the sake of testing. In practice it is better to fail early due to illegal input.
I can think of the following as good solutions of the problem:
(a) Remove the trailing space from " test 7 "
- i.e. make " test 7"
out of it, and do the same [ test 7 ]
-> [ test 7]
with the glob in whitespace.in
. This should pass on Windows and the leading space would still serve the purpose of checking that the spaces between square brackets matter.
(b) Leave spaces_in_section_name
as is but skip it on Windows
(c) Combine (a) and (b): Add a new test with leading space and without trailing space (something like " test 71"
[ test 71]
) and skip the existing spaces_in_section_name
test on Windows.
I tend to prefer (c) because it does not narrow the coverage and still makes it possible to pass on Windows.
I am ready to prepare a patch if I get green e.g. for (c).
cc @angelozerr
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.