kunaltyagi / nsiqcppstyle Goto Github PK
View Code? Open in Web Editor NEWCpp style checker in python
License: GNU General Public License v2.0
Cpp style checker in python
License: GNU General Public License v2.0
The NsiqCppStyle parser incorrectly identifies the starting and ending LBRACE/RBRACE tokens for a FUNCTION_BLOCK when one of the initialization variables uses an LBRACE/RBRACE pair.
Here is a minimal snippet of C++ code showing the problem:
1 // Resides in ~/junk/test_class.min.cpp
2 test_class::test_class(int argc_, char* argv_[]) :
3 argc(argc_),
4 _UTP{},
5 _videoEnabled(0)
6 {
7 _properties = NULL;
8 }
Notice that the class initialization list includes _UTP{}
, which uses list (or brace) initialization to initialize it to all default values. We'll see that NsiqCppStyle gets confused about the LRACE/RBRACE of the list initialization.
Running this snippet file through trace-callbacks:
cd <nsiqcppstyle root folder>
/usr/bin/python3.11 trace-callbacks.py ~/junk/test_class.min.cpp > ~/junk/test_class.min.cpp.log
results in the following (you can find the entire output attached to this comment):
======================================================================================
Processing: /homes/mosheru/junk/test_class.min.cpp
FileStart (lexer, filename='test_class.min.cpp', dirname='/homes/mosheru/junk')
--------------------------------------------------
Line (lexer, line='// Resides in ~/junk/test_class.min.cpp', lineNumber=1)
Comment (lexer, token=LexToken(CPPCOMMENT,'// Resides in ~/junk/test_class.min.cpp',1,1,0, False, None))
--------------------------------------------------
Line (lexer, line='test_class::test_class(int argc_, char* argv_[]) :', lineNumber=2)
Token (lexer,
contextStack (empty),
token=LexToken(ID,'test_class',2,1,40, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(DOUBLECOLON,'::',2,11,50, False, None))
FunctionName (lexer,
fullName='test_class::test_class',
decl='False',
contextStack (empty),
context='FUNCTION_BLOCK, 'test_class::test_class', LexToken(LBRACE,'{',4,9,116, False, None), LexToken(RBRACE,'}',4,10,117, False, None)')
Although the parser correctly recognizes "test_class::test_class" as a FunctionName, its context shows it believes the function's LBRACE and RBRACE are on line 4, column 9 (LBRACE) and 10 (RBRACE). This is, of course, incorrect: the correct function LBRACE/RBRACE are lines 6 and 8, respectively.
The parser continues to get things wrong:
Line (lexer, line=' _UTP{},', lineNumber=4)
Token (lexer,
contextStack (empty),
token=LexToken(ID,'_UTP',4,5,112, False, None))
FunctionScope (lexer,
contextStack (1))
FUNCTION_BLOCK, 'test_class::test_class', LexToken(LBRACE,'{',4,9,116, False, None), LexToken(RBRACE,'}',4,10,117, False, None)
Token (lexer,
contextStack (1),
FUNCTION_BLOCK, 'test_class::test_class', LexToken(LBRACE,'{',4,9,116, False, None), LexToken(RBRACE,'}',4,10,117, False, None)
token=LexToken(LBRACE,'{',4,9,116, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(RBRACE,'}',4,10,117, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(COMMA,',',4,11,118, False, None))
In the end, the parser interprets "_videoEnabled" to be a function name with LBRACE/RBRACE tokens on lines 6 and 8:
--------------------------------------------------
Line (lexer, line=' _videoEnabled(0)', lineNumber=5)
FunctionName (lexer,
fullName='_videoEnabled',
decl='False',
contextStack (empty),
context='FUNCTION_BLOCK, '_videoEnabled', LexToken(LBRACE,'{',6,1,141, False, None), LexToken(RBRACE,'}',8,1,165, False, None)')
Token (lexer,
contextStack (empty),
token=LexToken(FUNCTION,'_videoEnabled',5,5,124, False, None))
Token (lexer,
contextStack (1),
PARENBLOCK, '', 137, 139
token=LexToken(LPAREN,'(',5,18,137, False, None))
Token (lexer,
contextStack (1),
PARENBLOCK, '', 137, 139
token=LexToken(NUMBER,'0',5,19,138, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(RPAREN,')',5,20,139, False, None))
--------------------------------------------------
Line (lexer, line='{', lineNumber=6)
FunctionScope (lexer,
contextStack (1))
FUNCTION_BLOCK, '_videoEnabled', LexToken(LBRACE,'{',6,1,141, False, None), LexToken(RBRACE,'}',8,1,165, False, None)
Token (lexer,
contextStack (1),
FUNCTION_BLOCK, '_videoEnabled', LexToken(LBRACE,'{',6,1,141, False, None), LexToken(RBRACE,'}',8,1,165, False, None)
token=LexToken(LBRACE,'{',6,1,141, False, None))
--------------------------------------------------
The parser should handle brace initializations correctly, accurately identifying the opening and closing braces of the function.
Hello,
I was going though the code and noticed a weird indentation here :
https://github.com/kunaltyagi/nsiqcppstyle/blob/master/nsiqcppstyle_exe.py#L266
This else
is currently attached to the previous for
instead of the if
as the comment suggests :
https://github.com/kunaltyagi/nsiqcppstyle/blob/master/nsiqcppstyle_exe.py#L218
This change was introduced by this commit : f5767c102
In function PrepareReport
the open functions try to open the files as binaries. This leads to problems, when the output xml/csv is chosen. I think this is python 2 style ? Think about using csvfile = open(outputPath, "w")
. This fixed it for me
In a future PR, let's see if we can reduce the duplication by:
def RunPreprocessRule(self, lexer: Lexer, contextStack: ??) -> None:
""" Run rules which runs in the preprecessor blocks """
_RunLexerRules(self.preprocessRules, lexer, contextStack)
def _RunLexerRules(self, rules: List[Callable], lexer, *args, **kwargs) -> None
for rule in rules:
data = lexer.Backup()
rule(lexer, *args, **kwargs)
lexer.Restore(data)
Originally posted by @kunaltyagi in #35 (comment)
This could be done for just some of the callbacks
NsiqCppStyle should provide a convenient and easy-to-use tool to show precisely which rule callback functions will be invoked for a given sequence of source code.
Writing an NsiqCppStyle rule requires understanding which callback functions will be invoked when the engine is processing a known sequence of source code. Currently, knowing which callback functions to take advantage of can be tedious. In addition, a rule writer may not be aware of which callback functions are actually invoked, often missing an elegant method of implementing a rule.
The request is to write a tool that, given a snippet of source code, shows the rule writer the precise set of callback functions invoked, and their order. Each callback function displayed should provide important information such as the token type and value, any existing context stack, and other valuable data passed via the callback function.
The tool is written in the style of an NsiqCppStyle rule file, hooking in to the callback function system.
This tool is similar to Clang's ast-dump
command line option.
Suppose we want to see what callback functions NsiqCppStyle will invoke for the following C++ source code:
// Your First C++ Program
#include <iostream>
int main() {
std::cout << "Hello World!";
return 0;
}
Here is what the above tool's output might look like:
D:\nsiqcppstyle>python nsiqcppstyle.py -f rules\filefilter.txt "D:\Junk\NsiqCppStyle\hello-world.cpp"
nsiqcppstyle: N'SIQ Cpp Style ver 0.3.0.1
======================================================================================
= Analyzing hello-world.cpp
======================================================================================
- RULE_50_99_A_test_nsiqcppstyle_callbacks is applied.
======================================================================================
LexToken contain six fields: (type, value, lineno, column, lexpos, inactive, pp)
SessionStartRule()
Filter Scope "default" is applied.
Current Filter Setting (Following is applied sequentially)
1. \.cvs\ is excluded
2. \.svn\ is excluded
Current File extension and Language Settings
C/C++=cpp,mm,cc,c,m,h,hpp,cxx,hxx,hh
======================================================================================
Processing: D:\Junk\NsiqCppStyle\hello-world.cpp
FileStartRule(lexer, filename='hello-world.cpp', dirname='D:\Junk\NsiqCppStyle')
--------------------------------------------------
LineRule(lexer, line='// Your First C++ Program', lineNumber=1)
CommentRule(lexer, token=LexToken(CPPCOMMENT,'// Your First C++ Program',1,1,0, False, None))
--------------------------------------------------
LineRule(lexer, line='#include <iostream>', lineNumber=3)
PreprocessRule(lexer,
contextStack,
token=LexToken(PREPROCESSOR,'#include',3,1,27, False, True)
PreprocessRule(lexer,
contextStack,
token=LexToken(LT,'<',3,10,36, False, True)
PreprocessRule(lexer,
contextStack,
token=LexToken(ID,'iostream',3,11,37, False, True)
PreprocessRule(lexer,
contextStack,
token=LexToken(GT,'>',3,19,45, False, True)
--------------------------------------------------
LineRule(lexer, line='int main() {', lineNumber=5)
Rule(lexer,
contextStack,
token=LexToken(INT,'int',5,1,48, False, None))
FunctionNameRule(lexer,
fullName='main',
decl='False',
contextStack,
context='FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)')
Rule(lexer,
contextStack,
token=LexToken(FUNCTION,'main',5,5,52, False, None))
Rule(lexer,
contextStack,
token=LexToken(LPAREN,'(',5,9,56, False, None))
Context stack:
PARENBLOCK, , 56, 57
Rule(lexer,
contextStack,
token=LexToken(RPAREN,')',5,10,57, False, None))
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(LBRACE,'{',5,12,59, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
--------------------------------------------------
LineRule(lexer, line=' std::cout << "Hello World!";', lineNumber=6)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(ID,'std',6,5,65, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(DOUBLECOLON,'::',6,8,68, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(ID,'cout',6,10,70, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(LSHIFT,'<<',6,15,75, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(STRING,'"Hello World!"',6,18,78, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(SEMI,';',6,32,92, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
--------------------------------------------------
LineRule(lexer, line=' return 0;', lineNumber=7)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(RETURN,'return',7,5,98, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(NUMBER,'0',7,12,105, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
FunctionScopeRule(lexer, contextStack)
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
Rule(lexer,
contextStack,
token=LexToken(SEMI,';',7,13,106, False, None))
Context stack:
FUNCTION_BLOCK, main, LexToken(LBRACE,'{',5,12,59, False, None), LexToken(RBRACE,'}',8,1,108, False, None)
--------------------------------------------------
LineRule(lexer, line='}', lineNumber=8)
Rule(lexer,
contextStack,
token=LexToken(RBRACE,'}',8,1,108, False, None))
FileEndRule(lexer, filename='hello-world.cpp', dirname='D:\Junk\NsiqCppStyle')
ProjectRule(targetName='D:\Junk\NsiqCppStyle\hello-world.cpp')
=================================== Summary Report ===================================
** Total Available Rules : 54
** Total Applied Rules : 1
** Total Violated Rules : 0
** Total Errors Occurs : 0
** Total Analyzed Files : 1
** Total Violated Files Count: 0
** Build Quality : 100.00%
================================ Violated Rule Details ===============================
================================ Violated File Details ===============================
SessionEndRule()
A few comments:
[RULE|TOOL|UTIL]_nn_nn_text
or some other standard? If the user standard, what number corresponds to NsiqCppStyle-related rule files?nsiqcppstyle -r > filefilter.txt
. We need to modify the "-r" option NOT to output tool/util scripts, as they are not rules for finding warnings in C++ source code.Links to nsiqcppstyle.appspot.com yield 404. If no new site exists to replace the now-defunct, please consider creating the wiki for this project instead.
I would also greatly appreciate if anyone could link me / send me the information about the rules. I would like to take this tool into use as a part of automated student submission checking.
nsiqcppstyle.py
currently accepts INCLUDE/EXCLUDE filters via the filefilter.txt
file. Using the README example, if filefilter.txt
contains the following lines:
- \
+ \src\
- \src\test
NsiqCppStyle will analyze all source files under \src\
except \src\test
.
The INCLUDE/EXCLUDE filters are checked against a relative-path candidate file (e.g., \src\graph-logic.h
) to see if the file path starts with the filter string. So in this example, the path begins with \src\
so it will be included.
The problem is that there is no way to exclude a file name exhibiting a certain pattern. For example, I wanted to process all files under a folder, but did not want to process any file ending with .pb.h
(i.e., generated by the Google Protobuf Buffer mechanism). As far as I can see, there is no way to do this using NsiqCppStyle's filters in filefilters.txt.
There are two possible solutions:
Just a question, is this the new "official" home for nsiqcppstyle or is it just your personal copy ?
nsiqcppstyle_exe.py lines: 293 - 294 should have \ to continue the line. This can lead to compilation errors:
File "/usr/lib/nsiqcppstyle/nsiqcppstyle_exe.py", line 293
if fileExtension in cExtendstionSet and
^
SyntaxError: invalid syntax
Existing Code:
if fileExtension in cExtendstionSet and basefilelist.IsNewOrChanged(eachFile) and
Suggested Replacement:
if fileExtension in cExtendstionSet and
basefilelist.IsNewOrChanged(eachFile) and \
I experience many false positives in all functions that return a struct or a pointer to struct. Here is a very simple example.
filefilter.txt:
~ RULE_6_1_A_do_not_omit_function_parameter_names
nsiq_test.c:
struct test_struct
{
int a;
int b;
};
void func1(int *p, int x);
struct test_struct func2(void)
{
struct test_struct ts = {1, 2};
return ts;
func1(NULL, 0);
}
This gives the following false positive:
/home/pnin/src/skunk/nsiq_test.c(14, 5): function (func1) has non named parameter. use named parameter. [RULE_6_1_A_do_not_omit_function_parameter_names]
If I change the return type of func2
to int
instead of struct test_struct
the error goes away. I suspect that func2
is treated as a struct instead of a function. If I dump contextStack
during each iteration in nsiqcppstyle_checker.py, I get this when func2
returns an int:
FUNCTION_BLOCK, func2, LexToken(LBRACE,'{',10,1,91, False, None), LexToken(RBRACE,'}',15,1,165, False, None) >>
And this when func2
returns a struct:
STRUCT_BLOCK, func2, LexToken(LBRACE,'{',10,1,106, False, None), LexToken(RBRACE,'}',15,1,180, False, None) >>
NsiqCppStyle misidentifies the function name, in both declarations and definitions, when GCC's function attribute specifier follows the true function name. In the following code snippets:
static void finiUtracer_1() __attribute__((destructor));
void finiUtracer_2() __attribute__((destructor)) {}
NsiqCppStyle incorrectly identifies the token "__attribute__" as the function name.
This is contrasted by NsiqCppStyle correctly identifying the function name when the GCC attribute specifier precedes the true function name:
static void __attribute__((destructor)) finiUtracer_3();
void __attribute__((destructor)) finiUtracer_4() {}
The __attribute__
modifier in GCC is a way to specify special attributes of functions or variables. This feature allows developers to attach characteristics to function declarations to allow the compiler to perform more error checking or code optimization.
Here are a few examples of what you can do with __attribute__
:
__attribute__((constructor))
and __attribute__((destructor))
: These attributes, when used with a function, ensure that the function is called before main()
when the program starts, or after main()
when the program exits, respectively.
__attribute__((deprecated))
: This attribute, when used with a function, produces a warning whenever the function is used elsewhere in the code, indicating that the function is deprecated.
__attribute__((packed))
: This attribute, when used with a structure or union type definition, specifies that each member (after the first) is placed to minimize the memory required.
__attribute__((noreturn))
: This attribute, when used with a function, tells the compiler that the function does not return. The compiler can then optimize the code with this assumption.
Remember that __attribute__
is specific to GCC and may not work with other compilers. If you're writing portable code, you should use it conditionally.
I created a test file ~/junk/test.cpp
:
static void finiUtracer_1() __attribute__((destructor));
void finiUtracer_2() __attribute__((destructor)) {}
static void __attribute__((destructor)) finiUtracer_3();
void __attribute__((destructor)) finiUtracer_4() {}
CD-ing to the nsiqcppstyle root folder and running the following command:
/usr/bin/python3.11 trace-callbacks.py ~/junk/test.cpp > ~/junk/test.log
produced a full trace of all NsiqCppStyle callback function parameter lists (the complete output file is attached to this ticket). Here is the relevant output to illustrate this problem:
======================================================================================
Processing: /homes/mosheru/junk/test.cpp
FileStart (lexer, filename='test.cpp', dirname='/homes/mosheru/junk')
--------------------------------------------------
Line (lexer, line='static void finiUtracer_1() __attribute__((destructor));', lineNumber=1)
Token (lexer,
contextStack (empty),
token=LexToken(STATIC,'static',1,1,0, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(VOID,'void',1,8,7, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(ID,'finiUtracer_1',1,13,12, False, None))
Token (lexer,
contextStack (1),
PARENBLOCK, '', 25, 26
token=LexToken(LPAREN,'(',1,26,25, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(RPAREN,')',1,27,26, False, None))
FunctionName (lexer,
fullName='__attribute__',
decl='True',
contextStack (empty),
context='None')
Token (lexer,
contextStack (empty),
token=LexToken(FUNCTION,'__attribute__',1,29,28, False, None))
Token (lexer,
contextStack (1),
PARENBLOCK, '', 41, 54
token=LexToken(LPAREN,'(',1,42,41, False, None))
Token (lexer,
contextStack (2),
PARENBLOCK, '', 41, 54
PARENBLOCK, '', 42, 53
token=LexToken(LPAREN,'(',1,43,42, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(ID,'destructor',1,44,43, False, None))
Token (lexer,
contextStack (1),
PARENBLOCK, '', 41, 54
token=LexToken(RPAREN,')',1,54,53, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(RPAREN,')',1,55,54, False, None))
Token (lexer,
contextStack (empty),
token=LexToken(SEMI,';',1,56,55, False, None))
The true function name ("finiUtracer_1
") is identified as a regular token:
Token (lexer,
contextStack (empty),
token=LexToken(ID,'finiUtracer_1',1,13,12, False, None))
while the "__attribute__
" value is identified as the FunctionName:
FunctionName (lexer,
fullName='__attribute__',
decl='True',
contextStack (empty),
context='None')
I have not investigated NsiqCppStyle's parser code, but I would wager that the function attribute specifier ("\_\_attribute\_\_((destructor))
") looks too much like a function name and a parenthesized argument list. NsiqCppStyle might be taking the latest such match on the line as the function name.
NsiqCppStyle processes C++ files that have an extension found in the list (see nsiqcppstyle_exe.py
for the extension name logic):
["cpp", "h", "c", "hxx", "cxx", "hpp", "cc", "hh", "m", "mm"]
When I attempted to process a file (under Windows) with the external name of "ACCESSDB.CPP", NsiqCppStyle did not process it.
nsiqcppstyle_exe.py
defines accepted extensions:
extLangMap = {
"Html": set(["htm", "html"]),
"Java": set(["java"]),
"Javascript/ActionScript": set(["js", "as"]),
"JSP/PHP": set(["jsp", "php", "JSP", "PHP"]),
"C/C++": set(["cpp", "h", "c", "hxx", "cxx", "hpp", "cc", "hh", "m", "mm"])
}
...
cExtendstionSet = extLangMap.get("C/C++")
...
# if the target is file, analyze it without condition
if os.path.isfile(targetPath):
`= targetPath[targetPath.rfind('.') + 1:]
if fileExtension in cExtendstionSet:
ProcessFile(ruleManager, targetPath, analyzedFiles)
The search for fileExtension
in cExtendstionSet
is case-sensitive. This is correct for Linux, but for Windows the engine should perform a case-insensitive search.
If running under Windows, the search should be case-insensitive.
Add two new rule callback functions to notify a user rule when NsiqCppStyle is about to start/end its session.
In NsiqCppStyle, a session
is defined as the time range beginning just before the first source file is processed, and ending right after the last file is processed. Within this time range, NsiqCppStyle may have processed multiple projects
(processing of a single file or an entire directory tree). There is always only one session for a given run of NsiqCppStyle.
Currently, NsiqCppStyle has two sets of callbacks to notify a user rule when processing events start and/or end:
for targetPath in targetPaths:
if targetPath is a file:
FileStartRule()
<process the file>
FileEndRule()
else:
# targetPath is a directory ("project")
for (each file in the directory tree):
FileStartRule()
<process the file>
FileEndRule()
ProjectRules()
In a recent user rule I wrote, I needed to "setup" and "teardown" global data at the start and end of the session: at session start I defined required data structures, while at session end I wrote out the accumulated information to an external JSON file.
The request is to define new rule callback functions to notify a user rule when the session starts and ends (e.g., SessionStartRule
and SessionEndRule
). When implemented, rule writers will have three types of time range notifications to work with:
SessionStartRule()
for targetPath in targetPaths:
if targetPath is a file:
FileStartRule()
<process the file>
FileEndRule()
else:
# targetPath is a directory ("project")
for (each file in the directory tree):
FileStartRule()
<process the file>
FileEndRule()
ProjectRules()
SessionEndRule()
(I have already implemented the above, and hope to create a pull request with the changes in the near future.)
The fmt library in C++ offers a streamlined implementation of the boost::format libraries. Among other things it offers an sprintf() function. This function safely generates an std::string without buffer overflow issues.
nsiqcppstyle reports:
Do not use burfferoverflow risky function(sprintf [RULE_10_1_A_do_not_use_bufferoverflow_risky_function_for_unix]
for this line of code:
string s = fmt::sprintf("%2s%012.8f", year, doy_fraction);
The simple solution might be to check if sprintf( is preceded by a "::" (two colons), although that would run in to problems if you have "using namespace fmt;" and thus drop the fmt::. Still, I think it would be quite OK.
Prepare top-notch, visually appealing GitHub wiki documentation for NsiqCppStyle. The documentation should be as clear, simple, and appealing as possible, and should provide everything an NsiqCppStyle user needs to write style rules.
Most GitHub projects either have no wiki-based documentation, or the documentation is of poor quality. The goal is to make NsiqCppStyle attractive to potential users by providing world-class documentation.
It will take time and patience to implement this issue, given the wealth of technical information within the project. We should plan the wiki page structure before diving in. This page should serve as the whiteboard for developing the best page structure. Once decided, it will be easier to tackle the wiki pages one at a time.
Following is a short list of GitHub projects that have attractive and high-quality wiki documentation (by all means, if you find a great example, add it to the list!). We should feel free to emulate the best of what there is out there.
To see the raw markdown of a wiki page you like, slightly manipulate the project wiki page URL to generate the desired URL. Note that the URL is case-sensitive. Here are two examples to show how.
Example 1: View raw markdown for the wiki home page of Netflix/Hystrix
The URL to the GitHub wiki home page of the Netflix/Hystrix project is https://github.com/Netflix/Hystrix/wiki
. Looking at that page shows the page title is "Home":
https://raw.githubusercontent.com/wiki/
https://raw.githubusercontent.com/wiki/Netflix/Hysterix/
https://raw.githubusercontent.com/wiki/Netflix/Hysterix/Home
.md
: https://raw.githubusercontent.com/wiki/Netflix/Hysterix/Home.md
Example 2: View raw markdown of d3/d3's Tutorials wiki page
The URL to the GitHub wiki home page of the Netflix/Hystrix project is https://github.com/d3/d3/wiki
. Looking at that page shows the page title is "Tutorials":
https://raw.githubusercontent.com/wiki/
https://raw.githubusercontent.com/wiki/d3/d3/
https://raw.githubusercontent.com/wiki/d3/d3/Tutorials
.md
: https://raw.githubusercontent.com/wiki/d3/d3/Tutorials.md
Hi all,
we use Doxygen with the following syntax :
/*!
* \fn
* \param
* \brief
*/
But it is considered as violation by RULE_5_3_A ...
Though it is a proper way to use Doxygen :
https://www.doxygen.nl/manual/docblocks.html
Is it a big deal to fix?
Thanks a lot.
Lambdas seem to always produce violations of RULE_4_5_A_matching_braces_inside_of_function_should_be_located_same_column
For example, using async boost callbacks:
auto onAccept = [this](boost::system::error_code ec) {
if (!ec) {
RxSessionType::create(this)->start();
} else {
std::cout << "Experienced error on accept: " << ec << std::endl;
}
_acceptConnections();
};
In the above, it would report both the lambda body braces and if/else braces.
NsiqCppStyle
has anachronistic code related to updating the local NsiqCppStyle code base. Apparently, NsiqCppStyle was meant to check the code repo at http://nsiqcppstyle.appspot.com
. Now that NsiqCppStyle is hosted on GitHub, this "feature" is no longer relevant and should be removed.
Vestiges of remotely updating NsiqCppStyle can be found in three source files:
nsiqcppstyle_exe.main()
is the starting point:
...
import updateagent.agent
...
def main(argv=None):
...
updateNsiqCppStyle = False
for o, a in opts:
...
elif o == "--update":
updateNsiqCppStyle = True
elif o == "--no-update":
updateNsiqCppStyle = False
...
if updateNsiqCppStyle:
console.Out.Ci(console.Separator)
try:
updateagent.agent.Update(version)
except Exception as e:
console.Out.Error(e)
...
The above code makes a call to updateagent.agent.Update(currentVersion)
, which does the heavy work. Update()
also imports updateagent.minjson
, which is used to read in JSON data. Now that JSON is fully supported in Python 3, minjson.py
is no longer relevant.
In short, the entire updateagent
folder should be removed, and related changes made to the remaining code base.
The RULE_3_2_F_use_representitive_classname_for_cpp_filename which check filename against class name declared in it. When the class is final, the parser interpret "final" as the classname :-(
The following test reproduce the problem :
def test9(self):
self.Analyze("test/CamRecorderFactory.h",
"""
class CamRecorderFactory final
{
};
""")
assert not CheckErrorContent(name)
NsiqCppStyle is a serious Open Source software project, intended for general use. Every time a change is made to the engine code, there is a real risk of breaking the existing implementation and/or features. NsiqCppStyle requires high-quality and reliable regression tests to ensure that changes to the existing engine code base won't affect the existing features of the project.
As expected of a well-designed product, NsiqCppStyle has a dedicated unit/regression test source file: nsiqunittest/nsiqcppstyle_unittest.py
. The file defines a class unitTest
with member functions, each one representing a unit/regression test. This script file currently has a small number of tests:
Test Name | Description |
---|---|
unitTest.__testFunctionSpecifier | Internal utility function used by the testIgnore function |
unitTest.testIgnoreFinalFunctionSpecifier | Verifies that the final token is ignored |
unitTest.testIgnoreOverrideFunctionSpecifier | Verifies that the override token is ignored |
unitTest.testIgnoreNoexceptFunctionSpecifier | Verifies that the noexcept token is ignored |
unitTest.testGetPrevMatchingLT | Verifies lexer.GetPrevMatchingLT() works correctly for simple variable definition/declaration statement using '<...>' |
unitTest.testGetPrevMatchingLTWithInner | Verifies lexer.GetPrevMatchingLT() works correctly for variable definition/declaration statement using nested '<...>' pairs, no occurrence of adjacent LT or GT |
unitTest.testGetPrevMatchingLTWithInnerOnes | Verifies lexer.GetPrevMatchingLT() works correctly for variable definition/declaration statement using nested '<...>' pairs, has occurrence of adjacent LT or GT |
unitTest.test2 | Verifies all tokens can be iterated using GetNextToken |
unitTest.test3 | Verifies all tokens can be iterated using GetNextTokenSkipWhiteSpaceAndComment |
unitTest.test4 | Same as test3 with console level set to console.Level.Verbose |
unitTest.test5 | Verifies the first token of a complex definition/declaration has type ID and expected value |
It is clear that these tests are the proverbial "tip of the iceberg", and that many more such tests should be added to ideally provide complete coverage of internal engine code.
Effort should be invested to write as many targeted unit/regression tests for NsiqCppStyle internals as possible. As more people submit code changes, having a rich set of regression tests will give us the confidence needed to make changes without breaking the existing code base.
The writing of such tests should be an ongoing task: every time any contributor (or the maintainer) touches a section of NsiqCppStyle engine code, time should be allocated for enriching the set of unit/regression tests, all related to the section/features modified.
Similarly, while implementing a new feature, should a contributor break the product during development, s/he should stop and write the necessary unit/regression tests to never allow that break to happen again.
IMHO, this should be a high-priority task as soon as the current pull requests have settled and are merged.
I'd prefer these to be valid Path to remove a bunch of corner cases due to using str (eg: some filesystems don't allow unicode names, but others do), but let's postpone that to later
Originally posted by @kunaltyagi in #35 (comment)
When NsiqCppStyle parses the following code snippet:
Event& Event::operator=(OUT Event& other)
{
}
The internal token list (accessed via the lexer.tokenlist
member variable) looks like this:
tokenlist = {list: 19}
00 = {LexToken} LexToken(ID,'Event',1,1,0, False, None)
01 = {LexToken} LexToken(AND,'&',1,6,5, False, None)
02 = {LexToken} LexToken(SPACE,' ',1,7,6, False, None)
03 = {LexToken} LexToken(ID,'Event',1,8,7, False, None)
04 = {LexToken} LexToken(DOUBLECOLON,'::',1,13,12, False, None)
05 = {LexToken} LexToken(FUNCTION,'operator',1,15,14, False, None)
06 = {LexToken} LexToken(EQUALS,'=',1,23,22, False, None)
07 = {LexToken} LexToken(LPAREN,'(',1,24,23, False, None)
08 = {LexToken} LexToken(ID,'OUT',1,25,24, False, None)
09 = {LexToken} LexToken(SPACE,' ',1,28,27, False, None)
10 = {LexToken} LexToken(ID,'Event',1,29,28, False, None)
11 = {LexToken} LexToken(AND,'&',1,34,33, False, None)
12 = {LexToken} LexToken(SPACE,' ',1,35,34, False, None)
13 = {LexToken} LexToken(ID,'other',1,36,35, False, None)
14 = {LexToken} LexToken(RPAREN,')',1,41,40, False, None)
15 = {LexToken} LexToken(LINEFEED,'\n',1,42,41, False, None)
16 = {LexToken} LexToken(LBRACE,'{',2,1,42, False, None)
17 = {LexToken} LexToken(LINEFEED,'\n',2,2,43, False, None)
18 = {LexToken} LexToken(RBRACE,'}',3,1,44, False, None)
The problem is that NsiqCppStyle returns the overloaded operator function name ("operator=") as two tokens, i.e., a FUNCTION ("operator" at offset 5) and an EQUALS ("=" at offset 6). This is incorrect - the overloaded operator function name should be returned as one FUNCTION token with a value of "operator=":
tokenlist = {list: 18}
00 = {LexToken} LexToken(ID,'Event',1,1,0, False, None)
01 = {LexToken} LexToken(AND,'&',1,6,5, False, None)
02 = {LexToken} LexToken(SPACE,' ',1,7,6, False, None)
03 = {LexToken} LexToken(ID,'Event',1,8,7, False, None)
04 = {LexToken} LexToken(DOUBLECOLON,'::',1,13,12, False, None)
05 = {LexToken} LexToken(FUNCTION,'operator=',1,15,14, False, None)
06 = {LexToken} LexToken(LPAREN,'(',1,24,23, False, None)
07 = {LexToken} LexToken(ID,'OUT',1,25,24, False, None)
08 = {LexToken} LexToken(SPACE,' ',1,28,27, False, None)
09 = {LexToken} LexToken(ID,'Event',1,29,28, False, None)
10 = {LexToken} LexToken(SPACE,' ',1,34,33, False, None)
11 = {LexToken} LexToken(AND,'&',1,35,34, False, None)
12 = {LexToken} LexToken(ID,'other',1,36,35, False, None)
13 = {LexToken} LexToken(RPAREN,')',1,41,40, False, None)
14 = {LexToken} LexToken(LINEFEED,'\n',1,42,41, False, None)
15 = {LexToken} LexToken(LBRACE,'{',2,1,42, False, None)
16 = {LexToken} LexToken(LINEFEED,'\n',2,2,43, False, None)
17 = {LexToken} LexToken(RBRACE,'}',3,1,44, False, None)
This is the correct way to parse the overloaded operator function name - tokenlist[5] shows the FUNCTION token with a value of "operator=".
NsiqCppStyle currently supports ten (10) types of token callback functions. The rule writer can register one or more of these callback functions with the engine. Currently, the following are the supported callback functions:
Rule Type | Registration Function | Description | Comment |
---|---|---|---|
File End | AddFileEndRule | End-of-file of processed C++ file encountered | |
File Start | AddFileStartRule | C++ source file about to be processed | |
Function Name | AddFunctionNameRule | Function name in definition/declaration encountered | |
Function Scope | AddFunctionScopeRule | Token resides in a function scope | |
Line | AddLineRule | A new line has been read | |
Preprocess | AddPreprocessRule | A token with a preprocessor statement encountered | |
Project | AddProjectRules | Processing of project (i.e., an entire directory tree) completed | Can be invoked multiple times within a session. The registration function should be renamed from AddProjectRules (plural) to AddProjectRule (singular). |
Rule | AddRule | Token encountered (not in a preprocessor statement) | We should consider renaming to TokenRule / AddTokenRule. We should retain the old name for backwards compatibility, while providing a new alias. |
Type Name | AddTypeNameRule | Token is one of the strings [class / struct / enum / namespace / union] | |
Type Scope | Add | Token resides in a class / struct / enum / namespace / union scope |
When the NsiqCppStyle engine processes a token, one or more callback functions may be invoked. If a user rule has registered callback functions, the rule's callback functions will be invoked. This enables the user rule to perform its work.
Here is the loop that processes the tokens and invokes the callback functions:
def RunRules(ruleManager, lexer):
...
while(True):
try:
t = lexer.GetNextTokenSkipWhiteSpaceAndComment()
...
ruleManager.RunLineRule(
lexer, lexer.GetCurTokenLine(), currentLine)
if t.pp == True:
ruleManager.RunPreprocessRule(lexer, t.contextStack)
else:
if t.type == 'TYPE':
ruleManager.RunTypeNameRule(lexer, t.value.upper(), t.fullName,
t.decl, t.contextStack, t.context)
elif t.type == 'FUNCTION':
ruleManager.RunFunctionNameRule(lexer, t.fullName, t.decl,
t.contextStack, t.context)
elif t.contextStack is not None and t.contextStack.SigPeek() is not None:
sigContext = t.contextStack.SigPeek()
if sigContext.type == "FUNCTION_BLOCK":
ruleManager.RunFunctionScopeRule(lexer, t.contextStack)
elif sigContext.type in ["CLASS_BLOCK", "STRUCT_BLOCK", "ENUM_BLOCK", "NAMESPACE_BLOCK", "UNION_BLOCK"]:
ruleManager.RunTypeScopeRule(lexer, t.contextStack)
ruleManager.RunRule(lexer, t.contextStack)
...
try:
ruleManager.RunFileEndRule(lexer, os.path.basename(lexer.filename),
os.path.dirname(lexer.filename))
...
There is no rule callback when a COMMENT (C-Style) or CPPCOMMENT (C++-style) comment token is encountered. Thus, a user rule cannot handle a comment token as encountered in the token stream.
The request is that:
The RULE_5_3_A_provide_doxygen_function_comment_on_function_in_header
rule checks that function declarations are preceded with a Doxygen comment. Because the rule was written without a COMMENT callback, it waits until a FunctionName callback is invoked. At that point, it calls lexer.GetPrevTokenInType("COMMENT")
to find the preceding COMMENT token, etc.
Sometimes back-searching for the previous COMMENT token is not convenient. A while back I wrote a proprietary rule that processes hundreds of source files, identifying all Google Test unit test functions. Our coding standard for these functions was to document the functions with a sequence of multiple COMMENT (not CPPCOMMENT) lines. Rather than processing the multiple COMMENT lines post-facto, I wanted to process the comment lines as they were encountered. I, therefore, implemented what is requested in this issue, and would like to merge the implementation back into the repo.
In any case, I believe that a COMMENT or CPPCOMMENT is as valid a token as any, and should have its own dedicated callback function.
Currently any rule file used in an NsiqCppStyle session (e.g., RULE_3_1_A_do_not_start_filename_with_underbar
) must reside in the nsiqcppstyle\rules
package folder. You can see this in nsiqcppstyle_runmanager.py:
def LoadRules(self, checkingRuleNames):
...
for ruleName in checkingRuleNames:
ruleModule = __import__("rules." + ruleName)
self.loadedRule.append(ruleModule)
...
I have written many proprietary rules that I do not want to return to the NsiqCppStyle repository. To include them in my NsiqCppStyle checking runs, I must copy my rules to the rules
folder. Should I upgrade NsiqCppStyle to a newer version I may need to copy all of my proprietary rules again. Should I push changes through a GitHub pull request, I have to be careful not to add my proprietary rules to the commit. In general, I want to keep my proprietary rules totally separate from the NsiqCppStyle package.
The proposal is to support a new command-line option for specifying one or more additional paths that should be searched for rules. These paths can be anywhere in the file system. For all rule names in the filefilter.txt
file, NsiqCppStyle should search first the rules
folder, followed by any additional folders. If found, the engine should import the rule file. The following Stack Overflow post provides Python code that can be used to import the rule file from anywhere. Here is a test script I wrote:
d:\junk\import-from-anywhere\main.py
import importlib.util
import sys
spec = importlib.util.spec_from_file_location("test.py", "d:\\junk\\test.py")
foo = importlib.util.module_from_spec(spec)
sys.modules["test.py"] = foo
spec.loader.exec_module(foo)
print("foobar=%d" % (foo.foobar))
d:\junk\test.py
foobar = 5
Running main.py shows the import in action:
D:\Junk\NsiqCppStyle\Import-From-Anywhere>python main.py
foobar=5
There is an error on the indentation: look at https://github.com/kunaltyagi/nsiqcppstyle/blob/master/nsiqcppstyle_exe.py
line: 380 to 384.
elif line.startswith("+"):
arg = line[1: ].strip()
if arg != "": filter.AddInclude(arg)
elif line.startswith("-"):
arg = line[1: ].strip()
if arg != "": filter.AddExclude(arg)
elif line.startswith("%"):
arg = line[1: ].strip()
if arg != "": filter.AddVarMap(arg, "\"" + arg + "\" of filefilter.txt")
The verification for "-" and "%" should be at the same level than "+"
Thanks in advance!
I don't see any setuptools or autotools for this project, so I wonder if there is a preferred way to prepare a source tarball. I have packaged RPMs in the past, with a tarball made by hand and a custom spec file which added a few homegrown rules.
But now I'm also preparing debian packages, and I'd like to have a more reproducible way to go from repository to debian or RPM package.
Is your thought that nsiqcppstyle should eventualy have one of these formal distribution approaches? Would you welcome a setup.py approach? Or is there already something and I have just not found it?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.