How i can get an output with .csv format？ In python，i use "flawfinder ./test"

Thanks a lot.

Encoding Error on Windows 10

git clone https://github.com/Kitware/CMake.git

git clone https://github.com/david-a-wheeler/flawfinder.git

python3 flawfinder\flawfinder.py CMake\Source\CTest\cmCTestBuildHandler.cxx

No Encoding Error on WSL

File Content

Hello,

I have following warning during flawfinder scan:

src/[-]/CsvParser.cpp:34:  [2] (misc) open:
  Check when opening files - can an attacker redirect it (via symlinks),
  force the opening of special file type (e.g., device files), move things
  around to create a race condition, control its ancestors, or change its
  contents? (CWE-362).

I tried to check if filepath is symlink and play accordingly, however the warning is still there:

CsvParser::CsvParser(const bfs::path& csvFilePath)
{
    mCsvFile.exceptions(std::ifstream::badbit);
    mCsvFile.open(csvFilePath.string(), std::fstream::in);
    // ...
}

bfs is boost:filesystem it can be replaced with std::filesystem. How should I resolve the warning? IMO, flawfinder should offer examples how to solve certain warnings - this will be perfect learning opportunity to get knowledge in software security.

Klugier

I've tried flawfinder on my zt library:

zt.c:35:  [4] (format) printf:
  If format strings can be influenced by an attacker, they can be exploited
  (CWE-134). Use a constant for the format specification.

This refers to https://github.com/zyga/libzt/blob/main/zt.c#L35 - reproduced below for simplicity:

#define ZT_FORMAT_PRINTF(a, b) __attribute__((format(printf, a, b)))

Hi, I was wondering whether ioctl was evaluated and if not to understand why it would not be included in the analysis. It's my understanding ioctl is frequently targeted in malicious activities and was hoping to gain some clarity on this. Thanks in advance.

This is a tracking item to describe next possible steps with Flawfinder SARIF + HDF support.

Observations:

• The HDF format currently requires encoding tool -> CWE/other NIST standard mappings in a non-standard format which is currently persisted to the HDF tooling repo.
• If we solve this problem using SARIF taxonomies, it appears we could create a straightforward SARIF -> HDF converter that would unlock the SARIF eco-system for Heimdall.
• This interoperability isn't free, however: tools that opt into HDF support will need to explicit map their rule ids to the NIST standards. Several existing tools (Flawfinder being one) already have mapping data in some form.

As preparatory work, we are building a CWE Sarif taxonomy, which s/be published to a well-known web location. We will also need to create a taxonomy for the NIST standard (and create relationships from CWE to NIST IDs, as shown in the heimdall_tools repo).

It'd be interesting to discuss how these definitive taxonomy files are published on the web.

Once this core work is complete, we should update Flawfinder to emit its rule id -> CWE/NIST mappings in the log file. The Flawfinder log will also contain a link to the external CWE/NIST web-hosted taxonomies.

With a finished log such as this, a new SARIF -> HDF converter will be able to produce HDF that can flow into the various Heimdall tools. That will be very nice! All of this work will demonstrate a good path for other tools to follow.

Thoughts? :)

@eddynaka @yongyan-dh @david-a-wheeler

I've tried flawfinder on my zt library:

zt.c:47:  [4] (format) vfprintf:
  If format strings can be influenced by an attacker, they can be exploited
  (CWE-134). Use a constant for the format specification.

This refers to https://github.com/zyga/libzt/blob/main/zt.c#L47 - reproduced below for simplicity:

static void zt_logv(FILE* stream, zt_location loc, const char* fmt, va_list ap)
{
    if (stream != NULL) {
        if (loc.fname != NULL && loc.lineno != 0) {
            fprintf(stream, "%s:%d: ", loc.fname, loc.lineno);
        }
        vfprintf(stream, fmt, ap); /* This is line 47 */
        fprintf(stream, "\n");
    }
}

This function is used inside zt_logf reproduced below:

static void zt_logf(FILE* stream, zt_location loc, const char* fmt, ...)
{
    va_list ap;
    va_start(ap, fmt);
    zt_logv(stream, loc, fmt, ap);
    va_end(ap);
}

This is a common pattern in C, where a printf like function calls into vprintf like function.

It is technically true that vfprintf(stream, fmt, ap) requires agreement between fmt and ap but this is unavoidable in this specific case.

First - thanks.

Next, a question about a potential false positive (or at least aged out positive)

https://github.com/david-a-wheeler/flawfinder/blob/master/flawfinder#L1266

flags EnterCriticalSection as

"On some versions of Windows, exceptions can be thrown in low-memory situations",
"Use InitializeCriticalSectionAndSpinCount instead

However, the doc makes no mention of that.

The page from the book Writing Secure Code describes EnterCriticalSection as something that will not throw errors on XP, .NET Server, and later. Considering that XP EOL in April 8, 2014; .Net Server EOL 14 July 2015 if people are using OSes earlier than that - they have bigger issues than what will be flagged with flawfinder...

The doc for InitializeCriticalSection does indicate:

Windows Server 2003 and Windows XP:  In low memory situations, InitializeCriticalSection can raise a STATUS_NO_MEMORY exception. 
Starting with Windows Vista, this exception was eliminated and InitializeCriticalSection always succeeds, even in low memory situations.

Windows Server 2003 operating system EOL'ed on July 14, 2015.

What's the goal from flawfinder for managing the versions of Windows?

Thanks again.

It'd be great to have a GitHub Actions integration. Now that flawfinder has SARIF output, that should in theory be easy.

More info here about creating such integrations:
https://docs.github.com/en/code-security/secure-coding/integrating-with-code-scanning

Function names are searched in source code.
Would you dare to perform such source code searches by any higher level technology than variations of regular expressions?

When using LOAD_LIBRARY_SEARCH_SYSTEM32 on LoadLibraryEx, the developer is explicitly telling the function to load the DLL only from the System32 folder, therefore the warning emitted on the following snippet:

HMODULE user32 = LoadLibraryEx(L"user32.dll", nullptr, LOAD_LIBRARY_SEARCH_SYSTEM32);

is completely superfluous and even potentially misleading.

In order to improve the project structure, I would propose to put all test-related files (e.g. correct-results*.{txt,csv,html}, test*.{c,patch} ) into a subdirectory (e.g.tests/). Eventually, putting the relevant makefile parts into a separate makefile within the test directory would even further improve the clarity of the overall project (or at least of the makefile).

What do you think about this change? As soon as you agree to the general idea, I will create a pull-request.

This approach seems to be backed by PyPA and most answers on SO tend to suggest a similar structure for Python-based projects:

• https://github.com/pypa/sampleproject
• https://stackoverflow.com/questions/193161/what-is-the-best-project-structure-for-a-python-application

FF1057's text is missing CWEs in the text:

    "InitializeCriticalSection":
    (normal, 3, "Exceptions can be thrown in low-memory situations",
     "Use InitializeCriticalSectionAndSpinCount instead",
     "misc", "", {}, "FF1057"),

As a result, when the SARIF file is generated, the helpUri field becomes an invalid URI:

{
  "id": "FF1057",
  "name": "misc/InitializeCriticalSection",
  "shortDescription": {
    "text": "Exceptions can be thrown in low-memory situations."
  },
  "defaultConfiguration": {
    "level": "warning"
  },
  "helpUri": ")",
  "relationships": []
}

This leads to GitHub Code Scanning failing to accept the SARIF file:

Error: Unable to upload "flawfinder.sarif" as it is not valid SARIF:
- instance.runs[0].tool.driver.rules[8].helpUri does not conform to the "uri" format

Hi Expert,

I noticed that flawfinder will report security issue when using modern C++ std::istream::read().
https://en.cppreference.com/w/cpp/io/basic_istream/read

OpenAPITools/openapi-generator#10349

The check rule in Flawfinder is as below:

https://github.com/david-a-wheeler/flawfinder/blob/d9ddc06b7efea4c626205e5d39157255d112bfe4/flawfinder.py#L1282~L1287

    # fread not included here; in practice I think it's rare to mistake it.
    "getchar|fgetc|getc|read|_gettc":
    (normal, 1,
     "Check buffer boundaries if used in a loop including recursive loops (CWE-120, CWE-20)",
     "",
     "buffer", "dangers-c", {'input': 1}, "FF1029"),

Is this a false alert when using modern C++ std::istream::read() function?
If no, how to fix this issue?
https://github.com/OpenAPITools/openapi-generator/blob/master/samples/client/petstore/cpp-restsdk/client/ModelBase.cpp#L519

Regards,
Alex

Integrating flawfinder in my github repo I revieved a .sarif report generated, that has the following entry:

{
  "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json",
  "version": "2.1.0",
  "runs": [
    {
      "tool": {
        "driver": {
          "name": "Flawfinder",
          "version": "2.0.19",
          "informationUri": "https://dwheeler.com/flawfinder/",
          "rules": [
            {
[...]
              "id": "FF1057",
              "name": "misc/InitializeCriticalSection",
              "shortDescription": {
                "text": "Exceptions can be thrown in low-memory situations."
              },
              "defaultConfiguration": {
                "level": "warning"
              },
              "helpUri": ")",
              "relationships": []
            },

Uploading the result .sarif file caused an error in the workflow:

{
    "property": "instance.runs[0].tool.driver.rules[3].helpUri",
    "message": "does not conform to the \"uri\" format",
    "schema": {
      "description": "A URI where the primary documentation for the report can be found.",
      "type": "string",
      "format": "uri"
    },
    "instance": ")",
    "name": "format",
    "argument": "uri",
    "stack": "instance.runs[0].tool.driver.rules[3].helpUri does not conform to the \"uri\" format"
  }
Error: Unable to upload "/home/runner/work/wfn2fchk/wfn2fchk/flawfinder_results.sarif" as it is not valid SARIF:
- instance.runs[0].tool.driver.rules[3].helpUri does not conform to the "uri" format

Thank you for looking into this!

Hey there. I was told about your tool earlier today, and have already used it to identify some problem issues in code I'm looking at. However, I had to figure out the best way to convert the source files to UTF-8, in order to complete the testing. Maybe as you migrate further into using Python3, you can adopt use of another module to facilitate that transformation?

My project has lots of C code with embedded SQL that is processed by the Oracle Pro*C pre-compiler. We use the ".sc" filename extension for these files. Could you please add ".sc" to your c_extensions list? Or, could you add an option to modify recognized filename extensions?

Thanks...

saved hit file seems to be ignored when using --diffhitlist

To reproduce :

./flawfinder -S -C --quiet --omittime  --savehitlist /tmp/hits test.c
./flawfinder -S -C --quiet --omittime  --diffhitlist /tmp/hits test.c

The 2nd run should not find any hit, but it is.

hit.source_location defaults to 2.

But in the case of swprintf format string is argument 3. Thus it results in false positives.

Hit: [4] (format) swprintf: Potential format string problem (CWE-134). Make format string constant.

I have noticed that source code like “var = var + X” was specified at a few places so far.
I suggest to use augmented assignment statements instead.

Could you please address my stackoverflow issue?
https://stackoverflow.com/questions/64305167/how-to-limit-string-input-size-and-protect-it-from-overflow
Thanks!!
Lior

So far, I really like flawfinder. One of its limitations though is that it's somewhat difficult to integrate with linting plugins due to a lack of stream support, especially used when parsing buffers instead of saved files. The most common ways I've seen this done is either by accepting stream input or with a special "--stdin" flag.

"An svg icon file is required to show the flawfinder workflow in security tab." in GitHub actions per #49 This isn't an .ico file.

Do we have actual artistic capabilities? Otherwise I'll create a trivial icon, and we can replace it later when someone with talent can create it.

https://app.codacy.com/gh/vtorri/entice/issues?categoryType=Security

FCodacy findd potential security problems in strlen calls :

Does not handle strings that are not \0-terminated; if given one it may perform an over-read (it could cause a crash if unprotected) (CWE-126)

code :

if (!strncasecmp(arg, "file://", strlen("file://")))

but strlen("file://") is always 7 as we pass a static string which is always correctly 0-terminated

I have the following line in a C++ header file:

void HandleWrite(const boost::system::error_code &error);

Flawfinder gives me this output:

Agent.h:82:  [4] (shell) system:
  This causes a new program to execute and is difficult to use safely
  (CWE-78). try using a library call that implements the same functionality
  if available.

This is obviously wrong, it's just triggering on the word system even though it's just a namespace and not system().

Please add support of C++11.
We scanned our cpp project and faced issues that neither lambdas no string literals (https://en.cppreference.com/w/cpp/language/string_literal) are not parsed correctly by ./flawfinder.py
I can dd more error logs if needed, but I think it's clear from parsing cycles (they are too simple to catch complicated syntax with ; and " symbols)

On this line of code:

sscanf(ptr, "%08" PRIx32, &mask[i - 1]);

I'm getting a flagged:

The scanf() family's %s operation, without a limit specification, permits buffer overflows (CWE-120, CWE-20). 

The limit spec is there. "%08
The format is defined in: /usr/include/inttypes.h as # define PRIx32 "x"

The PRI format conversion specifies are a pretty common way of writing portable code now... They are part of the Open Group as part of Issue 5, released 1997. (So you know these reserved words are not a variable in the code).

I can understand why it should warn about using the PRI printing formats when I should be using the SCN scanning (should be using SCNx32) - but I think the existing fault is wrong. (I think).

Hi,

This is a nice little project. However, as you might know, regexes are not that great for finding flaws. Would you consider rewriting this tool to use a data-flow analysis platform like Joern? Joern is also open-source. Below is a link to the query database which should give an idea.

https://queries.joern.io

Thanks in advance.

Hi, I am using flawfinder to scan source code files in my mac terminal, but when I use the --csv option from the documentation, the hits are not being stored in a csv file. I am running the command flawfinder, then the name of the source code file in my directory followed by --csv, and the hits are displayed in my terminal but are not being saved to a csv file. Am I doing the command incorrectly?

It would be useful to have the capability to ignore test directories that are subdirectories to the source code.
i.e., --ignore=test
would ignore any subdirectory named "test" anywhere.

https://dwheeler.com/flawfinder/#downloading doesn't seem to have the 2.0.14 tarball.
The GitHub generated tarball appears to be missing the .ps files used in the makefile.
If I just use the setup.py I get: can't copy 'flawfinder.1.gz': doesn't exist or not a regular file

Since we only have the flawfinder.1 but not the generated tarball made with the makefile.

Maybe I'm misunderstanding how it should work?

Warn when PostgreSQL's PQExec is called with a non-constant, to warn about SQL injection.

The PostgreSQL libpq C interface provides several functions, as explained in the PostgreSQL (Command Execution Functions) documentation:

• PQexec directly runs a single string command and returns a result.
• PQexecParams implements a parameterized statement.
  Placeholders are represented in the command as $1, $2, etc.,
  and the parameter values are supplied as separate parameters in the same call.
• PQprepare implements a prepared statement.
  It takes a statement with placeholders and
  submits it to the database to be prepared.
  Users can later use the separate PQexecPrepared call
  to provide the placeholder parameter values and execute the resulting
  command.

Users should use the latter two for non-constant queries.

Flawfinder was integrated in our CI pipeline. It fails with the directories that have symlinks. "scp" directory is successful as it does not symlinks and "sdk" directory fails silently.

`$ mkdir flawfinder_report

38$ find /builds/scp/scpfirmware/scpos14ask/charter-scp-device/scp -type l

39$ flawfinder --minlevel=4 --html --context /builds/scp/scpfirmware/scpos14ask/charter-scp-device/scp > flawfinder_report/flawfinder_test_report_scp.html

40Warning: Skipping directory with initial dot /builds/scp/scpfirmware/scpos14ask/charter-scp-device/scp/.gitlab

[41](https://<gitlab>/scp/scpfirmware/scpos14ask/charter-scp-device/-/jobs/1868651#L41)$ find /builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk -type l

42/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/skales/boards/8996

43/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/skales/boards/7230

44/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/skales/boards/7x30

45/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/skales/boards/8x60

46/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/skales/boards/g1

47/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/skales/boards/7201a

48/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/package/network/config/netifd/files/sbin/ifdown

49/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/wireshark-github/.bzrignore

50/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/include/dt-bindings/input/linux-event-codes.h

51/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/vphn/vphn.c

52/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/vphn/vphn.h

53/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/primitives/asm/asm-compat.h

54/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/primitives/word-at-a-time.h

55/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/copyloops/copyuser_power7.S

56/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/copyloops/memcpy_power7.S

57/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/copyloops/memcpy_64.S

58/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/copyloops/copyuser_64.S

59/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/tools/testing/selftests/powerpc/stringloops/memcmp_64.S

60/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/arch/mips/boot/dts/include/dt-bindings

61/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/arch/metag/boot/dts/include/dt-bindings

62/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/arch/arm64/boot/dts/include/dt-bindings

63/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/arch/arm64/boot/dts/arm/vexpress-v2m-rs1.dtsi

64/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/arch/cris/boot/dts/include/dt-bindings

65/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/arch/powerpc/boot/dts/include/dt-bindings

66/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/arch/arm/boot/dts/include/dt-bindings

67/builds/scp/scpfirmware/scpos14ask/charter-scp-device/sdk/qualcomm_sdk/qsdk/qca/src/linux-4.4/arch/arm/boot/dts/sun8i-a33-et-q8-v1.6.dts

68

Running after_script

00:01

84Running after script...

85$ echo "completed"

86completed

87

Cleaning up project directory and file based variables

00:01

88ERROR: Job failed: exit code 1`

Similar to #11, I would like to propose to move all documentation to a new subdirectory called docs. If I got it correctly, this directory will only contain flawfinder.1. This directory could also contain a makefile to build the documentation.

I admit, it might be disputed to create one directory containing one file, but I see it as an API for a new developer trying to join this project and reading (/searching) the documentation is probably one of the most important things in the beginning and should be as easy as possible. Additionally, a manpage is perhaps (! - this might be wrong) not the most prominent documentation language for a Python-based project and hence, makes it harder to spot it at first.

If you agree with this suggestion, I will create a pull-request with the related changes.

First-of-all: Thank you for FlawFinder! I find it very useful.

My main development is on Windows machines and currently flawfinder works good on Unix/OSX and reported to work on Cygwin. However I would like to use it as a regular executable in my path as well. Currently FlawFinder uses a shebang and is a executable script.

As explained here by the people at click. For a good cross-platform experience, a script shouldn't rely on shebangs but on entry_points in setup.py:

The magic is in the entry_points parameter. Below console_scripts, each line identifies one console script. The first part before the equals sign (=) is the name of the script that should be generated, the second part is the import path followed by a colon (:) with the Click command.

As a proof-of-concept I've created a fork and implementation in a fork.

This would make it possible to do the following on Windows (without relying on cygwin or msys2):

• pip install . (inside a cloned repo)
• pip install flawfinder (to install it from PyPi)

And then you can call flawfinder inside any cmd or Powershell session, without breaking current unix/osx functionality.

Similar to #11 and #12, I would like to propose to move the source code (right now only flawfinder) into a separate folder called flawfinder/. This would lead to a directory structure which clearly separates source code, tests and documentation from meta data (such as Contribution.md, License etc.).

This structure is also used by e.g. PyPa's sample project or Open Sourcing a Python Project the Right Way.

Problem

Flawfinder includes plaintext error messages in CSV output, which prevents one from parsing it as CSV.

I can't reopen #46 so I'm creating this new issue. Feel free to close it and reopen #46 .

In flawfinder 2.0.18 the error messages are still mixed with the CSV output.

Could this small change fix the problem?:

(in lib/python3.6/site-packages/)
diff -cw flawfinder.py.orig flawfinder.py
*** flawfinder.py.orig 2021-07-21 14:47:55.037477771 -0400
--- flawfinder.py 2021-08-18 09:30:34.945143483 -0400
***************
*** 675,681 ****

def internal_warn(message):
! print(h(message))

# C Language Specific
--- 675,682 ----

def internal_warn(message):
! # print(h(message))
! print(h(message), file=sys.stderr)

# C Language Specific

This is the CSV output showing the problem:

File,Line,Column,DefaultLevel,Level,Category,Name,Warning,Suggestion,Note,CWEs,Context,Fingerprint,ToolVersion,RuleId,HelpUri
Parsing failed to find end of parameter list; semicolon terminated it in ((buf)
# endif

/* copy string (truncating the result, strings must not be NULL) */
#if _MSC_VER >= 1400
# define soap_strcpy(buf, len, src) (void)strncpy_s((buf), (len), (src), _TRUNCATE)
#elif defin
./claims/tools/src/tools.c,4232,9,5,5,buffer,gets,"Does not check for buffer overflows (CWE-120, CWE-20).",Use fgets() instead.,,"CWE-120, CWE-20", gets(buffer);,8574681bcf016b459efe0a123d75643927688f096b753ca762978b7c7aa81c29,2.0.18,FF1014,https://cwe.mitre.org/data/definitions/120.html

Validity Period
  Issued On	Wednesday, January 16, 2019 at 8:00:00 AM
  Expires On	Thursday, January 28, 2021 at 7:59:59 AM

It's Jan 28, 2021 today. Would you like to renew it?

As we all know, most open source code on the Internet (and GitHub specifically) is malware, mostly spyware.

Before or after every Git clone you have to spend a lot of time checking the source code for malicious code. I've been searching for a way to automate that. Can this project be extended to that?

I suggest to reuse a higher level build system than your current make script so that powerful checks for software features will become easier.

• CMake
• Autotools

Been using the tooling for years....this is a must have in our code electron microscope ;)

Any suggestions to output to JSON?

The answer “Flawfinder works by using a built-in database of C/C++ functions with well-known problems, …” was puplished.

Can these source code analysis approaches be structured so that they can be better described and extended?

Downloaded source package from https://dwheeler.com/flawfinder/flawfinder-2.0.16.tar.gz as directed by website https://dwheeler.com/flawfinder/

Was trying to understand why on windows the <pythondir>/scripts/flawfinder file was now <pythondir>/scripts/flawfinder.exe
(Should this be an issue as well, or is this expected?

Found that the 2.0.16 source download doesn't contain flawfinder or flawfinder script at all.

Expect this was due to recent refactor, which left this behind in the makefile:

distribute: clean flawfinder.pdf flawfinder.ps
	...<snip>...
	# We don't need both "flawfinder" and "flawfinder.py":
	rm -f ,tempdir/flawfinder.py

which removes flawfinder.py (which in v2.0.15 was named just flawfinder

Issue

Flawfinder generates artifact locations starting with a / slash or ./. At the same time and for portability reasons Flawfinder uses the uriBaseId property in SARIF. However, the idea of the uriBaseId is, that it is a placeholder to allow the artifactLocation to be relative. The placeholder of the SRCROOT can be replaced easily to make absolute paths.

Example

Test project: https://github.com/pwk4m1/Damn_Vulnerable_Device_Driver

Calling Flawfinder like this:

flawfinder --sarif .

Will result in:

{
  "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json",
  "version": "2.1.0",
  "runs": [
    {
      "tool": {
        "driver": {
          "name": "Flawfinder",
          "version": "2.0.19",
          "informationUri": "https://dwheeler.com/flawfinder/",
          "rules": [
            {
              "id": "FF1022",
              "name": "buffer/strlen",
              "shortDescription": {
                "text": "Does not handle strings that are not \\0-terminated; if given one it may perform an over-read (it could cause a crash if unprotected) (CWE-126)."
              },
              "defaultConfiguration": {
                "level": "note"
              },
              "helpUri": "https://cwe.mitre.org/data/definitions/126.html",
              "relationships": [
                {
                  "target": {
                    "id": "CWE-126",
                    "toolComponent": {
                      "name": "CWE",
                      "guid": "FFC64C90-42B6-44CE-8BEB-F6B7DAE649E5"
                    }
                  },
                  "kinds": [
                    "relevant"
                  ]
                }
              ]
            },
            {
              "id": "FF1015",
              "name": "format/sprintf",
              "shortDescription": {
                "text": "Potential format string problem (CWE-134)."
              },
              "defaultConfiguration": {
                "level": "error"
              },
              "helpUri": "https://cwe.mitre.org/data/definitions/134.html",
              "relationships": [
                {
                  "target": {
                    "id": "CWE-134",
                    "toolComponent": {
                      "name": "CWE",
                      "guid": "FFC64C90-42B6-44CE-8BEB-F6B7DAE649E5"
                    }
                  },
                  "kinds": [
                    "relevant"
                  ]
                }
              ]
            }
          ],
          "supportedTaxonomies": [
            {
              "name": "CWE",
              "guid": "FFC64C90-42B6-44CE-8BEB-F6B7DAE649E5"
            }
          ]
        }
      },
      "columnKind": "utf16CodeUnits",
      "results": [
        {
          "ruleId": "FF1022",
          "level": "note",
          "message": {
            "text": "buffer/strlen:Does not handle strings that are not \\0-terminated; if given one it may perform an over-read (it could cause a crash if unprotected) (CWE-126)."
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "./Damn_Vulnerable_Device_Driver/DVDD.c",
                  "uriBaseId": "SRCROOT"
                },
                "region": {
                  "startLine": 102,
                  "startColumn": 6,
                  "endColumn": 28,
                  "snippet": {
                    "text": "\tif (strlen(buff) != len) {"
                  }
                }
              }
            }
          ],
          "fingerprints": {
            "contextHash/v1": "e4347673c7b5f2f2fe6fbcca2fce047b1823698f122199f1f2c7c602f62d581f"
          },
          "rank": 0.2
        },
        {
          "ruleId": "FF1015",
          "level": "error",
          "message": {
            "text": "format/sprintf:Potential format string problem (CWE-134)."
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "./Damn_Vulnerable_Device_Driver/DVDD.c",
                  "uriBaseId": "SRCROOT"
                },
                "region": {
                  "startLine": 106,
                  "startColumn": 2,
                  "endColumn": 34,
                  "snippet": {
                    "text": "\tsprintf(diode->data, buff, len);"
                  }
                }
              }
            }
          ],
          "fingerprints": {
            "contextHash/v1": "f6bfebb1bd7afaa981eb091cc3ddf89686b6dc50e4ede92d76c48046e4ae266c"
          },
          "rank": 0.8
        }
      ],
      "externalPropertyFileReferences": {
        "taxonomies": [
          {
            "location": {
              "uri": "https://raw.githubusercontent.com/sarif-standard/taxonomies/main/CWE_v4.4.sarif"
            },
            "guid": "FFC64C90-42B6-44CE-8BEB-F6B7DAE649E5"
          }
        ]
      }
    }
  ]
}

Instead of "uri": "./Damn_Vulnerable_Device_Driver/DVDD.c", it should be "uri": "Damn_Vulnerable_Device_Driver/DVDD.c",`.

Solution

Improve the SARIF output by removing the slash / in front of the result.

              "physicalLocation": {
                "artifactLocation": {
                  "uri": "Damn_Vulnerable_Device_Driver/DVDD.c",
                  "uriBaseId": "SRCROOT"
                },

Problem

Flawfinder includes plaintext error messages in CSV output, which prevents one from parsing it as CSV.

Flawfinder manpage says the --csv option will:

Generate output in comma-separated-value (CSV) format. This is the recommended format for sending to other tools for processing. It will always generate a header row, followed by 0 or more data rows (one data row for each hit). Selecting this option automatically enables --quiet and --dataonly`.

However, if theres a "Parsing failed to find..." error during the flawfinder scan, this error message is included in the output between the header row and data rows.

When sending this CSV to tool other tools for processing, they fail to parse just after the header because of the error message.

Steps to reproduce

1. docker run -it python:3 bash
2. pip install flawfinder
3. echo $'# define SOAP_SNPRINTF_SAFE(buf, len) void)sprintf((buf)\n;' > test.h
4. flawfinder --csv test.h

Output:

root@python3:/# flawfinder --csv test.h
File,Line,Column,Level,Category,Name,Warning,Suggestion,Note,CWEs,Context,Fingerprint
Parsing failed to find end of parameter list; semicolon terminated it in ((buf)
;

test.h,1,45,4,buffer,sprintf,Does not check for buffer overflows (CWE-120),"Use sprintf_s, snprintf, or vsnprintf",,CWE-120,"#  define SOAP_SNPRINTF_SAFE(buf, len) void)sprintf((buf)",5bc94035b35d1ac8aec5dacd2cbde0fee2867a6d9610b3c0aed514824bd0eb1c

Proposal

Flawfinder's --csv option should only output CSV to STDOUT.

Basically flawfinder --csv . | grep ',' but native and in Python. 😅 🐍

Context

• https://gitlab.com/gitlab-org/gitlab/-/issues/327032
• https://gitlab.com/gitlab-org/gitlab/-/issues/330018

I have noticed that plus operators were specified between string literals of Python source code.
I find such specifications unnecessary because the programming language “Python” supports such concatenation without the usage of extra operators directly.

Given this source file:

#include <stdio.h>
#include <inttypes.h>

int main(char *arg)
{
	sscanf(arg, PRIu32);
	return 0;
}

Flawfinder 2.0.17 outputs:

FINAL RESULTS:

test.c:6:  [4] (buffer) sscanf:
  The scanf() family's %s operation, without a limit specification, permits
  buffer overflows (CWE-120, CWE-20). Specify a limit to %s, or use a
  different input function. If the scanf format is influenceable by an
  attacker, it's exploitable.

Note: no issue is generated if line 4 is changed to e.g. sscanf(arg, "I32u");

Hello! The 'Static Analysis Results Interchange Format' (SARIF) is an OASIS Standard proposed by Microsoft and designed in collaboration with many industry/government partners. The standard is universally adopted in Microsoft's public and internal tooling eco-system. GitHub requires SARIF ingestion for its Advanced Security offering.

This feature and Microsoft's ever-increasing use of (cross-platform) open source static analysis tooling has led us to propose and help drive SARIF support for various projects, most recently Brakeman and Spotbugs.

There's been a notable uptake of FlawFinder use at Microsoft and so we recently authored an open source FlawFinder CSV to SARIF converter. Even better, of course, would be direct support for producing SARIF in the FlawFinder tool. :)

The nice thing about having access to an existing converter is that it can be applied to any FlawFinder CSV to demonstrate equivalent SARIF, which should help minimize work required to understand this format (which is complex). The SARIF home page also contains link to other useful things, including a SARIF tutorials repo, an online validator for candidate SARIF logs, pointers to the actual specification, etc.

Any thoughts on implementing this feature and/or accepting a contribution for it?

@jhutchings1

The CONTRIBUTING.md states:

To make the program easy to install everywhere, the main executable is exactly one self-contained file.

This is definitely a valid argument. Still, I think you spent a lot of effort in making the installation as easy as possible using common tools like pip or apt. Additionally, 2k LoC are really not easy to parse (from a human perspective). Hence I would argue that it is already easy enough to install flawfinder in comparison to the readability of the source code. Thus, it might be more important to improve latter.

How do you think about this?

Here are the steps to repeat this using the Docker image supplied here:

Run the image:

docker run -it registry.gitlab.com/gitlab-org/security-products/analyzers/flawfinder:2 /bin/sh

Enter the following in the container:

apk add git
git clone https://github.com/microsoft/vcpkg.git
export SECURE_LOG_LEVEL=debug
cd vcpkg
../analyzer run

The following (truncated) output is generated:

[INFO] [Flawfinder] [2020-12-02T21:35:02Z] > GitLab Flawfinder analyzer v2.12.0
[INFO] [Flawfinder] [2020-12-02T21:35:02Z] > Detecting project
[INFO] [Flawfinder] [2020-12-02T21:35:02Z] > Found project in /vcpkg/ports/alac-decoder
[INFO] [Flawfinder] [2020-12-02T21:35:02Z] > Running analyzer
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
Error: File ended while in string.
[DEBU] [Flawfinder] [2020-12-02T21:35:17Z] > /usr/local/bin/flawfinder -m 1 --csv .
File,Line,Column,Level,Category,Name,Warning,Suggestion,Note,CWEs,Context,Fingerprint
Parsing failed to find end of parameter list; semicolon terminated it in (
            lhs.begin(), lhs.end(), rhs.begin(), rhs.end(), [](const std::string& lhs, const std::string& rhs) {
                return Strings::trim(StringView(lhs)) == Strings::trim(StringView(rhs
Parsing failed to find end of parameter list; semicolon terminated it in (lhs.feature_paragraphs.begin(),
                          lhs.feature_paragraphs.end(),
                          rhs.feature_paragraphs.begin(),
                          rhs.feature_paragraphs.end(
...
(list of hits)
...
[INFO] [Flawfinder] [2020-12-02T21:35:17Z] > Creating report
[FATA] [Flawfinder] [2020-12-02T21:35:17Z] > record on line 2: wrong number of fields

The report generation fails due to the two parsing errors. The second one can be found starting on line 63 in vcpkg/toolsrc/src/vcpkg/sourceparagraph.cpp:

return std::equal(lhs.feature_paragraphs.begin(),
                  lhs.feature_paragraphs.end(),
                  rhs.feature_paragraphs.begin(),
                  rhs.feature_paragraphs.end(),
                  [](const std::unique_ptr<FeatureParagraph>& lhs,
                     const std::unique_ptr<FeatureParagraph>& rhs) { return *lhs == *rhs; });

The error results from the parsing being terminated by the ; within the body of the lambda. I believe a potential solution is to update extract_c_parameters to track the current curly brace level and ignore the occurrences of ; at a level of 1 or greater. I'll test this and submit a pull request if successful.

-Ken