https://github.com/indygreg/PyOxidizer

Add E2E tests with real binaries, and test JSON output

The main progress bar total count is the total of the files walked in the filesystem:

count = sum(1 for i in walk_filepath_list(filepath_list, recursive))
check_output.enumerating_tasks_stop(count)

It's not representative of the smaller amount of files that will be analyzed by checksec (ELF/PE)

TODO

• publish on Pypi
• JSON export
• hide errors
• documentation
• ci badge
• pypi package badge
• python version badge

│ /home/wenzel/local/pe/win7/System32/DriverStore/FileRepository/prnca00z.inf_amd64_ne… │ No │

When using checksec.py to output to JSON format, the output does not show the same level of detail as the rich output formatter. Specifically, the HighEntropyVA and SafeSEH results for a binary will default to True or False and not "/" as in the rich output formatter. The same checks to determine the architecture and determine whether these flags should be implemented on the JSON formatter.

https://docs.oasis-open.org/sarif/sarif/v2.0/csprd01/sarif-v2.0-csprd01.html

Some of the "ELF" safety mechanisms are actually quite generic and found in other formats. Like PE, if you use GCC to compile an exe:

# This is MSYS2, "MSYS" environment.
printf '%s\n%s' '#include <stdio.h>' 'int main(){char e[64]; gets(e); puts(e);}' > 1.c
gcc -O2 -fstack-protector-all 1.c
nm a.exe | grep stack

You get:

0000000100403010 r .rdata$.refptr.__stack_chk_guard
0000000100403010 R .refptr.__stack_chk_guard
0000000100403010 R __fu0___stack_chk_guard
00000001004080e4 I __imp___stack_chk_fail
00000001004080ec I __imp___stack_chk_guard
00000001004080ec I __imp___stack_chk_guard
000000010040819c I __nm___stack_chk_guard
0000000000001000 A __size_of_stack_commit__
0000000000200000 A __size_of_stack_reserve__
00000001004010a0 T __stack_chk_fail

(Well, it's exactly one of the ELF mechanisms for now. But there are some not-yet-implemented things like SafeStack and selfrando that will also use simple symbol or section existence. Maybe clang CFI too...)

if lief.ELF.DYNAMIC_FLAGS.BIND_NOW in self.bin.get(lief.ELF.DYNAMIC_TAGS.FLAGS): in elf.py line 121

Glibc supports ELF.DYNAMIC_FLAGS_1.NOW to indicates BIND_NOW, I guess this case should be supported
otherwise the results of RELRO will differ from checksec.sh

• No PIE
• PIE enabled
• DSO
• REL

Hi, can you implement checksec.py in bash script? Maybe using such commands: readelf, objdump and grep.

I don't think the "Force Integrity" check is working. It appears to report everything as "yes" regardless of what the target has.
For testing purposes, on a Windows 10 box, an example file with Force Integrity enabled is c:\windows\system32\CameraSettingsUIHost.exe

An example file without Force Integrity enabled is c:\windows\system32\calc.exe

But checksec.exe says "yes" to both:

Add tests in Github CI with pypy interpreter
https://www.pypy.org/

Hi,

as suggested by @Wenzel , here a brainstorming post regarding a potentially interesting new feature:

Adding checks to detect:
-unstripped symbols
-unremoved debug information
-unremoved profiling instrumentation
-unremoved coverage instrumentation
-unremoved embedded LLVM bitcode
...

Rationale:
Besides checking for compiler hardening flags, it could be useful for developers to check for above points, before releasing a binary as to not disclose sensitive information.
On the other hand, this can be also useful for reverse engineering, as those points may give further inside into the code.

Implementation:
Check for presence of corresponding section/segment names "debug_", "_llvm_prf", ... or certain symbols "_llvm_profile*", ...

Agree/Disagree/Discussion or concrete ideas for checks?

It's currently the only stuff checksec.sh has on the chart that this thing does not. And compared to other things it's actually quite easy with a wrapper function in the shell. Not so easy in python though since you got to change the file input bits -- I figured that this could be a reason for not implementing it.

Something like:

scan_proc(){ local pid=$1; shift; sudo checksec /proc/$1/exe "$@"; }
scan_proc_libs(){
   # yes this final expansion is fragile. I don't care
   sudo bash -c '
     pid=$1
     shift
     checksec $(tr -s ' ' < /proc/$pid/maps | cut -d ' ' -f 6 | sort -u | xargs file | grep ELF | cut -d: -f1) "$@"'
} 

Also, the terminal is not restore, the progress bar is waiting to be updated. Related issue: #31

Some help from winchecksec:
https://github.com/trailofbits/winchecksec/blob/v2.0.0/checksec.cpp#L172

• nx: has_nx
• pie: is_pie
• /GS: https://stackoverflow.com/a/26498521/3017219
• ALSR
  • /DYNAMICBASE
  • /HIGHENTROPYVA
• Code integrity/signing
  • /INTEGRITYCHECK
  • Authenticode
• FORCE_INTEGRITY
• /ALLOWISOLATION
• SEH
• SafeSEH
• Control Flow Guard
• Return Flow Guard: https://github.com/trailofbits/winchecksec/pull/11/files

https://clang.llvm.org/docs/ControlFlowIntegrity.html

See this thread: https://twitter.com/DanielMicay/status/1312871240798023680?s=20

https://clang.llvm.org/docs/ShadowCallStack.html

The LIEF can be surprisingly slow on big binaries.

One example: pandoc (118 MB)

upstream issue: lief-project/LIEF#442

Article by Tencent lab:
https://xlab.tencent.com/en/2016/11/02/return-flow-guard/

Implementation by winchecksec:
https://github.com/trailofbits/winchecksec/blob/v2.0.0/checksec.cpp#L262

Got error while running checksec.exe (v0.7.1) on win11. Not sure if it support windows 11.
Just dont understand why is it looking for linux /lib/libc.so? Or i have missed out some important steps?

2023-05-24 00:45:15,756 DEBUG:root:Libc object not set
2023-05-24 00:45:15,756 DEBUG:root:Finding libc path: ldconfig
2023-05-24 00:45:15,756 DEBUG:root:Finding libc path: ldd
2023-05-24 00:45:15,771 DEBUG:root:Finding libc path: hardcoded paths
2023-05-24 00:45:15,772 DEBUG:root:Testing libc at /lib/libc.so.6
Can't open '\lib\libc.so.6'
2023-05-24 00:45:15,773 DEBUG:root:Testing libc at /lib/libc.so.7
Can't open '\lib\libc.so.7'
2023-05-24 00:45:15,779 DEBUG:root:Testing libc at /lib/libc.so
Can't open '\lib\libc.so'
2023-05-24 00:45:15,782 DEBUG:root:Testing libc at /lib64/libc.so.6
Can't open '\lib64\libc.so.6'
2023-05-24 00:45:15,785 DEBUG:root:Testing libc at /lib/i386-linux-gnu/libc.so.6
Can't open '\lib\i386-linux-gnu\libc.so.6'
2023-05-24 00:45:15,787 DEBUG:root:Testing libc at /lib/x86_64-linux-gnu/libc.so.6
Can't open '\lib\x86_64-linux-gnu\libc.so.6'
2023-05-24 00:45:15,792 DEBUG:root:Testing libc at /lib/arm-linux-gnueabihf/libc.so.6
Can't open '\lib\arm-linux-gnueabihf\libc.so.6'
2023-05-24 00:45:15,795 DEBUG:root:Testing libc at /lib/aarch64-linux-gnu/libc.so.6
Can't open '\lib\aarch64-linux-gnu\libc.so.6'
2023-05-24 00:45:15,796 DEBUG:root:Testing libc at /usr/x86_64-gentoo-linux-musl/bin/ld
Can't open '\usr\x86_64-gentoo-linux-musl\bin\ld'
2023-05-24 00:45:15,798 DEBUG:root:Failed to init Libc object: Cannot find a suitable libc path on your system
2023-05-24 00:45:15,799 DEBUG:root:{'libc': None}
2023-05-24 00:45:15,800 DEBUG:root:Could not locate libc. Skipping fortify tests for ELF.
Can't open '\lib\libc.so.6'
Can't open '\lib\libc.so.7'
Can't open '\lib\libc.so'
Can't open '\lib64\libc.so.6'
Can't open '\lib\i386-linux-gnu\libc.so.6'
Can't open '\lib\x86_64-linux-gnu\libc.so.6'
Can't open '\lib\arm-linux-gnueabihf\libc.so.6'
Can't open '\lib\aarch64-linux-gnu\libc.so.6'
Can't open '\usr\x86_64-gentoo-linux-musl\bin\ld'
2023-05-24 00:45:16,240 DEBUG:root:test_file\app.po.ts: Not an ELF/PE. Skipping

I ran into a bit of an issue with this library being licensed under the GPL-3.0; currently I'm in the process of releasing some code as open source under a (likely to be) more permissive license (e.g. MIT/BSD-3), but while reviewing libraries, I realized that one of the examples for how to write a plugin uses checksec.py as a dependency. As a result of the GPL license being infectious, I believe that means we are effectively being forced to release the software (or the example plugin) under the GPL-3.0 license as well.

Unless the goal by choosing the GPL license for checksec.py is to completely block any closed-source (written in Python, heh) commercial products from being released, LGPL seems like it would be a better fit to ensure changes commercial software makes to checksec.py are available to their users, while not making a complicated mess to figure out licenses for other open source projects that use it as a dependency (and thus their users).

That said, in our case given that it is a plugin example (completely optional and easy to pull out into its own separate repository), our workaround is probably going to be to release just that plugin example as a whole under the GPL-3.0 to satisfy the checksec.py license requirements, but with the code we wrote for the plugin example (which is all of it since it uses a pip installed copy of checksec.py as a library) under a permissive license so our users can freely use the example code we wrote without being affected by the virality of the GPL-3.0 (unless they also use checksec.py as a dependency for their plugin).

The other possibility is that we keep that example plugin as internal only, though that seems like a lose-lose situation all around.

https://docs.microsoft.com/en-us/cpp/build/reference/cetcompat?view=vs-2017

https://techcommunity.microsoft.com/t5/windows-kernel-internals/understanding-hardware-enforced-stack-protection/ba-p/1247815

see lief-project/LIEF#537

Hi,

I work for Keysight Technologies. We're creating a tool to create an inventory of the various security related properties of the files. We've been asked to collect the checksum and filetype in addition to the information your tool is collecting. Would you be open to accepting a PR adding the checksum and filetype based on command line flags? For the file type, I would use the python-magic library which will introduce a depencency you may or may not be comfortable with so I wanted to call that out.

LIEF 0.11.0 added support for Authenticode signature checking:
https://lief.quarkslab.com/blog/2021-01-19-lief-0-11-0/

When comparing results against checksec.sh, I noticed that checksec.py reports PIE: No whereas checksec.sh reports PIE: DSO for the following Ubuntu 20.04 binaries:

• /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (should already be present in any x86_64 Ubuntu 20.04 distro)
• /lib/x86_64-linux-musl/libc.so (installable via apt install musl-dev)

I believe checksec.py is wrong and checksec.sh is correct here because the above binaries are reported as shared objects by the file command.

I think the key feature with the above two binaries is that they do not contain an interpreter path, which looks to be a deciding factor in the LIEF code for whether the binary is PIE:

https://github.com/lief-project/LIEF/blob/255d09c92d09ebacb9473744550e1b90a8f4fd4e/src/ELF/Binary.cpp#L1184

I think this issue could be worked around in checksec.py by looking first to determine whether the binary is a shared object or not before checking with LIEF.

I wanted to get your opinion first before also potentially opening an issue on the LIEF repo.

https://lwn.net/Articles/761462/
"The GNU C Library can now be compiled with support for Intel CET, AKA Intel Control-flow Enforcement Technology."

release #57 broke the fortify scores somehow.

The str representation of ErrorParsingFailed needs to be changed

• https://github.com/etke/checksec.rs
• https://github.com/koczkatamas/checksec-js/tree/master/examples

Name                   Stmts   Miss Branch BrPart  Cover
--------------------------------------------------------
checksec/__init__.py       0      0      0      0   100%
checksec/__main__.py      93     93     40      0     0%
checksec/binary.py        19      7      4      1    57%
checksec/elf.py          150     87     38      0    34%
checksec/errors.py         6      2      0      0    67%
checksec/output.py       213    213     74      0     0%
checksec/pe.py            53     53      6      0     0%
checksec/utils.py         72     57     32      0    14%
--------------------------------------------------------
TOTAL                    606    512    194      1    12%

Coverage very low, and even pe.py is at 0%, which makes no sense
The fix --concurrency=multiprocessing doesn't seem to work.

Related: https://coverage.readthedocs.io/en/coverage-5.1/subprocess.html