xoreaxeaxeax / sandsifter Goto Github PK

View Code? Open in Web Editor NEW

This project forked from battelle/sandsifter

4.9K 4.9K 351.0 5.04 MB

The x86 processor fuzzer

License: BSD 3-Clause "New" or "Revised" License

Makefile 1.08% Python 66.42% Shell 0.50% C 32.00%

sandsifter's People

Contributors

Stargazers

Watchers

Forkers

maldiohead chubbymaggie viafreekab knooow cgiogkarakis miguel550 johnjohnsp1 yhzx2013 suriya73 purpleidea tisma rupran zaolin paddymahoney kelu124 simple555a paulmenzel shengbinzhou zeptancode shekkbuilder 3bm fishline pythonus idkwim benzaku sysopfb girror cnstudios 4144 puppycodes hloeffler nkiraly rootless4real jaggedsoft hotelzululima olivierh59500 motte daveti techlord-rce lysunder thomasking2014 michaeljclark sanguinawer jameslinus kputtur outlaw6 zhengsyou artisdom johnhubcr online-9 humeafo varunram hbcbh1999 hhy5277 davehardy20 nsk own2pwn neo4reo duvitech-llc openube nimdakey voidpx shekharsorot je2854 zephrax dafyddcrosby samuell ieswxia security-goodies sebasstrogg ttamna sec-db dfirgeek mctavi5h threatinteltest opntr theuncanny dimchansky tweksteen cloudgate313 ayowel rigred yashasvisharma lewiscowles1986 joncampbell123 tchernicum yikez978 enascimento bellcjt calebcase jesse-propfi esotericnomen reality9 linky-fan ozmitter efluffy carbolymer chenglujin matepeter90 initpwn

sandsifter's Issues

Sandsifter stops running. Capstone installed via pip.

I downloaded a copy of sandsifter from github. I then compiled with sudo make -j8.
sudo make -j8 cc -c injector.c -o injector.o -Wall injector.c:321:2: warning: excess elements in array initializer [enabled by default] .start={.bytes={0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, .len=0}, ^ injector.c:321:2: warning: (near initialization for 'total_range.start.bytes') [enabled by default] injector.c:322:2: warning: excess elements in array initializer [enabled by default] .end={.bytes={0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff}, .len=0}, ^ injector.c:322:2: warning: (near initialization for 'total_range.end.bytes') [enabled by default] cc injector.o -O3 -Wall -l:libcapstone.a -o injector -pthread

I ran sandsifter and it gave me this error.
sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t Traceback (most recent call last): File "./sifter.py", line 842, in <module> main() File "./sifter.py", line 815, in main stderr=subprocess.PIPE File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory
I ran it again but now it gave me a new problem. Sandsifter opens but it doesn't continue to run.
`sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t

./sifter.py --unk --dis --len --sync --tick -- -P1 -t

./injector -P1 -t -t -R -0 -s 2540215650

insn tested: 0

artf found: 0

runtime: 00:00:00.05

seed: 2540215650

arch: 64

date: 2017-09-02 21:39:54

cpu:

processor : 0

vendor_id : GenuineIntel

cpu family : 6

model : 45

model name : Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz

stepping : 7

microcode : 4294967295

v l s c`

Crash near 660fa4c40d

sudo ./sifter.py --unk --dis --len --sync --tick --save -- -P1 -t -i 660fa4c40d0000000000000000000000

If I unpause the program will crash to commandline faster than I can can see.
In my first run which took almost 4 hours it ended up probably at this exact moment too, maybe already before that though, since there was some weird stuff going on in the few seconds before the crash.

I since I didn't use the --save command I tried botching a resume file and I got some overflow error that looked funny so I renamed the data-folder to give the program a fresh start.

Core i5 3450.
I guess there is some instruction here that ends up crashing it.

Generate basic docs from summarizer; put in wiki; crowdsource more detailed docs

Maybe someone will come along and doc everything.

make fail - injector

Hi,
using Manjaro Linux
but the build fail.

python-capstone
python2-capstone
Python 2.7.13
Python 3.6.2

What did i wrong ?

$ make
cc -c injector.c -o injector.o -Wall
injector.c:321:93: warning: excess elements in array initializer
.start={.bytes={0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, .len=0},
^~~~
injector.c:321:93: note: (near initialization for 'total_range.start.bytes')
injector.c:322:91: warning: excess elements in array initializer
.end={.bytes={0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff}, .len=0},
^~~~
injector.c:322:91: note: (near initialization for 'total_range.end.bytes')
cc injector.o -O3 -Wall -l:libcapstone.a -o injector -pthread
/usr/bin/ld: injector.o: relocation R_X86_64_32S against undefined symbol `dummy_stack' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
make: *** [Makefile:35: injector] Error 1

ask for good command line in end

Helo dear IT worker,

If I do the command line sandsifter with -X It will remove bad thing in my processor ?

"sudo ./sifter.py --unk --dis --len --sync --tick --low-mem -- -P1 -t -N -X"

Works in progress...

Thank you to bring your brain with me.

Regards.

RWX mprotect

I am using Gentoo Hardened (and so I have W^X protection with the PaX/grsecurity patches).

$ su -c './sifter.py --unk --dis --len --sync --tick -- -P1 -t'
Password:
injector: injector.c:1410: int main(int, char **): Assertion `!mprotect(packet_buffer,PAGE_SIZE,PROT_READ|PROT_WRITE|PROT_EXEC)' failed.
$ su -c dmesg
Password:
…
[246009.553043] grsec: denied RWX mprotect of <heap> by /mnt/gentoo/home/haelwenn/Sources/git/github.com/xoreaxeaxeax/sandsifter/injector[injector:5204] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/python2.7[python2:5201] uid/euid:0/0 gid/egid:0/0
[246009.553135] grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /mnt/gentoo/home/haelwenn/Sources/git/github.com/xoreaxeaxeax/sandsifter/injector[injector:5204] uid/euid:0/0 gid/egid:0/0,
parent /usr/bin/python2.7[python2:5201] uid/euid:0/0 gid/egid:0/0

summarize fails to parse objdump output

The output of my objdump for a certain instruction the sifter found looks like:

o:     file format binary


Disassembly of section .data:

00000000 <.data>:
   0:	prefetch BYTE PTR [rax]

However the things that try to parse it "swallow" the instruction, in particular the sed ones which I am not at all sure what they are supposed to do

256-color terminal required (make it work with putty)

Hello

I get errors when running the program from putty: terminal output is messed up and I get this message:

_curses.error: init_pair() returned ERR

as explained here: https://stackoverflow.com/questions/18551558/how-to-use-terminal-color-palette-with-curses

I had TERM=xterm-256color

To make it work I had to set TERM=xterm-256color

You could add a warning that the TERM value is not OK, or fall back to simpler colors.

Requires an outrageous amount of RAM to summarize

I have 32GB (with an 8GB VM running, 24GB available) and I ran out of memory in summarize.py!

_curses.error: init_pair() returned ERR

Python 2.7.6

Traceback (most recent call last):
File "./sifter.py", line 842, in
main()
File "./sifter.py", line 833, in main
gui = Gui(ts, injector, tests, args.tick)
File "./sifter.py", line 376, in init
self.init_colors()
File "./sifter.py", line 423, in init_colors
self.COLOR_BLACK
_curses.error: init_pair() returned ERR

Convert into a BOINC distributed testing project with no disassembler (initially)

Convert into a BOINC distributed testing project with no disassembler (initially).
-- Yes, as in SETI @ Home or ROSETTA @ Home etc ...

Having to replicate programs and scripts locally via git is beyond most Linux users skill level, and that goes for Android and MacOS versions as well.

Obviously this means some kind of back end collection database, but not out of the limits of say mySQL or any other open source DB.

The x86 CPUID would have to be the paramount signature mechanism, and something else but similar for ARM as it is RISC and has much fewer instructions.

This probably should be funded via maybe the Electronic Freedom Foundation or others ...

Injector hang

Hi,

Injector seems to be hanging on Fedora 23 after ~1B instruction tests, selinux is in permissive mode, cpu is Intel(R) Core(TM) i7-4800MQ CPU @ 2.70GHz.

I've tried twice, going to attach using strace this time and see if anything is happening but in the meantime, any ideas?

thanks

Warnings while compiling with clang-4 on Fedora 26 x64

CC=clang make

clang  -c injector.c -o injector.o -Wall
injector.c:321:93: warning: excess elements in array initializer
        .start={.bytes={0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, .len=0},
                                                                                                   ^~~~
injector.c:322:91: warning: excess elements in array initializer
        .end={.bytes={0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff}, .len=0},
                                                                                                 ^~~~
2 warnings generated.
clang  injector.o -O3 -Wall -l:libcapstone.a -o injector -pthread

clang -v

clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64

Unable to run under Arch

Hi,

I'm trying to test sandsifter on Arch. After manually specifying python2, I get:

$> sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t
Traceback (most recent call last):
  File "./sifter.py", line 842, in <module>
    main()
  File "./sifter.py", line 817, in main
    arch = re.search(r".*(..)-bit.*", injector_bitness).group(1)
AttributeError: 'NoneType' object has no attribute 'group'

Please let me know if you need any more info or testing.

Can't build sandsifter

Context

I'm trying to build sandsifter on Debian 8, zsh, tmux,
Env variables:

XDG_SESSION_ID=1
DISPLAY=:0
XDG_RUNTIME_DIR=/run/user/1000
GDM_LANG=en_US.utf8
SHELL=/usr/bin/zsh
SSH_AGENT_PID=2211
XDG_SEAT=seat0
_=/usr/bin/printenv
GDMSESSION=lightdm-xsession
GPG_AGENT_INFO=/run/user/1000/keyring/gpg:0:1
WINDOWID=52455024
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-J3nUQZdZKZ,guid=b53a4d3fa8cc46849b2ddc925978a17c
SHLVL=1
TERM=xterm
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
VTE_VERSION=3801
PATH=/home/user/environments/cap/bin:/usr/local/bin:/usr/bin:/bin:/home/user/.vimpkg/bin:/home/user/.bin://home/user/.cargo/bin:/home/user/projects/go/bin/:/usr/sbin:/home/user/.fzf/bin:/home/user/go/bin:/usr/local/go/bin
LANG=en_US.UTF-8
XDG_VTNR=7
LANGUAGE=en_US:en
PWD=/tmp/sandsifter
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
XAUTHORITY=/home/user/.Xauthority
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session0
DESKTOP_SESSION=lightdm-xsession
XDG_GREETER_DATA_DIR=/var/lib/lightdm/data/user
OLDPWD=/home/user
USER=user
ZSH=/home/user/.oh-my-zsh
UPDATE_ZSH_DAYS=5
PAGER=less
LESS=-R
LC_CTYPE=en_US.UTF-8
LSCOLORS=Gxfxcxdxbxegedabagacad
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:
WORKON_HOME=/home/user/environments
VIRTUALENVWRAPPER_PROJECT_FILENAME=.project
VIRTUALENVWRAPPER_WORKON_CD=1
VIRTUALENVWRAPPER_SCRIPT=/usr/local/bin/virtualenvwrapper.sh
VIRTUALENVWRAPPER_HOOK_DIR=/home/user/environments
MANPATH=:/home/user/.fzf/man
GOPATH=/home/user/go
EDITOR=vim
VIRTUAL_ENV=/home/user/environments/cap
PS1=(cap) ${ret_status} %{$fg[cyan]%}%c%{$reset_color%} $(git_prompt_info)

Steps to Reproduce

Clone latest commit
Install dependencies from capstone page: sudo apt-get install libcapstone3 libcapstone-dev
mkvirtualenv cap
pip install capstone
Run make command.

Current Result

(cap) ➜  sandsifter git:(master) make
cc  -c injector.c -o injector.o -Wall
injector.c:321:2: warning: excess elements in array initializer
  .start={.bytes={0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, .len=0},
  ^
injector.c:321:2: warning: (near initialization for ‘total_range.start.bytes’)
injector.c:322:2: warning: excess elements in array initializer
  .end={.bytes={0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff}, .len=0},
  ^
injector.c:322:2: warning: (near initialization for ‘total_range.end.bytes’)
injector.c: In function ‘print_asm’:
injector.c:554:3: warning: implicit declaration of function ‘cs_disasm_iter’ [-Wimplicit-function-declaration]
   if (cs_disasm_iter(
   ^
injector.c: In function ‘main’:
injector.c:1435:2: warning: implicit declaration of function ‘cs_malloc’ [-Wimplicit-function-declaration]
  capstone_insn = cs_malloc(capstone_handle);
  ^
injector.c:1435:16: warning: assignment makes pointer from integer without a cast
  capstone_insn = cs_malloc(capstone_handle);
                ^
cc  injector.o -O3 -Wall -l:libcapstone.a -o injector -pthread
injector.o: In function `print_asm':
injector.c:(.text+0x7a7): undefined reference to `cs_disasm_iter'
injector.o: In function `give_result':
injector.c:(.text+0x19c6): undefined reference to `cs_disasm_iter'
injector.o: In function `main':
injector.c:(.text+0x24d1): undefined reference to `cs_malloc'
collect2: error: ld returned 1 exit status
Makefile:35: recipe for target 'injector' failed
make: *** [injector] Error 1

Expected result

Sandsifter redy for usage

capstone/capstone.h missing

Hello,

when I do a make on the folder SandSifter it speak about capstone/capstone.h is missing...
And my processor is a i386 also is there the problem ?

Thank you in advance to answer my ask,
Best Regards.

(sorry if my english is bad I am french !)

./sifter.py command missing

Hello dear IT worker,

./sifter.py command is missing...
also in sandsifter-master there is a sifter.py too what can I do ?

Thank you in advance to answer my ask.

Best Regards
(I have Ubuntu 16.04.3 on a old system pentium 4 with only one dies !)

Typo in sandsifter paper

https://github.com/xoreaxeaxeax/sandsifter/blob/master/references/domas_breaking_the_x86_isa_wp.pdf
"it is easy for malicious softer to mask..."
should be
"it is easy for malicious software to mask..."

sifter.py fails with "OSError: [Error 2] No such file or directory"

When running sifter.py as per example in the README it fails with [Errno 2] No such file or directory

# ./sifter.py --unk --dis --len --sync --tick -- -P1 -t
Traceback (most recent call last):
  File "./sifter.py", line 842, in <module>
    main()
  File "./sifter.py", line 815, in main
    stderr=subprocess.PIPE
  File "/usr/lib/python2.7/subprocess.py", line 390, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1024, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

sandsifter and capstone have been freshly build from git sources
capstone-bindings for python were installed via pip

OS: Alpine Linux 3.5 in an LX-Zone on smartOS (SunOS 5.11 joyent_20170511T001921Z)

I couldn't really figure out what file sifter.py or subprocess.py is failing to open - a quick trace for open* syscalls on the host shows these files being accessed (and existent):

# dtrace -n 'syscall::open*:entry { printf("%s %s",execname,copyinstr(arg0)); }'
dtrace: description 'syscall::open*:entry ' matched 4 probes
CPU     ID                    FUNCTION:NAME
 15   8834                       open:entry sifter.py /var/ld/64/ld.config
 15   8834                       open:entry sifter.py /native/lib/64/libc.so.1
 15   8834                       open:entry sifter.py /native/usr/lib/64/libmapmalloc.so.1
 15   8834                       open:entry sifter.py /native/lib/64/librpcsvc.so.1
 15   8834                       open:entry sifter.py /native/lib/64/libnsl.so.1

A full trace of a failed sifter.py execution is available on pastebin: https://pastebin.com/YP2wUZHy

Parallelization

Given how long it takes to evaluate a processor, it would be great to break the search into multiple processes.

Resume improvements

On my i5-7600K I'm getting relatively frequent crashes of the application.
When using the --resume option to work around this and keep scanning, the results so far are cleared.

So my request for resume would be to:

Preserve the number of scanned instructions
Preserve the number of anomalies
Continue the log in append mode (perhaps requiring/implying --sync)
Take multi-threading with -l and -j into account

At that point, running with resume and a docker with a restart policy would perhaps be enough to run this overnight and get a more-or-less complete result to summarize.

Making use of this work

Hello,
thanks for that awesome project.
Are you part of this subproject : https://github.com/rigred/sandsifter-tests?
Either way, an idea : rigred/sandsifter-tests#10
BR

Edit for a noob question : Does the interaction between undocumented instructions and microcode look like an interesting research area to you?

_curses.error: init_pair()

On some versions of MacOS you need to #include <sys/ucontext.h> instead of #include <ucontext.h>

instruction information

0f 18 /4-7 work on a Pentium 3 and later. I assumed that they were 0f 18 /0-3 aliases.

db e0 is feni db e1 is fdisi
df c0 - df c7 is ffreep st(i)
c0/c1/d0/d1/d2/d3 3x/7x/bx/fx are sal r/m, imm8 That's just shl with another name.
f6/f7 /1 is truly an alias for f6/f7 /0.
f1 is icebp.

From the whitepaper:
66e90000 jmpw 4f5
This is a four-byte instruction, and it jumps to a 16-bit address (i.e. 0x0000xxxx).
Intel CPUs do this, too. The disassembler is not wrong.

Port to ARM

Just want to understand if you have thought of porting this to ARM or not.

Please consider adding a license

Hi! Please consider adding some sort of license to this repo,
so that others can properly fork and contribute under clear terms
(and perhaps even package and distribute the code in distro package
managers).

There's a handy guide to choosing a license here if you haven't done so
before: https://choosealicense.com/

summarizer does not properly write temporary files to forward to disassemblers

When the summarizer writes data into temporary files for the disassembler, the disassembler sees 0 byte files. Adding a temp_file.flush() after the write in disassemble fixes this issue for me, at least for ndisasm

Doesn't work under TMUX od SCREEN session

I get this when I run under tmux or screen session:

injector: injector.c:1410: main: Assertion `!mprotect(packet_buffer,4096,0x1|0x2|0x4)' failed.

Linker error on ArchLinux

I'm getting a linker error when running make (Additionally I've tried adding -fPIC to the compilation options however this hasn't solved the issue.)

See below command output and versions.
From commit dff6324
make output:

cc  -c injector.c -o injector.o -Wall                                                                                                                                                        
injector.c:321:93: warning: excess elements in array initializer                                                                                                                             
  .start={.bytes={0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, .len=0},                                                                                 
                                                                                             ^~~~                                                                                            
injector.c:321:93: note: (near initialization for ‘total_range.start.bytes’)                                                                                                                 
injector.c:322:91: warning: excess elements in array initializer                                                                                                                             
  .end={.bytes={0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff}, .len=0},                                                                                   
                                                                                           ^~~~                                                                                              
injector.c:322:91: note: (near initialization for ‘total_range.end.bytes’)                                                                                                                   
cc  injector.o -O3 -Wall -l:libcapstone.a -o injector -pthread                                                                                                                               
/usr/bin/ld: injector.o: relocation R_X86_64_32S against undefined symbol `dummy_stack' can not be used when making a shared object; recompile with -fPIC                                    
/usr/bin/ld: final link failed: Nonrepresentable section on output                                                                                                                           
collect2: error: ld returned 1 exit status
make: *** [Makefile:35: injector] Error 1

cc --version : cc (GCC) 7.1.1 20170630

ld --version: GNU ld (GNU Binutils) 2.28.0.20170506

_curses.error: cbreak() returned ERR when using --sync or --save or --resume together

Whenever I use more than one of the mentioned flags I get the following error:

Traceback (most recent call last):
File "./sifter.py", line 842, in
main()
File "./sifter.py", line 833, in main
gui = Gui(ts, injector, tests, args.tick)
File "./sifter.py", line 369, in init
curses.cbreak()
_curses.error: cbreak() returned ERR

This does not go away by setting their xterm256 TERM environment variable

This is especially unfortunate for the case of trying to use --sync and --resume

Compiling on macOS fails

On macOS make yeilds:

/usr/include/ucontext.h:43:2: error: The deprecated ucontext routines require _XOPEN_SOURCE to be defined
#error The deprecated ucontext routines require _XOPEN_SOURCE to be defined
 ^
...

Using

cc -D_XOPEN_SOURCE -c injector.c -o injector.o -Wall

diff --git a/injector.c b/injector.c
index 75848b5..280ae4e 100644
--- a/injector.c
+++ b/injector.c
@@ -13,7 +13,7 @@
 #include <time.h>
 #include <execinfo.h>
 #include <limits.h>
-#include <ucontext.h>
+#include <sys/ucontext.h>
 #include <sys/types.h>
 #include <stdint.h>
 #include <stdbool.h>

fixes the first issue. But, it looks like there is still an issue with the portability of ucontext structures:

cc  -c injector.c -o injector.o -Wall
injector.c:321:93: warning: excess elements in array initializer
        .start={.bytes={0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, .len=0},
                                                                                                   ^~~~
injector.c:322:91: warning: excess elements in array initializer
        .end={.bytes={0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff}, .len=0},
                                                                                                 ^~~~
injector.c:853:31: error: member reference type 'struct __darwin_mcontext64 *' is a pointer; did you mean to use '->'?
        ((ucontext_t*)p)->uc_mcontext.gregs[IP]+=UD2_SIZE;
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
                                     ->
injector.c:853:32: error: no member named 'gregs' in 'struct __darwin_mcontext64'
        ((ucontext_t*)p)->uc_mcontext.gregs[IP]+=UD2_SIZE;
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
injector.c:853:38: error: use of undeclared identifier 'REG_RIP'
        ((ucontext_t*)p)->uc_mcontext.gregs[IP]+=UD2_SIZE;
                                            ^
injector.c:81:13: note: expanded from macro 'IP'
        #define IP REG_RIP
                   ^
injector.c:866:29: error: member reference type 'struct __darwin_mcontext64 *' is a pointer; did you mean to use '->'?
                (uintptr_t)uc->uc_mcontext.gregs[IP]-(uintptr_t)packet-preamble_length;
                           ~~~~~~~~~~~~~~~^
                                          ->
injector.c:866:30: error: no member named 'gregs' in 'struct __darwin_mcontext64'
                (uintptr_t)uc->uc_mcontext.gregs[IP]-(uintptr_t)packet-preamble_length;
                           ~~~~~~~~~~~~~~~ ^
injector.c:866:36: error: use of undeclared identifier 'REG_RIP'
                (uintptr_t)uc->uc_mcontext.gregs[IP]-(uintptr_t)packet-preamble_length;
                                                 ^
injector.c:81:13: note: expanded from macro 'IP'
        #define IP REG_RIP
                   ^
injector.c:883:24: error: member reference type 'struct __darwin_mcontext64 *' is a pointer; did you mean to use '->'?
        memcpy(uc->uc_mcontext.gregs, fault_context.gregs, sizeof(fault_context.gregs));
               ~~~~~~~~~~~~~~~^
                              ->
/usr/include/secure/_string.h:65:27: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                          ^~~~
injector.c:883:25: error: no member named 'gregs' in 'struct __darwin_mcontext64'
        memcpy(uc->uc_mcontext.gregs, fault_context.gregs, sizeof(fault_context.gregs));
               ~~~~~~~~~~~~~~~ ^
/usr/include/secure/_string.h:65:27: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                          ^~~~
injector.c:883:45: error: member reference type 'mcontext_t' (aka 'struct __darwin_mcontext64 *') is a pointer; did you mean to use
      '->'?
        memcpy(uc->uc_mcontext.gregs, fault_context.gregs, sizeof(fault_context.gregs));
                                      ~~~~~~~~~~~~~^
                                                   ->
/usr/include/secure/_string.h:65:33: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                                ^~~
injector.c:883:46: error: no member named 'gregs' in 'struct __darwin_mcontext64'
        memcpy(uc->uc_mcontext.gregs, fault_context.gregs, sizeof(fault_context.gregs));
                                      ~~~~~~~~~~~~~ ^
/usr/include/secure/_string.h:65:33: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                                ^~~
injector.c:883:73: error: member reference type 'mcontext_t' (aka 'struct __darwin_mcontext64 *') is a pointer; did you mean to use
      '->'?
        memcpy(uc->uc_mcontext.gregs, fault_context.gregs, sizeof(fault_context.gregs));
                                                                  ~~~~~~~~~~~~~^
                                                                               ->
/usr/include/secure/_string.h:65:38: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                                     ^~~
injector.c:883:74: error: no member named 'gregs' in 'struct __darwin_mcontext64'
        memcpy(uc->uc_mcontext.gregs, fault_context.gregs, sizeof(fault_context.gregs));
                                                                  ~~~~~~~~~~~~~ ^
/usr/include/secure/_string.h:65:38: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                                     ^~~
injector.c:883:24: error: member reference type 'struct __darwin_mcontext64 *' is a pointer; did you mean to use '->'?
        memcpy(uc->uc_mcontext.gregs, fault_context.gregs, sizeof(fault_context.gregs));
               ~~~~~~~~~~~~~~~^
                              ->
/usr/include/secure/_string.h:65:59: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                                                          ^~~~
/usr/include/secure/_common.h:38:55: note: expanded from macro '__darwin_obsz0'
#define __darwin_obsz0(object) __builtin_object_size (object, 0)
                                                      ^~~~~~
injector.c:883:25: error: no member named 'gregs' in 'struct __darwin_mcontext64'
        memcpy(uc->uc_mcontext.gregs, fault_context.gregs, sizeof(fault_context.gregs));
               ~~~~~~~~~~~~~~~ ^
/usr/include/secure/_string.h:65:59: note: expanded from macro 'memcpy'
  __builtin___memcpy_chk (dest, src, len, __darwin_obsz0 (dest))
                                                          ^~~~
/usr/include/secure/_common.h:38:55: note: expanded from macro '__darwin_obsz0'
#define __darwin_obsz0(object) __builtin_object_size (object, 0)
                                                      ^~~~~~
injector.c:884:17: error: member reference type 'struct __darwin_mcontext64 *' is a pointer; did you mean to use '->'?
        uc->uc_mcontext.gregs[IP]=(uintptr_t)&resume;
        ~~~~~~~~~~~~~~~^
                       ->
injector.c:884:18: error: no member named 'gregs' in 'struct __darwin_mcontext64'
        uc->uc_mcontext.gregs[IP]=(uintptr_t)&resume;
        ~~~~~~~~~~~~~~~ ^
injector.c:884:24: error: use of undeclared identifier 'REG_RIP'
        uc->uc_mcontext.gregs[IP]=(uintptr_t)&resume;
                              ^
injector.c:81:13: note: expanded from macro 'IP'
        #define IP REG_RIP
                   ^
injector.c:885:17: error: member reference type 'struct __darwin_mcontext64 *' is a pointer; did you mean to use '->'?
        uc->uc_mcontext.gregs[REG_EFL]&=~TF;
        ~~~~~~~~~~~~~~~^
                       ->
injector.c:885:18: error: no member named 'gregs' in 'struct __darwin_mcontext64'
        uc->uc_mcontext.gregs[REG_EFL]&=~TF;
        ~~~~~~~~~~~~~~~ ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 warnings and 20 errors generated.
make: *** [injector.o] Error 1

SyntaxError

Hi,

When i run the program,
sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t

shows the error:

  File "./sifter.py", line 196
    if type(x) not in [type(0), type(0L)]:
                                                     ^
SyntaxError: invalid syntax

someone can help me?

Doesn't compile on Kali

Hey domas, take a look at it:

t@kali:~/Desktop/Tools/sandsifter# make
cc -c injector.c -o injector.o -Wall
injector.c:321:93: warning: excess elements in array initializer
.start={.bytes={0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, .len=0},
^~~~
injector.c:321:93: note: (near initialization for ‘total_range.start.bytes’)
injector.c:322:91: warning: excess elements in array initializer
.end={.bytes={0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff}, .len=0},
^~~~
injector.c:322:91: note: (near initialization for ‘total_range.end.bytes’)
injector.c: In function ‘inject’:
injector.c:817:2: warning: asm operand 7 probably doesn’t match constraints
asm volatile ("
^~~~~~~
injector.c:817:2: error: impossible constraint in ‘asm’
Makefile:38: recipe for target 'injector.o' failed
make: *** [injector.o] Error 1

Doesn't compile on Solus OS

Installed capstone from the Git repo, tried to run make in the sandsifter folder and got these error messages :

cc -g2 -O3 -pipe -fPIC -Wformat -Wformat-security -fno-omit-frame-pointer -fexceptions -D_FORTIFY_SOURCE=2 -fstack-protector --param ssp-buffer-size=32 -fasynchronous-unwind-tables -ftree-vectorize -feliminate-unused-debug-types -Wall -Wno-error -Wp,-D_REENTRANT -c injector.c -o injector.o -Wall
injector.c:321:93: warning: excess elements in array initializer
  .start={.bytes={0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}, .len=0},
                                                                                             ^~~~
injector.c:321:93: note: (near initialization for ‘total_range.start.bytes’)
injector.c:322:91: warning: excess elements in array initializer
  .end={.bytes={0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff}, .len=0},
                                                                                           ^~~~
injector.c:322:91: note: (near initialization for ‘total_range.end.bytes’)
injector.c: In function ‘inject’:
injector.c:778:2: warning: asm operand 15 probably doesn’t match constraints
  __asm__ __volatile__ ("\
  ^~~~~~~
injector.c:778:2: error: impossible constraint in ‘asm’
injector.c: In function ‘main’:
injector.c:1508:5: warning: ‘pid’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  if (pid!=0) {
     ^
injector.c:1503:3: warning: ‘null_p’ may be used uninitialized in this function [-Wmaybe-uninitialized]
   munmap(null_p, PAGE_SIZE);
   ^~~~~~~~~~~~~~~~~~~~~~~~~
make: *** [Makefile:38: injector.o] Error 1

Running make version 4.2.1 and CC version 6.4.0

License?

The project does not contain any licensing information, this legally prevents people from actually editing/redistributing the code since open-source software needs to be explicitly licensed as such.

Two undefined names

flake8 testing of https://github.com/xoreaxeaxeax/sandsifter on Python 2.7.13

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./gui/gui.py:202:27: F821 undefined name 'height'
        window.addstr(y + height - 1, x, '}', color)
                          ^

./gui/gui.py:203:37: F821 undefined name 'progress'
        window.addch(y, int(x + 1 + progress * (width - 2)), curses.ACS_BLOCK, color)
                                    ^

sudo make install on capstone-master/bindings/python doesn't work....

Hello dear IT worker,

"sudo make install" on capstone-master/bindings/python doesn't work....
"Make" works but "sudo make install" doesn't works
After I tried the sandsifter-master command line with the same error for sifter.py ...

Thank you in advance to answer my asks,

Best regards.

Run completes instantly

On a t2.medium with the following dockerfile, sandsifter finishes in under a second, whether I use --low-mem and -N or not as prescribed in the "legacy systems" section of the README. I think it's reporting that it's not testing any instructions. ("insn tested: 0")

FROM ubuntu
RUN apt-get update
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get install -y libcapstone-dev gcc git python-pip make
RUN pip install capstone
RUN git clone https://github.com/xoreaxeaxeax/sandsifter
WORKDIR sandsifter
RUN make
ENV TERM=xterm-256color

Output:

ubuntu@ip-172-30-1-40:~/sand$ sudo docker build -t sandsifter . 2>&1 > /dev/null
ubuntu@ip-172-30-1-40:~/sand$ sudo time docker run -it sandsifter ./sifter.py --unk --dis --len --sync --tick --low-mem -- -P1 -t -N
#
# ./sifter.py --unk --dis --len --sync --tick --low-mem -- -P1 -t -N
# ./injector -P1 -t -N -t -R -0 -s 3759600895
#
# insn tested: 0
# artf found:  0
# runtime:     00:00:00.00
# seed:        3759600895
# arch:        64
# date:        2017-08-04 04:54:08
#
# cpu:
# processor	: 0
# vendor_id	: GenuineIntel
# cpu family	: 6
# model		: 63
# model name	: Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
# stepping	: 2
# microcode	: 0x25
#                               v  l  s  c
0.00user 0.00system 0:00.40elapsed 1%CPU (0avgtext+0avgdata 16100maxresident)k
0inputs+0outputs (0major+1202minor)pagefaults 0swaps

need to make this work on windows as well

would be nice if it worked on windows

Add Intel XED as decoder/disassembler

Source at https://github.com/intelxed/xed. Version 3.

XED being a complete encoder/decoder for every architecture including AMD, KNC, KNL.

For instance, it can give details on some unofficial instructions:

{
ICLASS    : SALC
CPL       : 3
CATEGORY  : FLAGOP
EXTENSION : BASE
ISA_SET   : I86
FLAGS     : MUST [ cf-tst ]
PATTERN   : 0xD6 not64
OPERANDS  : REG0=XED_REG_AL:w:SUPP
COMMENT   : UNDOC - "The Undocumented PC", 2nd ed 1997, says it is present on all Intel CPUs of that time.
}

{
ICLASS    : INT1
CPL       : 3
CATEGORY  : INTERRUPT
EXTENSION : BASE
ISA_SET   : I86
PATTERN   : 0xF1
OPERANDS  : REG0=rIP():w:SUPP
COMMENT   : UNDOC by Intel, but in AMD's opcode map
}

{
ICLASS    : FSETPM287_NOP
CPL       : 3
CATEGORY  : X87_ALU
EXTENSION : X87
ATTRIBUTES: NOP NOTSX
PATTERN   : 0xDB MOD[0b11] MOD=3 REG[0b100] RM[0b100]
OPERANDS  :
COMMENT   : UNDOC
}
{
ICLASS    : FENI8087_NOP
CPL       : 3
CATEGORY  : X87_ALU
EXTENSION : X87
ATTRIBUTES: NOP NOTSX
PATTERN   : 0xDB MOD[0b11] MOD=3 REG[0b100] RM[0b000]
OPERANDS  :
COMMENT   : UNDOC
}
{
ICLASS    : FDISI8087_NOP
CPL       : 3
CATEGORY  : X87_ALU
EXTENSION : X87
ATTRIBUTES: NOP NOTSX
PATTERN   : 0xDB MOD[0b11] MOD=3 REG[0b100] RM[0b001]
COMMENT   : UNDOC
OPERANDS  :
}

{
ICLASS    : FFREEP
CPL       : 3
CATEGORY  : X87_ALU
EXTENSION : X87
ATTRIBUTES: X87_CONTROL NOTSX
FLAGS     : MUST [ fc0-u   fc1-u   fc2-u   fc3-u   ]
PATTERN   : 0xDF MOD[0b11] MOD=3 REG[0b000] RM[nnn]
OPERANDS  : REG0=X87():r:f80 REG1=XED_REG_X87TAG:w:SUPP REG2=XED_REG_X87POP:r:SUPP
COMMENT   : UNDOC
}

and so on.

Display hangs partway through

Displayed fine for a while, then stopped at 2e0f9a5be2. CPU usage remains at 100%. I can manually check the data/sync contents and I continue to see changes, so the program isn't hung, but the display remains constant.

Screenshot:

Command:
sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t

Code is from the rigred branch: https://github.com/rigred/sandsifter-tests
CPU: Intel i7-4790K
OS: Ubuntu MATE 16.04 LTS
libcapstone3, libcapstone3-dev, python-capstone: 3.0.4-0.2
Terminal: MATE Terminal 1.12.1-1

How to donate?

subj

Completion of error handling

Would you like to add more error handling for return values from functions like the following?

printf ⇒ help
pthread_mutex_init ⇒ main

Howto NOPs on Linux unknow instructions?

Small howto please. Something like this:

switch ( opcode) {
case AABBCC:
opcode = 0x90;
....
}

build fails with musl libc

as used by the popular alpine linux, but also on sabotage linux.

injector.c:14:22: fatal error: execinfo.h: No such file or directory
 #include <execinfo.h>

$ sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t
injector: injector.c:1410: main: Assertion `!mprotect(packet_buffer,PAGE_SIZE,PROT_READ|PROT_WRITE|PROT_EXEC)' failed.
#
# ./sifter.py --unk --dis --len --sync --tick -- -P1 -t
# ./injector -P1 -t -t -R -0 -s 3429684305
#
# insn tested: 0
# artf found:  0
# runtime:     00:00:00.11
# seed:        3429684305
# arch:        64
# date:        2017-07-29 18:14:29
#
# cpu:
# processor	: 0
# vendor_id	: AuthenticAMD
# cpu family	: 23
# model		: 1
# model name	: AMD Ryzen 7 1700 Eight-Core Processor
# stepping	: 1
# microcode	: 0x8001126
#                               v  l  s  c