Hi,

I'm trying to compile the agent (with Mingw dependences installed) and i'm having the following error:
`#make

make -C libseh -f GNUmakefile
make[1]: se entra en el directorio '/home/user/test/pyrebox/guest/win/libseh'
i686-w64-mingw32-gcc -O2 -Wall -g -std=c99 -fno-strict-aliasing -o build/common.o -c common/common.c
In file included from common/../seh.h:36:0,
from common/common.c:33:
common/../os/windows/seh-support.h:35:10: error fatal: ../../common/config.h: No existe el fichero o el directorio
#include "../../common/config.h"
^~~~~~~~~~~~~~~~~~~~~~~
compilación terminada.
make[1]: *** [GNUmakefile:36: build/common.o] Error 1
make[1]: se sale del directorio '/home/user/test/pyrebox/guest/win/libseh'
make: *** [Makefile:77: libseh/build/libseh.a] Error 2
`

I'm trying to compile it from ArchLinux, maybe this is the problem, but i think it shouldn't, everything else compiled and worked like a charm. This is such an awesome project!

Hi,

Is it possible to monitor user space memory read and write events per process?
I would like to generate memory reference trace for specific process, something similar to the one created by Pin tool: https://software.intel.com/sites/landingpage/pintool/docs/81205/Pin/html/index.html#MAddressTrace

Thanks

Hi,

I am trying to use your project to speed up malware unpacking. So, I am trying to set up malware monitor.

Following the instructions I have compile the DbGenerator from the project Deviare2 after patching it as you saw (first just this tool and after that the full project). The issue is that to use Database/DbBuilder/ run build_db32 you need a prepossess headers(in this case preprocessW32.h) or just a .h file with the headers you want to build the database for. The preprocessW32.h headers are missing, at least in my case. I have seem many headers file under the directory HeaderBuilder. What would be the proper header file to be used with Malware monitor in Pyrebox?

Cheers,

Vicente

I'm messing around with a module to bypass some anti-VM tricks and many of them depends on the instruction "CPUID" witch corresponds to the opcode "0x0FA2"
Example:

The question is that, with the callback OPCODE_RANGE_CB, many other opcodes like the "CALL" from the examples and some other random works perfect, but with this one it doesn't seem to catch nothing:

cm.add_callback(CallbackManager.OPCODE_RANGE_CB,cpuid_call,start_opcode=0x0F,end_opcode=0x0F) #0FA2 -> CPUID

i'm using Win7x86 and the tool Pafish (https://github.com/a0rtega/pafish) to make tests~~

Working in 64-bit.

After breaking on a MEM_READ_CB or MEM_WRITE_CB, I would like to know the memory address of the instruction that caused the read or write.

I know I could read cpu.RIP, but that gives me the value of the next instruction, which is of course not the same thing.

I considered the Instruction Register which might have been enough, but I couldn't find that exposed through the API(??).

So, is there any way of knowing the address of the current instruction being executed when a MEM_READ_CB or MEM_WRITE_CB breaks?

Thanks!

Hi

Is it possible to launch new process inside the vm?
Are there any options except launching it manually via VNC?

Tnx

I'm trying to add a MEM_READ_CB callback once a named process has started.

The new process is detected via CREATEPROC_CB and, as per the below, I see the Matched new process! message:

# Called via: cm.add_callback(CallbackManager.CREATEPROC_CB, new_proc, name="new_proc")
def new_proc(pid, cr3, name):
    global cm
    if name == 'WINSLL.EXE':
    	pyrebox_print("Matched new process! pid=%d, cr3=%d, name=%s\n" % (pid, cr3, name))
    	cm.rm_callback('new_proc')
    	cm.add_callback(CallbackManager.MEM_READ_CB, mem_read, name="mem_read")

def mem_read(cpu_index, vaddr, size):
	global cm
	pyrebox_print("READ! vaddr=%d, size=%d" % (vaddr, size))
	cm.rm_callback("mem_read")

The code seems fine in that there are no errors, but the mem_read function is seemingly never called, or at least, I never see my READ message.

Am I missing something? Is it essential to add a cm.add_trigger("mem_read", "triggers/trigger_bpr_memrange.so") trigger? I don't want to because I want any read.

Thanks!

I am interested in tracking the creation/deletion of files in a win32 guest, similar to the process create/delete tracking that is done in the script_example.py script. Is there a way to do that?

When running...

It fails at...

Adding logging for stdout/stderr for that subprocess call captured...
make: *** No rule to make target 'triggers/trigger_bpwh_memrange-i386-softmmu.so'. Stop.

Solved: The subprocess starts the make call in the current working directory rather than using the pyrebox directory. Those make calls fail if the pyrebox execution was started from other directories.

Hello,
well i would like to ask about the agent that its installed in Victim Vm.
Is it only for transfering the samples and to run python scripts?
Can i run python scripts without the agent?

Can i import the logs into elasticstack?

Hello,

I keep getting the noted error when running the build.sh but I have qemu installed and i can seem to locate those missing libraries.

Below is the screenshot to the error am getting.

As commented in PR #36, the block end event does not represent necessarily the end of a basic block.

As commented by f4rnham:

"""[...] issues with basic block ends being only subset of translation block ends.
Under certain conditions, like too long basic block or popf, sti, invlpg instructions, TB translation is terminated immediately even though basic block didn't end."""

Instead of dealing with all these individual cases, the documentation should be updated to reflect that block-begin and block-end refer to QEMU translation blocks and may not match basic-block boundaries.

Hi pyrebox developers,
The script which is loaded in pyrebox needs to get the initial time of the VM in the beginning, and then get the current VM time every time the callback function is called. When the time difference between the current time minus the initial time reaches the set maximum time, the callback function is removed.
But If the time library in python is called, we will get the runtime of the script instead of the running time of the VM. So I would like to ask if there is a way to get the total running time of the VM?
Best regards.

Fails to clean before attempting to build the update

pyrebox$ make clean
rm -f triggers/*.so triggers/*.o triggers/*.d exploit_detect/*.so exploit_detect/*.o exploit_detect/*.d
make[1]: Entering directory '/git/pyrebox/sleuthkit'
make[1]: *** No rule to make target 'clean'.  Stop.
make[1]: Leaving directory '/git/pyrebox/sleuthkit'
Makefile:78: recipe for target 'clean-sleuthkit' failed
make: *** [clean-sleuthkit] Error 2

Fails to build the update

/git/pyrebox$ make
make[1]: Entering directory '/git/pyrebox/qemu'
        CHK version_gen.h
  CC      i386-softmmu/pyrebox/qemu_glue_sleuthkit.o
In file included from /git/pyrebox/qemu/pyrebox/sleuthkit/tsk/libtsk.h:4:0,
                 from /git/pyrebox/qemu/pyrebox/qemu_glue_sleuthkit.c:26:
/git/pyrebox/qemu/pyrebox/sleuthkit/tsk/base/tsk_base.h:53:10: fatal error: tsk/tsk_incs.h: No such file or directory
 #include "tsk/tsk_incs.h"
          ^~~~~~~~~~~~~~~~
compilation terminated.
/git/pyrebox/qemu/rules.mak:66: recipe for target 'pyrebox/qemu_glue_sleuthkit.o' failed
make[2]: *** [pyrebox/qemu_glue_sleuthkit.o] Error 1
Makefile:478: recipe for target 'subdir-i386-softmmu' failed
make[1]: *** [subdir-i386-softmmu] Error 2
make[1]: Leaving directory '/git/pyrebox/qemu'
Makefile:72: recipe for target 'all' failed
make: *** [all] Error 2

Host : Ubuntu 16.04.03
Guest: Windows 7 SP1x64 KN Build.

Win_Agent_64 is running in Guest OS.

But Agent_strncmp called when it's crash

The VM agent allows easier full automation of tests, both for windows and linux. The tests should be improved so that they can be run automatically.

Hey,

i have read in the docu that is possible to write in the register from the VM. I use Qemu to simulate an old DOS system. Now i want to read/write a register from the VM. I found in your docu the function "api.w_r(cpu_index, regname, val)". In my special case i want to read/write the serial port COM1 (Port adress #3f8). So what is the correct way to initialize the parameter of the function api.w_r(cpu_index, regname, val) to read/write in the this register.

Thanks for helping

It would be very nice to list ntoskrnl.exe modules with lm command. In the other hand, it would be useful that x command could list ntoskrnl symbols.

Hi all,

It seems that even if I don't specify -vnc, PyREBox spawns vnc anyway:

(venv-pyrebox) root@3a6bf416bc2f:/pyrebox# ./pyrebox-x86_64 -m 1024 -drive file=/slw/panda-Win7SP1x64/hdd.img -monitor stdio -conf pyrebox.conf.Win7SP1x64

[*] Loading python component initialization script
[*] Platform: x86_64-softmmu
[*] Starting python module initialization
[*] Reading configuration from 'pyrebox.conf.Win7SP1x64'
[*] Searching for KDBG...
[*] Initializing scripts...
[*] Finished python module initialization
QEMU 2.10.0 monitor - type 'help' for more information
(qemu) VNC server running on 127.0.0.1:5900
[*] KPCR found at fffff80002841d00!!
[*] KDBG found at fffff800028400a0!!

But I don't want to use VNC. I want the window (GTK?) that opens, as PANDA does.

Is this possible?

Thanks,
Adam

Hey all,

Super interested to play with this. I see you have a Dockerfile created. Unfortunately, I cannot create an automated build against your github account since I do not own it. Could you guys setup an auto-build?

This issue tracks the status and updates related to the new callback format, to be pushed shortly into a development branch.

I have problems with function va_to_pa(pgd,va) in win7 sp1 64 bits. With win7 32 bits i haven't problems .. Volatility profile is correct.

[13] pyrebox(74c)> proc explorer.exe
Process set to 74c:acd4000:explorer.exe

[14] pyrebox(74c)> api.va_to_pa(0xacd4000,0x7fef9110000)

ValueError Traceback (most recent call last)
../pyrebox/ipython_shell.pyc in ()
----> 1 api.va_to_pa(0xacd4000,0x7fef9110000)

../pyrebox/api.pyc in va_to_pa(pgd, addr)
229 #If this function call fails, it will raise an exception.
230 #Given that the exception is self explanatory, we just let it propagate upwards
--> 231 return c_api.va_to_pa(pgd,addr)
232
233 def start_monitoring_process(pgd):

ValueError: Incorrect function parameters: pgd,addr

[15] pyrebox(74c)>

Execution related callbacks (BLOCK_BEGIN_CB, BLOCK_END_CB, INSN_BEGIN_CB, INSN_END_CB) can be triggered for pgd other than specified in cm.add_callback call.

Tested under Win7SP1x86, see test script.
https://github.com/f4rnham/pyrebox/blob/pgd_mismatch/scripts/block_end_test.py

My assumption is, that certain translation blocks are shared between processes. TB hashing (tb_hash_func) doesn't take CR3 / PGD into account and optimizations in emulated OS could result in a single ntdll.dll/kernel32.dll/... copy in physical memory being mapped into multiple processes.

Currently, callbacks are added into TBs during their translation based on currently executing pgd. This can result in TB being created (first time executed) by non-monitored process (without callback) and later reused by monitored process and missing callback trigger.

While I can't confirm the last part by test, based on my understanding of PyREBox it could occur.

Hello. Is it possible to support the esxi? I tried to try to implement it. But it did not work. I tried to understand why this did not happen. I learned about the fact that the download occurred only before the bootloader.

The agent can't detect that it's running in a pyrebox VM. Am I missing something?

Host OS: Ubuntu 18.04 LTS
Guest OS: Windows 7 Ultimate SP1 x86
Agent Name: win_agent_32.exe

pyrebox.conf

[Modues]
plugins.guest_agent: True
.
.
[AGENT]
name: win_agent_32.exe
conf: win_agent_32.exe.conf

win_agent_32.exe.conf

[BUFFER]
BufferOffset: 25568
BufferSize: 4096

Start machine using: ./pyrebox-i386 -monitor stdio -m 4096 -usb -device usb-tablet -drive file=images/win7sp1.qcow2,index=0,media=disk,format=qcow2,cache=unsafe -vnc 127.0.0.1:0

[*] Loading python component initialization script
[*] Platform: i386-softmmu
[*] Starting python module initialization
[*] Reading configuration from 'pyrebox.conf'
[*] Searching for KDBG...
[*] Initializing scripts...
[*]  Loading python module plugins.guest_agent
[plugins.guest_agent] [*]    Initializing guest_agent plugin
[*] Finished python module initialization
QEMU 2.12.0 monitor - type 'help' for more information

hi ! I'm setting up pyrebox but no have process anything :(

$ sudo ./start_x86_64.sh /mnt/hgfs/Shared/QEMU/imagesWin7x64

[] Loading python component initialization script
[] Platform: x86_64-softmmu
[] Starting python module initialization
[] Reading configuration
[] Importing scripts.script_example
[scripts.script_example] [] Initializing callbacks
[scripts.script_example] [] Initialized callbacks
[] Finished python module initialization
[*] Searching for KDBG... << no message about KDBG ? :(
QEMU 2.9.0 monitor - type 'help' for more information

[13] pyrebox> ps
CPU 0 PGD: 1ef111a0 InKernel: 1
+------+---------+-----------+-----+-----+
| Name | Running | Monitored | PID | PGD |
+------+---------+-----------+-----+-----+
+------+---------+-----------+-----+-----+

[14] pyrebox> vol pslist
VolShell: Error while executing volatility command
No suitable address space mapping found
Tried to open image as:
MachOAddressSpace: mac: need base
LimeAddressSpace: lime: need base
WindowsHiberFileSpace32: No base Address Space
MachOAddressSpace: MachO Header signature invalid
LimeAddressSpace: Invalid Lime header signature

screendump is works. windows normally booted. Thanks

I only have one cpu (index = 0). If i set cpu_index to 1 crash:

[5] pyrebox> api.is_kernel_running(1)
[1] 12954 segmentation fault ./pyrebox-x86_64 -m 512 -monitor stdio -usb -drive -loadvm 1

Hi,

After creating the image using:

qemu-img create -f qcow2 -o compat=0.10 images/Win7.qcow2 30G

I then ran the following to create the VM:

./pyrebox-x86_64 -m 2g -monitor stdio -usb -drive file=images/Win7.qcow2, index=0,media=disk,format=qcow2,cache=unsafe -cdrom images/Win7.iso -boot d -enable-kvm

This returns the qemu prompt, but the VM does not start up.

Am I missing something?

Thanks!

• Listing processes during system boot and system operation
• Monitoring individual address spaces (processes)
• Monitoring individual threads separately
• Extracting module information for user processes and kernel, as well as symbols
• Extracting symbols from .dynsym (dynamic symbols) from memory.
• Add documentation on how to generate volatility profiles for linux kernels.
• Add tests for linux VMI, process and module creation/removal detection, and other relevant callback types.

Got this error.

Also attaching a screenshot.
#####################
(qemu) sh
../Objects/longobject.c:998: bad argument to internal function
Traceback (most recent call last):
File "/root/pyrebox/pyrebox/init.py", line 124, in pyrebox_ipython_shell
from ipython_shell import start_shell
SystemError: ../Objects/longobject.c:998: bad argument to internal function
#####################

1. Operating system used to compile and run PyREBox.
   Ubuntu 16.04 16.04.1-Ubuntu SMP

2. The specific operating system version and emulation target you are using.
   Windows 7 Ultimate SP1
   using Win7SP1x64 volatility

3. Shell command / script / task you were trying to run.
   Just started the shell.

Attaching pyrebox.conf screenshot

I know this issue was closed earlier but the solutions described don't seem to work for me. The host is an Ubuntu Linux x86_64 with a KVM guest OS of WinXPSP3. The pyrebox.conf file contains:

[MODULES]
scripts.script_example: False
scripts.script_example_min: False
scripts.vol_test: False
scripts.proc_bp: False
scripts.mem_write_test: False
scripts.page_write_execute_trigger: False
scripts.getset_var_example: False
scripts.bp_range_test: False
pyrebox_test.test_0_vmi_create_proc: False
pyrebox_test.test_1_block_begin: False
pyrebox_test.test_2_opt_block_begin: False
pyrebox_test.test_3_opt_insn_begin: False
pyrebox_test.test_4_mem_write: False
pyrebox_test.test_5_mem_read: False
pyrebox_test.test_6_keystroke: False
pyrebox_test.test_7_nic_test: False
pyrebox_test.test_8_opcode_range: False
pyrebox_test.test_9_tlb_and_context_change: False
pyrebox_test.test_10_block_end: False
pyrebox_test.test_11_insn_end: False

[VOL]
profile: WinXPSP3x86

Below I am including the commands invoked and the results displayed:

root@erebor:~/pyrebox-master# ./pyrebox-i386 -monitor stdio -m 512 -usb -drive file=pyrebox/images/xpsp3.qcow2,index=0,media=disk,format=qcow2,cache=unsafe -net nic,model=rtl8139 -net user

[] Loading python component initialization script
[] Platform: i386-softmmu
[] Starting python module initialization
[] Reading configuration
[] Finished python module initialization
[] Searching for KDBG...
QEMU 2.9.0 monitor - type 'help' for more information
(qemu) sh

[14] pyrebox> print_cpu
Showing results for cpu 0, indicate cpu index otherwise

           CPU 0

========================================
EAX 0x5a189bbb
ECX 0xffdffc70
EDX 0x00000000
EBX 0xffdffc70

ESP 0x80550f34
EBP 0x80550f50

ESI: 0xffdffc50
EDI: 0x81fb3150

EIP: 0xf85e1d3e
EFLAGS: 0x00000246

CR0: 0x8001003b
CR1: 0x00000000
CR2: 0x01250000
CR3: 0x00039000
CR4: 0x00000698

ES: Sel: 00000023 Base: 00000000 Size: ffffffff Flags: 00cff300
CS: Sel: 00000008 Base: 00000000 Size: ffffffff Flags: 00cf9a00
SS: Sel: 00000010 Base: 00000000 Size: ffffffff Flags: 00c09300
DS: Sel: 00000023 Base: 00000000 Size: ffffffff Flags: 00cff300
FS: Sel: 00000030 Base: ffdff000 Size: 00001fff Flags: ffc093df
GS: Sel: 00000000 Base: 00000000 Size: 00000000 Flags: 00000000
LDT: Sel: 00000000 Base: 00000000 Size: 00000000 Flags: 00008200
GDT: Sel: 00000000 Base: 8003f000 Size: 000003ff Flags: 00000000
IDT: Sel: 00000000 Base: 8003f400 Size: 000007ff Flags: 00000000
TR: Sel: 00000028 Base: 80042000 Size: 000020ab Flags: 80008904

[15] pyrebox> vol pslist

[16] pyrebox> ps
CPU 0 PGD: 39000 InKernel: 1
+------+---------+-----------+-----+-----+
| Name | Running | Monitored | PID | PGD |
+------+---------+-----------+-----+-----+
+------+---------+-----------+-----+-----+

[17] pyrebox>

I downloaded the latest release and was able to get the volatility commands to work but the "ps" and "proc" commands do not work. I am running 64 bit Ubuntu host with a 32 bit WinXPSP2 guest.

Here is the pyrebox.conf file:

[MODULES]
scripts.script_example: False
scripts.script_example_min: False
scripts.vol_test: False
scripts.proc_bp: False
scripts.mem_write_test: False
scripts.page_write_execute_trigger: False
scripts.getset_var_example: False
scripts.bp_range_test: False
pyrebox_test.test_0_vmi_create_proc: False
pyrebox_test.test_1_block_begin: False
pyrebox_test.test_2_opt_block_begin: False
pyrebox_test.test_3_opt_insn_begin: False
pyrebox_test.test_4_mem_write: False
pyrebox_test.test_5_mem_read: False
pyrebox_test.test_6_keystroke: False
pyrebox_test.test_7_nic_test: False
pyrebox_test.test_8_opcode_range: False
pyrebox_test.test_9_tlb_and_context_change: False
pyrebox_test.test_10_block_end: False
pyrebox_test.test_11_insn_end: False

[VOL]
profile: WinXPSP3x86

Here is the output from several commands related to the issue:

./pyrebox-x86_64 -monitor stdio -m 512 -usb -drive file=pyrebox/images/xpsp3.qcow2,index=0,media=disk,format=qcow2,cache=unsafe -netdev user,id=network0 -device rtl8139,netdev=network0

[] Loading python component initialization script
[] Platform: x86_64-softmmu
[] Starting python module initialization
[] Reading configuration
[] Finished python module initialization
[] Searching for KDBG...
QEMU 2.9.0 monitor - type 'help' for more information
(qemu) sh

[1] pyrebox> vol pslist
Offset(V) Name PID PPID Thds Hnds Sess Wow64 Start Exit

0x821c8a00 System 4 0 75 235 ------ 0
0x82090368 smss.exe 312 4 3 19 ------ 0 2017-08-07 17:43:23 UTC+0000
0x81ff7320 csrss.exe 412 312 11 314 0 0 2017-08-07 17:43:24 UTC+0000
0x820a3020 winlogon.exe 436 312 24 531 0 0 2017-08-07 17:43:24 UTC+0000
0x81ffd330 services.exe 488 436 16 241 0 0 2017-08-07 17:43:25 UTC+0000
0x81fec978 lsass.exe 500 436 22 349 0 0 2017-08-07 17:43:25 UTC+0000
0x81fde020 svchost.exe 712 488 19 194 0 0 2017-08-07 17:43:32 UTC+0000
0x81fcb4a8 svchost.exe 788 488 9 215 0 0 2017-08-07 17:43:43 UTC+0000
0x81fb1b70 svchost.exe 904 488 66 1147 0 0 2017-08-07 17:44:02 UTC+0000
0x81f9db60 svchost.exe 1040 488 6 84 0 0 2017-08-07 17:44:14 UTC+0000
0x81f7d220 svchost.exe 1140 488 14 197 0 0 2017-08-07 17:45:07 UTC+0000
0x81fee468 spoolsv.exe 1368 488 13 116 0 0 2017-08-07 17:46:23 UTC+0000
0x81fdb8e0 explorer.exe 1524 1460 13 309 0 0 2017-08-07 17:46:45 UTC+0000
0x81f420a8 wscntfy.exe 832 904 1 28 0 0 2017-08-07 17:47:04 UTC+0000
0x82110c28 alg.exe 732 488 7 104 0 0 2017-08-07 17:47:06 UTC+0000
0x82103b88 cmd.exe 584 1524 1 30 0 0 2017-08-07 17:47:18 UTC+0000

[2] pyrebox> proc 584
Process 584 not found

[3] pyrebox> proc 1524
Process 1524 not found

[4] pyrebox> proc cmd.exe
Process cmd.exe not found

[5] pyrebox> ps
CPU 0 PGD: aed000 InKernel: 1
+------+---------+-----------+-----+-----+
| Name | Running | Monitored | PID | PGD |
+------+---------+-----------+-----+-----+
+------+---------+-----------+-----+-----+

[6] pyrebox> print_cpu
Showing results for cpu 0, indicate cpu index otherwise

           CPU 0

========================================
RAX 0x0000000000000543
RCX 0x00000000ffdffc50
RDX 0x0000000080010031
RBX 0x00000000ffdff000
R8 0x0000000000000000
R9 0x0000000000000000
R10 0x0000000000000000
R11 0x0000000000000000
R12 0x0000000000000000
R13 0x0000000000000000
R14 0x0000000000000000
R15 0x0000000000000000

RSP 0x0000000080549c54
RBP 0x00000000ffdff980

RSI: 0x0000000080552740
RDI: 0x00000000805529a0

RIP: 0x00000000806d89ba
RFLAGS: 0x0000000000000297

CR0: 0x000000008001003b
CR1: 0x0000000000000000
CR2: 0x0000000000e6fd20
CR3: 0x0000000000aed000
CR4: 0x00000000000006f8

ES: Sel: 00000023 Base: 0000000000000000 Size: ffffffff Flags: 00cff300
CS: Sel: 00000008 Base: 0000000000000000 Size: ffffffff Flags: 00cf9a00
SS: Sel: 00000010 Base: 0000000000000000 Size: ffffffff Flags: 00c09300
DS: Sel: 00000023 Base: 0000000000000000 Size: ffffffff Flags: 00cff300
FS: Sel: 00000030 Base: 00000000ffdff000 Size: 00001fff Flags: ffc093df
GS: Sel: 00000000 Base: 0000000000000000 Size: 00000000 Flags: 00000000
LDT: Sel: 00000000 Base: 0000000000000000 Size: 00000000 Flags: 00008200
GDT: Sel: 00000000 Base: 000000008003f000 Size: 000003ff Flags: 00000000
IDT: Sel: 00000000 Base: 000000008003f400 Size: 000007ff Flags: 00000000
TR: Sel: 00000028 Base: 0000000080042000 Size: 000020ab Flags: 80008904

[7] pyrebox>

Ubuntu 16.04:

ui/input-keymap.c:8:45: fatal error: ui/input-keymap-atset1-to-qcode.c: No existe el archivo o el directorio
compilation terminated.

Best regards.

If script callback is reading memory (through volatility or api.r_va) while user is typing to qemu console, deadlock can occur.
Can be consistently reproduced (Win7SP1x86) by running https://github.com/f4rnham/pyrebox/blob/pgd_mismatch/scripts/block_end_test.py

qemu_global_mutex and pyrebox_mutex mutexes are acquired in opposite order, see traces

Thread 1 (Thread 0x7ff3d765aac0 (LWP 16374)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007ff3d7244b95 in __GI___pthread_mutex_lock (mutex=0x55cb7fc95e00 <pyrebox_mutex>) at ../nptl/pthread_mutex_lock.c:80   <---------- locked, see T4 #51
#2  0x000055cb7f5fa840 in fd_chr_read (chan=0x55cb84f704e0, cond=<optimized out>, opaque=0x55cb84f702e0) at chardev/char-fd.c:47
#3  0x00007ff3d5a156aa in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#4  0x000055cb7f650a23 in glib_pollfds_poll () at util/main-loop.c:213
#5  os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:261    <---- util/main-loop.c:258 qemu_mutex_lock_iothread(); // locks qemu_global_mutex
#6  main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:515
#7  0x000055cb7f2c9e5b in main_loop () at vl.c:1919
#8  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4801

Thread 4 (Thread 0x7ff3b3a18700 (LWP 16386)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007ff3d7244b95 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x55cb7fc6a960 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2  0x000055cb7f65387a in qemu_mutex_lock (mutex=mutex@entry=0x55cb7fc6a960 <qemu_global_mutex>) at util/qemu-thread-posix.c:65  <---------- locked, see T1 #5
#3  0x000055cb7f306bac in qemu_mutex_lock_iothread () at /root/pyrebox/qemu/cpus.c:1581
#4  0x000055cb7f2ce0e5 in prepare_mmio_access (mr=<optimized out>, mr=<optimized out>) at /root/pyrebox/qemu/exec.c:2885
#5  0x000055cb7f2d31cf in address_space_read_continue (as=as@entry=0x55cb7fc672a0 <address_space_memory>, addr=addr@entry=655360, attrs=..., attrs@entry=...,
    buf=buf@entry=0x7ff3b3a156d0 "x\270\262\320\363\177", len=len@entry=1024, addr1=<optimized out>, l=<optimized out>, mr=0x55cb865585e0) at /root/pyrebox/qemu/exec.c:3009
#6  0x000055cb7f2d342e in address_space_read_full (as=as@entry=0x55cb7fc672a0 <address_space_memory>, addr=655360, addr@entry=94332510433952, attrs=...,
    buf=0x7ff3b3a156d0 "x\270\262\320\363\177", buf@entry=0xa0000 <error: Cannot access memory at address 0xa0000>, len=1024, len@entry=1) at /root/pyrebox/qemu/exec.c:3077
#7  0x000055cb7f2d359e in address_space_read (len=1, buf=0xa0000 <error: Cannot access memory at address 0xa0000>, attrs=..., addr=94332510433952, as=0x55cb7fc672a0 <address_space_memory>)
    at /root/pyrebox/qemu/include/exec/memory.h:1966
#8  address_space_rw (as=as@entry=0x55cb7fc672a0 <address_space_memory>, addr=addr@entry=655360, attrs=..., attrs@entry=..., buf=buf@entry=0x7ff3b3a156d0 "x\270\262\320\363\177",
    len=len@entry=1024, is_write=<optimized out>) at /root/pyrebox/qemu/exec.c:3091
#9  0x000055cb7f2d35d7 in cpu_physical_memory_rw (addr=addr@entry=655360, buf=buf@entry=0x7ff3b3a156d0 "x\270\262\320\363\177", len=len@entry=1024, is_write=is_write@entry=0)
    at /root/pyrebox/qemu/exec.c:3098
#10 0x000055cb7f39277c in cpu_physical_memory_read (len=<optimized out>, buf=0x7ff3b3a156d0, addr=655360) at /root/pyrebox/qemu/include/exec/cpu-common.h:83
#11 connection_read_memory (user_paddr=<optimized out>, buf=buf@entry=0x7ff3644178d0 "\270\006", user_len=user_len@entry=4096) at /root/pyrebox/qemu/pyrebox/qemu_glue.c:915
#12 0x000055cb7f393015 in py_vol_read_memory (dummy=<optimized out>, args=<optimized out>) at /root/pyrebox/qemu/pyrebox/api.cpp:697
#13 0x00007ff3d6958d3c in PyEval_EvalFrameEx () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
    ...
#47 0x00007ff3d6a9a538 in PyEval_EvalCodeEx () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#48 0x00007ff3d6a6a229 in ?? () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#49 0x00007ff3d6905583 in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#50 0x00007ff3d6a99a37 in PyEval_CallObjectWithKeywords () from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#51 0x000055cb7f39569b in CallbackManager::deliver_callback (this=<optimized out>, type=type@entry=BLOCK_END_CB, params=...) at /root/pyrebox/qemu/pyrebox/callbacks.cpp:831  <---------- pthread_mutex_lock(&pyrebox_mutex);
#52 0x000055cb7f395a76 in block_end_callback (params=...) at /root/pyrebox/qemu/pyrebox/callbacks.cpp:169
#53 0x000055cb7f396454 in helper_qemu_block_end_callback (cpu=0x55cb85129040, tb=0x7ff3b40293c0 <code_gen_buffer+172950>, from=2189689658, to=0)
    at /root/pyrebox/qemu/pyrebox/qemu_glue_callbacks.c:124
#54 0x00007ff3b40294b2 in code_gen_buffer ()
#55 0x000055cb7f3322ad in cpu_tb_exec (itb=<optimized out>, itb=<optimized out>, cpu=0x7ff3b48bb480 <code_gen_buffer+9159766>) at /root/pyrebox/qemu/accel/tcg/cpu-exec.c:167
#56 cpu_loop_exec_tb (tb_exit=<synthetic pointer>, last_tb=<synthetic pointer>, tb=<optimized out>, cpu=0x7ff3b48bb480 <code_gen_buffer+9159766>)
    at /root/pyrebox/qemu/accel/tcg/cpu-exec.c:579
#57 cpu_exec (cpu=cpu@entry=0x55cb85129040) at /root/pyrebox/qemu/accel/tcg/cpu-exec.c:684
#58 0x000055cb7f3073f2 in tcg_cpu_exec (cpu=<optimized out>) at /root/pyrebox/qemu/cpus.c:1270
#59 qemu_tcg_rr_cpu_thread_fn (arg=<optimized out>) at /root/pyrebox/qemu/cpus.c:1365
#60 0x00007ff3d7242494 in start_thread (arg=0x7ff3b3a18700) at pthread_create.c:333
#61 0x00007ff3d4f75abf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

The sample scripts "script_example_min.py" doesn't work for my setting.
My settings:

1. PyREBox installed on Ubuntu 16.04.
2. Guest system Windows 10 x64 1607.

My observations:

1. I can't see callback logs after PyREBox is started.
2. I checked the code of vmi.cpp and found that "os_index" parameter in method vmi_init() is set to VistaSP2x64 rather than Win10x64, when vol_profile equals to "Win10x64".
   My question is: is it because Windows 10 is not supported on current PyREBox, or Windows 10 works the same way as VistaSP2?
3. When I modified source code by changing "os_index" in vmi_init() from VistaSP2x64 to Win10x64. I can see one log message for the creation of first process, and then no logs for the remaining processes.

Please suggest on how to make PyREBox callbacks work in Windows 10 setting.
Thank you.

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./pyrebox/init.py:89:52: F821 undefined name 'module_name'
        pp_warning("[*]  Module %s not loaded\n" % module_name)
                                                   ^

https://github.com/Cisco-Talos/pyrebox/blob/master/pyrebox/init.py#L89

Hi pyrebox developers:

I am using api_tracer to extract the apis commonly used by malware.
When I run the calculator on a WinXP VM, the calculator runs normally and api_tracer gets function_calls.log.
But when I run Wannacry.exe, Wannacry.exe will automatically quit after running. I unload the mw_monitor module and re-run Wannacry.exe, it can successfully encrypt the files in the virtual machine.

Another ransomware is "Trojan-Ransom.Win32.Jaff.exe", its MD5 is "d349764bd5e16ee0e202b1e9dc057318", just like wannacry, it can also encrypt files in computers. When I use mw_monitor to monitor Jaff.exe's execution, unlike wannacry, Jaff.exe does not end the process, but it is stuck and doesn't have any movement, but the virtual machine can still operate normally.

My mw_monitor configuration files as follow:
mw_monitor.conf:

[general]

output_bundle: bundle.tar.gz
api_database: mw_monitor/third_party/deviare2_db/deviare32_populated.sqlite

[interproc]
bin_log_name: interproc.bin
text_log_name: interproc.log
basic_stats_name: basic_stats

[api_tracer]
text_log_name: function_calls.log
bin_log_name: function_calls.bin

[coverage]
cov_log_name: coverage.bin
cov_text_name: coverage.log

[dumper]

path = dumper/

mw_monitor_run.json:

{
    "api_tracer": {
        "bin_log": false,
        "exclude_apis": [],
        "exclude_modules": [],
        "exclude_origin_modules": [],
        "include_apis": [],
        "light_mode": true,
        "procs": null,
        "text_log": true
    },
    "coverage": {
        "procs": null
    },
    "dumper": {  
        "dump_at": "kernel32.dll!CopyFileA",
        "dump_on_exit": false
    },
    "general": {
        "files_path": "C:\\",
        "main_executable": "wannacry.exe",
        "files_bundle": "malware/malware.zip"
    },
    "interproc": {
        "basic_stats": true,
        "bin_log": true,
        "text_log": true
    },
    "modules": {
        "api_tracer": true,
        "coverage": false,
        "dumper": false,
        "interproc": false
    }
}

Hi pyrebox developers,

I am working on a project that needs to take a snapshot at the beginning of the VM, then use Guest Agent to run a malware and call my own function to analyze the memory state, finally when analyze is over, load the initial snapshot to recover VM. So, I use api.save_vm in initialize_callbacks() and api.load_vm in clean(), and set scripts.my_script:True in pyrebox.conf.

The problem is when I start VM in pyrebox, it always reports:"pyrebox-i386: -drive file=images/WinXP.qcow2,index=0,media=disk,format=qcow2,cache=unsafe: No block device can accept snapshots".

However, when the VM start completely and then I manually use qemu command "import_module my_script", the script works well. It can take a snapshot when use command "import_module", and load the initial snapshot when use command "unload_module".

But, I wonder is there a way to take and load snapshot automatically in the script instead of type commands to call my script manually every time?

Thanks a lot.

Hi pyrebox developer(s),

I am working a project that utilizes pyrebox to analyze binary programs. I followed pyrebox documentation and created a 32-bit Windows 7 virtual machine. I used Win7SP1x86 profile and commands such as "ps" and "vol pslist" worked fine. I could see the processes running in the VM, however when I tried to use guest_agent.copy(path_to_my_file, "C:\temp.exe"), the file did not appear in the VM. Moreover, assuming that the file was inside the VM "somehow" I executed guest_agent.execute_file("C:\temp.exe") and predictably nothing happened. It might have been a little mistake but it would be very helpful if you could point me to the right direction.

Thank you.

Reproduction steps:

• Share NTFS folder from Windows host to Ubuntu VM with VMware Workstation
• cd /mnt/hgfs/shared
• git clone https://github.com/Cisco-Talos/pyrebox.git
• cd pyrebox
• ./build.sh

[*] Patching qemu...
ln: failed to create symbolic link ‘qemu/pyrebox’: Operation not supported
[*] Configuring qemu...
ln: failed to create symbolic link ‘linux-headers/asm’: Operation not supported
ln: failed to create symbolic link ‘i386-softmmu/Makefile’: Operation not supported
ln: failed to create symbolic link ‘x86_64-softmmu/Makefile’: Operation not supported
[!] Oops... build failed!

When we are executing a callback from an instrumented TB, such as instruction begin (or a r/w/x execution breakpoint, etc), and we insert a breakpoint/callback on an address on the same translation block, the translation cache is not flushed until we finish executing the current translation block, and thus the callback is never delivered. When this happens, we should exit the cpu loop, flush the translation block, and continue executing to force the re-translation of the block before we continue.

i have always the next error with this 2 functions (sym_to_va and va_to_sym):

"Process with PGD %x not found"

In other functions PGD is correct. I have analyzed pyrebox/api.py and i think the next code exit very fast (never scan all procs, then never will find the proc):

else:
raise ValueError("Process with PGD %x not found" % pgd)

I have commented this else and functions finds the symbol or the va.

Greets,
JoseMi

I'm developing a module for pyrebox, but i have problems to reload (with changes) this module from Qemu console. My procedure is:

import_module funcap
..... (module execution and finish)
unload_module 1

After i want to load the same module but this module has changes. I have to reboot the machine to apply the changes .. is it this normal ?? other procedure to reload modules ??

Thank you!!

Hello, is it possible to attach PyREbox to an existing XEN domain?

Thank you

what about the performance ? Whole system emulation would be very slow , i think :(

Hi,

I have the following problem when trying to use mw_monit.

My set up is as follows

My host operating system is Ubuntu 16.04 (Linux vforensic 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux). My guest operating system is Windows version 6.1.7601.24094.

The volatility profile is Win7SP1x86 (with this one "(qemu) ps" and "pyrebox> vol pslist" work fine. I also tried Win7SP1x86_24000 but it doesn't work).

I ran the agent in the guest and leave it running with the message "Calling the get_version opcode", then I took an snapshot to be used with mw_monitor.
When I reload the afore mentioned vm with the agent and mw_monitor active in my pirebox.conf, my output is as follows:

[] Loading python component initialization script
[] Platform: i386-softmmu
[] Starting python module initialization
[] Reading configuration from 'pyrebox.conf'
[] Searching for KDBG...
[] Initializing scripts...
[] Loading python module plugins.guest_agent
[plugins.guest_agent] [] Initializing guest_agent plugin
[] Loading python module mw_monitor.mw_monitor
[] Module plugins.guest_agent already imported, did you want to reload it instead?
[mw_monitor.mw_monitor] Reading mw_monitor configuration...
[mw_monitor.mw_monitor] Copying host file to guest, using agent...
[mw_monitor.mw_monitor] Initializing callbacks
[mw_monitor.mw_monitor] No process matching that process name, deferring process detection
[mw_monitor.mw_monitor] Initializing process creation callback
[mw_monitor.mw_monitor] Adding dumper commands
[mw_monitor.mw_monitor] Initialized callbacks
[] Finished python module initialization
QEMU 2.11.1 monitor - type 'help' for more information
(qemu) [] KPCR found at 838f1000!!
[*] KDBG found at 82931378!!
[mw_monitor.mw_monitor] Starting monitoring process calc.exe
[mw_monitor.mw_monitor] Loading API tracer database...
[mw_monitor.mw_monitor] Started monitoring process with PGD 86fd3000 and name calc.exe

At this point the guest seems to be hanged and I have not the (qemu) prom back, so I stop it with Ctrl+C, with the following trace:

pyrebox-i386: terminating on signal 2
Traceback (most recent call last):
File "/home/vicente/pyrebox/pyrebox/api_internal.py", line 335, in function_wrapper
f(*args)
File "/home/vicente/pyrebox/mw_monitor/mw_monitor.py", line 64, in new_process
mw_monitor_start_monitoring_process(main_proc, insert_proc=False)
File "/home/vicente/pyrebox/mw_monitor/mw_monitor_classes.py", line 269, in mw_monitor_start_monitoring_process
new_proc.proc_num, "triggers/trigger_block_user_only_coverage.so")
File "/home/vicente/pyrebox/pyrebox/api.py", line 944, in add_trigger
raise ValueError("Could not correctly compile trigger\n")
ValueError: Could not correctly compile trigger

How should I trouble shoot this compilation error with the trigger? You mentioned in the documentation something as trying to compile them as shared objects, but I am afraid this is not clear enough for me. What should I write in the makefile for the compilation of these shared objects?

Cheers,

Vicente

Hi,

Tested with both Win7 x86 and x64 vms, on different environments.
Taking snapshot with savevm works with KVM (-enable-vm), but hangs without it.
What could be the reason?

Thanks

I've been trying to build pyrebox on Ubuntu 18.04 and i've been having this issue:

after a bit of reading, finally found the reason, that is related to a change in "mman.h" on the last version of Ubuntu.

Here are the changes to the "configure" and "util/memfd.c" files to get it working:
https://git.qemu.org/?p=qemu.git;a=commit;h=75e5b70e6b5dcc4f2219992d7cffa462aa406af0

I suppose that making this changes to the project may break qemu for the other distros, so it live this here, in case someone else has the same issue :)

wow, nice work, congrats! but your project description is still empty with "No description, website, or topics provided."

keep it up, cheers!