DPCs seem to be problematic especially on start up and on shut down. Sometimes the calling program will just crash (or another program might crash), this is usually due to memory management, i.e. cr3 load/store which isn't handled properly.

An alternative would be using exec_on_cpu (See here), there is only one issue with this, if this is called inside of an arbitary process context, we will get the CR3 of that process and use as the host cr3, and so obviously this is bad. This can be fixed using driver entry to get the kernel cr3 (or so), because DriverEntry is called from IopLoadDriver which is called inside some system thread.

This is quite rare, but it still nice to fix.

Hello,

Looks like there has not been any updates in a very long time and I am wondering if the project is still active?

Thanks

step:
1.build ksm.sys and ksm_um.exe
2.create service and start
3.run ksm_um.exe

below is windbg log.

please help me, thank you so much!!!

ksm: CPU 3: check_dynamic_pgtables: PXE: FFFFBBDDEEF77000 PPE FFFFBBDDEEE00000 PDE FFFFBBDDC0000000 PTE FFFFBB8000000000
ksm: CPU 3: check_dynamic_pgtables: Addr 0x1DA8440 0x1DA8440
ksm: CPU 3: DriverEntry: We're mapped at FFFFF80032D80000 (size: 61440 bytes (60 KB), on 15 pages)
ksm: CPU 3: ksm_init: EPT/VPID caps: 0x00000F0106714141
ksm: CPU 3: ksm_init: 9 physical memory ranges
ksm: CPU 3: ksm_init: Range: 0x0000000000001000 -> 0x00000000000A0000
ksm: CPU 3: ksm_init: Range: 0x0000000000100000 -> 0x000000000E367000
ksm: CPU 3: ksm_init: Range: 0x000000000E3B2000 -> 0x000000000E4D5000
ksm: CPU 3: ksm_init: Range: 0x000000000E504000 -> 0x000000000E58D000
ksm: CPU 3: ksm_init: Range: 0x000000000E5AC000 -> 0x000000000EF42000
ksm: CPU 3: ksm_init: Range: 0x000000000EF4B000 -> 0x000000000EF5E000
ksm: CPU 3: ksm_init: Range: 0x000000000EF64000 -> 0x000000000EF74000
ksm: CPU 3: ksm_init: Range: 0x000000000EF79000 -> 0x000000000FEE8000
ksm: CPU 3: ksm_init: Range: 0x000000000FF78000 -> 0x0000000080000000
ksm: CPU 3: ksm_init: 18 MTRR ranges (0 default type)
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000000000 -> 0x0000000000010000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000010000 -> 0x0000000000020000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000020000 -> 0x0000000000030000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000030000 -> 0x0000000000040000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000040000 -> 0x0000000000050000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000050000 -> 0x0000000000060000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000060000 -> 0x0000000000070000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000070000 -> 0x0000000000080000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000080000 -> 0x0000000000084000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000084000 -> 0x0000000000088000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000088000 -> 0x000000000008C000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x000000000008C000 -> 0x0000000000090000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000090000 -> 0x0000000000094000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000094000 -> 0x0000000000098000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000098000 -> 0x000000000009C000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x000000000009C000 -> 0x00000000000A0000 fixed: 1 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x0000000000000000 -> 0x0000001000000000 fixed: 0 type: 6
ksm: CPU 3: ksm_init: MTRR Range: 0x00000000C0000000 -> 0x0000000100000000 fixed: 0 type: 0
ksm: CPU 3: DriverEntry: ready
ksm: CPU 3: DriverEntry: ret: 0x00000000
ksm: CPU 3: DriverDispatch: open from ksm_um.exe
ksm: CPU 3: DriverDispatch: ksm_um.exe: IOCTL: 0x8008E008 of length: 0
ksm: CPU 2: __ksm_init_cpu: NisSrv.exe: Started: 1
ksm: CPU 3: __ksm_init_cpu: NisSrv.exe: Started: 1
ksm: CPU 0: __ksm_init_cpu: ksm_um.exe: Started: 1
ksm: CPU 1: __ksm_init_cpu: NisSrv.exe: Started: 1
KDTARGET: Refreshing KD connection

*** Fatal System Error: 0x0000007f
(0x0000000000000008,0xFFFFCE01F4C09F50,0x0000000000000001,0xFFFFF80032D810E9)

Break instruction exception - code 80000003 (first chance)

A fatal system error has occurred.
Debugger entered on first try; Bugcheck callbacks have not been invoked.

A fatal system error has occurred.

Connected to Windows 10 17763 x64 target at (Tue May 14 13:46:21.028 2019 (UTC + 8:00)), ptr64 TRUE
Loading Kernel Symbols
................................

Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
Run !sym noisy before .reload to track down problems loading symbols.

...............................
................................................................
....................................................
Loading User Symbols
.....
Loading unloaded module list
............
Loading Wow64 Symbols
............................................................

Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
Run !sym noisy before .reload to track down problems loading symbols.

....
................................................................
...............................

•                                                                         *
  

•                    Bugcheck Analysis                                    *
  

•                                                                         *
  

Use !analyze -v to get detailed debugging information.

BugCheck 7F, {8, ffffce01f4c09f50, 1, fffff80032d810e9}

"C:\Windows\System32\KERNELBASE.dll" was not found in the image list.
Debugger will attempt to load "C:\Windows\System32\KERNELBASE.dll" at given base 00000000`00000000.

Please provide the full image name, including the extension (i.e. kernel32.dll)
for more reliable results.Base address and size overrides can be given as
.reload <image.ext>=,.
Probably caused by : ksm.sys ( ksm!__vmx_entrypoint+72 )

Followup: MachineOwner

nt!DbgBreakPointWithStatus:
fffff800`2ddbd0a0 cc int 3
3: kd> !analyze -v

•                                                                         *
  

•                    Bugcheck Analysis                                    *
  

•                                                                         *
  

UNEXPECTED_KERNEL_MODE_TRAP (7f)
This means a trap occurred in kernel mode, and it's a trap of a kind
that the kernel isn't allowed to have/catch (bound trap) or that
is always instant death (double fault). The first number in the
bugcheck params is the number of the trap (8 = double fault, etc)
Consult an Intel x86 family manual to learn more about what these
traps are. Here is a portion of those codes:
If kv shows a taskGate
use .tss on the part before the colon, then kv.
Else if kv shows a trapframe
use .trap on that value
Else
.trap on the appropriate frame will show where the trap was taken
(on x86, this will be the ebp that goes with the procedure KiTrap)
Endif
kb will then show the corrected stack.
Arguments:
Arg1: 0000000000000008, EXCEPTION_DOUBLE_FAULT
Arg2: ffffce01f4c09f50
Arg3: 0000000000000001
Arg4: fffff80032d810e9

Debugging Details:

KEY_VALUES_STRING: 1

STACKHASH_ANALYSIS: 1

TIMELINE_ANALYSIS: 1

DUMP_CLASS: 1

DUMP_QUALIFIER: 0

BUILD_VERSION_STRING: 17763.1.amd64fre.rs5_release.180914-1434

DUMP_TYPE: 0

BUGCHECK_P1: 8

BUGCHECK_P2: ffffce01f4c09f50

BUGCHECK_P3: 1

BUGCHECK_P4: fffff80032d810e9

BUGCHECK_STR: 0x7f_8

BAD_STACK_POINTER: ffffce01f4c09648

CPU_COUNT: 4

CPU_MHZ: 8a0

CPU_VENDOR: GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: 9e

CPU_STEPPING: a

CPU_MICROCODE: 6,9e,a,0 (F,M,S,R) SIG: 84'00000000 (cache) 84'00000000 (init)

DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT

PROCESS_NAME: OneDrive.exe

CURRENT_IRQL: 0

ANALYSIS_SESSION_HOST: DESKTOP-M522AG6

ANALYSIS_SESSION_TIME: 05-14-2019 13:49:24.0341

ANALYSIS_VERSION: 10.0.17763.1 amd64fre

LAST_CONTROL_TRANSFER: from fffff8002de92cf2 to fffff8002ddbd0a0

STACK_TEXT:
ffffce01f4c09648 fffff8002de92cf2 : 0000000000000008 0000000000000003 ffffce01f4c097b0 fffff8002dd5d060 : nt!DbgBreakPointWithStatus
ffffce01f4c09650 fffff8002de92477 : 0000000000000003 ffffce01f4c097b0 fffff8002ddc9460 000000000000007f : nt!KiBugCheckDebugBreak+0x12
ffffce01f4c096b0 fffff8002ddb5547 : 0000000000000000 0000000000000000 000000000006362c 0000000000000000 : nt!KeBugCheck2+0x957
ffffce01f4c09dd0 fffff8002ddc6c69 : 000000000000007f 0000000000000008 ffffce01f4c09f50 0000000000000001 : nt!KeBugCheckEx+0x107
ffffce01f4c09e10 fffff8002ddc1ca8 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiBugCheckDispatch+0x69
ffffce01f4c09f50 fffff80032d810e9 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiDoubleFaultAbort+0x2a8
0000000000000001 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : ksm!__vmx_entrypoint+0x72 [C:\Users\Documents\Code\OpenSource\ksm\vmx.asm @ 266]

THREAD_SHA1_HASH_MOD_FUNC: 6b58434ef1ddf4f30217c266c8d33bb2905704d5

THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 42bbe0a8c5964c1490f9e75da735695266e806e4

THREAD_SHA1_HASH_MOD: 93e457b14469d1689dc46086e4e788eb4343ae51

FOLLOWUP_IP:
ksm!__vmx_entrypoint+72 [C:\Users\Documents\Code\OpenSource\ksm\vmx.asm @ 266]
fffff800`32d810e9 50 push rax

FAULT_INSTR_CODE: c3519d50

FAULTING_SOURCE_LINE: C:\Users\Documents\Code\OpenSource\ksm\vmx.asm

FAULTING_SOURCE_FILE: C:\Users\Documents\Code\OpenSource\ksm\vmx.asm

FAULTING_SOURCE_LINE_NUMBER: 266

FAULTING_SOURCE_CODE:
262:
263: ; Give them their stack pointer
264: mov rsp, rdx
265:

266: push rax
267: popfq ; eflags to indicate success
268:
269: push rcx ; return address (rip + instr len)
270: ret
271:

SYMBOL_STACK_INDEX: 6

SYMBOL_NAME: ksm!__vmx_entrypoint+72

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: ksm

IMAGE_NAME: ksm.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 5cda55e5

STACK_COMMAND: .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET: 72

FAILURE_BUCKET_ID: 0x7f_8_STACKPTR_ERROR_ksm!__vmx_entrypoint

BUCKET_ID: 0x7f_8_STACKPTR_ERROR_ksm!__vmx_entrypoint

PRIMARY_PROBLEM_CLASS: 0x7f_8_STACKPTR_ERROR_ksm!__vmx_entrypoint

TARGET_TIME: 2019-05-14T05:46:19.000Z

OSBUILD: 17763

OSSERVICEPACK: 0

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK: 272

PRODUCT_TYPE: 1

OSPLATFORM_TYPE: x64

OSNAME: Windows 10

OSEDITION: Windows 10 WinNt TerminalServer SingleUserTS

OS_LOCALE:

USER_LCID: 0

OSBUILD_TIMESTAMP: unknown_date

BUILDDATESTAMP_STR: 180914-1434

BUILDLAB_STR: rs5_release

BUILDOSVER_STR: 10.0.17763.1.amd64fre.rs5_release.180914-1434

ANALYSIS_SESSION_ELAPSED_TIME: 933

ANALYSIS_SOURCE: KM

FAILURE_ID_HASH_STRING: km:0x7f_8_stackptr_error_ksm!__vmx_entrypoint

FAILURE_ID_HASH: {e54130eb-8cc9-b505-6b94-54fc35ddda77}

Followup: MachineOwner

Hi,
I saw that you need KeSignalCallDpcSynchronize() to launch vmx for each cpu core. Is it right?
Do you know how to operate this on Linux Platform?
Thank you very very much!

Performed from following on Ubuntu 14.04 64bit in Virtual Box
Cross under Linux

Install the following packages:

• Debian/Ubuntu: [sudo] apt-get install gcc-mingw-w64-x86-64 binutils-mingw-w64-x86-64
  - ArchLinux: [sudo] pacman -S mingw-w64-gcc
  - Fedora: [sudo] yum install mingw64-gcc

Then `make -f Makefile.windows C=1 all`

Following Output Received:

ksm@ksm:~/ksm$ make -f Makefile.windows C=1 all
x86_64-w64-mingw32-gcc -shared -Wl,--subsystem,native -Wl,--dynamicbase -Wl,--stack=0x6000 -Wl,--file-alignment,0x1000 -Wl,--section-alignment,0x1000 -Wl,--entry,DriverEntry -Wl,--nxcompat -Wl,--exclude-all-symbols -Wl,--enable-stdcall-fixup -nostartfiles -nostdlib -o bin/ksm.sys obj/exit.o obj/htable.o obj/hotplug.o obj/introspect.o obj/ksm.o obj/sandbox.o obj/mm.o obj/main_nt.o obj/epage.o obj/print.o obj/resubv.o obj/vcpu.o obj/vmx.o -L/usr/x86_64-w64-mingw32/lib -lntoskrnl -lhal -lmingwex
obj/exit.o:exit.c:(.text+0x4fc): undefined reference to `__writecr2'
obj/exit.o:exit.c:(.text+0x10c3): undefined reference to `__writecr2'
obj/exit.o:exit.c:(.text+0x1294): undefined reference to `__writecr2'
obj/exit.o:exit.c:(.text+0x28e4): undefined reference to `__writecr2'
obj/exit.o:exit.c:(.text+0x2934): undefined reference to `__writecr2'
obj/exit.o:exit.c:(.text+0x2b07): more undefined references to `__writecr2' follow
/usr/bin/x86_64-w64-mingw32-ld: obj/exit.o: bad reloc address 0x334 in section `.rdata'
collect2: error: ld returned 1 exit status
make: *** [bin/ksm.sys] Error 1

How to fix this issue?

The following occurs after a bit of sandboxing an app, I haven't looked much into it, but I suspect that some service (svchost.exe in this case) is trying to duplicate handle of the sandboxed app and it somehow just fails... Here's the stack dump:

fffff880`07d1f650 fffff800`02b6481b : 00000000`00000002 fffff880`07d1f758 fffffa80`03d85170 fffffa80`040c1b30 : nt!ObpIncrementHandleCountEx+0x411
fffff880`07d1f710 fffff800`02b64d48 : fffff8a0`01ca9400 fffff800`00000040 fffffa80`018d6a20 00000000`00000001 : nt!ObDuplicateObject+0x21b
fffff880`07d1f9f0 fffff800`0288a8d3 : fffffa80`03d97910 00000000`0077ee38 fffff880`07d1fa88 fffffa80`00000000 : nt!NtDuplicateObject+0x138
fffff880`07d1fa70 00000000`76f716da : 000007fe`fd0f3b45 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
00000000`0077ee18 000007fe`fd0f3b45 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!ZwDuplicateObject+0xa
00000000`0077ee20 00000000`76e15e6b : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : KERNELBASE!DuplicateHandle+0x35
00000000`0077ee70 000007fe`f104a3af : 00000000`002a7458 000007fe`f0f96c98 00000000`002a7458 00000000`000000d0 : kernel32!DuplicateHandleImplementation+0x15b
00000000`0077ef90 000007fe`f1045445 : 00000000`00000648 00000000`00000001 00000000`00000648 000007fe`f1054070 : wersvc!ReportCrash+0x257
00000000`0077f020 000007fe`f1045230 : 00000000`0039dff8 00000000`00000000 00000000`00000000 00000000`002a7450 : wersvc!CWerService::DispatchPortRequestWorkItem+0x1cd
00000000`0077f690 00000000`76f32a21 : 00000000`003a5140 00000000`00000000 00000000`00000000 00000000`003a51f0 : wersvc!CWerService::StaticDispatchPortRequestWorkItem+0x18
00000000`0077f6c0 00000000`76f40c26 : 00000000`770245e8 00000000`003a3418 00000000`770245c0 00000000`77024610 : ntdll!TppSimplepExecuteCallback+0x91
00000000`0077f710 00000000`76e1652d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!TppWorkerThread+0x5ff
00000000`0077fa10 00000000`76f4c521 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`0077fa40 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d

Log:

ksm: CPU 0: DriverEntry: We're mapped at FFFFF88004B63000 (size: 114688 bytes (112 KB), on 28 pages)
ksm: CPU 0: ksm_init: 3 physical memory ranges
ksm: CPU 0: DriverEntry: ready
ksm: CPU 0: DriverEntry: ret: 0x00000000
ksm: CPU 0: DriverDispatch: ksm_um.exe: IOCTL: 0x8008E008
ksm: CPU 0: __ksm_init_cpu: ksm_um.exe: Started: 1
ksm: CPU 1: __ksm_init_cpu: vmtoolsd.exe: Started: 1
ksm: CPU 0: ept_handle_violation: 0: PA 00000000FD5C00C0 VA FFFFF8800377B0C0 (0 AR --- - 1 AC r--)
ksm: CPU 1: ept_handle_violation: 0: PA 00000000FD5C4000 VA FFFFF8800377F000 (0 AR --- - 1 AC r--)
ksm: CPU 0: ept_handle_violation: 0: PA 00000000FDFEC024 VA FFFFF880037B2024 (0 AR --- - 1 AC r--)
ksm: CPU 0: ept_handle_violation: 0: PA 00000000FD5EA008 VA FFFFF880009B3008 (0 AR --- - 1 AC r--)
ksm: CPU 0: ept_handle_violation: 0: PA 00000000FD5C2818 VA FFFFF8800377D818 (0 AR --- - 2 AC -w-)
ksm: CPU 0: ept_handle_violation: 0: PA 00000000FD5C5820 VA FFFFF88003780820 (0 AR --- - 1 AC r--)
ksm: CPU 0: ept_handle_violation: 0: PA 00000000FD5C40D4 VA FFFFF8800377F0D4 (0 AR --- - 1 AC r--)
ksm: CPU 0: ept_handle_violation: 0: PA 00000000FD5EB010 VA FFFFF880037AB010 (0 AR --- - 1 AC r--)
ksm: CPU 1: ept_handle_violation: 0: PA 00000000FD5EA1B8 VA FFFFF880009B31B8 (0 AR --- - 1 AC r--)
ksm: CPU 0: ept_handle_violation: 0: PA 00000000FD4EC040 VA FFFFF880009AF040 (0 AR --- - 2 AC -w-)
ksm: CPU 1: ept_handle_violation: 0: PA 00000000FD4EC040 VA FFFFF880009AF040 (0 AR --- - 2 AC -w-)
ksm: CPU 0: DriverDispatch: ksm_um.exe: IOCTL: 0x8008E000
ksm: CPU 0: ept_handle_violation: 3: PA 00000000FD5EB010 VA FFFFF880037AB010 (0 AR --- - 1 AC r--)
ksm: CPU 1: ept_handle_violation: 3: PA 0000000063957C80 VA 00000000002EDC80 (5 AR r-x - 2 AC -w-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 0000000063957C80
ksm: CPU 1: ept_handle_violation: 3: PA 0000000064888254 VA 00000000002EF254 (5 AR r-x - 3 AC rw-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 0000000064888254
ksm: CPU 1: ept_handle_violation: 3: PA 00000000647C3F9C VA 0000000000B7F9C (5 AR r-x - 3 AC rw-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 00000000647C3F9C
ksm: CPU 0: ept_handle_violation: 3: PA 000000005F9F30AC VA 000000013F7310AC (5 AR r-x - 3 AC rw-)
ksm: CPU 0: ksm_sandbox_handle_ept: allocating cow page for 000000005F9F30AC
ksm: CPU 0: ept_handle_violation: 3: PA 0000000064888368 VA 00000000002EF368 (5 AR r-x - 2 AC -w-)
ksm: CPU 0: ksm_sandbox_handle_ept: allocating cow page for 0000000064888368
ksm: CPU 0: ept_handle_violation: 3: PA 00000000632341B0 VA 000000013F7321B0 (5 AR r-x - 3 AC rw-)
ksm: CPU 0: ksm_sandbox_handle_ept: allocating cow page for 00000000632341B0
ksm: CPU 0: ept_handle_violation: 3: PA 0000000051A80068 VA 000007FFFFFDE068 (5 AR r-x - 2 AC -w-)
ksm: CPU 0: ksm_sandbox_handle_ept: allocating cow page for 0000000051A80068
ksm: CPU 0: ept_handle_violation: 3: PA 000000005FB6CA90 VA 00000000000BCA90 (5 AR r-x - 2 AC -w-)
ksm: CPU 0: ksm_sandbox_handle_ept: allocating cow page for 000000005FB6CA90
ksm: CPU 0: ept_handle_violation: 3: PA 00000000647C3F98 VA 00000000000B7F98 (5 AR r-x - 3 AC rw-)
ksm: CPU 0: ksm_sandbox_handle_ept: allocating cow page for 00000000647C3F98
ksm: CPU 0: ept_handle_violation: 3: PA 0000000063957DF0 VA 00000000002EDDF0 (5 AR r-x - 2 AC -w-)
ksm: CPU 0: ksm_sandbox_handle_ept: allocating cow page for 0000000063957DF0
ksm: CPU 1: ept_handle_violation: 3: PA 00000000643CDB00 VA 00000000002EEB00 (5 AR r-x - 2 AC -w-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 00000000643CDB00
ksm: CPU 1: ept_handle_violation: 3: PA 0000000064A4A440 VA 0000000077052440 (5 AR r-x - 3 AC rw-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 0000000064A4A440
ksm: CPU 1: ept_handle_violation: 3: PA 000000005EC71000 VA FFFFF8800A8E4000 (5 AR r-x - 2 AC -w-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 000000005EC71000
ksm: CPU 1: ept_handle_violation: 3: PA 00000000FD4EC040 VA FFFFF880009AF040 (0 AR --- - 2 AC -w-)
ksm: CPU 1: ept_handle_violation: 3: PA 0000000051A80068 VA 000007FFFFFD
68 (5 AR r-x - 2 AC -w-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 0000000051A80068
ksm: CPU 1: ept_handle_violation: 3: PA 000000005F9A0328 VA 0000000076F0B328 (5 AR r-x - 3 AC rw-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 000000005F9A0328
ksm: CPU 1: ept_handle_violation: 3: PA 00000000658E4BF8 VA 00000000000BABF8 (5 AR r-x - 2 AC -w-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 00000000658E4BF8
ksm: CPU 1: ept_handle_violation: 3: PA 000000005EEF2000 VA FFFFF70001081000 (5 AR r-x - 2 AC -w-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 000000005EEF2000
ksm: CPU 1: ept_handle_violation: 3: PA 000000005EBF3000 VA 0000000000060000 (5 AR r-x - 2 AC -w-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 000000005EBF3000
ksm: CPU 1: ept_handle_violation: 3: PA 0000000065001250 VA 000007FFFFFDF250 (5 AR r-x - 2 AC -w-)
ksm: CPU 1: ksm_sandbox_handle_ept: allocating cow page for 0000000065001250

Type of this issue (please specify)

This is a support matter (i.e. your own modified tree). I've removed CPU_DYING because it is not supported anymore.

System information

1. CPU: Intel (Codename: i5-4278U)
2. Kernel: Linux
3. Kernel version: 4.9.0-3-amd64

Issue description

After running sudo ./a.out the machine freeze, I was able to understand that "ioctl" causes the freeze.
Since the machine just become unresponsive there is no logs,

Nice to have, there is not really that much that needs to be done besides mainly converting calling conventions and some minor stuff. The assembly stuff in x64.S can be re-used, just need some macros to for the calling conventions (e.g. rsi in place of rcx, rdi in place of rdx, etc.). Obviously some intrinsics won't be available there, but those can be easily defined with GCC inline assembly, most of the painful ones are already there, just need stuff like __writecr3 etc, we should probably just ditch Linux defined ones and re-use the names not to confuse each other.

Seems to generate #DF for some reason on setup. Maybe the APIC access page is bad.

Since we use multiple EPT pointers, in most cases a lot of entries will be redundant (i.e. they map same page), so we can optimize memory footprint by merging them together across EPTPs, then we can split the entries when the mapping has changed for one or more EPTP.

Type of this issue (please specify)

• [x ] This is a bug in the upstream tree as-is unmodified.
• This is a support matter (i.e. your own modified tree)
• This is a technical question

System information

1. CPU: i7-4770
2. Kernel: Linux
3. Kernel version: 4.9

I have been trying to deploy KSM on my desktop using Linux kernel 4.9.
I actually manage to load the module but while subverting it, i get a lot of troubles.
-Sometimes it works (rarely)
-When it crushes, I have an exit with (see exit.c):
- Exit reason 10,
- exit reason 48
- Triple fault

• kernel panic
  128.232471] ksm: CPU 7: ksm_open: open() from a.out
  [ 128.237388] ksm: CPU 7: ksm_ioctl: ioctl from a.out: cmd(0x00004B02)
  [ 129.194462] ksm: CPU 0: __ksm_init_cpu: swapper/0: Started: 1
  [ 129.200279] ksm: CPU 0: vcpu_handle_exit: exit 48 prev=0
  [ 129.414808] ksm: CPU 0: vcpu_handle_exit: exit 48 prev=48
  [ 129.420264] ksm: CPU 0: vcpu_handle_exit: exit 48 prev=48
  [ 130.012442] ksm: CPU 0: vcpu_handle_exit: exit 48 prev=48
  [ 130.017919] ksm: CPU 0: vcpu_handle_exit: exit 48 prev=48
  [ 130.023348] ksm: CPU 0: vcpu_handle_exit: exit 48 prev=48
  [ 130.152079] ksm: CPU 1: __ksm_init_cpu: swapper/1: Started: 1
  [ 130.157905] ksm: CPU 1: vcpu_handle_exit: exit 48 prev=0
  [ 130.163337] ksm: CPU 1: vcpu_handle_exit: exit 2 prev=48
  [ 130.168676] Kernel panic - not syncing: bugcheck 00000000CCDDFF11 0000000033DDE83A 0x0000000000000002 0x0000000000000030
  [ 130.168676]
  [ 130.181083] CPU: 1 PID: 316 Comm: systemd-journal Tainted: G OE 4.9.260+ #1
  [ 130.189125] Hardware name: Dell Inc. Precision T1700/073MMW, BIOS A08 04/25/2014
  [ 130.196558] ffff9b1457a0de68 ffffffff8949496b ffff9b1457a0c000 ffffffffc04c7020
  [ 130.204032] ffff9b1457a0dee8 ffffffff89492e82 00007ffd00000028 ffff9b1457a0def8
  [ 130.211507] ffff9b1457a0de90 ffffffff88ce2559 00000000ccddff11 0000000033dde83a
  [ 130.218980] Call Trace:
  [ 130.221457] ksm: CPU 0: vcpu_handle_exit: exit 10 prev=48
  [ 131.165855] ksm: CPU 2: __ksm_init_cpu: swapper/2: Started: 1
  [ 131.171642] ------------[ cut here ]------------
  [ 131.176283] WARNING: CPU: 2 PID: 0 at arch/x86/kernel/smp.c:128 native_smp_send_reschedule+0x40/0x50
  [ 131.185462] Modules linked in: ksmlinux(OE) nls_iso8859_1 nouveau snd_hda_codec_hdmi intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp i915 snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec crct10dif_pclmul ghash_clmulni_intel snd_hda_core snd_hwdep aesni_intel snd_pcm aes_x86_64 lrw glue_helper ablk_helper mxm_wmi dcdbas cryptd wmi dell_smm_hwmon snd_seq_midi intel_cstate input_leds ttm snd_seq_midi_event snd_rawmidi snd_seq drm_kms_helper intel_rapl_perf serio_raw snd_seq_device snd_timer i2c_algo_bit snd mei_me fb_sys_fops mei syscopyarea sysfillrect sysimgblt soundcore video mac_hid sch_fq_codel[ 131.241858] Shutting down cpus with NMI
  [ 131.241869] Kernel Offset: 0x7c00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
  [ 131.241871] ksm: CPU 2: vcpu_handle_exit: exit 10 prev=0
  [ 131.322523] ---[ end Kernel panic - not syncing: bugcheck 00000000CCDDFF11 0000000033DDE83A 0x0000000000000002 0x0000000000000030
  [ 131.322523]

KSM path: https://github.com/asamy/ksm

Type of this issue (please specify)

• This is a bug in the upstream tree as-is unmodified. (only added debug print messages)

System information

1. CPU: Intel i7 3770K (Codename: Ivy Bridge)
2. Kernel: NT Kernel (ntkrnlmp.exe)
3. Kernel version: 10.0.14393 Build number: ?
4. OS: Windows 10 x64

Build Configuration

• ENABLE_DBGPRINT
• ENABLE_FILEPRINT
• EPAGE_HOOK
• INTROSPECT_ENGINE
• PMEM_SANDBOX
• ENABLE_RESUBV

I have to mention that this also happens when I just enable EPAGE_HOOK

Issue description

My issues is, that after I start the user mode application (ksm_um.exe), my PC just freezes.
I have waited up to about 10 minutes without anything happening. No crash, nothing.
The last log entry in the log file is right before the DPC call. After that, nothing gets through.
When I tested other hypervisors, I always got some kind of feedback (good or bad) like a BSOD which would help to track down the issue.

Is nested support functional? I would like to reproduce this scenario for academic purpose for RDTSC integrity:

L0 KSM -> L1 KSM (Malicious)
L0 controls RDTSC/RDTSCP exit handler, L1 tries to controls them.
Guest software call RDTSC, L0 returns the bare machine value WITHOUT invoking L1

Can I do this?

1. in memory hack module, when a page code read itself , ept voilation will happen again and agian...
   when set a page read/write only, the code reading the page that the code is in will cause voilation (because of executing)
   when set a page execute only, the code reading the page that the code is in will cause voilation (because of reading)

If i trigger the #VE in the other VMM(eg.kvm),how can i set the ve_except_info ->except_mask=0 to enable another #VE occurs in guest?

Need I say words?

After cross compiling windows driver on linux and loading it in windows, when user mode app is run Driver crashes with BUG_CHECK 0xD1 : DRIVER_IRQL_NOT_LESS_OR_EQUAL

Perhaps even completely create new tables for the Host, e.g. GDT, IDT, LAPIC, etc. This would make it a lot better and flexible, we could even use it as a boot time hypervisor that way (e.g. start before the kernel does).

This is quite a low priority issue, but I will play with this sometime later for fun.

Seems that the MinGW build of KSM does not produce output to DebugView at all, but outputs fine to WinDBG which is very weird...

Seems to just casually BSOD. It runs for a while fine with the beep sound (the BSOD-incoming sound), then crashes with DRIVER_IRQL_NOT_LESS_OR_EQUAL. Specifically in vcpu_sync_idt in the memcpy call.

Type of this issue (please specify)

• This is a bug in the upstream tree as-is unmodified.
• This is a support matter (i.e. your own modified tree)
• This is a technical question

Issue description

mtrr_ranges array has wrong size and may eventually overflow if user gets really REALLY unlucky

Current Behavior

Let's deconstruct this issue:
in ksm.h:
struct mtrr_range mtrr_ranges[MAX_MTRR];

in mm.h:

#define MAX_VAR_MTRR		255
#define MAX_FIXED_MTRR		11
#define MAX_MTRR		MAX_VAR_MTRR + MAX_FIXED_MTRR

in ksm.c:

	mm_cache_mtrr_ranges(&k->mtrr_ranges[0], &k->mtrr_count, &k->mtrr_def);

in mm.c:

		for (msr = __readmsr(MSR_MTRRfix64K_00000), offset = 0x10000, base = 0;
		     msr != 0; msr >>= 8, base += offset)
			make_mtrr_range(&ranges[idx++], true, msr & 0xff, base, base + 0x10000);

		for (val = MSR_MTRRfix16K_80000, offset = 0x4000; val <= MSR_MTRRfix16K_A0000; ++val)
			for (msr = __readmsr(val), base = 0x80000;
			     msr != 0; msr >>= 8, base += offset)
				make_mtrr_range(&ranges[idx++], true, msr & 0xff, base, base + 0x4000);

		for (val = MSR_MTRRfix4K_C0000, offset = 0x1000; val <= MSR_MTRRfix4K_F8000; ++val)
			for (msr = __readmsr(val), base = 0xC0000;
			     msr != 0; msr >>= 8, base += offset)
				make_mtrr_range(&ranges[idx++], true, msr & 0xff, base, base + 0x1000);

Here's the problem: for each fixed MTRR, there are actually 8 subranges.
Intel manual says:

11.11.2.2 Fixed Range MTRRs
The fixed memory ranges are mapped with 11 fixed-range registers of 64 bits each. Each of these registers is
divided into 8-bit fields that are used to specify the memory type for each of the sub-ranges the register controls:
• Register IA32_MTRR_FIX64K_00000 — Maps the 512-KByte address range from 0H to 7FFFFH. This range
is divided into eight 64-KByte sub-ranges.
• Registers IA32_MTRR_FIX16K_80000 and IA32_MTRR_FIX16K_A0000 — Maps the two 128-KByte
address ranges from 80000H to BFFFFH. This range is divided into sixteen 16-KByte sub-ranges, 8 ranges per
register.
• Registers IA32_MTRR_FIX4K_C0000 through IA32_MTRR_FIX4K_F8000 — Maps eight 32-KByte
address ranges from C0000H to FFFFFH. This range is divided into sixty-four 4-KByte sub-ranges, 8 ranges per
register.

Expected Behavior

I think fix is as simple as defining MAX_FIXED_MTRR to 11*8.

Inline code / patches to be used when reproducing

From 57ad5c11ab9d5d8c353379b21207902c3e47ba4d Mon Sep 17 00:00:00 2001
From: Petr Benes <[email protected]>
Date: Tue, 24 Jul 2018 00:43:12 +0200
Subject: [PATCH] fix MAX_FIXED_MTRR definition

---
 mm.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm.h b/mm.h
index 1a79519..d9863a4 100644
--- a/mm.h
+++ b/mm.h
@@ -406,7 +406,7 @@ struct mtrr_range {
 };
 
 #define MAX_VAR_MTRR		255
-#define MAX_FIXED_MTRR		11
+#define MAX_FIXED_MTRR		11*8
 #define MAX_MTRR		MAX_VAR_MTRR + MAX_FIXED_MTRR
 extern void mm_cache_mtrr_ranges(struct mtrr_range *ranges, int *count, u8 *def_type);
 
-- 
2.17.0.windows.1

Type of this issue (please specify)

• This is a bug in the upstream tree as-is unmodified.
• This is a support matter (i.e. your own modified tree)
• This is a technical question

System information

1. CPU: inrel core i5-6200u
2. Kernel: linux
3. Kernel version: 3.16.0-23-generic

Issue description

I want to execute read_vmcs(GUEST_CR3), but it occurs errors. I want to creat a model for EPT translating. what should i do? Thanks.

open_device();
do_ioctl(dev, KSM_IOCTL_SUBVERT, NULL, 0);
do_ioctl(dev, KSM_MY_EPT_START, NULL, 0);
do_ioctl(dev, KSM_MY_EPT_HANDLE, NULL, 0);
......

case KSM_MY_EPT_HANDLE:
......
cr3 = vmcs_read(GUEST_CR3);

For Linux

• ksmlinux.ko and ksmlinux.o
• Stack dump from dmesg or kernel panic
  [ 1113.715543] ksm: CPU 1: ksm_open: open() from a.out
  [ 1113.716271] ksm: CPU 1: ksm_ioctl: ioctl from a.out: cmd(0x00004B02)
  [ 1113.843726] ksm: CPU 0: vcpu_run: cpu[0]: vmxon succeed.
  [ 1113.845012] ksm: CPU 0: __ksm_init_cpu: systemd-udevd: Started: 1
  [ 1114.003092] ksm: CPU 1: vcpu_run: cpu[1]: vmxon succeed.
  [ 1114.003866] ksm: CPU 1: __ksm_init_cpu: a.out: Started: 1
  [ 1114.003904] ksm: CPU 1: ksm_ioctl: ioctl ret: 0
  [ 1114.005308] ksm: CPU 1: ksm_ioctl: ioctl from a.out: cmd(0x00004B0E)
  [ 1114.005315] ksm: CPU 1: ksm_my_ept_start: ksm_my_ept_starting!!
  [ 1114.005327] ksm: CPU 1: ksm_ioctl: ioctl ret: -22
  [ 1114.005338] ksm: CPU 1: ksm_ioctl: ioctl from a.out: cmd(0x00004B10)
  [ 1114.005339] ksm: CPU 1: ksm_my_ept_handle: vcpu activate is 2
  [ 1114.006281] invalid opcode: 0000 [#1] SMP
  [ 1114.006604] Modules linked in: linux_ksm(OE) vmhgfs(OE) vmw_vsock_vmci_transport vsock kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_ac97_codec ac97_bus gameport snd_pcm snd_seq_midi snd_seq_midi_event snd_rawmidi serio_raw snd_seq snd_seq_device vmwgfx snd_timer ttm drm_kms_helper snd drm soundcore vmw_vmci i2c_piix4 shpchp bnep rfcomm bluetooth 6lowpan_iphc mac_hid parport_pc ppdev lp parport hid_generic usbhid hid psmouse mptspi mptscsih ahci libahci mptbase e1000 scsi_transport_spi pata_acpi vmw_pvscsi vmxnet3 [last unloaded: linux_ksm]
  [ 1114.006886] CPU: 1 PID: 6901 Comm: a.out Tainted: G OE 3.16.0-23-generic #31-Ubuntu
  [ 1114.006888] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
  [ 1114.006893] task: ffff8800362c5bb0 ti: ffff880008d9c000 task.ti: ffff880008d9c000
  [ 1114.006895] RIP: 0010:[] [] ksm_my_ept_handle+0x30/0x54 [linux_ksm]
  [ 1114.006924] RSP: 0018:ffff880008d9feb0 EFLAGS: 00000282
  [ 1114.006925] RAX: 0000000000000032 RBX: ffffffffffffffea RCX: 0000000000006802
  [ 1114.006927] RDX: 0000000000000007 RSI: 0000000000000046 RDI: 0000000000000246
  [ 1114.006928] RBP: ffff880008d9feb0 R08: 0000000000000845 R09: 0000000000000082
  [ 1114.006932] R10: 00007fd9aa68e6a0 R11: 796d5f6d736b203a R12: 0000000000000000
  [ 1114.006933] R13: ffff8800584566b8 R14: 0000000000004b10 R15: 0000000000000000
  [ 1114.006935] FS: 00007fd9aa89e740(0000) GS:ffff88007c620000(0000) knlGS:0000000000000000
  [ 1114.006937] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  [ 1114.006938] CR2: 00007fd9aa8b5000 CR3: 0000000077c6f000 CR4: 00000000003407e0
  [ 1114.006999] Stack:
  [ 1114.007001] ffff880008d9fed0 ffffffffc05d6133 ffff8800362c61a0 ffff880078d53600
  [ 1114.007006] ffff880008d9ff38 ffffffff811f4bc8 ffff8800164bc600 ffff88007c634800
  [ 1114.007008] ffff8800164bc600 0000000000000001 ffff880008d9ff60 ffffffff8178294d
  [ 1114.007010] Call Trace:
  [ 1114.007020] [] ksm_ioctl+0x93/0x260 [linux_ksm]
  [ 1114.007062] [] do_vfs_ioctl+0x2c8/0x4a0
  [ 1114.007104] [] ? __schedule+0x39d/0x890
  [ 1114.007107] [] SyS_ioctl+0x81/0xa0
  [ 1114.007113] [] system_call_fastpath+0x1a/0x1f
  [ 1114.007115] Code: 55 8b 0f 48 c7 c2 10 d6 5d c0 48 c7 c7 50 e2 5d c0 31 c0 48 89 e5 65 8b 34 25 84 b0 00 00 e8 74 f8 19 c1 b9 02 68 00 00 0f 78 c9 <0f> 96 c0 48 c7 c2 10 d6 5d c0 48 c7 c7 80 e2 5d c0 31 c0 65 8b
  [ 1114.007140] RIP [] ksm_my_ept_handle+0x30/0x54 [linux_ksm]
  [ 1114.007144] RSP
  [ 1114.007237] ---[ end trace 66246c1b37ae79a0 ]---
  [ 1114.010532] ksm: CPU 1: ksm_release: release() from a.out

Expected Behavior

read cr3 of guest and translate EPT from gva to hpa.

Hello, i want to hook epage but i always got BSOD when do it, can you help me solve it?here my snippet:

PVOID hkMmMapIoSpace(
PHYSICAL_ADDRESS PhysicalAddress,
SIZE_T NumberOfBytes,
MEMORY_CACHING_TYPE CacheType
)
{
DbgPrint("hook mapio\n");
vcpu_vmfunc(EPTP_NORMAL, 0);
void *ret = MmMapIoSpace(PhysicalAddress, NumberOfBytes, CacheType);
vcpu_vmfunc(EPTP_EXHOOK, 0);
return ret;
}

then i hook epage after ksm ready :

RtlInitUnicodeString(&deviceLink, KSM_DOS_NAME);
if (NT_SUCCESS(status = IoCreateSymbolicLink(&deviceLink, &deviceName))) {
KSM_DEBUG_RAW("ready\n");
ksm->host_pgd = __readcr3();
ksm_hook_epage(MmMapIoSpace, hkMmMapIoSpace);
goto out;
}

then BSOD, here the log:

Microsoft (R) Windows Debugger Version 10.0.17763.132 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Windows\MEMORY.DMP]
Kernel Bitmap Dump File: Kernel address space is available, User address space may not be available.

Symbol search path is: srv*
Executable search path is:
Windows 10 Kernel Version 17763 MP (8 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 17763.1.amd64fre.rs5_release.180914-1434
Machine Name:
Kernel base = 0xfffff80232e0f000 PsLoadedModuleList = 0xfffff8023322a790
Debug session time: Fri May 3 07:22:57.372 2019 (UTC + 7:00)
System Uptime: 0 days 0:03:14.058
Loading Kernel Symbols
.....................................Page 2002ed57e too large to be in the dump file.
Page 2002efe7d too large to be in the dump file.
..........................
....Page 2002fa078 too large to be in the dump file.
............................................................
................................................................
................
Loading User Symbols
PEB is paged out (Peb.Ldr = 000000a2`fb19b018). Type ".hh dbgerr001" for details
Loading unloaded module list
.........

•                                                                         *
  

•                    Bugcheck Analysis                                    *
  

•                                                                         *
  

Use !analyze -v to get detailed debugging information.

BugCheck 1E, {ffffffffc000001d, fffff8023813111c, ffff9605aba08080, 0}

Probably caused by : ksm.sys ( ksm!__vmx_vmcall+0 )

Followup: MachineOwner

1: kd> !analyze -v

•                                                                         *
  

•                    Bugcheck Analysis                                    *
  

•                                                                         *
  

KMODE_EXCEPTION_NOT_HANDLED (1e)
This is a very common bugcheck. Usually the exception address pinpoints
the driver/function that caused the problem. Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: ffffffffc000001d, The exception code that was not handled
Arg2: fffff8023813111c, The address that the exception occurred at
Arg3: ffff9605aba08080, Parameter 0 of the exception
Arg4: 0000000000000000, Parameter 1 of the exception

Debugging Details:

KEY_VALUES_STRING: 1

STACKHASH_ANALYSIS: 1

TIMELINE_ANALYSIS: 1

DUMP_CLASS: 1

DUMP_QUALIFIER: 401

BUILD_VERSION_STRING: 17763.1.amd64fre.rs5_release.180914-1434

SYSTEM_MANUFACTURER: Micro-Star International Co., Ltd.

SYSTEM_PRODUCT_NAME: GT72 2QD

SYSTEM_SKU: To be filled by O.E.M.

SYSTEM_VERSION: REV:0.C

BIOS_VENDOR: American Megatrends Inc.

BIOS_VERSION: E1781IMS.316

BIOS_DATE: 09/23/2015

BASEBOARD_MANUFACTURER: Micro-Star International Co., Ltd.

BASEBOARD_PRODUCT: MS-1781

BASEBOARD_VERSION: REV:0.C

DUMP_TYPE: 1

BUGCHECK_P1: ffffffffc000001d

BUGCHECK_P2: fffff8023813111c

BUGCHECK_P3: ffff9605aba08080

BUGCHECK_P4: 0

EXCEPTION_CODE: (NTSTATUS) 0xc000001d - {EXCEPTION} Illegal Instruction An attempt was made to execute an illegal instruction.

FAULTING_IP:
ksm!__vmx_vmcall+0 [E:\Source\ksm\vmx.asm @ 287]
fffff802`3813111c 0f01c1 vmcall

EXCEPTION_PARAMETER1: ffff9605aba08080

BUGCHECK_STR: 0x1E_c000001d

CPU_COUNT: 8

CPU_MHZ: a86

CPU_VENDOR: GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: 47

CPU_STEPPING: 1

CPU_MICROCODE: 6,47,1,0 (F,M,S,R) SIG: 1D'00000000 (cache) 1D'00000000 (init)

BLACKBOXBSD: 1 (!blackboxbsd)

DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT

PROCESS_NAME: NVDisplay.Container.exe

CURRENT_IRQL: 2

ANALYSIS_SESSION_HOST: DESKTOP-6P18NJQ

ANALYSIS_SESSION_TIME: 05-03-2019 07:24:27.0595

ANALYSIS_VERSION: 10.0.17763.132 amd64fre

DPC_STACK_BASE: FFFF938C8EE37FB0

LAST_CONTROL_TRANSFER: from fffff8023309d07e to fffff80232fc2730

FAILED_INSTRUCTION_ADDRESS:
ksm!__vmx_vmcall+0 [E:\Source\ksm\vmx.asm @ 287]
fffff802`3813111c 0f01c1 vmcall

STACK_TEXT:
ffff938c8ee36a38 fffff8023309d07e : 000000000000001e ffffffffc000001d fffff8023813111c ffff9605aba08080 : nt!KeBugCheckEx
ffff938c8ee36a40 fffff80232fcb222 : fffff802332f2000 fffff80232e0f000 0005be3c00a6e000 000000000010001f : nt!KiFatalExceptionHandler+0x22
ffff938c8ee36a80 fffff80232f24240 : ffff938c8ee370d0 0000000000000000 ffff938c8ee36ff0 0000000000000000 : nt!RtlpExecuteHandlerForException+0x12
ffff938c8ee36ab0 fffff80232e31ac4 : ffff938c8ee379e8 ffff938c8ee37730 ffff938c8ee379e8 ffff938c93bdf630 : nt!RtlDispatchException+0x430
ffff938c8ee37200 fffff80232fd3f42 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiDispatchException+0x144
ffff938c8ee378b0 fffff80232fce78e : ffff960500000000 ffffad008836af08 0000000000000000 fffff80232ef68af : nt!KiExceptionDispatch+0xc2
ffff938c8ee37a90 fffff8023813111c : fffff802381312ba 0000000000400a02 0000000000000000 0000000000000000 : nt!KiInvalidOpcodeFault+0x30e
ffff938c8ee37c28 fffff802381312ba : 0000000000400a02 0000000000000000 0000000000000000 ffffad0088367f90 : ksm!__vmx_vmcall [E:\Source\ksm\vmx.asm @ 287]
ffff938c8ee37c30 fffff80232e88577 : ffffad0088367f80 ffff9605a0bbb000 ffff938c8ee37d60 0000000000000000 : ksm!__percpu___do_hook_page+0x1a [e:\source\ksm\epage.c @ 95]
ffff938c8ee37c60 fffff80232e87bbe : ffffad0088365180 0000000000000000 0000000000000002 0000000000000004 : nt!KiExecuteAllDpcs+0x2e7
ffff938c8ee37da0 fffff80232fc9595 : 0000000000000000 ffffad0088365180 ffff938c90617b40 0000000000000000 : nt!KiRetireDpcList+0x1ae
ffff938c8ee37fb0 fffff80232fc9380 : 0000000000000102 0000000000000000 00007ffad95897f0 0000000000000560 : nt!KxRetireDpcList+0x5
ffff938c90617a90 fffff80232fc8a6c : ffff9605aba08080 000000a2fbffca98 ffff938c90617a98 ffff9605aba7fc60 : nt!KiDispatchInterruptContinue
ffff938c90617ac0 00007ffad970995d : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiDpcInterrupt+0x2dc
000000a2fbffd470 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : 0x00007ffa`d970995d

THREAD_SHA1_HASH_MOD_FUNC: a8a9c7b3fbf112c4a80644be074a7883be953ef0

THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 93e92b7c547e03d379abd7930fd410f934b1f49c

THREAD_SHA1_HASH_MOD: 390a9432c7ea7ec685200dfa4bb581b24db674b6

FOLLOWUP_IP:
ksm!__vmx_vmcall+0 [E:\Source\ksm\vmx.asm @ 287]
fffff802`3813111c 0f01c1 vmcall

FAULT_INSTR_CODE: fc1010f

FAULTING_SOURCE_LINE: E:\Source\ksm\vmx.asm

FAULTING_SOURCE_FILE: E:\Source\ksm\vmx.asm

FAULTING_SOURCE_LINE_NUMBER: 287

FAULTING_SOURCE_CODE:
283: hlt ; not reached
284: jmp do_hlt
285: __vmx_entrypoint ENDP
286:

287: __vmx_vmcall PROC
288: ; assumes:
289: ; rcx = hypercall
290: ; rdx = data
291: vmcall
292: setna al

SYMBOL_STACK_INDEX: 7

SYMBOL_NAME: ksm!__vmx_vmcall+0

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: ksm

IMAGE_NAME: ksm.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 5ccb89d0

STACK_COMMAND: .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET: 0

FAILURE_BUCKET_ID: 0x1E_c000001d_BAD_IP_ksm!__vmx_vmcall

BUCKET_ID: 0x1E_c000001d_BAD_IP_ksm!__vmx_vmcall

PRIMARY_PROBLEM_CLASS: 0x1E_c000001d_BAD_IP_ksm!__vmx_vmcall

TARGET_TIME: 2019-05-03T00:22:57.000Z

OSBUILD: 17763

OSSERVICEPACK: 0

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK: 272

PRODUCT_TYPE: 1

OSPLATFORM_TYPE: x64

OSNAME: Windows 10

OSEDITION: Windows 10 WinNt TerminalServer SingleUserTS

OS_LOCALE:

USER_LCID: 0

OSBUILD_TIMESTAMP: 2005-12-02 14:58:59

BUILDDATESTAMP_STR: 180914-1434

BUILDLAB_STR: rs5_release

BUILDOSVER_STR: 10.0.17763.1.amd64fre.rs5_release.180914-1434

ANALYSIS_SESSION_ELAPSED_TIME: 1031

ANALYSIS_SOURCE: KM

FAILURE_ID_HASH_STRING: km:0x1e_c000001d_bad_ip_ksm!__vmx_vmcall

FAILURE_ID_HASH: {dad81c61-e5b3-d117-68a0-fb68d4438f5e}

Followup: MachineOwner

I have learned the code about EPTP switching, and I found the vcpu_vmfunc function is called in ksm_introspect_start(Introspect.c), and it is called in ksm_ioctl(Main_linux.c). But I don't know understand how to call ksm_ioctl, and how to use EPTP switching VMFUNC.

I am trying to transplant ksm to linux platform. I had a question that why you didn't disable the interrupt in vcpu_handle_exit()? When I tried to disable the interrupt in my codes (linux-version of ksm), the os crashed and had no response!

cross compiled driver on linux fails driver signing. As a result unsigned driver needs to be loaded.

This was encountered under Windows 10, but I think it's probably a general AT&T assembly error, see vmx.S.

Hey,

Edit: I see you updated the project, I will check and update here again.

I've taken the latest release of this project and tried to compile it with visual studio 2015,
Trying to compile with release x64 will give the following compile errors:

....\exit.c(1680): error C2065: 'curr_handler': undeclared identifier
....\exit.c(1684): error C2065: 'curr_handler': undeclared identifier
....\exit.c(1685): error C2065: 'curr_handler': undeclared identifier

Which happen because

#if defined(NESTED_VMX) || defined(DBG)
static u16 curr_handler = 0;

not happening.

Compiling in debug x64 will work, however trying to run it result in bsod.
Stack:

This crash is on physical computer, Windows 10 x64 Version 10.0.14393, all virtualization and stuff are on.

Also i have a question:
I want to use the vmfunc to change eptp when a selected program is running, so basically I need to check context switches and to run vmfunc whenever the context switch is toward giving run time to my process.

I have all the code about the process creation and such but not really sure how to "hook" the context switches, I could for example exit when cr3 load (which happens when program context switch) but it's not really bullet proof since any kernel driver can just change the cr3 register to the program cr3 (in which this case I don't want to call the vmfunc to switch the eptp).

I was wondered if you have any idea how to do this? (preferably someway with not much overhead since context switches are happen very frequent)

Type of this issue (please specify)

• This is a bug in the upstream tree as-is unmodified.
• This is a support matter (i.e. your own modified tree)
• This is a technical question

In mm.c, function mm_cache_mtrr_ranges

...
	if ((cap >> 8) & 1 && (def >> 10) & 1) {
		/* Read fixed range MTRRs.  */
		for (msr = __readmsr(MSR_MTRRfix64K_00000), offset = 0, base = 0;
		     msr != 0; msr >>= 8, offset += 0x10000, base += offset)
			make_mtrr_range(&ranges[idx++], true, msr & 0xff, base, base + 0x10000 - 1);

		for (val = MSR_MTRRfix16K_80000, offset = 0; val <= MSR_MTRRfix16K_A0000; ++val)
			for (msr = __readmsr(val), base = 0x80000;
			     msr != 0; msr >>= 8, offset += 0x4000, base += offset)
				make_mtrr_range(&ranges[idx++], true, msr & 0xff, base, base + 0x4000 - 1);

		for (val = MSR_MTRRfix4K_C0000, offset = 0; val <= MSR_MTRRfix4K_F8000; ++val)
			for (msr = __readmsr(val), base = 0xC0000;
			     msr != 0; msr >>= 8, offset += 0x1000, base += offset)
				make_mtrr_range(&ranges[idx++], true, msr & 0xff, base, base + 0x1000 - 1);
	}
...

offset should not be incremented each inner loop - it should be set once (with base, in the initial part of the for expression).

This error creates this pattern:

ksm: CPU 1: ksm_init: MTRR Range: 0x0000000000000000 -> 0x000000000000FFFF fixed: 1 type: 6
ksm: CPU 1: ksm_init: MTRR Range: 0x0000000000010000 -> 0x000000000001FFFF fixed: 1 type: 6
ksm: CPU 1: ksm_init: MTRR Range: 0x0000000000030000 -> 0x000000000003FFFF fixed: 1 type: 6 // should be 0x0000000000020000 -> 0x000000000002FFFF
ksm: CPU 1: ksm_init: MTRR Range: 0x0000000000060000 -> 0x000000000006FFFF fixed: 1 type: 6 // should be 0x0000000000030000 -> 0x000000000003FFFF
ksm: CPU 1: ksm_init: MTRR Range: 0x00000000000A0000 -> 0x00000000000AFFFF fixed: 1 type: 6 // etc
ksm: CPU 1: ksm_init: MTRR Range: 0x00000000000F0000 -> 0x00000000000FFFFF fixed: 1 type: 6 // ...

MSR_IA32_FEATURE_CONTROL.lock[0bit] = 1 is set in kernel version 5.x by default.

which means no modification to MSRs is allowed until the system reboot.

the code in ksm.c is invalid anymore.

// ksm.c    int __ksm_init_cpu(struct ksm *k)

feat_ctl = __readmsr(MSR_IA32_FEATURE_CONTROL);
	if ((feat_ctl & required_feat_bits) != required_feat_bits) {

Hey,

I'm trying to run the project (compiled using vs2015 since I couldn't make it compile on mingw using the Makefile for some reason)

And I tried to run it on vm (VMWare) on windows 10 x64 with hyper-v on (complained the first time it wasn't installed)

and I get BSOD, opening the dump in Windbg shows the crash is in the function ept_check_capabilitiy

on the line of u64 vpid = __readmsr(MSR_IA32_VMX_EPT_VPID_CAP);

I don't think it has to do with the fact I compiled with vs2015 instead of gcc.

Any idea how to solve it?
and also it will help a lot if you could write which version of mingw your using so i would be sure the version isn't the problem of compiling.

Thank you for your help!