Comments (59)
probably needs more syscalls
needs reading the dynamic linker section
dyld is WEIRD
x86_64 linux test currently fails to load symbols - it jumps to 0 from PLT
from usercorn.
currently static and dynamically linked glibc all crash on load for various reasons. haven't tried any other libc implementations, and osx dynamic linking isn't supported yet either.
from usercorn.
fixed glibc crashes (was bad interp entry point)
from usercorn.
fixed incorrect auxv for musl and glibc.
musl crashes in decode_vec
glibc is jumping to a random unmapped address
from usercorn.
glibc is crashing at elf_get_dynamic_info()
oh, I need to point to the program header inside the loaded section, not copy my own phdr in...
from usercorn.
@lunixbochs can you share your code to build x86.darwin.macho?
from usercorn.
It's just the test binary from https://github.com/lunixbochs/lib43 - if you want to build the 32-bit version specifically, do cmake -DCPU=i386
I'm assuming it works because lib43 doesn't care about macho auxv :)
from usercorn.
osx auxv equivalent is supported on x86_64 now. the primary blocker is now unimplemented mach syscalls
from usercorn.
This is what happens on i386 after trying to run a dynamically linked binary:
Inconsistency detected by ld.so: rtld.c: 1141: dl_main: Assertion `_rtld_local._dl_rtld_map.l_libname' failed!`
from usercorn.
That's a regression. I'm not sure why it's happening yet. Wouldn't be surprised if it's a problem in auxv.
If you want to crack open the glibc source and take a look, narrowing this down would be helpful.
from usercorn.
You meant this?
http://code.woboq.org/userspace/glibc/elf/rtld.c.html#1128
from usercorn.
from usercorn.
Would any of usercorn's tracing options be of any help here?
from usercorn.
Yeah. -trace should enable all of the useful tracing options. I don't recommend posting entire traces as they can be quite large (and I see the same error when I run one)
from usercorn.
No worries - I would've attached a file if you needed a trace. :)
-original message-
Subject: Re: [usercorn] support dynamic linkers (#21)
From: Ryan Hileman [email protected]
Date: 20.11.2015 22:29
Yeah. -trace should enable all of the useful tracing options. I don't recommend posting entire traces as they can be quite large (and I see the same error when I run one)
Reply to this email directly or view it on GitHub:
#21 (comment)
from usercorn.
I can offer this backtrace from gdb:
(gdb) catch syscall exit
(gdb) catch syscall exit_group
(gdb) run
Starting program: /home/petevine/unpacked/usercorn-unstable/usercorn /usr/bin/ls
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
Inconsistency detected by ld.so: rtld.c: 1141: dl_main: Assertion `_rtld_local._dl_rtld_map.l_libname' failed!
[New LWP 2387]
[New Thread 0xb317fb40 (LWP 2386)]
[New Thread 0xb39bfb40 (LWP 2385)]
[Switching to Thread 0xb1608b40 (LWP 2387)]
Catchpoint 4 (call to syscall exit), 0xb3c9c729 in start_thread ()
from /lib/libpthread.so.0
(gdb) bt
#0 0xb3c9c729 in start_thread () from /lib/libpthread.so.0
#1 0xb3bf758e in clone () from /lib/libc.so.6
(gdb) c
Continuing.
[Thread 0xb1608b40 (LWP 2387) exited]
[Switching to Thread 0xb3ab0700 (LWP 2384)]
Catchpoint 5 (call to syscall exit_group), 0xb7fd9b30 in __kernel_vsyscall ()
(gdb) bt
#0 0xb7fd9b30 in __kernel_vsyscall ()
#1 0x0811701f in syscall.Syscall ()
at /home/petevine/go/src/syscall/asm_linux_386.s:23
#2 0x08114d49 in syscall.Exit (code=127)
at /home/petevine/go/src/syscall/zsyscall_linux_386.go:376
#3 0x080b5c6f in os.Exit (code=127) at /home/petevine/go/src/os/proc.go:54
#4 0x0804b8db in main.main ()
at /home/petevine/go/src/github.com/lunixbochs/usercorn/go/usercorn/cli.go:85
from usercorn.
gdb isn't interesting here because it's attached to Usercorn (which I know very well) and not the target.
Once I have #10 we'll be able to attach GDB to the target, but that shouldn't be necessary here (as Usercorn should be able to provide a symbolicated traceback if the binary has symbols). It might not work on dynamic symbols? (#79)
from usercorn.
okay, looks like phnum is zero for some reason, which makes phdr impossible to parse.
edit: was a regression, is fixed in 089b6fa - back to fixing TLS and weirdness
from usercorn.
Could you explain what's going on here? (I chose wc
cause it seems to be working in busybox)
./usercorn /bin64/ldd /bin64/wc --help
panic: Could not identify file magic.
goroutine 1 [running]:
main.main()
/home/petevine/go/src/github.com/lunixbochs/usercorn/go/usercorn/cli.go:55 +0x636
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/home/petevine/go/src/runtime/asm_386.s:1662 +0x1
or
./usercorn /bin64/wc --help
invalid read: @0x8, 0x8 = 0x0
Registers:
fs 0x0000000000000000 rbx 0x000000000012a000 rsi 0x0000000000000000 r11 0x00000000607fe620
gs 0x0000000000000000 rcx 0x000000000040108d r8 0x000000000012a000 r12 0x000000000012a000
rax 0x0000000000000000 rdi 0x00000000607fed5d r9 0x0000000000000001 r13 0x0000000000000000
rbp 0x00000000607fe770 rdx 0x000000000d696913 r10 0x000000000101cdc1 r14 0x000000000d696913
r15 0x000000000040108d
Thx!
from usercorn.
You can't run ldd under usercorn because it's a shell script that basically does LD_TRACE_LOADED_LIBRARIES=1 /bin64/wc --help
In the other one, that's the bug this issue is about. Look at the stacktrace after -etrace to see where it's breaking.
from usercorn.
Oh, the --help
was a silly paste mistake from trying to run the binary; provided the bug wasn't there should invoking LD_TRACE_LOADED_LIBRARIES=1 /bin64/wc
under usercorn work at present?
I'll be adding the trace later.
from usercorn.
Oh, it's actually LD_TRACE_LOADED_OBJECTS
, and that doesn't seem to work because it's currently breaking before link finishes. LD_DEBUG=all
works for me.
This is my stacktrace:
0x100fd10 match_symbol (dl-version.c:76)
0x10100f7 _dl_check_map_versions+0x27 (dl-version.c:191)
0x10100d0 _dl_check_map_versions (dl-version.c:174)
0x1010562 _dl_check_all_versions+0x12 (dl-version.c:384)
0x1010550 _dl_check_all_versions (dl-version.c:380)
0x1004790 version_check_doit
0x100eb10 _dl_receive_error (dl-error.c:205)
0x100173a dl_main+0xca
0x1001670 dl_main
0x1015c42 _dl_sysdep_start+0x42 (dl-sysdep.c:111)
0x1015c00 _dl_sysdep_start (dl-sysdep.c:86)
0x1004825 _dl_start+0x55
0x10047d0 _dl_start
0x1001180 _start
from usercorn.
Yeah, that's the trace. How do you pass vars of the guest environment to usercorn?
from usercorn.
Environment is inherited from host. It ends up on the stack above argv.
from usercorn.
I asked because on my 32-bit system all I'm getting with LD_DEBUG=all ./usercorn /bin64/wc
is info about the host system.
from usercorn.
Oh, ld.so might be unsetting that env var. I'm testing on OS X so my host linker won't do things like that.
from usercorn.
Unless this tiny part is what I was after?
file=libc.so.6 [0]; needed by /bin64/wc [0]
23137: find library=libc.so.6 [0]; searching
23137: search cache=/etc/ld.so.cache
23137: search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64 (system search path)
23137: trying file=/lib64/tls/x86_64/libc.so.6
23137:
23137: file=libc.so.6 [0]; generating link map
23137: dynamic: 0x00000000009ceb60 base: 0x000000000060a000 size: 0x00000000003caec0
23137: entry: 0x000000000062a990 phdr: 0x000000000060a040 phnum: 10
23137:
invalid read: @0x8, 0x8 = 0x0
from usercorn.
This is my output: https://bochs.info/p/njj89z
from usercorn.
Is the usercorn env=
part the answer to my original question or just the debug output?
from usercorn.
~/p/usercorn
is my shell path. It's more obvious in a screenshot because it's colored.
from usercorn.
In that case I think a feature like this would be a good idea.
I got tons of debug info from usercorn (which honoured the variable) whereas I was merely interested in some info about the guest environment.
from usercorn.
For reference, qemu-user takes these arguments. What you're proposing is the -E / -U interface. This will be pretty easy, so I'm adding issue #106 for it.
Argument Env-variable Description
-h print this help
-g port QEMU_GDB wait gdb connection to 'port'
-L path QEMU_LD_PREFIX set the elf interpreter prefix to 'path'
-s size QEMU_STACK_SIZE set the stack size to 'size' bytes
-cpu model QEMU_CPU select CPU (-cpu help for list)
-E var=value QEMU_SET_ENV sets targets environment variable (see below)
-U var QEMU_UNSET_ENV unsets targets environment variable (see below)
-0 argv0 QEMU_ARGV0 forces target process argv[0] to be 'argv0'
-r uname QEMU_UNAME set qemu uname release string to 'uname'
-B address QEMU_GUEST_BASE set guest_base address to 'address'
-R size QEMU_RESERVED_VA reserve 'size' bytes for guest virtual address space
-d item[,...] QEMU_LOG enable logging of specified items (use '-d help' for a list of items)
-D logfile QEMU_LOG_FILENAME write logs to 'logfile' (default stderr)
-p pagesize QEMU_PAGESIZE set the host page size to 'pagesize'
-singlestep QEMU_SINGLESTEP run in singlestep mode
-strace QEMU_STRACE log system calls
-version QEMU_VERSION display version information and exit
from usercorn.
From the changelog:
elf/dl-version.c (match_symbol): Use D_PTR to access relocated entries in l_info.
and the source:
const char *strtab = (const void *) D_PTR (map, l_info[DT_STRTAB]);
So l_info is relocated. I did some research and "modern kernels apply relocations to ld.so".
This step of startup should be fixed by #107
from usercorn.
I'd emailed you about this thinking I broke something with a little hack but it wasn't my fault after all:
./usercorn /bin64/wc --help
hi 0
Inconsistency detected by ld.so: ../sysdeps/x86_64/dl-machine.h: 497: elf_machine_rela_relative: Assertion `((reloc->r_info) & 0xffffffff) == 8' failed!
I'm on 32-bit i386 linux.
from usercorn.
What's hi 0
? I'm pretty sure the error we're getting on glibc is actually relocation related across the board, so I'm not sure your binary is anything special (besides the fact it prints an assert). If you do think something changed, can you try git bisecting?
from usercorn.
No idea about hi 0
(probably wc is confused and trying to count something in spite of being asked not to).
I don't have that binary any longer and rebuilt from the latest unstable source. That's what you meant, right?
from usercorn.
Can you git bisect to before this message was being printed, and see which usercorn -trace is longer? (fyi, make usercorn
is way faster than make
because it doesn't try to download deps)
from usercorn.
I can't do that now but bisecting from memory it would have to be this one:
I'll try to provide an actual trace comparison from before that later.
from usercorn.
Alright, if that's the case can you get me a trace from before and after (well, before and current unstable) when you get a chance?
from usercorn.
Ok, I tried building @ 95806ec but it failed with:
# github.com/lunixbochs/usercorn/go/usercorn
go/usercorn/cli.go:53: cannot use absPrefix (type string) as type *usercorn.Config in argument to usercorn.NewUsercorn
go/usercorn/cli.go:57: corn.Verbose undefined (type *usercorn.Usercorn has no field or method Verbose)
go/usercorn/cli.go:58: corn.TraceSys undefined (type *usercorn.Usercorn has no field or method TraceSys)
go/usercorn/cli.go:59: corn.TraceMem undefined (type *usercorn.Usercorn has no field or method TraceMem)
go/usercorn/cli.go:60: corn.TraceMemBatch undefined (type *usercorn.Usercorn has no field or method TraceMemBatch)
go/usercorn/cli.go:61: corn.TraceReg undefined (type *usercorn.Usercorn has no field or method TraceReg)
go/usercorn/cli.go:62: corn.TraceExec undefined (type *usercorn.Usercorn has no field or method TraceExec)
go/usercorn/cli.go:67: corn.TraceMatchDepth undefined (type *usercorn.Usercorn has no field or method TraceMatchDepth)
go/usercorn/cli.go:69: corn.TraceMatchDepth undefined (type *usercorn.Usercorn has no field or method TraceMatchDepth)
go/usercorn/cli.go:72: corn.TraceMatch undefined (type *usercorn.Usercorn has no field or method TraceMatch)
go/usercorn/cli.go:72: too many errors
make: *** [usercorn] Error 2
Any use going back further?
from usercorn.
For now, just the latest binary's trace:
And an older one's @ 2b80f6a which just reads:
panic: Could not identify file magic.
goroutine 1 [running]:
main.main()
/home/petevine/go/src/github.com/lunixbochs/usercorn/go/usercorn/cli.go:55 +0x636
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/home/petevine/go/src/runtime/asm_386.s:1662 +0x1
which, going in the opposite direction, was still the case @ d3f09a2
and the latest 3976cd6 brings back the same error.
from usercorn.
woah, my x86_64 dynamic musl-libc test works on the latest master, and my static glibc binary stopped working, so maybe it's fixed and I broke something else
bisect shows this broke in 089b6fa
from usercorn.
It seems the error above was caused by inadvertent truncation of the wc
binary - the trace is the same invalid read as in the old one.
from usercorn.
Even though every trace shows:
Inconsistency detected by ld.so: do-rel.h: 116: elf_dynamic_do_Rela: Assertion `map->l_info[(34 + 0 + (0x6fffffff - (0x6ffffff0)))] != ((void *)0)' failed!
some binaries get stuff like this:
$ ./usercorn /bin64/ls
/bin64/ls: error while loading shared libraries: tls/x86_64/libcap.so.2: file too short
Is it about symlink handling?
from usercorn.
Symlinks are handled. I think there's actually a linux-specific bug. It's way more reliable on OS X.
from usercorn.
You probably mean darwin host; anyway could you point me to a minimal darwin lib/bin package for client testing? I have never tried the -prefix
switch and a different kernel yet.
from usercorn.
maybe puredarwin http://www.puredarwin.org/downloads/
from usercorn.
I tried mounting the downloaded vmdk image with qemu-nbd
, but all I got eventually was an hfsplus: invalid secondary volume header
error. Maybe you could email me just a few essential libs and a test binary?
from usercorn.
I'm not sending you OS X binaries, and I don't have any other xnu/darwin setup right now.
from usercorn.
Here's a tarball of the HFS rootfs from PureDarwinNano_20091226.tar.xz: https://bochs.info/~aegis/PureDarwin.tar.gz
from usercorn.
I'd already tried that too - the mounted iso image contains just a few files for booting from cd. How's the actual data supposed to be stored?
from usercorn.
Huh? This is a working root filesystem, with dynamic binaries, dyld, libraries, etc. Should be more than sufficient for basic dynamic link tests.
from usercorn.
Yeah, sorry, lack of concentration! I'd already had a file named PureDarwinNano_20091226.tar.xz
which you mentioned above unfortunately :)
Anyway, thanks a lot and getting back on topic:
$ ./usercorn -prefix PureDarwin/ PureDarwin/bin/ls -l
+ block @0x8fe1fbb8
0x8fe1fbb8: 5a pop edx
0x8fe1fbb9: 89e1 mov ecx, esp edx 0x8fe1d33a
0x8fe1fbbb: 0f34 sysenter ecx 0x607fee2c
+ block @0x8fe1fbbd
0x8fe1fbbd: 0f1f00 nop dword ptr [eax]
0x8fe1fbc0: b806000000 mov eax, 6
0x8fe1fbc5: cd82 int 0x82 eax 0x00000006
panic: Syscall missing: 50331654
goroutine 17 [running, locked to thread]:
github.com/lunixbochs/usercorn/go.(*Usercorn).Syscall(0x106ca2a0, 0x3000006, 0x0, 0x0, 0x10856010, 0x0, 0x106122c0, 0x0, 0x0)
/home/odroid/go/src/github.com/lunixbochs/usercorn/go/usercorn.go:695 +0x130
github.com/lunixbochs/usercorn/go/arch/x86.DarwinSyscall(0xab29e740, 0x106ca2a0, 0x3)
/home/odroid/go/src/github.com/lunixbochs/usercorn/go/arch/x86/darwin.go:41 +0xfc
github.com/lunixbochs/usercorn/go/arch/x86.DarwinInterrupt(0xab29e740, 0x106ca2a0, 0x82)
/home/odroid/go/src/github.com/lunixbochs/usercorn/go/arch/x86/darwin.go:53 +0x80
github.com/lunixbochs/usercorn/go.(*Usercorn).addHooks.func5(0xab29e6b0, 0x106e04d0, 0x82)
/home/odroid/go/src/github.com/lunixbochs/usercorn/go/usercorn.go:553 +0x84
github.com/unicorn-engine/unicorn/bindings/go/unicorn.hookInterrupt(0x4c0148, 0x82, 0x108460b0)
/home/odroid/go/src/github.com/unicorn-engine/unicorn/bindings/go/unicorn/hook.go:45 +0x80
And it seems it's still just i386
which might explain the problem.
from usercorn.
// osfmk/i386/machdep_call.c
// DDD: the last two are BSD_CALL instead of CALL...
//#define __NR_thread_get_cthread_self VG_DARWIN_SYSCALL_CONSTRUCT_MDEP(0)
//#define __NR_thread_set_cthread_self VG_DARWIN_SYSCALL_CONSTRUCT_MDEP(1)
// 2 is invalid
#define __NR_thread_fast_set_cthread_self VG_DARWIN_SYSCALL_CONSTRUCT_MDEP(3)
//#define __NR_thread_set_user_ldt VG_DARWIN_SYSCALL_CONSTRUCT_MDEP(4)
//#define __NR_i386_set_ldt VG_DARWIN_SYSCALL_CONSTRUCT_MDEP(5)
//#define __NR_i386_get_ldt VG_DARWIN_SYSCALL_CONSTRUCT_MDEP(6)
So this is i386_get_ldt
. I'll need to update the syscall table in ghostrace with these (I don't have any machine-dependent syscalls in there atm)
LDT/GDT support on x86_32 aren't done yet, but I think if you want a 64-bit darwin root you need to build it yourself or find a mac.
from usercorn.
Ok, I'll explore the former option :)
from usercorn.
http://web.mit.edu/darwin/src/modules/xnu/osfmk/man/i386_get_ldt.html
http://web.mit.edu/darwin/src/modules/xnu/osfmk/man/i386_set_ldt.html
There's a Unicorn register LDTR which points at the LDT memory address. I'm actually not sure what the LDT does in kernel mode (as I'm not doing a usermode transition). A workaround for now would be to actually set a GDT.
There's experimental code in x86/linux.go:gdtWrite()
to write GDT entries, but I don't remember if the upstream GDT problems were fixed.
To read the userspace segment descriptors, you can do s := u.StrucAt(addr)
, then s.Unpack(&descriptor)
, where descriptor is a struct type like...
type Descriptor struct {
Addr uint32
Size uint32
}
func() DoStuff() {
var desc Descriptor
s := u.StrucAt(addr)
for i := 0; i < count; i++ {
err := s.Unpack(&desc)
if err != nil {
return UINT64_MAX
}
}
}
You can do the opposite with s.Pack(&desc)
to write descriptors back to guest memory.
from usercorn.
gdtWrite is actually working now, so maybe PureDarwin could be supported with this.
from usercorn.
Thanks, I've just confirmed busybox-i686
is fully functional.
from usercorn.
Closing this. Will break out a couple of remaining tasks.
from usercorn.
Related Issues (20)
- dsymutil failed HOT 9
- Arm v7 emulation -> panic: Syscall missing: 18023 HOT 4
- add support for += in scripts
- request for `on exit`
- Custom filesystem HOT 4
- How to use usercorn with Lua HOT 1
- bpf: Add option for seccomp, and resolve meaning (e.g. syscall numbers)
- Regarding the python interface HOT 6
- Add instruction hooking
- Signed Register Functions HOT 7
- Makefile issues HOT 4
- unstable: OS X build is broken HOT 2
- Display More Symbols in Trace HOT 20
- Unmap over several pages not correctly unmapping each page individually
- "dl_main: Assertion 'main_map != NULL' failed!" when executing mipsel binary HOT 14
- incorrect display of self-modifying code in streamui
- N/A
- FATAL: kernel too old HOT 1
- the keystone and capstone libs now land in deps/lib64 (ld: cannot find -lkeystone) HOT 1
- Please provide darwin AARCH64 support (iOS, macOS M1)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from usercorn.