You have register RCX listed for the fourth parameter to syscalls but it should be R10.

"System V Application Binary Interface: AMD64 Architecture Processor Supplement"
https://github.com/hjl-tools/x86-psABI/wiki/x86-64-psABI-1.0.pdf
Page 148 "The kernel interface uses %rdi, %rsi, %rdx, %r10, %r8 and %r9."

Hi, thanks for this great resource, really helps when attempting to program in assembly for MacOS, there's not lot info out there.

I have hardly seen this mentioned anywhere, but the MacOS x86_64 syscall ABI seems to use the Carry Flag in the EFLAGS register to signal that an error happened, unlike Linux which uses negative numbers. I think that it would be cool if that was mentioned on the syscall page, because that info is not simply commonly available, I found it from here: https://stackoverflow.com/questions/47834513/64-bit-syscall-documentation-for-macos-assembly

Another worrying thing is the report that syscalls may clobber rdx, which I also haven't seen mentioned elsewhere...

Stumbled upon this just a moment ago.

Trying to issue posix_spawn system call. Here's the schematic:
rdi: pointer to a int where to store the spawned process' pid.
rsi: path
rdx: pointer to a struct of settings, can be null
rcx: pointer to argv
r8: pointer to envp

And here's the code:

global start
start:
	mov r9, [rsp] ; argc
	lea rcx, [rsp + 8] ; argv: {"./syscall2\0", "/bin/test\0"}
	lea r8, [rsp + 8 + r9*8 + 8] ; envp
	push 0 ; pid
	mov rax, 0x020000F4 ; posix_spawn syscall
	mov rdi, rsp ; pointer to pid
	mov rsi, [rcx+8] ; argv[1]
	mov rdx, 0
	syscall
	mov rax, 0x02000001
	syscall

However, trying this out doesn't work. Spying the syscall in another terminal windows, by:

sudo dtrace -n 'syscall::posix_spawn*:entry { printf("%s %p %s %p %p %p",execname,arg0,copyinstr(arg1),arg2,arg3,arg4); }'

And launching the assembly program with
nasm -f macho64 syscall2.asm && ld syscall2.o -static -o syscall2 && ./syscall2 /bin/test

dtrace finds that the call looks slightly off:
posix_spawn:entry syscall2 7ff7bfeff6a0 /bin/test 0 0 7ff7bfeff6c8
The arg2 after /bin/test is supposed to be zero, but the arg3 is not! Clearly it's expecting arg3 in some other register!

After trial and error, I noticed that this code works:

global start
start:
	mov r9, [rsp] ; argc
	lea r10, [rsp + 8] ; argv: {"./syscall2\0", "/bin/test\0"}
	lea r8, [rsp + 8 + r9*8 + 8] ; envp
	push 0 ; pid
	mov rax, 0x020000F4 ; posix_spawn syscall
	mov rdi, rsp ; pointer to pid
	mov rsi, [r10+8] ; argv[1]
	mov rdx, 0
	syscall
	mov rax, 0x02000001
	syscall

The only difference is that rcx is changed to r10.

I don't have a clue, when this change has taken place or does it only happen on specific versions / hardware.