Stackpointer subtraction in listings 14-4, 14-5, 14-6 about low-level-programming HOT 2 CLOSED

Thank you for your question. I was able to reproduce the book listings with -fno-stack-protector flag. Apparently this should be added to errata, which I am doing right now.

1. The strange things at you are seeing at 0x40055d-0x40056a and 0x400588-0x400597 are parts of the stack buffer overflow protection mechanism, explained in 14.8.1. See how this value from fs:0x28 is being placed next to the return address and old rbp value: this is a security cookie, if it gets overwritten the program craches by calling 400420 <__stack_chk_fail@plt>.

2. rsp is not subtracted enough because of the red zone, which is explained in 14.1.4.

The red zone is an area of 128 bytes that spans from rsp to lower addresses. It relaxes the rule “no data below rsp”; it is safe to allocate data there and it will not be overwritten by system calls or interrupts.

So, 0xf90 is decimal 3984, 3984 + 128 = 4112 = 0x1010 so that is how much place we really get if we subtract 0xf90 from rsp. Your listing shows a close enough number: 0x1020, which is different by 16. Why so?

You compiled the code with stack protection implicitly turned on. Because of that, we need an extra 8 bytes in stack immediately next to old rbp value in order to store the security cookie (the bloat you mentioned).

We subtract a bit more to keep rsp well aligned in case of future calls, such as call 400420 <__stack_chk_fail@plt>. When interacting with C code, the stack should usually be 16-bit aligned (a part of calling convention described in ABI).

When maximum is being called, the stack is 16-bit aligned. Then we do the following:

• call rsp -= 8
• push rbp rsp -= 8
• sub rsp, x
  Were x equal to 0x1018 (enough place for locals + security cookie), we would have broken the 16-bit alignment. It leads to undefined behavior when calling <__stack_chk_fail@plt>.

In case of stack protection turned off, this function performs no calls (leaf function), and we don't need to keep rsp aligned.

from low-level-programming.

Thanks for explaning this.

from low-level-programming.