Skip to content
Instructors: {{ course.instructors }}

Memory Exploits

What an attacker does with a memory bug. First by seizing the instruction pointer to run code of their choosing, and then without touching control flow at all.

Builds on  Spatial & Temporal errors
Part A · Hijacking control flow

Control-Flow Hijacking

That completes the bugs themselves, spatial and temporal both. We now turn from the bugs themselves to what an attacker does with them: seizing the instruction pointer to run code of their choosing. Two families follow: code injection, then code reuse (return-to-libc and ROP).

01 · From bug to exploit

Control-flow hijacking I: code injection

A control-flow hijacking exploit uses a memory bug to seize the instruction pointer. Aleph One wrote the technique up in 1996 {{ cite.aleph_one }} and it has not gone away since. The first flavor is code injection: a memory exploit that hijacks control to jump into the attacker's own data payload. We overflow the buffer, place machine code (“shellcode”) in it, and point the return address back at that code.

Two practical problems and their fixes appear in the walkthrough: you may not know the buffer's exact address, and your guess must land exactly on the first shellcode byte. The NOP sled, a long run of 0x90 (no-op) bytes before the shellcode, solves both: land anywhere in the sled and execution “slides” down into the payload.

↗ Lab, Program 1: four levels that end with a shell you wrote the bytes for

Code injection needs three things
REQ 1Write the attack payload somewhere in memory.
REQ 2Have that payload be executable.
REQ 3Divert control flow to the payload.
02 · From bug to exploit

Control-flow hijacking II: code reuse

Injecting your own code has a catch: the payload must be written into memory and also be executable. A neater idea sidesteps that: instead of injecting new code, reuse code that is already in the program and already runnable. Code reuse is a memory exploit that hijacks control to jump to an attacker-chosen address of code that already exists. The classic instance is return-to-libc: overwrite the return address with the address of an existing function like execv, and fake the arguments it expects on the stack.

Think of it as adding new edges to the program's control-flow graph, call/return edges that the programmer never wrote. No new code is introduced at all, the exploit is assembled entirely from code already in the program. Pushed further, attackers chain many tiny existing snippets (“gadgets”) ending in ret. That is return-oriented programming (ROP) {{ cite.shacham_rop }}. Finding and chaining the gadgets can itself be automated {{ cite.pshape }}.

↗ Lab, Program 2: four levels on the same bug with the stack no longer executable

03 · Code reuse, advanced

Return-oriented programming (ROP)

ROP is not a separate category of attack, it is the most general form of code-reuse control-flow hijacking. Return-to-libc reused one whole function. Return-oriented programming generalizes the idea to its limit: chain together dozens of tiny existing snippets (gadgets), each just a couple of instructions ending in ret (opcode 0xc3). String enough of them and you can compute anything, all from code that is already in the program.

Key observation: x86 instructions overlap

x86 instructions are variable-length, and the CPU will start decoding at any byte you jump to. So the same bytes decode into completely different instructions depending on where you enter, and a huge program contains an enormous supply of unintended sequences that happen to end in c3 (ret).

Here is the raw byte stream in memory. Two different entry points carve out two overlapping windows, and each window decodes into a completely different instruction sequence:

◄ window A · enter at byte 0 ────────────────────────►
f7
c7
07
00
00
00
0f
95
45
c3
◄ window B · enter at byte 1 (shifted +1) ──────►
Window A: decoded from byte 0, as intended
f7 c7 07 00 00 00   test   $0x00000007, %edi
0f 95 45 c3         setnzb -61(%ebp)   ; c3 = the −61 disp
Window B: decoded from byte 1, where the attacker jumps
c7 07 00 00 00 0f   movl   $0x0f000000, (%edi)
95                  xchg   %ebp, %eax
45                  inc    %ebp
c3                  ret    ; same byte, now a gadget!
The byte stream decoded as intended: test and setnzb instructions The same byte stream decoded one byte later: movl, xchg, inc, ret
The overlap, from the lecture slide: one byte stream, entered one byte apart, yields different instructions, and the shifted decoding ends in a usable ret.

A ROP gadget is any such sequence ending in a control transfer. Because every gadget ends in ret, the attacker just fills the stack with a list of gadget addresses: each ret pops the next one and execution flows down the chain. Step through the running example from Carlini & Wagner's paper {{ cite.carlini_rop }}:

ROP chain (Carlini et al., USENIX Security 2014) · step {{ ropNum }} of 8
Goal: *(0x4a304120) = *(0x4a304120) + 0x00032400, no injected code
The Stack grows down ↓ · high addresses on top {{ stackTag }}
address   :  4 bytes stored on the stack
{{ c.esp }}
{{ c.addr }}:{{ c.val }}
{{ c.instr }}
low addresses ↓ · each ret pops ESP upward
CPU state
EIP{{ regEip.val }}
EAX{{ regEax.val }}
EBX{{ regEbx.val }}
ECX{{ regEcx.val }}
MEM{{ regMem.val }}
MEM = *(0x4a304120)
Memory at the gadget addresses
{{ m.addr }}{{ m.bytes }}{{ m.disasm }}
Data memory: the write target
{{ dataMem.addr }}{{ dataMem.val }}
A gadget address points at its first byte, not a ret. 0x080485d9 holds 01 d8 (add eax,ebx), and the c3 ret sits two bytes later at 0x080485db.

{{ ropCaption }}

The attack procedure
  1. Pre-identify useful gadgets in the program's executable code.
  2. Corrupt the stack so it holds the chain of gadget addresses (plus any immediates).
  3. Hijack control once, to jump to gadget A. A ends in ret, which reads the next address B off the stack.
  4. The ret jumps to B, whose ret jumps to the next, and the chain runs itself, gadget after gadget.

↗ Lab, Program 3: four levels building this chain by hand, on x86-64

One question decides how seriously to take all of this: how much can a chain actually compute? Anything. Given a large enough body of code to draw gadgets from, the set available is Turing-complete, and gadgets of no more than three bytes are enough to get there {{ cite.microgadgets }}. Return-oriented programming is not a trick that happens to work on some programs, it is a general way to run a program of the attacker's choosing out of code that is already in memory.

04 · Code reuse, advanced
Lab in Lecture 2

Vtable hijacking

A distinct flavor of code reuse, separate from return-to-libc. Every C++ object with virtual methods holds a pointer to its vtable, a table of function pointers {{ cite.sekar_oop }}. Getting a pointer to the wrong type is its own bug class, and tools exist to catch it {{ cite.caver }}. Corrupt that pointer (often via a heap or type-confusion bug) so it points at attacker-controlled data, and the next virtual call transfers control to an address of the attacker's choosing, hijacking control flow without ever touching a return address. Where it sits among the other ways to redirect control is what makes it worth a section of its own: a return address is one word on the stack and every defense since has watched that word, while a vtable pointer is one word inside an object, reached by whatever bug reached the object. And unlike a return, a virtual call hands the callee the object as its first argument, so arriving somewhere is not the same as arriving with something to say.

↗ Lecture 2, Program 2: four levels reaching this through a use-after-free, with the address written down in advance

↗ Lab, Program 4: seven levels that find out where everything is first, then forge the table out of what they found

Object memory layout with a virtual method table (VMT) for a class under inheritance
Object layout and the virtual method table: the target of the corruption. Figure credit: R. Sekar, Stony Brook CSE307.
Part B · Leaving control flow alone

Non-Control-Flow Attacks

Everything so far (overflows, ROP, bad casts) ultimately hijacked control flow. But an attacker can win without ever changing where the program jumps, by corrupting (or merely reading) ordinary data.

01 · Non-control-flow attack

Data-oriented attacks

Ask a sharper question: do you actually need to divert control flow to win? Often the answer is no. A data-oriented attack corrupts ordinary non-control data and leaves the control flow completely intact.

Control data
Return addresses, function pointers, vtables: they decide where execution goes. What control-flow hijacking targets.
Non-control data
A permission flag, a user ID, a length, a filename: never executed, but corrupting it can be just as devastating.

The state of the art is to corrupt security-critical data, leave control flow unchanged, and still do significant damage. A classic example is the Internet Explorer SafeMode bypass {{ cite.yu_blackhat }}. In IE, JScript runs in a locked-down mode called SafeMode, the guard that forbids web-page scripts from reaching dangerous capabilities such as the WScript.Shell object. Any web page you visit runs untrusted JavaScript in your browser, and WScript.Shell would let that page's script run operating-system commands on your machine (e.g. launch calc.exe, or download and run malware), so SafeMode normally blocks it. Whether that guard is up is stored as a single flag, safemode, at a fixed offset inside the JScript object:

// jsobj + 0x188 = the SafeMode flag (JScript 5.8)
safemode = *(DWORD *)(jsobj + 0x188);   // read the guard flag
if ( (safemode & 0xB) == 0 ) {          // guard DOWN when the flag is 0 ← the check
    Turn_on_God_Mode();                 // dangerous COM objects now allowed
}

The guard is that if on line 3: normally safemode is non-zero, the test fails, and the privileged path stays closed. The attack never touches this code. Instead, using an arbitrary-write primitive it simply zeroes the safemode field in memory. The next time IE's own, unmodified code runs this check the test now passes, the guard is down, and WScript.Shell becomes available to run arbitrary commands: no control-flow hijack, no ROP. Another canonical case: the wu-ftpd seteuid bug, where corrupting a stored user-ID gives root without touching any code pointer.

You may not need to corrupt anything at all

Push the idea further. Some attacks just read memory they shouldn't. Heartbleed (2014) is the poster child {{ cite.openssl_advisory }} {{ cite.heartbleed_site }}.

The setting. OpenSSL is the open-source library that most web servers use to speak TLS/HTTPS, the encryption behind the padlock in your browser. When a client (your browser) and a server hold a TLS connection open, either side can send a heartbeat: a small "are you still there?" message. You send a short piece of data plus a number saying how long that data is, and the other end simply echoes the same data back. The bug: The OpenSSL code parsing the heartbeat request trusted the claimed length received over the Internet, instead of measuring the data it actually received. The result is that an unauthorised client can leak secrets from the server, such as its TLS private key.

The cartoon below is the intuition. Imagine the client sends a 4-letter word but claims it's 500 letters, and the server dutifully copies 500 bytes into its reply: your 4 letters plus 496 bytes of whatever sat next to them in server memory (session cookies, passwords, even the private key). No corruption, no control-flow change, just a missing bounds check on a read.

Cartoon contrasting a normal Heartbeat request with a malicious one that lies about its length
A cartoon of Heartbleed: the “give me 500 letters” analogy, not the real code. The actual bug is the few lines of OpenSSL shown below. Figure by FenixFeather, via Wikimedia Commons, CC BY-SA 3.0, after xkcd 1354.

The real bug: a missing bounds check

Stripped of the cartoon, here is the actual vulnerable function (tls1_process_heartbeat / dtls1_process_heartbeat, OpenSSL 1.0.1). A heartbeat request carries a type, a 16-bit length, and a payload, and the length is taken straight from the attacker:

/* OpenSSL 1.0.1 · ssl/t1_lib.c → tls1_process_heartbeat() */
unsigned char *p = &s->s3->rrec.data[0], *pl;
unsigned short hbtype;
unsigned int   payload;                /* length the request claims */

hbtype = *p++;                         /* 1. read message type      */
n2s(p, payload);   /* 2. payload = length the CLIENT claims  ← attacker-controlled */
pl = p;                                /* 3. pl → real payload bytes */

buffer = OPENSSL_malloc(1 + 2 + payload + padding);
bp = buffer;
*bp++ = TLS1_HB_RESPONSE;
s2n(payload, bp);
memcpy(bp, pl, payload);   /* BUG: copies 'payload' bytes, but never checks
                             the request actually held that many */

Nothing compares payload against rrec.length, the number of bytes actually received. So memcpy reads up to 65,535 bytes starting at pl, walking off the end of the real request into adjacent heap memory (private keys, session cookies, passwords) and ships it back to the attacker in the response. A buffer over-read, no write required.

The fix (OpenSSL 1.0.1g) is a single bounds check: reject any request whose claimed length exceeds the bytes actually received.

hbtype = *p++;
n2s(p, payload);
if (1 + 2 + payload + 16 > s->s3->rrec.length)
    return 0;              /* claimed length > bytes received → discard */
pl = p;

Two directions lead out of here. Chain the corruptions instead of using one at a time and non-control data becomes Turing-complete, the data-only counterpart of ROP {{ cite.dop }}. The other route to the same end is a bad cast, which makes the program read one type as another, and Lecture 1B works through it.

06 · Lab

Do it yourself

Four programs, nineteen levels. The first three follow the three sections above: inject your own code, then reuse the code already there, then assemble it out of fragments. The fourth removes what the other three were given, which is knowing where anything is.

Every level needs a shell rather than a browser tab, because none of it can be done without reading the binary. From level 2 the payloads are raw bytes rather than text, so they come out of python3 and a pipe. Programs and levels fold away once you are done with them. Hints and answers stay folded until you open them.

A level goes like this. Read the task. Copy exploit.py, change the payload, run it. Some levels need no payload and are answered with readelf, objdump, or by running the program twice and comparing. Open the fold that says how to tell only once you think you have it: it confirms an answer, it does not give one.

Six cards follow. Four are the programs you attack. The other two contain no bugs: exploit.py is the starting point and runs unchanged, and driver.py, which it imports, handles the process. Both ship in the download. Anything saved under /vagrant/src survives a logout.

The first three programs are the same program: one unbounded read into a 64-byte buffer, the saved return address 88 bytes away. Only the build flags change, and how much you are allowed to bring. All three are built with -no-pie, which is why an address found with objdump can go into a payload literally while the printed address of buf cannot: the image is fixed, the stack and libc are not. Program 4 is different. Its bug is on the heap, it has no saved return address, it takes counts on the command line, and it prints a memory dump instead of a list of addresses.

Numbers quoted in the answers were measured on the lab machine. The addresses you need are not among them: those change on every run and you read them from the program as it starts.

{{ p.name }} {{ p.title }}
The program {{ p.codeLines }}
{{ p.codeNode }}
$ {{ p.build }}

{{ p.buildNote }}

Level {{ l.n }} {{ l.title }} {{ l.runsLabel }} {{ l.stars }} {{ l.diffLabel }}

{{ l.taskNode }}

Hint

{{ l.hintNode }}

You have it when

{{ l.winNode }}

Go further

{{ l.moreNode }}

End of section

Memory exploits, in brief

  • A control-flow hijack turns a memory bug into control of the instruction pointer, running code the attacker chose.
  • Code injection writes shellcode and jumps to it, while code reuse (return-to-libc, ROP) needs no injected code at all.
  • ROP chains tiny existing gadgets ending in ret, enough to compute anything, entirely from code already present.
  • All of it assumes you know where things are. Randomization takes that away, so a modern chain starts by leaking one pointer and subtracting a measured offset, and a vtable pointer is hijacked the same way a return address is, without ever touching the stack.
  • You can win without hijacking control flow: corrupt non-control data (a flag, an ID, a length) and the program's own code does the damage {{ cite.chen_noncontrol }}.
  • Sometimes you needn't corrupt anything at all: a missing bounds check on a read (Heartbleed) leaks secrets outright.
  • Because control flow never changes, control-flow defenses never fire.

{{ course.notes }} · Instructors: {{ course.instructors }}

06 · Sources

References

[{{ r.n }}] {{ r.cite }} ↗ {{ r.linkLabel }}