Two processes, one shuffle
WhiteHat Contest 11 RE1: a fork-and-pipe crackme with a shuffled key
RE1 hides a plain string comparison behind fork(), two pipes and ptrace checks. Once the control flow is split into parent and child, the whole task reduces to undoing one block permutation.
- 01
Inspect
The program prints a prompt and answers Wrong for any input. Decompiled, main forks, connects parent and child with two pipes, and only the child knows the key.
- 02
Hypothesize
The parent prints True only if the child writes a nonzero number back, so the child's comparison path is the only thing worth reading. ptrace checks can be ignored by reading statically.
- 03
Test
The child accepts only a 42-character string with fixed fragments at both ends, then permutes four 10-character blocks and compares the result with a constant.
- 04
Solve
Apply the inverse permutation to the constant. The reordered string is the accepted key.
RE1 was a 100-point reverse engineering task in WhiteHat Contest 11, a wargame-style contest run through the WhiteHat Wargame platform. The download is a single Linux ELF executable. Run directly, it prints a prompt, reads a line and answers Wrong !!! no matter what is typed. That behavior alone says little, so the first job is to find out where the verdict is produced.
First look at the binary
Loaded into a decompiler, main turns out to be unusual for a crackme. It starts with a call to a helper that detects tracing, then creates two pipes and calls fork(). Everything after that splits into two branches. The parent closes the ends of the pipes it does not use, prints Input key :, reads a line with gets, writes the raw bytes into the first pipe, and then blocks on a read from the second pipe. The child does the mirror image: it reads from the first pipe and writes a single character back through the second.
The parent never compares the input to anything. It only inspects the reply. If the reply is a string of digits and atoi of it is nonzero, the parent prints True and exits cleanly. Otherwise it prints Wrong !!! and loops for another attempt. Two more conditions sit in front of that branch, a stored result of the ptrace helper and a second routine that looks for a debugger process by name, and either one forces the wrong answer. Both are anti-analysis measures aimed at someone stepping through the program, not at someone reading it.
This structure explains the stubborn failure. Because the answer travels over a pipe, a debugger attached to the parent sees nothing interesting, and a debugger attached to the child has to follow the fork first. In gdb that means set follow-fork-mode child, and the ptrace check then fires because the process is already traced. Static reading avoids the whole problem.
The child’s decision tree
The child’s code is a deeply nested chain of if statements, each of which calls the same failure function when it does not hold. The failure function writes the ASCII character 0 into the second pipe, and the success function writes 1. Flattened, the conditions the input must satisfy are these:
| Check | Required value |
|---|---|
| Debug state from the child’s own helper | not traced |
| First byte | { |
| Length | exactly 42 |
| Bytes 1 to 10 | start with 5, end with 1 |
| Last byte | } |
| Bytes 31 to 40 | start with 4, end with d |
| After the shuffle routine | equals a 42-byte constant starting {da and ending 8c} |
The pattern checks at the two ends look like a hint, and they are. They are consistent with the constant at the bottom of the table once the shuffle is understood. The constant itself is printed in the binary’s data section, so it can be read directly. The page that described the original task redacted the middle of the key, and the same convention is kept here: the value is not reproduced, only the method.
The shuffle routine
The routine that sits between the pattern checks and the final comparison was named confuseKey in the decompiled output. It declares four 15-byte local buffers and zeroes them. After verifying again that the input is 42 characters long and starts with {, it copies four slices of 10 bytes each, starting at offsets 1, 11, 21 and 31, into those buffers. It then clears the input buffer, writes { back, and concatenates the buffers in the order third, fourth, first, second. The last step sets byte 41 to }.
In other words the 40 characters between the braces are cut into four blocks A, B, C, D and reassembled as C, D, A, B. Because the blocks are of equal length, this is the same as swapping the two halves of the 40-character body. The operation is its own inverse: applying it twice returns the original.
That is the whole trick. The comparison constant is the string after shuffling, so the accepted input is the constant with its halves swapped. In Python the inversion is one line:
target = "{da...8c}" # the 42-byte constant from the binary
body = target[1:-1]
key = "{" + body[20:] + body[:20] + "}"
Checking the result against the earlier pattern table is a useful sanity step. The block that ends up first in the key must begin with 5 and end with 1, and the block at offset 31 must begin with 4 and end with d. If those fragments appear after the swap, the arithmetic on offsets is right.
Wrong turns worth knowing about
Several things look important and are not. The gets call suggests an overflow, but the child reads at most 200 bytes into a 200-byte buffer, and nothing in the task requires memory corruption. The ptrace tricks consume time if a debugger run is attempted first. Reading the child’s code first, as a pure control-flow exercise, is faster than fighting them.
Another common mistake is to treat the nested conditions as the key space. They are not a search problem. Fixed fragments plus one permutation leave no freedom beyond the redacted middle, and that middle comes from the constant, not from brute force.
What transfers to other challenges
Any program that forks and talks to itself over pipes can be reduced to one question: which process holds the decision. Find the writer of the success byte and follow its conditions backward. Input transforms like the one here, a fixed shuffle of equal blocks, are always invertible by applying the same slicing to the target. The same reading approach applies to the Python-based RE task from the same contest, covered in the RE3 walkthrough, and to the layered decryption in Zwiebel. For a visual way to spot where code, strings and padding sit inside an ELF before opening a disassembler, see binary data visualization.
Questions readers ask
Why does the program always print Wrong when run normally?
The parent process only prints True when the child sends back the byte for 1. The child checks the input against a fixed 42-character pattern, so any ordinary string fails at the first length or prefix test and the child replies 0, which the parent turns into a Wrong message.
Do the ptrace checks need to be defeated?
Not for the intended solve. The key logic is readable in a disassembler, and the final comparison constant sits in the binary, so the answer can be derived statically. Patching or bypassing ptrace matters only if a debugger run is wanted to confirm the result.
What makes the shuffle easy to invert?
It is a fixed permutation of four equal-length blocks with the braces re-added. A permutation of equal blocks is undone by applying the matching swap, and in this case the blocks end up exchanged in pairs, so one slicing operation reverses it.
New to capture the flag? Formats, teams and a first-month plan in one place.
Read the CTF guide