Python inside an exe

WhiteHat Contest 11 RE3: recovering Python from a py2exe executable

RE3 ships a Windows executable that crashes with a Python traceback. That traceback gives the task away: the program is bundled Python 2, and the rest is decompilation and a factoring exercise.

Hexfield editorial

EventWhiteHat Contest 11
CategoryRE, 100 points
Packerpy2exe (Python 2)
FlagSHA1 of three room numbers
Reverse100 pts
  1. 01

    Inspect

    A Windows console program draws ASCII art and offers two menu options. Choosing either one crashes with a urllib2 traceback that names digital_fortrees.py.

  2. 02

    Hypothesize

    The file names and the urllib2 frames show Python 2 packed with py2exe. The program downloads two helper scripts and runs them, so the logic is in plain sight.

  3. 03

    Test

    unpy2exe extracts the main .pyc, uncompyle2 restores the source, and the two downloaded scripts show that the second one multiplies directory names and compares the product to a constant.

  4. 04

    Solve

    Factor the 28-digit constant into its three prime factors and feed them into the flag formula.

Byte histogram of a Python 2 compiled .pyc file, with peaks at opcode values and a long tail of rarely used bytes
A compiled Python module has a recognizable byte profile: a few opcode bytes dominate, which separates it from packed or encrypted data at a glance.

The statement for RE3 is short: an executable named digital_fortrees.exe, and a flag defined as the SHA-1 of three strings joined with colons, one per room. Nothing else is given, which pushes the work toward finding out what kind of program it is before deciding how to analyze it.

Running it first

In a Windows virtual machine the program prints a large ASCII drawing of a castle, a banner reading Digital Fortress and a two-item menu: draw an infinity map, or go through all rooms on the map. Input is validated. Random text is rejected and the menu repeats, so there is no cheap way in through the prompt.

Picking either valid option ends the run abruptly with a traceback. It lists the script name digital_fortrees.py, a call to urllib2.urlopen, and the standard library frames underneath it, ending in URLError with a getaddrinfo failed message (the error code is the Winsock one for a host that cannot be resolved). That single screen answers three questions. The program is Python 2, because urllib2 does not exist in Python 3. It was packaged for Windows, because tracebacks show module names with .pyc suffixes, which is how py2exe reports frames from its bundled library archive. And it needs network access, because the menu choice triggers a download.

Getting the source back

py2exe stores compiled bytecode inside the executable, often in a resource section or in an appended archive. A small tool called unpy2exe walks that structure and writes out the .pyc files. Running it on the task binary yields the main script as digital_fortrees.py.pyc. A raw .pyc from Python 2.7 starts with a four-byte magic number and a four-byte timestamp, followed by a marshalled code object; the decompiler uncompyle2 handles that format and returns source that looks like what the author wrote, with names intact.

The recovered main module is short. It prints the ASCII art, reads a choice, and on a valid choice downloads one of two scripts from a fixed URL and executes it. The URLs are plain string constants, so the follow-up scripts can be fetched by hand without running anything. This matters for safety as well as speed: executing downloaded code from a wargame server on a personal machine is unnecessary when the same code can be read.

The two helper scripts

The first script, drawmap.py, runs an endless loop. It searches for prime numbers and, for each one it finds, creates a directory named after that prime. On its own it produces nothing useful, but it explains the menu text: drawing the map fills the current directory with rooms.

The second script, letgo.py, is the checker. It lists all directories in the current folder, multiplies their names together into a running product, opens a console window per room, and finally compares the product with a hard-coded integer of 28 digits. If the values match it prints a congratulation, otherwise a failure message.

So the check is not about running the game at all. It asks only for a set of directory names whose product equals one constant. Since each directory name is a prime created by the first script, the constant must be a product of primes, and the intended answer is the set of those primes.

Factoring the constant

A 28-digit integer is approximately 2^93. With three prime factors of similar size, each is around nine to ten digits, which is trivial for Pollard rho and even for trial division up to the square root of the smallest factor. A computer algebra system returns the factorization instantly; the original solution used a public web tool for it. In Python 2 or 3, sympy.factorint(n) returns the dictionary of primes in a fraction of a second, and gmpy2 offers a similar speed.

The result is exactly three primes, each with multiplicity one. These are the room numbers. The flag formula takes the first, second and third room in order, joined by colons, and hashes the string with SHA-1. Order matters, and the statement does not say whether it is ascending. Ascending order is the natural reading, because drawmap.py generates primes in increasing order and the first rooms created are the smallest ones.

Common wrong turns

Opening the exe in a native disassembler is the obvious first reaction and a poor one. The entry point is the py2exe launcher stub, and the code worth reading is bytecode in a resource. A second trap is to skip the helper scripts and look for the key in the main module, where only the menu exists. A third is overthinking the arithmetic: the constant is not encrypted, not salted and not a hash, only a product.

What transfers to other challenges

Whenever a Windows binary prints Python frames, the work is extraction, not reverse engineering of machine code. The tools differ by packer: unpy2exe for py2exe, pyinstxtractor for PyInstaller, and a direct marshal load for bare .pyc files. Version mismatches between the bytecode and the decompiler are the main practical issue, since a decompiler built for 2.7 fails on 3.x opcodes. The same read-the-checker approach solves the fork-and-pipe task in the RE1 walkthrough, and the layered ELF in Zwiebel shows what happens when the logic is hidden by code encryption instead. To see how compiled bytecode differs from packed or encrypted data in a byte histogram, read compression versus encryption.

Questions readers ask

How can a traceback reveal that an exe is Python?

py2exe bundles the interpreter and the compiled modules, and an unhandled exception prints the original module names and line numbers. Frames such as urllib2.pyc and a script name ending in .py are not produced by a native compiler, so they identify the runtime immediately.

Why does uncompyle2 matter here?

It turns Python 2 bytecode back into readable source. The extracted .pyc contains the menu logic and the URLs of the helper scripts, and reading that source is faster and less error-prone than stepping through bytecode by hand.

Is factoring a 28-digit number hard?

No. A number of that size is well within reach of trial division with a modest sieve, of a Pollard rho implementation, or of any computer algebra system. The task relies on the key parts being primes of moderate size, not on the difficulty of factoring.

New to capture the flag? Formats, teams and a first-month plan in one place.

Read the CTF guide