Challenge Description#

Given challenge. UPX-packed binary expecting license file.

Analysis#

After unpacking: binary requires LICENSE-<hex> format. Has anti-debug (ptrace, /proc/self/status), SIGILL handler for VM execution.

VM logic recovered:

out[i] = CONST[i] XOR license[i mod 10]   for i = 0..45

Recovered 46-byte constant: dbbc3342678166ae9a08e0c6154e46ac7fb9c245aa87386814a07fa0984ead83547d7bb8598ac30ffa87542611a8

Only the first 10 license bytes matter for the transformed output.

Solution#

Derive first 8 key bytes from BITSCTF{ prefix, brute-force remaining 2 bytes:

#!/usr/bin/env python3
import re

CONST = bytes.fromhex(
    "dbbc3342678166ae9a08e0c6154e46ac7fb9c245aa873868"
    "14a07fa0984ead83547d7bb8598ac30ffa87542611a8"
)

PREFIX = b"BITSCTF{"


def decode_with_key10(key10: bytes) -> bytes:
    return bytes(CONST[i] ^ key10[i % 10] for i in range(len(CONST)))


def main():
    key = [None] * 10
    for i, ch in enumerate(PREFIX):
        key[i] = CONST[i] ^ ch

    pattern = re.compile(r"^BITSCTF\{[A-Za-z0-9_]+\}$")

    for k8 in range(256):
        for k9 in range(256):
            key[8] = k8
            key[9] = k9
            key10 = bytes(key)
            pt = decode_with_key10(key10)

            try:
                s = pt.decode("ascii")
            except UnicodeDecodeError:
                continue

            if pattern.fullmatch(s):
                print(f"key10_hex={key10.hex()}  flag={s}")


if __name__ == "__main__":
    main()

Output:

key10_hex=99f5671124d520d5f63c
flag=BITSCTF{l4y3r_by_l4y3r_y0u_unr4v3l_my_53cr375}

Valid license: LICENSE-99f5671124d520d5f63c