Initial public release: x86-32 to C static recompiler and macOS ARM64 runtime

Author: TalhaMudo

.gitignore

@@ -0,0 +1,24 @@
+# --- Oyunun kendisi ve ondan tureyen her sey: ASLA repoya girmez ---
+# Oyun verisi + SPEED2.EXE (EA'nin telifli icerigi)
+/Need for Speed Underground 2*/
+# Lift edilmis C kaynagi: SPEED2.EXE'nin makine kodunun cevirisi (turev eser).
+# tools/reloc2.py + tools/genall.py + tools/argcount.py ile birebir yeniden uretilir.
+/gen/
+# Derlenmis ikililer, .app paketleri, dagitim zip'i, gomulu oyun verisi
+/out/
+
+# --- Kisisel / oturum kayitlari ---
+/session.jsonl
+/Claude outputs/
+# Ham oturum ekran goruntuleri (secme/ ve rapor/ haric)
+/sunum_ss/*
+!/sunum_ss/secme/
+!/sunum_ss/rapor/
+
+# --- Ortam / gecici ---
+/.venv/
+__pycache__/
+*.pyc
+.DS_Store
+*.o
+*.log

D3DX_HLE.md

@@ -0,0 +1,31 @@
+Copyright (c) 2026 Talha Müderrisoğlu. All rights reserved.
+
+This software and its source code, documentation, tools, images and other
+files in this repository (the "Software") are proprietary.
+
+1. Personal use. You may download, build, run and modify the Software on
+   your own computers, for your own personal, non-commercial use only, with
+   your own legally obtained copy of the game.
+
+2. Everything else requires permission. You may not publish, distribute,
+   share, upload, host, sell, sublicense or otherwise make available the
+   Software, any modified version of it, or any work derived from it, in
+   whole or in part, in source or binary form, and you may not use it for
+   any commercial purpose, without the prior written permission of the
+   copyright holder. This includes forks, ports, mirrors and builds made from
+   the Software.
+
+3. Requesting permission. Open an issue at
+   https://github.com/TalhaMudo/NFSU2Recomp to ask for permission. Any
+   permission given applies only to what it explicitly states.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+Earlier copies: earlier versions of this project that were distributed under
+other license terms remain subject to those terms only for those exact copies.
+This license applies to this version and all later versions.

GRAFIK_ARASTIRMA.md

@@ -0,0 +1,232 @@
+# NFSU2 on Apple Silicon — static recompilation of a Win32/x86 game to native ARM64
+
+This repository contains the tools and runtime that turn the original Windows
+executable of *Need for Speed: Underground 2* (2004, 32-bit x86, Direct3D 9)
+into a **native ARM64 macOS binary**. The game's x86 machine code is lifted to
+C ahead of time and compiled with clang. There is no Wine, no emulator, no JIT
+and no Windows involved at run time.
+
+> **This repository contains no game code and no game assets.** It contains the
+> recompiler and the runtime only. You need your own copy of the game; the C
+> code generated from its executable and the resulting binaries are derived
+> from the game and are not, and must not be, distributed here.
+
+![Race, native ARM64 build](sunum_ss/secme/27_20260924_guncel-yaris.jpg)
+
+## Status
+
+The game boots, reaches the menus, and races are playable with sound.
+
+| Area | State |
+|---|---|
+| x86-32 → C lifter (integer, x87, MMX, SSE/SSE2, 3DNow!, string ops) | 23,278 functions, 132.5 MB of generated C |
+| Win32 layer (CRT startup, heap, TLS, threads, files, registry, window) | working |
+| Direct3D 9 → Metal, shader bytecode (vs/ps 1.x–2.x) → MSL | working |
+| DirectSound → CoreAudio, DirectInput (keyboard, mouse) | working |
+| Windowed, fullscreen, Retina-native and 1080p widescreen | working |
+| Double-clickable `.app` (`./pack.sh`) | working |
+| Optional "HD" build: 4× MSAA, SSR, ray-traced reflections | working, see `GRAFIK_ARASTIRMA.md` |
+
+### Known limitations
+
+- **Exactly one build of `SPEED2.EXE` is supported** (see [Requirements](#requirements)).
+  Game addresses are hard-coded in the runtime, and the pipeline relies on a
+  property of that particular file (below).
+- Videos are skipped: there is no VP6 decoder.
+- Keyboard and mouse only; no gamepad or wheel support. No network play.
+- Structured exception handling is not implemented (`RtlUnwind` /
+  `RaiseException` are stubs).
+- x87 is modelled with 64-bit doubles, not 80-bit extended precision.
+- One workaround with an unknown root cause remains: the front end's
+  world-view-projection matrix arrives broken and an identity matrix is injected
+  instead (`FIXMAT`, see `run.sh`).
+- 400 instructions in rarely used opcodes are still emitted as traps. None are
+  hit in the measured race scene.
+- Some visuals are unverified against a Windows reference (crowds / skinned
+  meshes, HUD seams). See "Kalan grafik maddeleri" in `PROJECT.md`.
+
+## How it works
+
+```
+SPEED2.EXE (PE32, x86)
+   │  tools/reloc2.py     recover the relocation table  → out/relocs.txt
+   │  tools/genall.py     recursive-descent discovery + tools/lift.py
+   ▼                      x86 → C, one C function per guest function
+gen/all/shard_*.c  (128 files) + fntab.c (guest address → native function)
+   │  build.sh            clang -O3, in parallel
+   ▼
+runtime/  (hand-written)  CPU state, guest memory, Win32, D3D9→Metal, audio
+   │  link.sh
+   ▼
+out/nfsu2_boot            46 MB arm64 Mach-O
+```
+
+- **Pointers vs. constants.** The hard problem of x86 static recompilation is
+  telling a 4-byte address from a 4-byte number. This executable has its
+  relocation *directory entry* zeroed, but the relocation *data* is still
+  physically present in an unnamed section. `tools/reloc2.py` parses it by hand:
+  91,106 absolute addresses, all of which point into the image. Function
+  discovery and jump-table recovery are seeded from it.
+- **Guest memory.** The 32-bit guest address space is reserved with `mmap`;
+  guest→host translation is a single ARM64 addressing mode
+  (`ldr w8, [x_base, w_addr, uxtw]`).
+- **Indirect calls** go through an O(1) table indexed by guest address.
+- **Code generated at run time.** The game writes small x86 thunks to its stack
+  and calls them. `runtime/genexec.c` detects these and runs them in a small
+  interpreter, handing calls back to lifted functions.
+- **D3DX effects.** The statically linked D3DX9 effect framework took ~47% of
+  guest time when executed as lifted code; `runtime/d3dx_hle.c` short-circuits
+  parts of it (`D3DX_HLE.md`).
+
+| Path | What it is |
+|---|---|
+| `tools/lift.py` | x86-32 → C lifter, with lazy EFLAGS (flag liveness) |
+| `tools/genall.py` | function discovery, parallel generation, sharding |
+| `tools/reloc2.py` | relocation table recovery |
+| `tools/argcount.py` | stdcall argument counts for imports |
+| `tools/isatest.py`, `tools/fuzzisa.py` | instruction-semantics tests and differential fuzzer |
+| `runtime/cpu.h`, `mem.c`, `dispatch.c`, `genexec.c` | CPU state, guest memory, dispatch, run-time code |
+| `runtime/win32.c`, `win32_ext.c`, `fileio.c`, `res.c` | Win32 API layer |
+| `runtime/com.c`, `metal.m`, `vsemu.c` | Direct3D 9 / DirectInput / DirectSound COM layer, Metal renderer, shader translation |
+| `runtime/audio.m` | CoreAudio output |
+| `runtime/overrides.c`, `d3dx_hle.c` | hand-written replacements for specific game functions |
+| `runtime/enhance.c` | HD build only (`-DNFS_ENHANCED=1`) |
+| `runtime/bundle.m`, `pack.sh` | `.app` behaviour and packaging |
+| `tests/boot.c` | program entry point (runs the game from its PE entry point) |
+
+## Verification
+
+- **Instruction semantics:** 169 hand-written tests, 439 checks
+  (`tools/isatest.py` → `out/isatest`).
+- **Differential fuzzing against real x86:** `tools/fuzzisa.py` assembles random
+  instruction sequences, runs them as real x86-64 code under Rosetta 2 and as
+  lifted C on ARM64, and compares registers, flags and memory.
+- **Function-level tests:** 15 checks against lifted game functions
+  (`out/nfsu2_all`).
+- **Reproducible generation:** the generated C is a pure function of the
+  executable and the tools; regenerating gives byte-identical output.
+
+`./run.sh test` runs the first and third.
+
+## Performance
+
+Measured on an M1 Pro at 1280×960, same scene and settings, alternating runs
+(method, raw numbers and caveats in `WINE_KIYAS.md`):
+
+| | This port | Wine 11.18 (wined3d, Rosetta 2) |
+|---|---|---|
+| Race | 119 fps (capped by the 120 Hz display) | 10–15 fps |
+| 3D menus | 119 fps | 45 fps |
+| CPU time per frame | ~12.6 ms | ~105 ms |
+
+## Requirements
+
+- Apple Silicon Mac, macOS 11 or later. Developed on an M1 Pro, macOS 27.
+- Xcode Command Line Tools (`clang`).
+- Python 3 with `capstone` 5.x and `pefile` (`requirements.txt`).
+- Rosetta 2, only for `tools/fuzzisa.py`.
+- Your own installation of the game, with this exact executable:
+
+  | | |
+  |---|---|
+  | File | `SPEED2.EXE`, 4,788,224 bytes |
+  | SHA-256 | `8f6a9ae2c4cfa01181d2fd755d99c6f4b7103f60273908ea1de1e719dfe2a297` |
+
+  Other builds have not been tested and will not work without changes.
+
+## Build and run
+
+Run everything from the repository root.
+
+```bash
+export NFSU2_DATA="/path/to/Need for Speed Underground 2"   # folder containing SPEED2.EXE
+EXE="$NFSU2_DATA/SPEED2.EXE"
+
+python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
+
+# 1) relocation table -> out/relocs.txt
+.venv/bin/python tools/reloc2.py "$EXE"
+
+# 2) lift every function to C -> gen/all/   (~20 s)
+#    OVERRIDE lists the functions that runtime/overrides.c and
+#    runtime/d3dx_hle.c wrap; it must match them or the link fails.
+OVERRIDE=60fab0,61e390,61f130,5c7ed0,50d510,65ae35,6623ea,66aa3f,66ace2,6691b1,669296,664a2f,663a42,688e89,663275,662edc,660a52,668e0d,664792,66ac27,65addb,65acf5,66d13a,66d303,66d7e9,66e8a9,66f501,66ca38,65ab24 \
+  .venv/bin/python tools/genall.py "$EXE" 128
+
+# 3) import argument table -> gen/all/impargs.c
+.venv/bin/python tools/argcount.py "$EXE"
+
+# 4) compile the shards, then build the runtime and link
+./build.sh && ./link.sh
+
+./run.sh test        # ISA + function tests
+./run.sh             # windowed
+./run.sh 1080p       # 1920x1080, widescreen-corrected
+./run.sh hd          # HD build
+./pack.sh            # out/NFSU2.app
+```
+
+`run.sh` documents the environment switches; most are diagnostics.
+
+## Related work
+
+Static recompilation of x86 is not new, and this is not the first project to do
+it:
+
+- [M-HT/SR](https://github.com/M-HT/SR) statically recompiles DOS and Windows
+  game executables (Septerra Core, Battle Isle 3 and others) at assembly level,
+  for targets including macOS arm64.
+- [sp00nznet/xboxrecomp](https://github.com/sp00nznet/xboxrecomp) lifts original
+  Xbox (x86) games to C with a Python + Capstone pipeline very similar to this
+  one.
+- [N64: Recompiled](https://github.com/N64Recomp/N64Recomp) and
+  [XenonRecomp](https://github.com/hedge-dev/XenonRecomp) established the
+  approach for MIPS and PowerPC consoles, and are where the method here comes
+  from.
+- McSema, rev.ng, RetDec and BinRec are IR-based binary lifters from the research
+  side.
+
+What this project adds is narrower: recovering a relocation table whose
+directory entry was zeroed and using it to separate pointers from constants; a
+differential instruction fuzzer that uses Rosetta 2 as the oracle; a hybrid
+interpreter for x86 code the game generates at run time; and an end-to-end,
+playable Win32 + Direct3D 9 → macOS ARM64 + Metal port of a 23,000-function 3D
+game.
+
+## Documentation
+
+The engineering notes are in Turkish, as are the code comments.
+
+- `PROJECT.md` — chronological engineering log: findings, bugs and root causes,
+  measurements.
+- `D3DX_HLE.md` — high-level emulation of the D3DX effect framework.
+- `GRAFIK_ARASTIRMA.md` — graphics research and the HD build.
+- `WINE_KIYAS.md` — comparison against Wine.
+- `sunum_ss/rapor/nfsu2-rapor.html` — illustrated project report.
+
+## How this was built
+
+Development was carried out with an AI coding assistant (Claude Code) under the
+direction of the project owner. Goals, constraints and priorities, observations
+from playing the game, Windows reference material and direction decisions came
+from the project owner; implementation, measurement and debugging were done
+together with the assistant.
+
+## Legal
+
+Copyright (c) 2026 Talha Müderrisoğlu. All rights reserved.
+
+This repository is **not open source**. You may download, build, run and
+modify it for your own personal, non-commercial use with your own copy of the
+game. Publishing, distributing or sharing it — or any modified version, port,
+fork or build of it — and any commercial use require prior written
+permission. See `LICENSE`.
+
+**Permission:** to publish, share or build on this code, ask first by opening
+an issue in this repository.
+
+This project is not affiliated with or endorsed by Electronic Arts. *Need for
+Speed* and *Need for Speed: Underground 2* are trademarks of Electronic Arts
+Inc. The repository contains none of the game's code or data. Screenshots show
+the game running through this port and are included for documentation; the
+game content in them belongs to its owners.

LICENSE

@@ -0,0 +1,63 @@
+# Wine kıyas ölçümü (2026-10-03)
+
+Projenin çıkış noktası olan "oyunu Wine ile çalıştırma" yoluna karşı native
+portun ölçümü. Aynı makine (M1 Pro, macOS 27.0.1, 120 Hz ekran), aynı ayarlar
+(`~/.nfsu2_registry`: yarış çözünürlüğü indeks 3 = 1280x960, tüm efektler açık),
+aynı sahne (Quick Race → Circuit → Resort Loop, `TRACKS/STREAML4RA.BUN`), ses
+iki tarafta da kapalı. Native ve Wine **dönüşümlü** iki tur koşuldu
+(N → W → N → W) ki ısıl sürükleme iki tarafa eşit dağılsın.
+
+## Sonuç
+
+| | Native port | Wine 11.18 (wined3d, Rosetta) | oran |
+|---|---|---|---|
+| Yarış, +10..60 s | **119.3 / 119.0 fps** (8.34 ms medyan) | **11.3 / 10.0 fps** (~88–100 ms) | ~11x |
+| Yarış, +60..140 s | 119.8 / 119.3 fps | 14.6 / 12.8 fps (~68–78 ms) | ~8.7x |
+| Kare süresi p99 (yarış) | 9.4–10.6 ms | 10 sn'lik pencerelerde 4–5 fps'e düşüşler | |
+| 3D menüler | 119.5 / 118.7 fps | 45.4 / 45.9 fps | ~2.6x |
+| Oyun süreci CPU (yarış ortası) | %151 / %148 | %142 / %144 (tüm Wine süreçleri %147–151) | |
+| **Kare başına CPU zamanı** | **~12.6 ms** | **~105 ms** | **~8x** |
+| Enerji etkisi (`top` POWER) | 123 / 121 | 116 / 117 | kare başına ~8x |
+| RSS (oyun süreci) | ~575 MB | ~345 MB (+ wineserver) | |
+| Açılış güvenilirliği | 4/4 | 11 açılışın 7'si 15 sn içinde kare üretmedi (otomatik yeniden denendi) | |
+
+**Native yarışta 120 Hz vsync tavanına dayalı** (`NOVSYNC=1` ile de 119–120);
+gerçek tavan payı bu tablodan yüksek, yani oran **en az** 8–11x.
+
+Wine tarafında hiçbir thread doymuş değil: oyun thread'i ~%70–77 meşgul,
+`wined3d_cs` (komut thread'i) ~%44–48. Toplam CPU ~1.5 çekirdek iken kare
+başına 8x daha fazla CPU harcanıyor; geri kalan zaman muhtemelen thread'ler
+arası senkronizasyon ve GL→Metal sürücü beklemesi. Bu bir çıkarım: Rosetta
+altındaki kod `sample` ile sembollenmedi.
+
+## Wine tarafında karşılaşılanlar (Wine yolunun gerçek maliyeti)
+
+1. Homebrew'ün `wine-stable`/`wine@devel` cask'ları 2026-09-01'de kapatıldı;
+   macOS 27 güncellemesi Rosetta 2'yi kaldırmıştı (elle yeniden kuruldu).
+2. Tam ekran 1280x960 macdrv'de reddediliyor (`ChangeDisplaySettings -2`) →
+   `CreateDevice` başarısız → oyun `0x5D2790`'da null okumasıyla çöküyor.
+   Yalnızca Wine sanal masaüstüyle açılıyor.
+3. Pencere Mac'te öne geçemezse (ekran kilitliyken ya da kullanıcı başka
+   uygulamadayken) tam ekran D3D cihazı "kayıp" durumunda kalıyor, oyun
+   donuyor. macOS 14+ kooperatif aktivasyonu ve kilitli ekran yüzünden
+   programla öne alınamıyor.
+4. Açılışta aralıklı çökme (`7BF21239`, Wine DLL'i) ve sessiz kapanmalar.
+
+## Yöntem ve sapmalar (dürüstlük notu)
+
+- Kilitli ekran yüzünden Wine koşuları **pencereli modda** yapıldı. Bunun için
+  oyunla gelen ThirteenAG widescreen fix ASI'si (`dinput8=n,b`) yüklendi ama
+  `WindowedMode=2` dışındaki tüm müdahaleler kapatıldı (FixHUD/FOV, FPSLimit,
+  SingleCoreAffinity, ConsoleGamma, SkipIntro vb. = 0). Native taraf ASI'siz.
+- Wine'da menü girişi zamanlamalı `keybd_event` enjektörüyle yapıldı
+  (`~/wine-olcum/tools/mkinject.py`, clang + elle PE32), native'de `AUTOSEQ`.
+  Pist her iki tarafta da L4RA; Wine'da araba koşudan koşuya değişti
+  (206 / 106), native'de hep Peugeot 206.
+- Native fps: `FPSCSV`, yarış başı = çizimi 1000'i aşan ilk sürekli kare,
+  `cizim > 400` kareleri. Wine fps: `WINEDEBUG=+timestamp,+fps` (~1.5 sn'lik
+  ortalamalar, kare ağırlıklı birleştirildi), yarış başı = menülerden sonraki
+  ilk <20 fps raporu.
+- CPU/RSS `ps` ile 1 sn'de bir, enerji `top -stats power` ile 5 sn'de bir.
+- Ham veriler ve betikler: `~/wine-olcum/` (`ab.sh`, `run_wine.sh`,
+  `run_native.sh`, `tools/nstat.py`, `tools/wstat.py`, `tools/summary.py`,
+  `n_ab*.csv`, `w_ab*.log`).

PROJECT.md

@@ -0,0 +1,38 @@
+#!/bin/bash
+# 23k lift edilmis fonksiyonu paralel derle
+set -e
+ROOT="$(cd "$(dirname "$0")" && pwd)"
+# -O3 + -mcpu=native olculdu: yaris karesi ortanca 18.7 -> 15.7 ms
+# (53.5 -> 63.9 fps).  Lifted kod cok sayida kucuk temel bloktan olusuyor,
+# daha agresif inline/sadelestirme dogrudan geri donuyor.
+# DAGITIM tabani:
+#  -mcpu=apple-m1  : M1 tum Apple Silicon'un ortak paydasi.  "native" ile
+#                    derlenirse yeni bir Mac'te uretilen ikili M1'de gecersiz
+#                    komutla coker.  M1'de olculen fark sifir.
+#  -mmacosx-version-min=11.0 : bayrak verilmezse ikili derleyen makinenin OS
+#                    surumuyle damgalanir (minos 26.0) ve eski macOS'ta
+#                    "daha yeni surum gerekiyor" diyerek hic acilmaz.
+MCPU="${MCPU:-apple-m1}"
+MINOS="${MINOS:-11.0}"
+OPT="${OPT:--O3 -mcpu=$MCPU -mmacosx-version-min=$MINOS}"
+J="${J:-$(sysctl -n hw.ncpu)}"
+mkdir -p "$ROOT/out/obj"
+cd "$ROOT/gen/all"
+# $OPT'u xargs komut satirina GOMME: -I{} ile kurulan satirin uzunlugu
+# sinirli ("command line cannot be assembled, too long").  Disaridan
+# ortamla gecir, ic kabuk genisletsin.
+#
+# Derleyici ciktisini DOGRUDAN "| head -5"e BAGLAMA: head 5 satirdan sonra
+# kapaninca clang bir sonraki uyariyi yazarken SIGPIPE ile oluyor ve nesne
+# dosyasi HIC yazilmiyordu (shard_0006/0019: 3 denemede 3 kez), ustelik
+# build.sh yine 0 donuyordu -> link "symbol(s) not found" ile kiriliyordu.
+# Cikti once degiskene alinir, sonra kisaltilir; clang'in cikis kodu korunur
+# (herhangi bir shard derlenemezse xargs, dolayisiyla build.sh hata doner).
+export OPT ROOT
+ls shard_*.c fntab.c stubs.c 2>/dev/null | \
+  xargs -P "$J" -n 1 sh -c '
+    o=$(clang $OPT -c -I"$ROOT/runtime" -I. -Wno-unused-label -Wno-parentheses-equality \
+          -o "$ROOT/out/obj/$(basename "$1" .c).o" "$1" 2>&1); r=$?
+    [ -z "$o" ] || printf "%s\n" "$o" | head -5
+    [ $r -eq 0 ] || echo "DERLENEMEDI: $1" >&2
+    exit $r' sh