In March 2016, a beta of a small project I was working on showed up on Ars Technica. The headline was "How a new emulator generates 3D scenes from 2D NES games." The emulator was mine. It only ran in Firefox. It was full of bugs. I was honestly surprised that anyone wrote about it at all.
That project shipped as 3dSen 1.0 on Steam in 2025, ten years later. I'd never planned for it to take this long. This is a brief look back at what changed in those years and what I got wrong along the way.
How it started
I grew up in Vietnam in the 1980s. The NES was my first console. Like a lot of kids my age, I used to imagine what those flat 8-bit worlds would look like in real space. The pipes you went into. The pits you fell down. The towers you climbed up.
By 2015 I was a developer at an outsourcing company. The work didn't fit me well. The work was repetitive and I lost most of my motivation. I kept asking myself why I wasn't building something I actually cared about. I had no plan and no business case. I just wanted to see Mario in 3D, and as far as I could tell, nobody else was doing it.
So I started.
The first idea was full automation
The first version of 3dSen tried to do everything automatically. The same algorithm should look at any ROM, identify shapes on screen, and reconstruct them in 3D in real time. No per-game data. No manual work.
When Ars Technica asked me about it, I told them: "Everything is calculated automatically in runtime. Nothing is done manually." I believed it, and at the time it was true.
The approach seems promising for the initial set of testing games. Pipes in Super Mario Bros. became cylinders. Ladders in Mega Man stayed in the background while wall enemies came forward. The shape-recognition algorithm clustered tiles by colour patterns. That worked roughly 50% of the time.
The other 50% was bad. Some games rendered as floating cubes. Others looked correct on a title screen and broke as soon as you started a level. The algorithm wasn't buggy. The algorithm was doing exactly what I told it to do. The problem was that I was asking it to do something it couldn't.
The thing I didn't see at first
The NES doesn't actually know what it's drawing.
The hardware just paints 8×8-pixel tiles. It doesn't know which tiles are walls, which are decorations, which are characters. It doesn't know a pipe is a pipe.
When you ask "is this tile a wall, or a backdrop?" — there's no answer hiding in the ROM. The same 8×8 tile pattern can be a brick in a wall, a cloud in the sky, or a UI element. Three different 3D interpretations. The NES sees them as identical.
A few years later I told Kotaku: "I often have difficulties to fully understand the pixelated graphics in NES games. They have a high level of abstraction, so sometimes it takes a lot of time to fully understand what some graphic elements represent." Even I can't always tell what a chunk of NES pixels is supposed to represent. I've been staring at sprite sheets for over a decade. If I struggle, an algorithm definitely will.
I was trying to solve, automatically, a problem that doesn't have a single right answer.
What 3dSen actually does
By the time I gave a video interview around the 2020 Early Access launch, the engine had settled into a four-step pipeline. It still works this way:
Step 1. Sample the PPU's output and divide it into 8×8 tiles. Fully automatic, game-agnostic.
Step 2. Cluster adjacent tiles into shapes — a pipe, a wall, a character. The algorithm makes the first pass. A profile maker can adjust the result. Semi-automatic.
Step 3. Map each 2D shape into a 3D shape and a position in space. Walls become walls, pipes become cylinders, characters get volume.
Step 4. Run the profile script. Apply per-game adjustments — rotations, animations, depth tweaks, hidden shapes, added shapes. This is where most of the work lives.
A profile is two things. Static shape settings that say "this tile is a wall, that tile is a backdrop, this sprite is a cylinder." And runtime scripts that handle situations the static rules can't. The scripts matter because shape properties aren't fixed across a game. The same on-screen tile can need to behave like a wall in one room and a foreground prop in the next. Only by inspecting the running game state can a profile decide what's correct in any given frame.
That's why the work scales the way it does. Simple games like Mario Bros., Dr. Mario, Donkey Kong, Galaga take a few days each. Bigger games like Super Mario Bros. 3, The Legend of Zelda, Excitebike depend heavily on context and need substantial scripting. Those take weeks, sometimes months. The profile is a small program that runs alongside the emulator.
Making peace with profiles
It took a long time to feel okay about profiles. They felt like a defeat. The 2016 vision of universal automation was elegant. Per-game work seemed like giving up.
What helped me was understanding that the interpretation of a game is a creative decision, and the implementation of that interpretation is just code. Choosing whether a cluster of pixels is a tower, a hat, or a window is something a person has to decide. Writing the script that detects the cluster and renders it correctly is engineering. Both halves are needed. A profile is where they meet.
Once I let go of "fully automatic," other things became possible. The conversions got better. As I told Kotaku, every supported game in 3dSen has been "painstakingly handmade." I don't think that work could have been done by an algorithm. The polish that makes Mario feel like Mario in 3D requires someone to sit with the game and decide.
There's something about this approach I find genuinely nice. There isn't one right answer. With the 3dSenMaker tool, anyone can author a profile. You don't have to agree with my interpretation of Castlevania. You can build your own. Different profile makers will produce different Zeldas. None of them is wrong.
That's already happening. The roughly 100 profiles that ship with 3dSen are the ones I built myself. The games I've spent the last decade tuning. There are now also over 30 community-made profiles in the public repository on itch.io, built by people I've never met, for games I might not have prioritised. Some of those community profiles are collaborations. A community member starts one, hits a wall, and we work through the hard parts together.
That part still surprises me. I made the tool because I needed it. I didn't expect anyone else to use it the way they have.
Looking back
Ten years on a single project is not what I planned in 2015. I thought 3dSen would be a one-year hobby that I'd finish and move on from.
What actually happened is that every problem I solved revealed two more behind it. Games that swap tile graphics every frame. Games that abuse mid-frame PPU tricks. Z-layer ambiguities. Animation timing. And, oddly, games that already use perspective or pseudo-3D effects. Those turn out to be the hardest of all. Trying to 3Dify a game that's already trying to look 3D is its own special kind of headache.
The original 2016 framing wasn't a mistake either. The fully-automatic version was a real attempt at a hard problem. I learned a lot from watching it fail. If I'd started with profiles, I don't think I would have understood why they were necessary.
3dSen 1.0 ships now with the games I built profiles for over the years, plus a growing list contributed by the community. Together they cover well over a hundred titles. More arrive every few weeks. It's smaller than I once hoped, larger than I had any right to expect, and the result of a lot of late nights. The kid in 1980s Vietnam who wanted to see Mario's pipes in 3D would, I think, be quietly happy.
And I think that's okay.