In March 2016, Ars Technica covered the beta of a small project I was building. The headline was "How a new emulator generates 3D scenes from 2D NES games." That emulator was mine. It only ran in Firefox, it was full of bugs, and I was surprised that anyone wrote about it.
Ten years later, that project shipped as 3dSen 1.0 on Steam in 2025. I never expected it to take that long. This is a short look at what changed, and what I got wrong along the way.
How it started
I grew up in Vietnam in the 1980s, and the NES was my first console. Like many kids my age, I imagined what its flat 8-bit worlds might look like in real space: pipes you went into, pits you fell down, and towers you climbed up.
By 2015, I was working as a developer at an outsourcing company. The repetitive work did not suit me, and I had lost most of my motivation. I kept wondering why I was not building something I cared about. There was no plan or business case. I simply wanted to see Mario in 3D, and as far as I knew, nobody else was doing it.
So I started.
The first idea was full automation
The first version of 3dSen tried to automate everything. One algorithm would read any ROM, identify the shapes on screen, and rebuild them in 3D in real time, with no per-game data or manual work.
When Ars Technica asked me about it, I told them: "Everything is calculated automatically in runtime. Nothing is done manually." I believed that, and at the time it was true.
The approach looked promising with the first test games. Pipes in Super Mario Bros. became cylinders. Ladders in Mega Man stayed in the background while wall enemies moved forward. The shape-recognition algorithm grouped tiles by colour pattern, and it worked roughly half the time.
The other half was a mess. Some games became clouds of floating cubes. Others looked right on the title screen and broke as soon as a level began. The algorithm was not broken; it was doing exactly what I had asked. I was asking it to solve a problem it could not solve.
The thing I didn't see at first
The NES doesn't actually know what it's drawing.
The hardware draws 8×8-pixel tiles. It does not know which are walls, decorations or characters. It does not know that a pipe is a pipe.
When you ask "is this tile a wall, or a backdrop?" — there's no answer hiding in the ROM. The same 8×8 tile pattern can be a brick in a wall, a cloud in the sky, or a UI element. Three different 3D interpretations. The NES sees them as identical.
A few years later I told Kotaku: "I often have difficulties to fully understand the pixelated graphics in NES games. They have a high level of abstraction, so sometimes it takes a lot of time to fully understand what some graphic elements represent." After more than a decade of studying sprite sheets, I still cannot always tell what a group of pixels represents. An algorithm will struggle too.
I was trying to automate a problem with no single right answer.
What 3dSen actually does
By the time I gave a video interview around the 2020 Early Access launch, the engine had settled on the four-step pipeline it still uses today:
Step 1. Sample the PPU output and divide it into 8×8 tiles. This step is fully automatic and game-agnostic.
Step 2. Group adjacent tiles into shapes such as a pipe, wall or character. The algorithm makes the first pass, and a profile maker can adjust the result.
Step 3. Map each 2D shape to a 3D form and position. Walls become walls, pipes become cylinders, and characters gain volume.
Step 4. Run the profile script to apply rotations, animations, depth adjustments, and any hidden or added shapes. Most of the work happens here.
A profile has two parts. First, static shape settings that say "this tile is a wall, that tile is a backdrop, this sprite is a cylinder." Second, runtime scripts for cases those rules cannot cover. A tile may be a wall in one room and a foreground prop in another, so the script checks the current game state and chooses the right interpretation for each frame.
This is why the workload varies so much. Simple games such as Mario Bros., Dr. Mario, Donkey Kong and Galaga take a few days each. Larger, context-heavy games such as Super Mario Bros. 3, The Legend of Zelda and Excitebike need substantial scripting and can take weeks or months. Each profile is a small program running alongside the emulator.
Making peace with profiles
It took me a long time to accept profiles. The 2016 idea of universal automation was elegant, and per-game work felt like giving up.
What changed my mind was realizing that interpreting a game is a creative decision, while implementing that interpretation is engineering. A person must decide whether a cluster of pixels is a tower, hat or window. Code can then detect it and render it correctly. A profile is where those two jobs meet.
Once I let go of "fully automatic," the conversions improved. As I told Kotaku, every supported game in 3dSen has been "painstakingly handmade." I do not think an algorithm could provide the judgment needed to make Mario still feel like Mario in 3D.
I like that this approach leaves room for interpretation. With the 3dSenMaker tool, anyone can make a profile. You do not have to agree with my version of Castlevania; you can build your own. Different makers will create different Zeldas, and none of them is necessarily wrong.
That is already happening. I built the roughly 100 profiles that ship with 3dSen over the past decade. There are also more than 30 community-made profiles in the public repository on itch.io, covering games I might not have prioritised. Some become collaborations: a community member starts a profile, reaches a difficult case, and we solve it together.
That still surprises me. I built the tool because I needed it, without expecting other people to use it this way.
Looking back
I did not plan to spend ten years on one project. In 2015, I thought 3dSen would be a year-long hobby that I would finish and leave behind.
Instead, every solved problem revealed two more: games that swap tile graphics every frame, use mid-frame PPU tricks, create ambiguous Z layers, or depend on exact animation timing. The hardest are often games that already use perspective or pseudo-3D effects. Converting a game that is already trying to look 3D brings its own kind of headache.
The original 2016 approach was not a mistake. It was a serious attempt at a hard problem, and I learned from watching it fail. If I had begun with profiles, I might never have understood why they were necessary.
3dSen 1.0 now includes the profiles I built over the years and a growing set from the community. Together they support well over a hundred games, with more arriving every few weeks. It is smaller than I once hoped, larger than I expected, and the result of many late nights. I think the kid in 1980s Vietnam who wanted to see Mario's pipes in 3D would be quietly happy.
And I think that's okay.