An in-depth look at Link’s Awakening render loop

This article is part of an ongoing “Disassembling Link’s Awakening” series, where I attempt to gain some understanding on how special effects were implemented in this game.

We’ve seen previously how Link’s Awakening renders the opening cutscene. This time, let’s take a step back, generalize, and have a look at the main render loop.

The game’s core

Games are usually based on a main loop. Conceptually, it looks like this:

while (true) { // loop forever

In plain text, the main loop repeats the same operations, once per frame, over and over:

Of course this is a simplification, and a few key elements have been omitted (for instance there is no audio there). For more information, you can read this much more extensive article about the render loops generic structure.

Link’s Awakening makes no exception, and has its own render loop, right after the initialization code. Let’s see how it is handled.

Read more…

WideGB now available on Windows

Good news: the WideGB emulator is now available on Windows. See below for download links.

The WideGB Game Boy emulator running on Windows

What took so long?

When the first WideGB release was published a few weeks ago, only macOS builds were available.

Theoretically, the emulator WideGB is based on (SameBoy) is also available on Windows—so it should have been a simple matter of booting a Windows machine, and compiling the code.

So what took so long? Well, several small things.

First, setting up a Windows development environment in a Virtual machine is somehow painful. Downloading a Windows VM, installing Visual Studio, configuring the build environment, setting up dependencies, getting the vanilla SameBoy to build, and so on.

Second, I wanted to share the same source folders between my macOS host and the Windows guest. Which means having shared folders, which are mounted on Windows as an external network drive. And it turns out that on Windows, relative paths in Makefiles work by accident… except when mixing different drives, where they stop working entirely. More scripting and fixing required.

Last thing, I thought C was portable, and recompiling wouldn’t require major changes. Ha. Of course many POSIX features are not available on Windows (except when using a POSIX compatibility layer—but the SameBoy build system doesn’t). So remplacements or compatibility stubs were found for creating, enumerating and deleting directories, generating random numbers, etc.

As always, this is very much a work in progress. Glitches are to be expected.

What’s next?

Now that the Windows version compiles, integrating WideGB in the libretro port should be interesting. SameBoy already features a working libretro core—so maybe WideGB could be integrated into this.

The heuristic used to detect a new scene is still finicky: sometimes it detects too little, and sometimes too much. A better perceptual hashing implementation could be less sensitive to scrolling (which hardly ever signals a scene change), and more sensitive to hard-cuts and fades-to-white.

To fine-tune the algorithm, a good tool would be to dump some typical frames of different games into PNG files—and then use a scripting language to test different algorithms. New algorithms could be applied to the dumped frames, to compare the results and check for regressions.

Also, my C-fu is still not as good as I’d like to: there are some inefficiencies, crashes, and low-hanging performance optimisations that would definitely improve the stability of WideGB.

And of course, contributions are welcome!

WideGB: playing Game Boy games on wide screens

A few months ago, Daniel Prilik released WideNES. This clever emulator hack allows to peek beyond the screen boundaries of NES games, to see more of the game, and to play in wide screen formats.

To celebrate the 30th anniversary of the Game Boy release, today I’m releasing a WideNES-inspired emulator, WideGB.

What is this thing?

WideGB is an emulator. Well, it is actually a modified version of SameBoy, an excellent and very accurate Game Boy emulator made by Lior Halphon.

When running a build of SameBoy compiled with WideGB, an extra option appears: “Use Widescreen”. Enabling this option extends the screen boundaries of the game: you can then resize the window, and use the aspect ratio you wish.

How does it work?

WideGB is very similar to WideNES. It basically records the screen as it moves, and keeps the parts of the screen previously drawn in place.

When starting a game for the first time, or when reaching a new area of the game, WideGB doesn’t know yet about any parts of the screen. But as soon as you move, it starts recording the area graphics, and you can see the places you’ve been to appearing gradually.

When the player moves to a new location (such as entering inside a house), WideGB detects that the picture changed suddenly: it saves the previous scene, and starts a new one. It even looks for previously encountered locations, so that if a scene was already visited, it can be restored automatically.

Additionally, WideGB attempts to draw the HUD of the game (with timers, lifes count, etc.) with a translucency effect, so that the part of the game under the HUD are still visible.

A limitation of this method is that sprites are not recorded. Sprites are often used for NPC and ennemies: recording them causes still characters to appears at the edge of the screen, and doesn’t look very good.

However, this method has the benefit of being (in theory) compatible with every game. Truth to be told, some heuristics needs to be tuned on a per-game basis (such as “how much does a picture need to change to signal a new scene?”). But should work with most games of the Game Boy and Game Boy Color library. It has been tested with Pokémon Red, Gold, Super Mario Land 2, and Zelda: Link’s Awakening.

For more informations on the inner workings of this emulating method, see the original article describing the internals of WideNES.

Download it

Want to try this at home?

What needs to be improved

This is very much a work in progress.

As you can see, the Windows build is not out yet. Some Windows-specific code has to be adjusted (like enumerating directories), and a build should be available soon. EDIT: the Windows port is now available.

A limitation of the current engine is the heuristic used to detect a new scene. A perceptual hash of the current and next frame is used–but sometimes it detects too little, and sometimes too much. A better perceptual hashing implementation could be less sensitive to scrolling (which harding ever signals a scene change), and more sensitive to hard-cuts and fades-to-white.

On the long run, I’d like to merge WideGB into the officiel SameBoy tree. However, it could also be easily integrated into other emulators. WideGB is designed as a platform-agnostic library. This means it can be used with any kind of emulator, regardless of the rendering method: give it frames, and it will split out the data required to render the extended screen. It has already been used as a backend for two different rendering methods (Cocoa and SDL), and should be quite flexible.

If you are interested, have a look at the source code. And, of course, contributions are welcome.

Happy 30th birthday, Game Boy!

Link’s Awakening disassembly progress report – week 4

This article is part of an ongoing “Disassembling Link’s Awakening” series, where I attempt to gain some understanding on how special effects were implemented in this game.

After a six-month pause, disassembling efforts have resumed again! And quite a lot got done this week.

Dump the game’s dialogs

Back to December, a surprise contribution arrived from Sanqui. He submitted a pull request containing a clean dump of all the game’s dialogs.

Sanqui even explained how he extracted the dialog’s texts and indexing table from the ROM:

I found the text in the intro (“What a relief!”), set a breakpoint on it, and traced back the relevant code to figure out where the pointer table is. From there I was ready to write a script to dump all the text. :)

He may make it simpler than it sound though: the Python script he wrote is quite a piece of work, and can extract all the dialogs in a readable format.

Anyway you can now browse through the game’s dialog, from Marin’s iconic opening lines to the texts added specifically for the DX version.

Link's Awakening First dialog lines

There is a lot going on to display these letters on screen.

Figure out how the dialog system works

Beside dumping the dialog’s data, Sanqui also reverse-engineered how the game actually prints a dialog on-screen.

As many things in the game, the dialog system is driven by a state-machine, dispatching the execution according to all the states the dialog can be in.

; Values for wDialogState
DIALOG_CLOSED              equ $00
DIALOG_OPENING_1           equ $01
DIALOG_OPENING_2           equ $02
DIALOG_OPENING_3           equ $03
DIALOG_OPENING_4           equ $04
DIALOG_OPENING_5           equ $05
DIALOG_LETTER_IN_1         equ $06
DIALOG_LETTER_IN_2         equ $07
DIALOG_LETTER_IN_3         equ $08
DIALOG_BREAK               equ $09 ; press A to continue
DIALOG_SCROLLING_1         equ $0A
DIALOG_SCROLLING_2         equ $0B
DIALOG_END                 equ $0C ; press A to close
DIALOG_CHOICE              equ $0D ; press A to choose
DIALOG_CLOSING_1           equ $0E
DIALOG_CLOSING_2           equ $0F

The dialog system takes advantage of an interesting data-transfert system used throughout the game. It allows a function to define an asynchronous data request to update the Video Background data. During the next vertical-blank, this request will be executed by the VBlank handler, which will display the next letter of the dialog’s text.

Also, if you played the game, you probably remember how it is possible to steal one of the items from Mabe’s Village shop.

Animation of Link stealing an item in the shop

I wouldn’t advise going back to this shop again.

If you actually do this, your save file will be renamed to “THIEF” – without any way to change it back.

Well, turns out this behavior has been slightly obfuscated: in the code, the characters string "THIEF" is actually stored as 'T'+1, 'H'+1, 'I'+1, 'E'+1, 'F'+1. Which means that for ROM hackers looking at the data, all that will appear is "UIJFG", and no thief to be found.

Add disassembly for bank 2

For a long time, extracting the resources of the game (pictures, dialogs) made good progress–but disassembling the code kind of stalled.

The thing is, only some portions of the code are extracted yet (let alone labelled and documented). And it became more and more difficult to disassemble a new bank. Existing disassemblers were not good enough to produce a workable output, and often lacked the ability to use existing labelled symbols when disassembling a new bank.

I tried for many hours to fix the Python-based disassembler used for the Pokemon Blue/Red disassembly, but I found the code hard to edit and prone to unwanted changes when adding new features.

Fortunately, no more than two months ago, mattcurie released a new Game Boy disassembler, mgbdis, also written in Python. It already took advantages of symbol files to disassemble new banks, and I found it relatively easy to fix some minor issues, edit the output style, and add new features.

After spending some hours tweaking the output, a new bank was finally committed: we have the code for bank 2! This bank contains some part of the audio engine, plus gameplay-related code.

Of course much of it still remains to be documented. But the logic for selecting the music track to be played on the overworld has already been pretty well documented; you can check it out.

Code sample with the Overworld music data

These values map the Overworld. Can you recognize the Mysterious Forest on the left ($04), and the Tal Tal Mountain Range ($06) on the top?

What’s next

Now that the disassembler can produce high-quality output, before reverse-engineering more code, I would like to add disassemblies for the other banks. The trick is to identify which sections are code and which are data–but at least for some of these banks it should be relatively easy to figure it out.

Le côté obscur du cerveau

J’aime beaucoup la capacité des sciences, naturelles et sociales, à proposer des modèles mentaux qui rendent intuitif un comportement complexe. C’est particulièrement frappant en biologie, où le bon modèle mental peut permettre de donner du sens aux réactions de son propre corps.

Je sors justement d’une intervention de Mani Ramaswami, neurobiologiste, qui est venu à l’IISER Pune parler de ses recherches sur un mécanisme particulier du cerveau : l’inhibition des engrammes. Sous le titre accrocheur (« Le côté obscur du cerveau »), et le jargon scientifique, il s’agit en fait d’un modèle mental de « Comment ça se fait qu’on s’habitue à des stimulus de toute sorte ». Et ce modèle mental permet de donner une intuition de plein de choses : le filtrage des bruits de fond, la mémoire à long-terme, la fonction du sommeil, et certaines maladies mentales.

Ces recherches se basent sur un certain nombre d’expériences et de publications – mais pour être honnête, je n’ai pas tout retenu. Je vous laisse voir les expérimentations par vous même – et à la place je vais plutôt vous raconter le modèle mental avec les mains.

Comment le cerveau s’habitue-t-il ?

Le cerveau est une machine à filtrer. Plus précisément une des fonctions principales du cerveau est de séparer les informations pertinentes des détails pas importants : les bruits de fond, ou les bruits répétitifs, ou les mouvements perturbants, etc.

Ce filtrage se fait en bonne partie par la diminution de l’importance accordée aux évènements répétitifs et peu pertinents. Par exemple on rentre dans une pièce, et on remarque qu’il y a un bruit de fond, ou une odeur persistante – mais passé dix minutes on ne s’en rend plus vraiment compte.

Comment est-ce que ça fonctionne ? Eh bien on peut faire des expériences qui montrent que les stimulus complexes sont encodés comme des groupes de neurones qui, à force d’être stimulés, se déclenchent simultanément. Les neurobiologistes appellent ça des engrammes.

L’histoire, c’est qu’au fur et à mesure qu’un stimulus se répète, une réaction en miroir se construit : des neurotransmetteurs inhibent cet engramme. Ce n’est pas qu’il disparait : il est toujours activé, mais il y a également des inhibiteurs sur ce groupe de neurones. La réaction transmise in fine au cerveau est alors bien moindre : on s’est habitué.

Diagramme d’une exposition à un stimuli initial, puis une fois l’habituation déclenchée

C’est également ce qui explique (à très gros traits) comment un stimulus, même inhibé à force de répétition, peut revenir rapidement, dès qu’il y a un léger changement des conditions : les inhibiteurs disparaissent, l’engramme est actif à nouveau.

La mémoire à long-terme

Ce mécanisme de stimulus → répétition → habituation permet de modéliser plein de comportements différents.

C’est le cas par exemple du passage de la mémoire à court-terme à la mémoire à long-terme. On peut imaginer le processus de cette manière :

  1. On a une expérience ;
    (→ stockée comme un engramme)
  2. Le souvenir de cette expérience tourne dans la tête ;
    (→ l’engramme réagit très facilement, et est répété par le cerveau)
  3. Après un moment le souvenir s’atténue, et cesse d’être présent au quotidien ;
    (→ à force de répétition, l’engramme reçoit des inhibiteurs)
  4. Mais on peut toujours convoquer ce souvenir explicitement dans la mémoire à long-terme.
    (→ les inhibiteurs sont supprimés, et l’engramme redevient actif)

Schématiquement, ce passage d’une mémoire qui s’impose (court terme) à une mémoire convocable à la demande (long terme) se représenterait de cette façon :

Un souvenir nouveau est présent dans la mémoire à court terme ; l’inhibition par la répétition le fait passer dans la mémoire à court terme.

C’est donc l’inhibition par la répétition qui ferait passer un souvenir ou un stimulus de la mémoire court-terme à la mémoire long-terme. Et de la même manière, la dé-inhibition ré-activerait le souvenir ou le stimulus.

« Je vais dormir dessus »

Ce modèle mental donne également une intuition sur une des fonction du sommeil.

On dit souvent qu’on mémorise en dormant – et les expériences montrent effectivement des liens entre le sommeil et la mémoire. Mais que se passe-t-il concrètement ?

On peut utiliser une analogie. Quand on dort, on sait que le système moteur est déconnecté ; c’est pour cela qu’on peut rêver de jouer au foot sans bouger les jambes. Une hypothèse est que le système émotionnel serait également déconnecté de la même manière. Le sommeil serait alors le moment où on peut rejouer des souvenirs en boucle, sans pour autant susciter de réaction émotionnelle. Et la répétition permet justement de construire des inhibiteurs, et donc de faire passer le souvenir dans la mémoire à long terme.

Pour Ramaswami, ça permet aussi d’avoir une intuition sur le fonctionnement des rêves : ils seraient provoqués par des inhibiteurs qui lâchent aléatoirement pendant le sommeil. Des souvenirs disparates sont donc ramenés à la conscience — et ensuite le cerveau essaie de donner du sens avec ça (parce qu’il parait qu’une des zone du cerveau a pour fonction spécifique de donner du sens aux choses, ce qui n’a pas fini pas de m’étonner.)

En cas de pépin

Enfin, cette manière de représenter la mémoire ouvre une porte sur les mécanismes à l’œuvre dans certaines maladies mentales.

Par exemple, si la fonction d’inhibition ne marche plus correctement, on se retrouve avec des souvenirs qui tournent en boucle (parce que la répétition est nécessaire à la construction de l’inhibition), mais qui ne quittent pas la mémoire à court-terme. Cela peut donner une idée de ce qui se passe avec certaines formes de stress post-traumatique.

Et dans l’autre sens, si le mécanisme suppresseur de l’inhibition dysfonctionne, ça donnerait des formes d’amnésie. Ce qui peut aussi expliquer comment il est possible de se souvenir de choses qu’on croyait avoir oubliées : le souvenir était là, mais inhibé.

Je crois qu’une partie de ces intuitions sont encore des théories – même si le mécanisme fondamental de stimulus-inhibition a l’air bien documenté. Mais je trouve que ça donne de chouettes résultats, et une manière intéressante de penser le sommeil et les souvenirs.