Not All Rabbit Holes Lead To Wonderland

By Stuart Dotson

The cursor blinked on the terminal. Sweat dripped down my brow.

The A/C was turned up way too high. But I couldn't get out of my chair to change it. Not yet.

It was the moment of truth.

I'd written a utility function. I'd even written a suite of unit tests. All that stood in the way of opening a pull request, begging for reviews, and releasing this code in prod was running the tests on my machine.

I pressed enter. The seconds went by. I got bored. I looked at my phone. What's happening on Reddit? I thought to myself. There's a smiling manatee. Look at those long whiskers. What are those called again? Ah, yes. Google says those are vibrissae. Manatees can't see much of anything; they use vibrissae to interact with the world.

A co-worker pinged me about an unrelated project. I'll see what I can do to help. Oh, hey, look at this. Someone posted an amusing pet photo. That dog thinks it's a person. Anthropomorphizing animals sure is amusing. What time is it again?

Half an hour later, I realized that the tests finished twenty-eight minutes ago. Two minutes to import a util function and test it.

Thus began a long and terrible rabbit hole that nearly snuffed the holiday twinkle from my eyes.

How imports and exports work in Node

Node modules are singletons. The very first time a file imports a node module, Node executes the module and builds the exports. This initialized module is saved in memory and used for every subsequent import.

That might sound a little dense, so here's an example:

// file #1

import { myUtil } from './utils'

// file #2 

// ...other imports

console.log('module initialized')

export const myUtil = () => 'booyah'

// ...other exports

No matter how many times myUtil or any other file #2 function is imported, we'll only see that log statement once.

Most Node codebases aren't quite so simple and include lots of files with sometimes very nested module dependency trees. If you execute a file that references file #2 at any point in the dependency tree, we'll see that log statement. You might be saying, "What's the big deal?"

Imagine that instead of logging, we're connecting to a database. Or instantiating a large and complicated client. We're doing all this work whether we need to or not. All we wanted was the util function and we got the whole galaxy of application infrastructure bearing down on us like the famed Atlas, the mythological Titan tasked with bearing the weight of the celestial spheres on his shoulders.

Down the rabbit hole

A white rabbit scampered by my desk. "What was that!" I yelled out.

"Follow me, I'll show you what's slowing things down. You can fix it. You can speed everything up. You'll be a hero," said the rabbit.

"Won't it take a lot of time? Shouldn't I be working on a ticket in the sprint?"

"It'll be oh so fast!" the rabbit said before scampering into the closet.

I reflected on the last decade of programming, and thought to myself, surely this time will be different. I followed the rabbit.

The rabbit led me to module after module in the dependency chain. After each visit, I'd sprinkle in a few log statements.

I scored some early wins. I opened a PR to avoid instantiating a client in the global module scope, which a co-worker approved. Some time later I found a class instantiation in global scope with another expensive operation in the constructor. Score! Another PR.

The performance gains were disappointingly small after a handful of PRs. I tried to isolate why we're connecting to a database. Or Redis. Or AWS. Or sending out the deep space request to the hideous spider aliens orbiting Alpha Centauri. I found myself playing wack-a-mole in a complicated web of dependencies. The hideous spiders from Alpha Centauri pointed their legs at me and laughed, taunting me with those nightmarish hateful eyes.

I considered splitting files apart to isolate just the functions I needed. The world slanted beneath my feet. I started to slide into the abyss. My arms and legs thrashed about, struggling to find something to hold onto, until finally, I was falling into the void of space.

The poll

I woke up in a puddle of sweat, my hair disheveled, and my hand still clutching the mouse.

I decided to post a poll in Slack to gather some objective metrics on the slowness of the repo. I wrote instructions on how to run a single test file, measure the load time in milliseconds, and look up the operating system version and processor.

I posted my results to encourage others to do the same. It took me 45 seconds to run a single file of unit tests.

The first few results were slow and seemed to confirm my suspicion. My excitement grew as it felt more certain that I was bringing visibility to a real problem that needed to be addressed.

Then another co-worker posted his results: ten seconds, but with a much newer laptop with a faster processor.

"How did you get the newer laptop?" I asked him.

"Just requested an upgrade from IT," he responded.

The decision

A snowy-haired older gentleman walked into my home office.

"What are you doing here? Who are you?" I asked, startled by the security breach.

He took a step forward and said with a smile, "Two roads diverged in yellow wood—"

"What makes you think it's ok to show up in a stranger's home? Leave or I'm calling the cops," I pointed toward the door.

The snowy-haired man hung his head low and walked out.

I thought about the "two roads diverged in yellow wood": Request a laptop upgrade or dive back down the rabbit hole to undo several years of accumulated sin in the repository.

I chose a new laptop.

And that has made all the difference.