Back to the blog

I Can Haz Init Script

Vincent Viallet Vincent Viallet on

Something went awfully wrong, and a rogue process is eating up all of the resources on one of your servers. You have no other choice but to restart it. No big deal, really; this is the age of disposable infrastructure after all. Except when it comes back up, everything starts going awry. Half the stuff supposed to be running is down and it's screwing with the rest of your setup.

INIT SCRIPTS, Y U NO LIKE?

You don't get to think about them very often, but init scripts are a key piece of a sound, scalable strategy for your infrastructure. It's a mandatory best practice. Period. And there are quite a few things in the way of getting them to work properly at scale in production environments. It's a tough world out there.

What we're dealing with...

Packages

Often enough, you're gonna end up installing a service using the package manager of your distro: yum, apt-get, you name it. These packages usually come with an init script that should get you started.

Sadly, as your architecture grows in complexity, you'll probably run into some walls. Wanna have multiple memcache buckets, or several instances of redis running on the same box? You're out of luck buddy. Time to hack your way through:

Honestly: PITA.

Built from source

First things first: you shouldn't be building from source (unless you really, really need to).

Now if you do, you'll have to be thorough: there may be samples of init scripts in there, but you'll have to dig them out. /contrib, /addons, ... it's never in the same place.

And that makes things "fun" when you're trying to unscrew things on a box:

Great.

Why existing solutions suck

Well, based on what we've just seen, you really only have two options:

  1. DIY; but if you're good at what you do, you're probably also lazy. You may do it the first couple times, but that's not gonna scale, especially when dealing with the various flavors of init daemons (upstart, systemd...),
  2. Use that thing called "the Internet"; you read through forum pages, issue queues, gists and if you're lucky you'll find a perfect one (or more likely 10 sucky ones). Kudos to all those of whom shared their work, but you'll probably be back to option 1.

We can do better than this

You'll find a gazillion websites for pictures of kittens, but as far as I know, there is no authoritative source for init scripts. That's just not right: we have to fix it. A few things I'm aiming for:

I've just created a repo where I'll be dumping various init scripts that will hopefully be helpful to others. I'd love to get suggestions or help.

And by the way, things are not much better with applications, though we're trying our best to improve things there too with things like pm2 (fresh and shinny, more about it in a later post).