Member-only story
How to Survive Your First On-Call Rotation as a Software Engineer
I had been a software engineer at my last company for about a month when my manager let me know I’d be going on-call during the next rotation. “Cool” I replied. Not cool.
I installed PagerDuty, a popular app that is as necessary as it is hated, to alert me when any critical ticket was created. I had casually been keeping tabs on the kinds of bugs that were deemed critical to see what I could expect during my shift. Basically, anything that prevented our customers from normal usage of any one of our many applications would trigger an alert. Button to export a CSV is broken? Critical ticket. API returning 500’s on a Saturday at 2 a.m.? Extra critical ticket.
Now, we had a comprehensive test suite, incredibly high test coverage and a kick ass QA team but I had yet to see a week go by without at least two critical escalations. At my previous companies, critical tickets were few and far between, and as such, there wasn’t much formality around how to handle these show-stopping bugs. At one company, the rotation would change every day and on the weekends, if anything blew up, we would play the game of who is going to acknowledge this first. Not it.
Luckily, my new company had some decent protocols in place for handling on-call rotation, though this didn’t lessen my stress. If you are a new software engineer and nearing or currently handling your first on-call, you’re likely feeling that same anxiety. Don’t worry… too much.
You Won’t Know How to Fix Everything
And you shouldn’t. If the product you’re working on has any sort of complexity, there is a strong likelihood that you won’t know how to fix a bug in a part of the system you’ve never seen. Only Bob, the super-senior alpha-geek knows how that exact program works. He’s out on vacation of course.
Hopefully, your team is aware that you are a human and thus fallible, and will reach out when they see you dealing with a bug that is particularly difficult to find and squash. If they don’t, you need to raise your hand, no matter how uncomfortable and ask for help. Pairing with another developer can improve your knowledge of the codebase and perhaps theirs as well. The end goal is that the bug is resolved in a timely manner…
