How many times has this happened to you: You get some code working locally, have an awesome test suite that covers all the possible code paths, you deploy and tell your client/boss, and they come back with “I got an error right away, don’t you test this stuff?!?”

It is crushing to hear, and frustrating because no matter how great your test suite was, something went wrong and you look like a fool.

“Don’t you test this stuff?!?”

I hear this complaint from our clients every so often, and I was guilty of causing it myself[1] this week. It is so frustrating, especially knowing how well our team tests. Extra disheartening is when you have tested the app in the browser locally too. You know it works, but something isn’t right on your server, and you didn’t see it coming.

“We’ll do it live!”[2]

Testing on the server may seem counter-intuitive, but save yourself getting egg on your face by clicking through your new feature. Adding a new workflow? Click through the main sections. Adding payments? Test against your test gateway (or even live if you have to) to make sure your settings work.

After weeks/months of stable deployments, this is a step that can be too often left out, but it is indeed worth it. Especially anything that hits an external service or changes your data. I’ve seen servers error because of a forgotten Rails migration or a firewall issue. The simple stuff is what gets you.

Take the Stage

Production servers can be bad to test against, and I actually don’t recommend it. I said test on the server not the production server. Setup a staging server the exact same way you set up your production environment[3]. That way, you’ll reduce or eliminate the difference. You can also test against fake data without having to have fake data in production.

If you’re using external services, use a test login or another account, but have them work the same way. For example, if you’re posting to Twitter, actually post to Twitter, but have it be to a different (private?) account. Extract the differences (username, password, url) and set them separately for each environment, but have the code work the same[4].

Suck it up.

I know you don’t like to test on the server, and even I forget on occasion. Do it anyway. You’ll spend a little more time, but you’ll look much more like the expert you are.

[1] In my case, an environment variable wasn’t getting pulled into the asset pipeline on Heroku, even though the variable was set. See user-env-compile if this happens to you.

[2] http://knowyourmeme.com/memes/bill-oreilly-rant

[3] I recommend using Chef or similar for consistency. On Heroku, just add another app for staging.

[4] Again, Chef can help here. On Heroku, I use environment variables, but you have to remember to set them!