What makes a good software QA person?

We have a few quirks, a hint of obsessive compulsive disorder, and we’re brutally honest. We never lie to our developers. We don’t sugar-coat, soft-shoe, back-peddle, exaggerate, or outright lie. We can’t. Our job requires us to be utterly truthful. We’ll tell it to you straight, we’re paid to be honest.

We think this stuff is funny. And it is! You have to appreciate the bug found in the wild, the "disastrous misconfiguration" error message on Chrome when it finds a "weak ephemeral Diffie-Hellman public key" (it has to do with TLS encryption), or the mall TV screen showing a Blue Screen Of Death. It’s free entertainment, but it also makes you think: what got missed that this bug escaped into the real world? How do I find a bug like that? How would I reproduce this bug? We know other people don’t always appreciate the humour in a good bug find, but a QA always will.

We’re in the details. There’s a slight correlation between being a good Dot Net developer and having blue eyes, based on two companies I’ve worked at. I notice colours, including eye colours, and I’m always looking for patterns. I spot when someone has matched their glasses, fingernails, shoes, toenails, and purse. If a line on a user interface is bumped down by a pixel partway along, I’ll see it. The fact that there are different numbers of steps between floors in my office building is a source of amusement, and some floors are a prime number of steps apart. When you use the back staircase, you get different numbers of stairs between floors. In the main stairwell, the door out isn’t on a consistent side as you go up the stairs, sometimes it’s on the south side, sometimes on the west. We live for details and patterns, sequences and clean lines.

We need more input, more tools, more tricks, more ways to play. More languages, more front-end and more back-end knowledge, and the stuff in between. Once we know about injection attacks, we’ll test that everywhere we can. We’re the reason you have client-side and server-side validation. "Why" and "how" are the most used words we know, followed by "what if". We’re not trying to take your job, we want to know enough that we can see the weak spots in the armour, right where you think we could never shove a buffer overrun but we do it anyway. We have an insatiable need to learn and we’re not afraid to be the least-informed person in the room while we pick up something new.

We ask a lot of questions. Some of them we already know the answer to, we’re trying to show you something in a gentle and non-confrontational way. Some of them we have no idea what the answer is, and it may be an obvious or dumb question to a developer, but we need to know. Asking questions is one way to find out. Breaking stuff is another, and scenario questions ("what if the user does X or Y") are a third. We have an idea of how you’re feeling from how you talk and type and move, we pay attention to that.

Résumé do’s and don’ts

(Written for a friend who was teaching a class of young people.)

Do be selective. You don’t have to list every job you’ve ever had. Highlight the relevant ones, and highlight specific things you learned or did at those jobs.

The purpose of a résumé is to get you an interview, not tell me your life story.

Never send a ten page résumé, or a seven page one, or even a five page one. If you can’t prioritise what to put on a résumé, you’re showing that you don’t have that skill, which is really useful in many jobs.

No-one should need any more than three pages at the very longest for someone with decades of experience. Don’t make your résumé a wall of text, give me enough to get my attention and move on. Hit the highlights and the most relevant experience.

If you’re applying to be an X, make sure your résumé mentions being an X, or training as an X, or research into being an X. Never apply for a job as an X and send a résumé telling me you’re a Y.

Read the résumé out loud before you send it. When you read it aloud, odd sentence structure or awkward wording is a lot more obvious. Have someone else read it to you.

Don’t refer to yourself in the third person, it’s weird ("Mary is a creative and visionary professional"). It looks weird, it sounds weird, you wouldn’t speak like that.</p

Never put it on your résumé if you’re not prepared to talk about it in detail. I leave JavaScript off mine, because I can hack my way through with Google and Stack Overflow, but I’m no expert and I don’t want to get grilled on it. If it is on the résumé, it’s fair game for me asking you questions about it.

Don’t inflate your skills. If you say you’re an expert at X but you can’t answer basic questions about X, then that makes me wonder what else is untrue on your résumé. Don’t lie, exaggerate, or make stuff up.

There should not be ANY typos, spelling errors, or grammatical errors in your résumé. Print it out and red pen it. Then get someone else to red pen it.

Stick with basic fonts: Times, Georgia, Ariel, Helvetica. Stay away from Comic Sans and Papyrus. Don’t use clip art, or colours. If your résumé gets printed, it’ll almost always be on a black and white laser printer, colours make it pale and harder to read.

Make sure your contact details on the résumé are correct, it’s the only way to get hold of you. If you set up a separate email address for the résumé, check it multiple times a day.

Don’t belittle your past employers on your résumé. We know you want to leave, but bad-mouthing them on paper makes me wonder what you’ll say about my company later.

If you have gaps in your employment history, be prepared to explain why.

If you’re applying for an entry level job, research what the skills are for someone in that job, and showcase those in your résumé. Even if you don’t have the experience, it shows you did the research and know what you’re looking for.

How to have a meeting that doesn’t suck

Have a defined end-point for the meeting, and get there as soon as you can.

Take your own notes, by hand, on paper. You’ll notice what’s relevant to you and the act of writing helps you recall it.

Allow audience participation, it’s not a lecture or a class, it’s about collaboration. Someone else knows information that’s relevant and they can’t speak if you never leave space for other people’s words.

Count at least seven seconds of silence when you ask a question. People start squirming at that much dead air and someone will fill it. Some people can tolerate more silence than others, watch for them and ask them questions directly.

If your meeting room is a fridge, people are too busy being cold to pay attention to what you say, and they probably hate you for keeping on talking.

Multiple-hour info-dump meetings will be forgotten by morning. Multiple-hour meetings that are also cold will be remembered only as cold and too long.

Humans need regular bio breaks. Having to ask to be taken to the rest room as an adult, because you require a security badge to get back in and you’re not handing those out to visitors, is humiliating. Don’t be that company. If you can’t get around the badge issue, schedule a break every couple of hours and get people up and moving.

If it’s a meeting where people can phone in, make sure they can hear you over the phone. Repeat questions before answering them, and speak clearly and more loudly than usual.

For a video conference, have a dry run of the technology the day before to make sure everything is working OK. Do not make your meeting the first time you turn the video camera on.

If there’s any way possible you can get everyone in the same room, do that.

The Red Team Rules for Software QA

I found the Original 12 Red Team Rules recently (there’s profanity in the list), and a lot of them are relevant to life as a software quality advocate/analyst/engineer/assurance person with a bit of modification.

Rule 1: Know when the test will be over.
(Was: Always have an escape plan).

Rule 2: Be aware of the environment for your app, database, network, mobile devices, etc.
(Was: Be aware of your surroundings.) Different environments require different testing strategies and tools.

Rule 3: Assume nothing.
(Was: Assumption is the mother of all f***ups.) Get your acceptance criteria spelled out, and if there’s not code in place to stop you trying something, do it.

Rule 4: Vary your attack.
(Was: Always have a backup plan.) If you don’t find a bug the first time out, try something different. Someone that always tests the same way won’t find new kinds of defects.

Rule 5: Never get caught by surprise.
(Was: Never get caught.) Know your application, who will be using it, how they will use it, and its vulnerabilities.

Rule 6: Collaborate effectively.
(Was: Keep your mouth shut.) Quality is a collaborative process, but talk with respect, and don’t log a half-baked bug report that can’t be replicated.

Rule 7: KISS: Keep it simple, stupid.
This applies equally to QA and the Red Team. Write stories, acceptance criteria, bug reports, and regression test plans that a total stranger could follow. Because in six months time, that total stranger could be you.

Rule 8: Simple and light equals freedom, agility and mobility.
Don’t get bogged down in a document-heavy test process. Test Driven Development will help if you can read code (and QA people should be reading the code).

Rule 9: Do the job and then go home.
(Was: Plan, execute, and vanish.) Tired and cranky people do poor work.

Rule 10: Automate the boring bits.
(Was: You don’t have to like it – you just have to do it.) Sometimes we have to do repetitive stuff. Automate it if you can so you don’t have to do it again, but don’t duplicate automated tests.

Rule 11: Always invest in good quality stuff.
Get the right tool for the job and learn to use it well. JMeter is unfriendly but provides excellent information.

Rule 12: Trust your gut.
If you have a hunch there’s a bug in an area of the application, keep digging until you find it.

Candidates, things your interviewers want you to know

(This is close to my heart because I’ve done a bunch of interviews lately, some good, some not so good.)

If a skill or qualification is on your resume, I will ask you about it, especially if it’s one I also have. Claim to know C# and I’ll ask you C# questions. Claim to know Java and I have a set of Java questions ready to roll. If you say you’re an ISTQB certified tester but don’t know their core testing principles from the syllabus, that’s a problem.

Never lie to me. If you didn’t do your pre-interview essay questions yourself, it will be obvious when I talk to you. I’d rather you said "I don’t know how to do that, but here’s how I might try." Even if you’re on the wrong track, a valiant attempt is far better than a copied answer.

Never ever give me someone else’s answers. Seriously. I can’t believe I have to say this. Things you found on the internet, copied in wholesale, and tried to pass off as your own leave a bad taste in my mouth.

Have some questions to ask me. Because your questions to me are also part of how I assess you. If you ask about the vacation policy, required working hours, and working from home in the interview, those questions worry me.

Don’t have sections copied and pasted from elsewhere on your resume. Why bother telling me the same stuff twice or four times? Don’t waste space and don’t waste my time.

It’s a conversation, not an interrogation. Brainstorm with me, think out loud, if I draw on the whiteboard and hand you a marker, use it. Bad art skills are never a disqualifier for software quality jobs.

Have examples to common questions ready, because I’ll ask you for specifics.

Smile. Even if you’re nervous, fake it if you have to.

Don’t bad-mouth your past or current employers. Even if they’re horrible, find a way to say it that doesn’t involve saying awful things about them. We already know there’s some reason you want to work elsewhere.

You’re interviewing us too, and this job may not be a good fit for you. Better to find that out now than take the job and hate it.

Have a reason for why this job at this company. If you don’t want to be here, I find it odd that you’d go to the trouble of interviewing.

If it’s a phone interview, make sure you answer the phone when I call. If the line is engaged or I end up in voicemail, that doesn’t start the interview off well.

Getting both in to and out of the building are also important. I had one candidate call HR to say the building was locked and she couldn’t get in. It wasn’t locked. Same candidate also walked straight past the elevators and into the kitchen when I said goodbye, and just stood there looking confused for a bit. If you can’t figure out building navigation when you’re nervous, not a good sign.

Words to a new software QA

We’re not after the developers. We’re after their code. It’s not personal.

If they didn’t want us to find the bug, they shouldn’t have written it in the first place.

QA starts with the requirements, if those are vague, the result will not be what the customer wants.

Not finding a bug does not prove there are no bugs. It proves you didn’t find one right there, at that particular planetary alignment with that precise set of system and environment and time of day and test data.

There is always a bug somewhere.

Learn how to program in the language your developers use. Also learn SQL (Structured Query Language), HTML, CSS, and any other acronyms you hear often.

Your developers are an amazing resource and are usually very helpful when presented with a "Can you please help me…" request.

Keep learning. There is always a new language, or tool, or technique, or platform, or environment, or program.

Get up and walk around often. Drinking large amounts of water will help this. Give your eyes a rest from the screen and go talk to your team.

From QA to Dev

Dear programmers, a few notes from your QA team lead:

We don’t actually hate you.
Not by default, and not at first. It takes time and effort on your part to earn the ire of the QA team. But if you tangle with one of us, you tangle with all of us. We talk about you. We trade horror stories about that one bug that took weeks to fix because you stalled every step of the way. Yep, we remember that one well. You can redeem yourself, but it’s not easy.

We want the same thing.
We want the best possible product to go out of the door and our job is to make you look good, rock star good. Sometimes, you get in the way of that.

Sloppiness annoys the snot out of us.
Like that zip code field in the database that only accepts ten characters, but we can enter 255 in the app? That’s something you should have thought of before you checked in the code. Spell check is your friend when you’re doing UI stuff. Think before you type.

You and I think alike, but we’re different.
You build it, we break it. Good QA people can program in your language, so we understand what you’re talking about, and we mipght have a suggestion as to how it could be done differently, or something you missed. Try bouncing ideas off us once in a while. Also, we may have an idea of what you’re actually doing, so lay off the BS about how hard it is.

We are extremely picky because we’re paid to be picky.
It’s not personal, it’s just how we are. Details are important and we care about all of them. Even the annoying ones. We also like the pictures to hang straight on the walls, and shoelaces to be tied neatly.

We don’t expect you to be perfect.
If you were perfect, we’d be out of a job. We know you’re not perfect, and neither are we. What we’d like is for you to accept that you’re capable of making mistakes and missing details, and stop acting like we’re out to get you. Trust me, if we were out to get you, we’d be waiting in the parking lot with pitchforks (we keep them in the server room, just in case).

We’re not perfect either.
Guess what? We make mistakes too! It’s OK to tell us when you think we’re wrong, but please don’t be a jerk about it. It’s not a contest.

Just fix the darned bug already.
We don’t care about whose fault it is, or how hard it was to fix, or how Joe Coder did such sloppy work you have to refactor all his stuff to fix it (believe me, we already know all about Joe Coder). We just want the bug fixed so we can re-test it and get it out the door and we have no patience for whining.

Never, EVER break the build we’re working in.
Please make absolutely sure the app will both compile and run before AND after you check your stuff in. If you break the build we’re WORKING IN, we’re dead in the water till you fix it. We’ll come and sit in your cube and stare at you until it’s fixed and we can go back to work.

Test it before you send it to us.
If we can’t even get through the happy path, then dude, what were you thinking when you checked the code in? (See “Sloppiness annoys the snot out of us”)

If you don’t understand us, talk to us.
Sometimes we don’t communicate all that well. Some things just make more sense face to face. Come talk to us, we rarely bite programmers.

See you at the release party!

Love, your QA team.

(Yes, I really am a QA team lead.)