Thursday, September 5, 2013

Testing the hiring process for testers...

 tldr: Hiring is hard.

  Having now interviewed hundreds of people for testing positions. I can say, "hire the people you, or someone you trust, knows." Learn to adjust who you trust based on interviews you do, and the people they recommend. Occasionally you'll get a diamond in the rough and you'll interview someone who you walk out saying "please stay here, I'll be right back" and off you run to get an offer letter made quickly. More often than not you will walk out saying "another 30 minutes lost." It's all the people in between that are the hardest pieces to sort, people that are pluses on some things but minuses on others. Which pieces are more important? Can someone be a plus here and does that offset the minus over there enough to hire this person? If you're on the fence about a hire, say no, trust your instincts. I think this one piece has saved me more then anything else.

A good testing person is hard to identify.
   I've always been a 2-stage interview person. During the first interview ask some personality questions, some company culture questions. See if they are even going to fit within your company / team. Then I dive into tech questions until I get 2-3 "I don't know" answers, this is where I end most first interviews. Second interview I open with the 2-3 questions they missed, if they didn't research them or have a decent answer the second time that's a major strike against them.
  I've also tried the more overt tactic of telling an interviewee who I thought was good enough for a second, "Here are two topics, I would like you to research one and we'll talk at your second interview next week." The first person I tried this on blew my mind with what she was capable of learning (over a weekend due to a miscommunication with HR). She learned both topics and to a depth that I'll admit I wasn't able to verify if she was correct. We hired her immediately, which is what might of set me up for a serious failure of the next person to get this same test. The second person came back the next week and gave me nothing. They didn't even have excuses for why they didn't learn anything. I was more then disappointed, I was dumbstruck. Here I had given this individual the perfect chance to shine and they had slapped me in the face. I stopped performing this overt tactic as it caused me too much stress to think that people like that existed.
  Over time the questions have evolved from technology to one of personality, intelligence, self-motivation and open ended questions. I have written across the top of my interview sheet "Ask the question, then shut-up Isaac". I've had so many people impress me with their answers to simple questions, and just as many burn themselves when I just listen. Questions like: 
  "Tell me about the most memorable bug you've found."
  "Describe a difficult co-worker and how you handled it."
  "What causes bugs?"
  "How have you improved your testing skills in the last 3-6 months?"
  "How do you prefer to communicate to developers / Product Owners / management?"
  "Describe a tool you use, and how you would improve that tool." 
  Now none of these questions are better than others, they all fit different people and different situations. sometimes I have to ask a variety because someone is clearly not interested in one of two of them.

Hire for the position:
  If you are hiring for a senior level position the person who is going to fill that better be able to be shown a problem, left alone for 1-3 months and when you come back *poof* results. If you are hiring for a junior level position remember YOU are the one responsible for showing this person the ropes, and  responsibilities, through a mentor; and that's you or someone you assign to do that. 
  Don't forget individual team dynamics. Some teams are uber-quiet get-work-done types, bringing in a loud obnoxious likes-to-talk person probably isn't the best idea.

Test the process:
   Keep the resumes of everyone you hire, along with your notes on what they said when you interviewed them. That way, as time progresses and you find out who the really great testers are, you can look back and see what they answered for questions you asked. Same for people who turned out poorly, you can look for patterns in how they answered. If your people really, really trust you re-interview them with new questions and get a feel for how your rock-stars answer these types of questions. Do this with caution as most people will freak out if their boss asked them to re-interview.

A team of interviewers is required:
  While I generally perform the first interview, no one gets through the gate without at least a second set of eyes. I try and get at least three other sets of eyes on a candidate, depending on position, seniority, skill set and any other number of attributes. 
  This was sunk into me on the first time I passed someone through a first interview, just to have a squad of people I trusted walk out of the second interview, and none of them would look me in the eye. I knew immediately that they wanted to know, "How the hell did this guy get this far?." Truth be told though, I still think the guy was a decent candidate. However, with three current employees I trusted saying "No", I couldn't ignore them because I asked for their opinion.
  With a team of interviewers it's also handy to have a code phrase, that instantly ends an interview. When the phrase is said, you allow the candidate to ask some questions so they feel comfortable then end the interview. That way no one wastes time on someone that is a hell no. This practice is mainly for those who are uncomfortable saying, "Look, clearly you don't fit, thanks for coming in, goodbye".

Know when to break the rules:
  Yes, I've ignored my own rules as I've laid them out, but I've hired over 50 people now. I've learned when a position I'm filling can take a certain amount of risk on a new tester that is going to require lots of training. But that was always explained to the candidate while making them an offer. Yes, some of them got offended cause they had "5 years experience" which amounted to nothing for me. Not because 5 years is nothing, but clearly their 5 years was only not spent on what I needed them to know.

  Apologies to all those people who have endured my interviews. Mainly to those who I have hired and then used as guinea pigs to find the next level of great testers.

1 comment:

  1. Although I do not currently work for Isaac, I was hired by Isaac for my first position as a QA Engineer. I am currently in a position in which I have hired several people and have participated in interviews for even more.

    I agree with most of what Isaac has outlined above, particularly the importance of personality. A person can be a fantastic tester but if people don't like to work with them, they will fail. QA needs to be able to communicate effectively with others on their team to be successful and this means they need to fit the culture and not be unpleasant to work with.

    The two things that I think are important to look for at interviews that I don’t think Isaac emphasized enough are:

    1) Thought process: How testers think is hugely important. I like to ask interview questions to try to figure out how well then can break apart problems and see what is important. I love the "how would you test a ....." questions for this. Asking further defining questions as they go can really help see how people look at a system made up of interconnected pieces.

    2) Motivation: I am not a baby sitter. I don’t want to have to babysit my people or spoon feed work to them. I want to hire people that I feel are motivated enough to keep busy and motivated enough to figure out what things they should be doing to keep busy. In other words, they should know what the priorities are for testing and be working on those items. Motivation is tough to interview for. It might be possible to test for it by asking a follow-up question at multiple interviews as stated by Isaac. I have also found a series of situational questions helps. Looking for signs of eagerness or “hunger” at interviews is always tough. People can fake it. There is a certain amount of gut feel and people reading that needs to happen with this one.

    As Isaac stated, the best way to get better at interviewing and hiring is to do it, take notes. See where you went wrong and learn from mistakes. I feel that I have improved but I still have a long ways to go.

    Happy Testing!

    ReplyDelete