Wednesday, February 25, 2015

Talking Topics

So we had a Lean Coffee about 4 or 5 months ago, where everyone wrote down what they would like to hear at our local Testing meeting group. BoiseOnTest

Over that time, JCD and I have had those cards on my desk, and randomly gone through them and tried to find people to talk about the topics rated the highest (don't comprehend, see

I've grown tired of these sitting on my desk, and I think other people might benefit from having a list of topics to talk about for test meetings, blogs or use them as you see fit.

I have purposely NOT provided context or organization for these, as I think that stimulates more creativity.

Future of Testing / Ways to Evolve
Bridging the gap between manual and automation testing
Automation: What it can and shouldn't do
Mythbusting as part of testing
Ambition, Drive and Contribution
Change (Company, Position, Attitude)
Where do experienced tested provide value (vs monkeys)
Test Design
Test Strategies for continuous integration / deployment
Risk Management
How to find great testers
Test automation classification (unit, acceptance, BVT, smoke, etc)
Training resources for testing (recommended courses, books, blogs, etc)
Effecting change
Job satisfaction
SDET or Not
Advancement / Promotion in a Test organization
Time management
Speaking / Writing
Social Skills
What's hidden in the remaining 10%
Test technology (tools, trends, etc)
Web Security Testing
Testing in different states, conditions and combinations
Gathering useful information (logs, databases, etc)
Have speakers give talks on what they do on a day-to-day basis
Experience reports
Strategies to influence development teams to test more
Data Driven Testing
Session Based Test Management
Testing Approach / Strategies
Waltzing with Bears - Communication up (specifically talking with authority, management, C levels)
Load testing techniques
The Scientific Method
Risk Analysis
Logic and Rational Thought
Equivalence Classing
Deciding what to test when you're "Testing Everything"
Grey box vs White box testing

And now my desk is slightly cleaner.

Wednesday, February 11, 2015

The World Wants to Replace You With Something Better

In a world where technology is constantly getting better, with fewer jobs and more automation (speaking generally, not 'test automation'), the world appears threatening.  However, the world has always been this way, just less extreme.  Politicians have always looked for better funding, new ways of communicating and ways to look better.  They slowly edged each other out, but in doing so changed the world, each forging more and more ways of getting funding/communication channels/etc.  The world doesn't want you, it wants the next guy's better idea.  I speak of politicians because it is an easy way to see how people 'vote' for what they like (not what they need or should get, but that could be a whole blog post of its own).  Darwin speaks of this in terms of evolution.  Markets speak of this in economic terms using money as a voting mechanism.  One general term that is used is progress.

The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man. ― George Bernard Shaw

Automation of all stripes appears inevitable.  Will all jobs go away?  I doubt that, at least not in my life time, but what will the world look like?  Have you read and considered The New Luddite Challenge?  Like Bill Joy, I have and I still don't have a lot of answers.  I want to automate things because I hate wasting time doing the same boring things over and over again, but I don't think all work is a waste per-se.  Some people really like cooking, and so they are not wasting their time cooking, even if they cook similar dishes over and over again. On the other hand, just surveying some of my co-workers, I am sure cooking is not a pleasure for everyone. Even worse, waste in cooking is not just time, but also in burnt meals and trashed leftovers no one wanted.   In fact, in the last 50 or so years, we have started seeing food automated be it via fast food or food in a freezer.  While we are not forced to have those sorts of meals, automation has left many of us without particularly strong cooking skills.  The world has replaced what might have been superior meals for meals that require little-to-no washing up or cooking at all.

If you look at recent history, we had different people doing whatever it was that they enjoyed.  They would then hire other people to do the things they didn't want to do.  For example, I test software, but I don't much care to do my own garage repair, so I hire someone to do that.  If we automate some jobs or even just sub-tasks in jobs away, what do we do with those who want to still do the job that was then automated?  One of the easy examples is what happens to truckers when we automate driving?  Obviously children that don't know the 'joys of truck driving' won't dream of being truckers, so maybe this is a self-correcting problem, but I don't like saying 'wait until all those old school folks die off'.  I am not here to give answers to this topic, certainly not for such a complex set of questions, but I think if you write automation, you should seriously explore the effects and ethics of the profession and how it changes the world, before you create a world you don't like.

In fact, I would go so far as to say you should frequently examine what you are doing to see if it is what you want to see done.  That is to say, if you work for a company that gets cat pictures to people, is that something you care about?  Do you feel that another cat picture platform is what matters?  Perhaps you'd rather hack together a bunch of low-value apps and make a few hundred per app.

A Real World Example

Ultimately, our automation of simple tasks might eventually cost society some of our jobs.  Yes, test-related jobs can be lost by automation.  To get the regression I have produced with automation would require more people than we could hire, but to even reach something near that level, my company would have to hire 3-5 people with low levels of knowledge ("Beginners").  My wage might meet these theoretical low-level beginners, but they would be simple button pushers.  While there are pros and cons to automation (E.G. Testing vs Checking), I think the coverage could be argued as less for the human side, but to be fair, given enough training, the quality of the testing might go up.

Let us speculate in my circumstances.  There are 50+/- fields that have to be filled out, with different types of allowed values based upon different modes across 12 different version of the product.  These are all API-based, and with very simple text-feedback.  It took me about 6 to 12 months to automate to the level I expect these low-level testers would work at, and maybe 3 of those months were around discovery of how the system worked.  It was all regression work with only 2-3 new features over that time, all of which were covered by a different tester.  In my speculation, I would imagine these beginner testers would only do the regression, not the new features.

So let us consider what would happen if we hired those testers.  The testers would work for ~6 hours a day, and would be typing in input and checking the output.  It would increase headcount, and they might need another manager or at least a lead.  The other pieces of automation wouldn't have been able to leverage the work and we would have had to hire even more people.  Our organization would have ballooned into a factory school testing organization.  It would have been ineffective, but would have had more human-coverage.  I have worked in such an organization 10 or so years ago, we had ~100-150 people banging away at 20+k worth of written tests.

I believe automation in this case was the right choice.  I think it adds serious benefits to the organization and those benefits are long lasting.  I think it is best we never hired those fictional testers.  Yes, I recognize that I have created a false-dichotomy in that there could have been other possible choices in the hiring process, such as 2 mid-level exploratory testers.  I still think automation was a better choice.  Perhaps not testing this system could have been an option, but I think automation was a good idea... or using unit tests... or ....  Perhaps there was a better option, but they got me and my automation.

The Future Is Now

As I have documented before, many organizations are moving towards a No Tester view of the world.  In a very real possibility, some software may not need to be tested in the traditional manner.  The future may not need you. In some ways, I think that is reassuring, but for others it is indeed sad.  Charles de Gaulle popularized the quote, "The graveyards are full of indispensable men."  My efforts, as valuable as they are, are only temporary until the next innovation, but also my mistakes are only temporary until they are patched.

Everyone must leave something behind when he dies ...  It doesn't matter what you do, so long as you change something from the way it was before you touched it into something that's like you after you take your hands away. ― Ray Bradbury, Fahrenheit 451

As we keep moving forward, change will continue to come.  The lesson in this is not to give up.  It is not that you shouldn't go for a testing job.  It is not that you shouldn't bother coding.  Rather, go learn a new skill and tomorrow learn another.  The future wants a better world, and while none of us know exactly what that will ultimately mean, it certainly does not mean you should stagnate and quit learning.  Go learn.  Then find hard problems and work on them.

Tuesday, February 3, 2015

Resumes, Lying and Embellishment

I was reading how Raji Bhamidipati had noted how many people with little experience were using fake experience on their CVs (I will use the term resume).  They had several questions on why people choose to be unethical and fraudulent on their resume.  Furthermore, she states that many of these people end up being hired using their fake resume.  These hollow resumes, were not caught, but leave the person in question of being caught for the rest of their time at their job.  This entire article is speculative based upon my experience and research, but should not be taken as a sure thing.

History and Brains

I won't pretend this is a new issue, or something that 'our generation' started, but I suspect it has gotten worse over time.  First of all, economics and social history is a part of this.  Speaking specifically of the United States, after the 1950s, most of Europe was in a economic ditch due to the destruction from World War 2.  People could easily get jobs and brought up the 'baby boomer' generation.  The US invested a lot into tech, and made a large number of advances.  Without digging deep into the history, the boomer generation slowly amassed wealth and was able to assist their children to go to college and unlike any other time in history, the US population mostly has college education.  Unlike many areas in the world, in the United States it is often the case that you take out what is effectively a long-term loan for education.  While this is overly broad, many people excepted quick success or ignore the bill until after they graduated. The US slowly moved away from manufacturing jobs, leaving us with many 'service' jobs, like changing oil and child care.  One of the few other sectors with significant growth was tech.  Also part of the cultural zeitgeist was that the boomer generation encouraged a belief in their children of being special.  With large debts from school, few places to work (that pay well) and unreasonably high expectations in life, much of the younger generation has been forced to compete in an environment they were not prepared for.  Add in pressure from social networks where your successful friends are highly visible, and lying to get a job might seem like a way out and up.

There is another real set of reasons involving the brain.  The first one is, many of those who lie on their resume are young.  They have little life experience and don't have a sense of the possible consequences.  They also don't have the brain to handle the risks.  Literally, those who are young enough don't have as many connections to the executive part of the brain and make dumber choices.  It is also why crime trends towards youth as well.  Notice how the drop off age is about 25.  About the same time the brain develops.

Another factor involves the Dunning-Kruger effect, people don't know when they aren't very good and when they are very good they think less of themselves.  Those who don't know there lack of skill think they are amazing, so they tend to inflate their resume with less truthful statements.  Worse yet, since they are lying to themselves, they are in some ways more capable of lying to others.  Those who are more knowledgeable tend to know how little they really know.  They therefore seem less capable, which helps bolster the case of the less skilled.

Employers Worst Enemy: Themselves

The final factor is the automated checking of resumes that looks for keywords in a resume.  Sometimes playing alphabet soup can help you get a job.  If you have little to no experience, alphabet soup is about the best bet you can make.  If you can back up the claims even a little it might make sense to do so.  Sometimes this is referred to as 'Embellishment'.  However, that can become a slippery slope.  In what might have been the worst interview I ever observed, in looking to find an upside, my coworker choose a technology the candidate had listed and asked him to talk about it.  The resume had categories, like 'familiar' and 'expert', with this technology being marked as expert.  I believe it was Tomcat servers, something I think most of us were not expert in and was not used in that company, so we were not evaluating his actual knowledge.  His first comment was that he only used it in a testing context, and that was how he was an expert.  So we asked how he'd find a bug in said system, or how he'd get updates deployed.  The best answer we eventually got after digging into his background was other people did everything for him.

But what did we do?  If he was smart, we taught him perhaps how to answer that form of question better without ever learning any of the tech he claimed.  In essence, even though we didn't hire him, he could learn the wrong things from that interview.  Worse yet, while he didn't get our job, we had no way of notifying other employers of his behavior.  We couldn't say how we caught him in several lies because his former employers included employers we had, so we knew some of the work was not his.

From an employers point of view, when they are first hiring, it is not unlikely they don't know what a good tester looks like.  They do some web research and come up with a list of properties that sound reasonable to them and come up with a job description.  If you follow the CDT view where things depend on context, it will be hard for an employer to ask smart questions.  Worse yet, ISTQB is about the only thing an employer might be able to use to filter with.  Ignoring my personal opinion on the value of such a cert, since an employer probably doesn't know much about testing or  ISTQB, they are not likely to go doing the validation of such a certificate.  If someone feels ISTQB is of little value, they might justify in their head, "My employer won't know the difference."

The employer is more likely to offer on the low end of the pay scale, even if they want experienced people.  Sometimes this is because they can't afford anyone more advanced, else they would already have a test team.  Other times it is because they are small and thus don't have enough cash to afford senior people.  Even if they have a test team, not everyone treats this as more than a job, and might not take their interviewing seriously.  This allows beginners in, as well as those whom lie on their resume.  In my experience, often these sorts of companies also have death-march-like development processes because the company has not matured.  This makes it so that many newer recruits end up leaving.  Another area where this happens is in contracting.  When you need 30 testers in seats and you are contracted to have them, or your pay is tied to the number of people, you have a conflict of interest between hiring just anyone and keeping the customer happy.  You might hire a few 'iffy' people because you can hide them in the middle of some better employees.

 What Do We Do?

The truth of the matter is I don't know that we can as a society eliminate this sort of low-level corruption.  I worry as automation gets better and less high paying jobs come available what sort of world we will live in.  One likely result is with increased competition we will see an upswing in this sort of falsification.

In trying to evaluate my own predictions, I tried to find any data around trending, to see if this problem is in fact getting worse.  I couldn't find anything substantial, but a variety of different percentages were thrown around and nothing that felt authoritative.  The one thing I did notice was the numbers were typically large, like 40% of resumes have some form of fraud in them.  Okay, you caught me, it was 39% with other numbers on that page lower.  Oh, and that was percentage of organizations, not resumes.  Darn, you're reading up on my data now?  I just wanted to make the number seem more round and dramatic.  What's wrong with that?  The point is, we all have to be careful with our language and clear on what we expect.  I imagine you expect that I try to get my facts straight and while some rounding might be implied, that the unit of measure would remain consistent.  For example, culturally, Latin Americas commonly include pictures with their resumes, while in the United States, that is unusual.  So if you send a resume to Latin America, you better include a picture, as that is the norm.  I know no one of us can set such expectations, but you can set them for yourself and ask about unclear parts of resumes that don't seem to follow your standard.  We have to make honesty the norm by being honest ourselves.

Obviously if you are having people ask you about advice regarding lying on your resume, the answer should be no.  I have given recommendations for people, and I have had one friend list work he did for me in his resume when I was about 16-18 years old.  He actually did do some testing, but it was not for a formal company.  I said he could because he did the work, formal organization or not.  Some people might see that sort of buddy system as flawed because you can use contacts to get jobs, but those without contacts are disadvantaged.  I have even heard claims that this can be accidentally racist/sexist, based upon who you know and trust.  The fact of the matter is, that hiring people you know and trust has always been a thing.  The reality is that hiring based upon personal knowledge of someone else's previous work is one of the best filtering mechanisms.  Hiring based in part upon the recommendation of someone you know is probably helpful.  Dr. Kaner suggested a entire packet of stuff be used to track someone's career, but that doesn't help much for those starting off.  Perhaps a broader network, like that of Dr. Kaner or LinkedIn is also of value, but that only helps vet people you know.  I recognize that some people may disagree with the idea of hiring people you have formerly worked with, and I welcome discussion around that.  I think it is better to have an open discussion than for me to stand on my soap box and expect you to just accept what I say.

There are two things we can do today for every candidate.  The one easy thing we can do is be careful with our hiring.  Get to know the candidate, ask lots of questions, and in the end, if you aren't sure, don't hire them.  I know that can hurt candidates, but I think it is important to choose the right person rather than getting the wrong person but moving faster.  The second piece is to investigate the claims of a candidate.  While I think it is better to do so in an interview, even after an interview, if a candidate makes a claim, it isn't too late to go look into it.  Better to spend ten minutes now than waste 6 months with them and another 3 months trying to get rid of them.  If that means calling the candidate to ask them a question, get on the phone and call.

While most of this has been about the interviewer insuring they are not being lied to, the candidate too should be concerned about the company.  From a candidate side, ask detailed questions, and dig into the details.  What is your bug count?  How much does it vary?  What flaws are there at the company?  What strengths?  And again, don't assume the interviewer isn't embellishing in their answers.  I don't think any interviewer would say "It's hell here.  Go somewhere else."  So investigate the company and the workplace environment.

Ultimately, you have to work with either the candidate or the people interviewing you for 8 hours a day, for a long time, so choose wisely.