Showing posts with label Management. Show all posts
Showing posts with label Management. Show all posts

Wednesday, June 3, 2015

Happiness: On the Job and in Life

In this post, Matt Heusser does a great job describing some of the problems around hiring older workers and some of the reasons older workers become less valued.  Obviously, this is a concern for all technology workers in the field, regardless if you are in test, operations or development.  I have not had much experience with it, but I do feel some of the call that those who are in the 40s and 50s start to feel.  I don't want to invest hours upon hours doing development outside of work while working 40 hours a week.  This blog, which I write for often, requires time which I could use to get money from somewhere else.  I don't get paid to do this.  Sometimes I clarify my own thoughts, or use this to prompt me to learn, but more often than not I am actually trying to teach others the hopefully valuable information I have.

However, there is a underlying subject I think is more important than just ageism in the work place.  I want to talk about happiness.

First, a few words, I am not a doctor of any sort, I am not giving advice for depression and if you are experiencing depression you should see a doctor.  One other thing, I'm going to ask you to resist clicking all the exciting links until after you've finished reading this article, as there is a metric ton of data on the subject, but interrupting the narrative will likely distract you.  Now on with the show...

Were We Happy In The Past?


In the past, almost everyone did farming.  There were a few other jobs, like tailor, doctor and merchant, but most people were farmers.  These jobs were fairly well known and a town depended on one (hopefully) competent doctor, one tailor, etc.  A large town might have a few of each, but only in the cities were there really any large quantities of any specialties.  These specialties were all relied upon and considered vital.  While there was anger around the usage of power and prestige these ranks sometimes bestowed, in general they were good jobs.  Life could be brutish and short.  However, the idea of manifest destiny  and excitement of the renaissance show that life in the past could be seen as meaningful.  Is being a farmer happy making?  Or being a doctor or ...?  Sadly, there is little documentation I could find about happiness in the distant past.  However, I did find lots on various problems we see an experience today.  It seems that the general fighting we have today was going on in the past.

It is strange that there should be so little reading in the world, and so much writing. People in general do not willingly read, if they can have any thing else to amuse them. -  Richard Burke to Samuel Johnson, 1783.
The number of technical students has declined constantly... IF THE SAME TEACHER CAN INSTRUCT 500,000 OR MORE [students] SIMULTANEOUSLY? - Radio Electronics, 1956, http://www.americanradiohistory.com/Archive-Radio-Electronics/50s/1956/Radio-Electronics-1956-05.pdf

So while I have been unable to find anything quantitatively correlated to happiness in the distant past (and only a few data point in the recent past), it does seem that the same challenges facing us have been unchanged in the past few hundred years.  We are constantly looking for productivity changes while desiring to keep our hard gotten gains.  Furthermore, while I have yet to read this tome, Steven Pinker suggests that we were actually more violent in the past.  So if violence decreases happiness, which I have found lots of evidence for, it would appear there is little basis for the assumption we were happier in the past.  In fact, it appears arguable that collectively we might be over all happier now then in the past!

Do Jobs Affect Happiness?


The next question is, do we know if farmers of today are happier compared to the general population?
In a word, no.  Happiness and job are nearly unrelated, however there are a few key factors that matter.  One is that you have enough money to survive and not worry about where your next meal will come from.  Another is having meaningful work and yet another is having autonomy.  There are also some negative correlations with jobs.  For example, not getting enough sleep or too much mental stress (some physical stress is good for you).  Lacking creativity can be an indirect form of mental stress as well.  Often this comes in a form known as boredom.  Basically, I think I can sum it all up in a single sentence:

You want a job that pays a living wage with difficult but surmountable and clear goals in which you have enough control to get it done and in which you will get back useful unambiguous feedback and time to recharge.

Great, we're done, right?  Well, I want to explore some more topics around this and how I think the reason our job does not seem highly connected to happiness.  The first is that we don't know how to practice being happy, so why would ones choice of job have an affect?  That is to say, if happiness is fairly mysterious then choice of job will appear to be a random or not related to happiness.  So how do you practice happiness?  For that matter, how can we discuss happiness when there are so many varying view on how to create it.  Is it something that comes without you noticing it or is it something you can actively seek?  If significant increases in wealth doesn't change happiness but for a small and short period of time, what does?

Rather than going on with so many links, I'm going to stop citing so many sources.  The purpose of this essay is not to do research for you but rather to explore the subject and not just trends I have seen in the research.  Furthermore, I wish to explore how to have a happy life, not just being happy on the job.

How We "Need" Society for Happiness


Helping others can, in and of itself hurt those we hope to help, without regards to the permission they give. You might have heard the phrase "What doesn't kill you makes you stronger." The correlation to that would be, "What makes life easier makes you weaker."  Obviously things like chronic pain and long term mental stress eventually do damage to a human being, but in general these two ideas have some truth to them.  Consider the case of the blind, a blind person might appreciate being given an 'extra hand' but at the same time, that might also make them less able to do things on their own when that helping hand isn't around.  Married couples are known to have slightly lower IQ than the unmarried because of dependencies on each other.

In a sense, as John Donne said, no man is an island. That does not equal a learned dependency in my book. Babies have real dependencies. We are dependent on doctors during surgery. We depend on lots of things. However, learned dependencies are not things we are incapable of doing. Maybe we don't want to do them or maybe it would cost too much of our time, but a learned dependency is when you quit trying because you're not expected to try. I suppose you could say I quit trying to change my oil because Jiffy Lube says I'm not expected to try. However, I would argue that my dependency on oil changes is more a matter of time, energy and my personal interest. That being said, it is a fine line.

In regards to happiness, is happiness truly just an internal thing or does it have external dependencies?  Can you live and enjoy pure silence, or do we really need music?  It is an interesting question, but even if you depend on external things, they may not really require other humans.  Enjoying waterfalls or existing books for example require no (new) human work.

Learned dependency, be it having someone else get you around, using a calculator to do math or depending on others for happiness is a problem because now you need something else in order to do the things that you want. I don't know how to give an algorithmic way of judging when it is enabling and undermining independence and when it is a genuine hand up.  Perhaps it is even in part personal taste, as the hermit choose no or few dependencies while the city dweller depends on many people for city services.  The hermit might even see those city services as enabling the city dweller, as the city dweller needs the bus in order to go see the opera they desire to see.  That is always a struggle, and has been one for ages.

If our dependencies on others needs to be managed, it seems only reasonable to ask about the other way around.  How do I prevent asserting myself into other people's need for genuine challenge? The Bible famously asks the question:
Am I my brother's keeper? - Genesis 4:9
In that case it was a way of ignoring responsibility for the murder of his brother, but the question of responsibility and social value for watching out for others has been around for centuries.  I don't pretend I will answer the question today, but I think it is a balancing act.  Rarely do we know what we really want. Our simulators for happiness are poor at best. Dan Gilbert talked about that in his TED talk.  So if we have a hard time knowing what will make us happy, pretending we know what other people want is silly.  This can become a wicked problem when what you want to do is help others but worry about foisting your culture, will or opinions on others without giving them room to grow.  Teachers suffer this question all the time.

 Can Happiness be Treated as a Duty?


The Stoic's claimed virtue is sufficient for happiness and that virtue is created by a will that is aligned with nature.  Kant's Categorical Imperative, in a very rough and simplified form, linked duty to what you imagine a reasonable person would do.  So unless you imagine a reasonable person would choose to be unhappy most of the time, it would appear happiness might in fact be a duty.  There are other philosophical theories like Hedonism which say that you should look to maximize long term happiness.  That is to say, have the day to day discipline to make your life better in the long term.  All of these various ideas about optimizing life with consideration for happiness seem to have something of a pattern to them.  They all push us to grow.  Be it via virtue, by considering how a reasonable person would live or look at the long term and attempting to minimize pain.

So how do you generate growth?  Well one aspect is to not be in fear.  Fear often is the opposite of growth.  You need to feel safe enough that you can work.  Even when in fear, such as when in a castle surrounded by an army, you have enough pause to feel safe enough to gather your wits and think.  Safety by itself isn't enough.  While safety is important, you have to be safe to try experiments, pushing yourself beyond what you have done before or in new ways allows for the growth, which leads towards happiness.  The feeling of growth is in my view, at least one of the causes of happiness.  However, if one only attempts to grow in one area, it can become an addiction.  Growth also has to relate to goals, and in particular unending goals.  Goals you can meet can later feel unsatisfying.  For example,  when you think of buying something, you feel more satisfaction then after you have purchased the item.  Since having meetable goals is less likely to create happiness, how do you create a unmeetable goal?  You could say, "I will weigh 150 lbs in 3 months," but this is an unsatisfying goal.  If you make it, maintaining it is much more difficult.  Instead, I will go to the gym 3 days a week remains satisfying because there is no 'end point'.

In addition to safety, you also need time to develop happiness.  You need to have time to work on your goals, or else you stagnate into surviving life.  Time can mean exact and dedicated time, such as a day of the week or hour in the day, or it can mean just having open variable spots.  Time is perhaps one of the trickier things to give detail on because different people like doing things in different ways.  Some people like things well planned out while others like to be more sporadic.  I have not seen evidence demonstrating that one is superior to the other, but personally I rather have some vague planning rather than chaos or an exact fixed schedule.

While the goals don't change often, they should be allowed some variety.  If you always go to the gym on Monday, Wednesday and Friday, you may end up burnt out.  Instead, you probably should have some time allocated to 'breaking' the goal.  Going to the library once a month instead of your Wednesday workout.  You should feel the pull to get back into the routine which helps make the routine more satisfying.  If you don't feel excited to go back into the routine, it demonstrates something is wrong with your goal.

Finally, you need to be able to see that these goals are working.  Without being able to check up on your goals, without being able to study the results and see if they match expectations you get little satisfaction from 'achieving' them.  While weighing 150 lbs might be a very watchable (E.G. an easy metric), you don't need that to see if going to the gym is helping you or not.  Since using easy metrics tend to make goals that end, those are likely not the first type of metric choices to make.  Saving enough money to buy a house is an admirable goal, but learning the habit of saving should be the real goal.  So using 'enough money to buy a house' as the measurement means you're probably watching the wrong thing.

Now if you have been reading this far, you'll probably notice something interesting.  I'm going to break a general rule I have of not copying and pasting and repeat myself from above.  Basically, I think I can sum it all up in a single sentence:

You want a job that pays a living wage with difficult but surmountable and clear goals in which you have enough control to get it done and in which you will get back useful unambiguous feedback and time to recharge.

If you noticed what I wrote in the job section and in the life section, you can see the life section is very close to a repeat.  In fact I would use the job section as a sort of check sum from what was discussed in the broader life category.

The idea that you can treat happiness as a duty is not exactly correct.  Instead, just like with getting a job that creates happiness, you need to do similar things in life.  It maybe formulaic sounding, but it is more complicated in one's actual life than any author can generalize.  A duty implies it is required, but this is more of a method, backed by out current science and history.  Can other methods work?  Maybe.  Is what I have written 100% correct for your life.  Very likely not.  All situations vary.  But the general ideas seem to remain consistent throughout time.

Disagree with me?  Well, my goal is to engage my readers, so please leave a comment.  I'll be happy to reply.  Oh yes, and now feel free to go click all those links and enjoy all that awesome data.

Monday, April 20, 2015

Expectations...again...

This was written a long time ago...but finally edited to readability.

So I just finished the AST BBST Instructors course. In it, we started by introducing ourselves. I of course summarized my introduction from this blog cause I was being lazy, and it said what I thought I wanted to say.

In response to that I was asked:
"It sounds like you have the confidence and experience to be a manager, but I wonder if this might make it hard for you to teach? What do you think?" - Another Student

I responded with:
"Yes I find that my mental model has a hard time bonding with people who aren't also self-starters and/or newbies. However I've been specifically working on how do we 'as a testing community' get the first level of people into testing. I still have a problem dealing with people who just want to be spoon-feed information.
My experience is mainly in taking people with 3-5 years of experience and tuning (yes tuning) them into the next skill they need to be the best at their current position. However so far this has been limited to where I work, as I usually understand the context enough to say "if you could do this" it's the next skill you should learn.
So part of my goal is to be able to teach less experienced people. But I still struggle with if I should TRY to deal with non-self-starters…"


I’d like to take a more detailed look at my personal thoughts on the matter. Specifically around, “Does your own attitude and personality cause you difficulty in teaching people?” I’m going to try and detail an internal dialogue I’ve been having about this lately. I first tried to sum up some feelings in an older blog about Expectations.

This is what I mean when I say, "not-self starters". There have always been people at places I've worked that are comfortable doing just the basics, show up at work, do their job, go home. These are the people that when asked in an interview "What do you do to improve your skills", have an answer similar to "On the job training". These are the people that don't seem to want to take time outside of work to improve their skills (which to me translates to "I don't consider this job a career"). Some of these people seem bright, others are merely content at getting along and that's enough for them.

Then there are the self-starter people who will go out of their way to learn new things. These people ask you questions, and when you say, "tell me what you know about it so far", they blow your mind with the research they have done. Or alter your understanding of the subject with some new pieces that you haven't heard or seen of yet. Sometimes at a minimum they have clearly researched and/or understood the material, but don't change your level of knowledge.

I know mentors / leaders want to use terms like taught, educated or instructed. But when you have self-starters, that isn't the appropriate wording. The closest I've come is tuning them. When I chat with a self-starter cause they are asking "What should I learn next?" (and them asking is one of the key points). There are a couple of ways to encourage them. And generally that is all you can do. Point in a direction, say 'that way', and then get out of their way.
  • Have them learn a new skill. Sometimes these are easy for beginning self-starters. Don't know SQL, yep I have yet to meet any tester that doesn't benefit from knowing it. As the self-starters get more and more experience, this can get harder and harder to find for each individual. Currently I've had serious success with just allowing them the freedom to find new things. This is the real power of the self-starter, they aren't okay with sitting idly by and surfing reddit, they WANT to provide value, they WANT to solve problems.
  • Have them level up a skill they are already strong in. This helps those who have just finished something rigorous, demanding or seriously mentally intensive. (This can be work related like finishing a project or a mentally challenging class, or a non work related life changing event.) It allows them to learn something, but with the permission to not be as intensive with it. An example would be having someone learn the singleton pattern, instead of a whole new programming paradigm like LISP.
  • Have them up a skill they are weak in. I don't normally recommend this for very many people. I tend to follow the Good to Great idea, that weakness isn't inherently bad. But there are genuinely some weaknesses you need to either compensate for, or bring up to a minimum bar.


Sometimes even that isn't enough. I recall one situation where I was attempting to tune someone I classify as a full-blown self-starter. We were going over the OSI model, and I was attempting to explain how each layer could be an attack point for testing. Unfortunately we were not speaking the same language, after about 30 minutes, they were frustrated and on the verge of crying. I was frustrated and not understanding why they couldn't get the 'simple concepts'. We parted that day, and I don’t recall us ever trying to train together again. I took this as a personal failure…how could I not explain such simple concepts (which OSI is and is not) to someone who tries really hard when they self-educate and generally succeeds? It took me a long time to understand that I had a base of knowledge about computers that they had never had exposure to. Bad on me for assuming what they knew. The problem was that I know stuff they didn't. They also know stuff I don't. It takes time and effort to truly get to know people. I hadn't taken the time or maybe didn't have the perception to understand that they had no idea what I was talking about. They didn't have the trust to tell me 'What the hell are you talking about'.

S'long as you learn to temper what you tell people you want, with realistic acceptance of what people can really accomplish. It all comes down to do you trust your people to work hard, do they trust you to not hose them with unreasonable expectations. It takes some serious time to build that trust. It's why you see people follow a strong leader around to companies.

If you don't fall into the category of self-starter as I've laid out here. There is still hope, you can become one. Start today, motivation breeds motivation, find something interesting, and learn it. When you're done learning that, find another. Then repeat ad nauseum.

I've going to wrap up and caveat this entire article with a thinking exercise for you.
Can your expectations be too high? Is this a bad thing?
How else would we ever achieve something new?

Monday, April 6, 2015

Tainter's Composite

Introduction


First let me start by defining my terms.  Dr. Joseph Tainter, is an anthropologist who looked into the question of why societies collapse.  A composite is combining multiple things into a single image.  I believe using Tainer's mode of thinking, one can create a model or system including for software and organizations.  This model, using a lot of systems thinking includes the work of the Dreyfus model, The Gervais Principle and general economics, but the framework all rests on is from Tainter.  I will explain Tainer's ideas, however, if you don't know about the Dreyfus model or Gervais Principle, you will need to read up on those before continuing further.  In fact, without those you will be lost.  I realize it is a huge amount of reading just to read this post, but I promise you it is worth your while if you want to understand the inner workings of business culture.

Tainter asserts that societies become more complex as the societies needs for solutions to problems increase.  That is to say, in order to solve a problem, the society adds complexity.  For instance, when a tax loophole is discovered, society might create a rule around that loophole to end it.  This complexity means that someone must create the law, the people who are affected must learn of it, there must be a means of enforcing the law, someone has to interpret the law and when violated someone must enforce the penalty.  Most laws might add minor complexity, but at some point the laws become uncountable, making it very difficult to follow the law.  Now this by itself might not be a problem, unless the value received from additional complexity declines compared to the cost.  When enough of these sorts of mounting complexities cost more to the society than the society produces, eventually the society will fall.

A Corporate Example

Let me consider a different form of this, one that is very easy to understand.  Let us suppose that you have a piece of software that requires 9 engineers but has an income stream sufficient to pay for 10 engineers.  Customers keep demanding new features, and the complexity of the system rises.  Each new features has both cross-cutting concerns and interlaces with existing features.  More bugs are found, but few new customers are added.  Eventually, those whom knew the project will leave, but the complexity of the project does not.  The company finds itself needing more engineers in order to support the product.  The new engineers conclude that the best thing to do is to rewrite the application.  Even if you ignore Joel's words (from the last link about refactoring vs rewrite), you know that then they had to hire even more engineers to support the old application while the rewrite was in progress.  The company had not been making that much money, didn't appear to be able to make much more and so the obvious thing to do is the close shop.  A sociopath from The Gervais Principle would clearly do so.  They would not feel bad for the engineers who lost their jobs, for the customers who lost the product or even the history of the company.

This scenario has not happened to me personally, but I know that at least twice my entire automation code set was abandoned because the person whom replaced me was either not an expert or was at least less experienced compared to me in their programming skills and the company had no one else beside myself who knew the automation code base.  Instead of trying to learn the code base, the less knowledgeable people threw it away and started over.  Why do that?  My guess is the person doesn't understand the complexity in the system and decide that the last guy or gal must have been an idiot.  I too have inherited code bases, and not once have I started over from scratch.  I have thrown away pieces and parts, but never the whole, so I can't directly answer it.  I suppose, I once discover there was former automation scripts about 6 months after I started a project written in a different programming language.  In that particular case, the code developed had already outstripped all the functionality from the existing discovered code.  Even in this case, I still reviewed the work to ensure we had capture the same set of solutions.  This leads into an interesting question of losing complexity by forgetting, however, I am not going to cover that topic.

Company Growth & Avoid Responsibility


This also applies to big companies and as I alluded to earlier, governments.  According to Tainer, complexity growth without equal or greater forms of production end up with an eventual collapse.  Consider HR policies of different organizations.  With self-employment, there is no HR, thus zero complexity.  With a company of just a few people there is likely still no HR, but rather just a group of people all working towards a unified goal.  Often these people are very close.  Eventually there is a need for someone to manage the complexity and you hire someone into HR.  As you keep growing, someone does a boneheaded thing, like not taking time off and not call in sick for a week and then show up expecting that they still have a job.  There is no policy saying they will be fired and so they demand that they either keep their job or get unemployment.  So a policy is added, no big deal...

Then you get to be a mega corp with 100,000 employees.  You have divisions bigger than most medium sized companies and you have a handbook big enough to use as a deadly weapon.  Big organizations are often not run nearly as kindly towards employees because (in part) no one is empowered enough to go do that or if they are, it is spread unevenly causing jealousy, which causes things to then be 'not allowed'.  Having a pizza party for success now makes 500 others unhappy as they smell your reward.  You made 50 people happy and 500 grumpy.  An expert would have seen the problem ahead of time and planned on making it outdoors or away from the other employees, but the person who had the party didn't know better. So the company decides to make a rule saying you can't do that.  Eventually all these rules add up.  They hurt morale because the Dreyfus expert knows its stupid but has to think about how to get around the rule, making their 'expert' skill less valuable (as the point of the expert is they don't use rules).  They, like most Gervais losers (economic losers) flee, making such companies lose their best employees. It also encourages people to not try to make work better as that would break norms and likely added more rules to the already overly large set of rules they have to deal with.  Finally it encourages people to quit thinking because there is a rule, making them even less effective.

There is yet another reason that rules are created.  As The Gervais Principle describes, the sociopaths are always interested in getting the clueless to accept as much responsibility as possible while giving limited credit or authority.  The clueless are a buffer for the sociopaths so that they get as much value as possible from the losers without having to deal with the losers.  This is a form of protection that a zealous clueless person is willing to do, however, the clueless often don't know how to actually run an organization.  The clueless are in a position in which they use baby talk to pretend they know what they are doing.  Since they don't actually know what they are doing, the sociopaths use rules to force the clueless to do what the sociopath thinks is right.  These types of rules create a different type of complexity.  This is a social complexity, where the sociopath must juggle personalities to get the most production.  The clueless is actually a cost, but a small cost considering they are a sort of hedge that if things go wrong, the clueless can be blamed.  In a larger sense, this means that the person with bad ideas but who is good at manipulation can maintain power for a long time in spite of making bad choices because those choices aren't attributed to that person.  This can eat away at an organization or society, building up costs from complex institutions with mild production value, further corrupting the society.

Cat & Mouse


Capitalism is an attempt to alleviate the ineffective manager who somehow remains employed via competition.  The problem with capitalism, is like any system, the complexity builds up to protect the system from being hacked.  This can be anything from monopoly laws to defenses from regulatory capture.  However, this sort of question is mostly capture by Tainter himself, so I won't dive into it.

Ultimately the Tainter Composite describes how complexity not only infects society but the institutions around society.  The final piece I want to describe is how this leads to a cat and mouse set of activities.  Often rules are made because there is a real and legitimate concern.  Some of my examples might seem silly but are actually true (I have experienced some of them).  However, even 'smart' rules can get you into trouble.

Consider black hat crackers and the white hat hackers.  Code is in fact just a form of rules, often very useful rules.  You are currently reading this using a system with more lines of rules than one person could memorize.  The trouble is, there is always a group who wants to enjoy finding ways around these rules for their own purposes, just like some of the troubles in capitalism.  It might be someone who just adds everyone to your friends list or it might be to take all the money out of your bank account.  As security gets better, more rules have to be applied to protect the system.  Will code ever get to the point where maintaining and adding rules are too expensive for one person ?  Will it ever get to the point where building up all the rules is more costly than accepting the attacks?  To some degree that has happened where most people have traded up freedom (complexity) for a walled garden (less value).  Once again, this gets into reducing complexity, which is a topic for another day.

Rather than telling you how this applies to testing, effectively creating rules for your brain, I am going to try not to add complexity and let you interpret it for yourself.  If you found this useful, please write a comment and I will write more on the topic, otherwise, I will leave this dense, difficult topic alone for a while.

Friday, December 12, 2014

Thinking about No Testers

In attempting to hear out the various viewpoints of those whom subscribe to the idea that Agile should move towards removing the role of tester.  I have yet to see anyone who actually suggested eliminating testing completely, if that is even possible.  So let us unbox this idea and examine it, with as little bias towards keeping my own job as possible.  Here are the rough set of arguments I have seen/heard in TL/DR (Too long, didn't read) form:
  • Limited deploys (to say .01% of the users and deploy to more over time) with data scientists backing it will allow us to get things done faster with more quality.
    • Monitoring makes up for not having testers.
  • Hiring the right team and best developers will make up for the lack of testers.
    • Writing code that "just works" is possible without testing, given loose enough constraints (which leads to the next argument).
  • Since complete testing is impossible, bench testing + unit testing + ... is enough.
  • Quality cannot be tested in.  That is the design/code.
    • Testers make developers lazy.
  • It is just a rename to change the culture so that testers are treated as equals.
    • Testers should be under a dev/team manager, not in a silo.
  • It is a minor role change where testers now work in a more embedded fashion.
  • We hire too many testers who add inefficiencies as we have to wait <N time> before deploying.
    • We only hire testers to do a limit set of activities like security testing.
  • Testers do so many things the name/profession does not mean much of anything.
  • With the web we can deploy every <N time unit>, so if there are bugs we can patch them quickly.
  • As a customer I see those testers as an expensive waste.

That is sure a lot of different reasons, and I'm sure it is not an exhaustive list.  Now I shall enumerate the rough responses I have seen:

  • A little genuine anger and hurt that 'we' are unwanted.
  • A little genuine disgust that 'we' are going through the 80's again.
  • Denial ('this will never work')
  • It can work for some, but probably not all.  (This follows a very CDT 'it depends' point of view)
    • Not everything can easily be instrumented/limited deployed.  For example, I have heard Xbox 360 games cost ~10+k to do an update.
    • In some software, having less bugs is more important than the cost of testing.
  • This is why we need a standard (I never heard this one, but it is an obvious response)
  • Concern about the question of testing as a role vs task.
  • Different customers have different needs they want to fulfill.
  • Good teams create defects too.
  • We need science to back up these claims.
  • Testing should move towards automation to help eliminate the bottlenecks that started this concern.
  • It is very difficult to catch your own mistaken assumptions.
    • If you need someone else to help catch mistakes, why not a specialist?
While I don't have a full set of links I have heard this back and forth viewpoints, but I think it is an interesting subject to consider.  I am going to go through my bulleted list top down, starting with the "no testing" side of the world.  I believe I have seriously considered the limited deployment and letting the tester be the customer in a previous post.  In that same post I noted that at least not all teams can make up for a lack of testers, so again, I think it has been addressed.

Complete Testing is Impossible & Quality Cannot be Tested in


The premise is true, complete testing is in fact impossible, I do agree with that.  The conclusion that hiring testers is not needed is the part I find interesting.  Quality of code and design cannot be tested in is also in interesting statement.  The idea that testers make developers lazy feels very unsubstantiated, with possible correlation-causation issues (and I have seen no studies on the subject), so I am going to leave that part alone for now.

I write software outside of work.  Mostly open source projects, of which I have 5 projects to my name.  Most of them were projects in which I was looking to solve a problem I had and wanted to give to the world.  In more than half of them I wrote no formal tests, no unit tests and to be honest the code quality in some of them is questionable.  There are bugs I know how to work around in some cases, or bugs that never matter to me and thus the quality for me is good enough.  I have had ~10k downloads from my various projects and very few bugs have ever been filed.  Either most people used it and then deleted it or the quality was good enough for them from a free app.  I hired no testers and as the sole dev I was the only one testing the app.  I think this is a +1 for the no tester group, if you work on an open source app for yourself, no tester (maybe) required.  CDT even would agree with this as it does depend.  Context matters.  I am sure other contexts exist, even if I have not seen them.  What I have not seen is specifics around contexts that might require no testers, with the exception of the Microsoft/Google sized companies.

Now can you test code quality in?  If you are about to design the wrong thing and a tester brings that up, did the tester 'test' code quality in?  Well he didn't test any software, the no tester side would say.  Sure, but he did test assumptions, which is often part of testing.  What about the feature the tester notes the competition has that you don't, which was discovered when comparing competitor protect quality?  What if I suppose the dev knows exact what to develop, can the tester add value?  Well I know I have made code suggestions that have gotten a developer thinking.  However, I could have just as easily been an XP developer sitting beside him as he coded rather than a tester.  Well what about TDD?  Is that testing quality in?  If I write tests first, I have to think about my design.  That is testing quality in, but again that is just the developer.  What if the tester and the developer co-design the TDD or the tester recommends additional tests added during development.  Again, this could be a developer, but a tester is often thinking about the problem differently.  So while I think the statement is false, if I look at it from the spirit, I think it might be true in some cases.  There are some really great developers who do think like testers and they could do it.  However, I think that is only some of the developers.  There will always be junior developers and there are always other mind set factors that play in.

 What is in a Name & Quick Deploys?


What's Montague? it is nor hand, nor foot,
Nor arm, nor face, nor any other part
Belonging to a man. O, be some other name!
What's in a name? that which we call a rose
By any other name would smell as sweet;
- Juliet, Romeo and Juliet
Does the fact that we are in a in-group called 'test' and some see developers as an outsider matter?  Visa-versa is probably true too.  In fact, the other day Isaac and I had a small debate about correcting a tester in some very minor minutia about deep mathematical and development theory.  To me, perhaps a developer who was neck deep in building new languages would find such a correction helpful, but a tester who was doing some exploration around the subject didn't need such a correction.  In fact such a correction might make things more confusing to anyone who didn't have a deep understanding of the subject.  I myself am guilty of the insider/outsider mentality.  We all categorize things and use those categories to make choices.  Perhaps renaming us all as human beings would make things better.  If an organization is particularly combative, this might be a useful change.  That being said, there is also a group identity for people who see themselves as testers or QA.  There is a specific field of study around it, and that also adds value.  Also, by having a sub-group, the tension, even if minor, might possibly help people grow.

This is a mushy, non-business oriented view of the world, and perhaps those who are just interested in the how fast things go today would prefer avoiding the conflict in order to get things done.  The fear of doing this from a QA point of view comes from the 1980s, where defects were a big deal.  Renaming people feels like going back to that era.  The argument back is that this isn't the 80s, we can ship software almost instantly.  The argument to this is that some software can't just be shipped with broken code.  Work on many different areas must be 'mostly right' the first time.  Also, this grand new world that people want to move into has not been vetted by time.  Wait a while and see if it is the panacea it is believed to be.  I recognize the value in renaming positions in order to improve team cohesion and believe it comes from a genuine desire to improve.  Just keep in mind what is lost and the risks in doing so.

I should note, I have seen some of this implemented before in organizations I have worked in.  Working under a single development manager, the development manager would often ignore the advice given by the "QA" person.  Perhaps a full rename would have fixed things, but often development wants to move forward with the new shiny thing.  Having no testers is perhaps a new shiny, and so development would be on board.  The people who are more aware of risk of change are the QA and operations people.  This is a generalization, unsubstantiated by anything but my observation, but it feels like QA is now calling this a risk and because it threatens our jobs we are being told we are biased, so we can be ignored.  Psychology and politics are complex things and anything I write will not adequately cover this subject.  I don't object to people trying this, maybe it will work.  I still see it as a risk.  I hope some people take it and report back their observations.

Perhaps the final argument about us is that we are many different roles tied with one name.  This is a complexity that came up at WHOSE.  We talked less about testing than you might expect.  Communication and Math and Project Management and ...  We had a list of about 200 skills a tester might need.  We are a swiss-army knife.  We are mortar for the projects developed, and because every project is different, we have to fill in different cracks for each project.  We happened to do that best in some ways because we happen to see many different areas of a system.  We have to worry about security, the customers who use it, the code behind it, how time works, the business, and lots more.  That is ignoring the testing pieces, like what to test, when to test and how to justify the bugs you find.  Maybe we could remove the justification piece, but sometimes the justification matters even to the developer, because you don't know if it is worth fixing or what fix to choose.  I think we can't help being many different things because what we do is complicated.

Customers Don't Want QA?


While that is a lie in regards to me personally (I get excited when I see QA in the credits of video games), perhaps it is different for others.  I have written elsewhere how users of mobile devices are accepting more defects.  Perhaps that is true in general.  We know there is such a thing as too much QA.  To go to the absurd, when QA bankrupts the company, it is clearly too much QA.  We also know there is too little QA, when the company goes bankrupt from quality issues.  Perhaps we could say when there are regulatory constraints that close the company down because of a lack of QA could also be there.  Those are the edge cases, and like a typical tester, I like starting at boundary cases.  Customers clearly want some form of quality, and as long as testers help enhance quality (E.G. Value to someone who matters), testers are part of that equation.  Are they the only way to achieve quality?  Maybe not, but that involves either re-labeling (see above) or giving the task to others, be they the developers or the customers or someone else.  Some customers like the cheaper products with less quality, and that does exist.  Having looked at the games that exist on mobile devices that are free, my view is the quality is not nearly as good.  Then again, I want testers.  Maybe I'm biased.  Even if I was a developer for a company, I would like to have testers.

Should the ideas of no testers be explored?  I have no objection, but I would say that it should be done with caution and be carefully examined.  Which makes the companies who try it...well, testers.

 Brief Comment on the Community


I have heard a lot of good responses, albeit limited.  I have also seen some anger and frustration.  Sometimes silly things are said in the heat of the moment, likely from both sides.  We also have some cultural elements whom are jagged edged.  These can create an atmosphere of conflict, which can be good, but often just causes shouting without much communication.  One of the problems is most people don't have the time or energy to write long winded detailed posts like this.  We instead use twitter and use small slogans like #NoTesting.  I personally think twitter as a method for communicating is bad for the community, but I acknowledge it exists.  How can we improve the venue of these sorts of debates?  How can we improve our communication?  I don't pretend to have an answer, but maybe the community does.

UPDATE: I wrote a second piece around this topic.

Monday, November 17, 2014

Introspection on why I left...

This is a story about a job long, long ago. 
In having gone through this, I developed a system of heuristic flags to look at an organization that you work for. While these are context sensitive, I tried to keep a little of the context in the flags to give the reader some idea of what I saw. Seeing these things as they happen is extremely difficult. When you are in the moment they can look like a compromise, yet remove yourself from the situation for 2-3 months and suddenly they become “OH MY GOD, I can’t believe they even asked us to do that!”
The reason that some of us persevered through lots and lots of these yellow and red flags is many. People don't like change, interviewing is change. People like to see things through to the end, quitting feels like you failed. I genuinely liked the people I worked with, or at least lots of them, but that doesn’t change the culture of the sociopaths in charge (Gervais Principle). Hindsight is 20/20, looking back, I should have quit way earlier, but recognizing the signs in real time is much more difficult then it would seem when writing this down years later.

(the stuff in italics are the warning signs)

It all started slowly. That’s the best way to disguise failure. It’s how we can convince ourselves, “we're just learning"…
I had been working for this company about three months when it was decided that we needed to go Agile. We hired 2 product owners did a couple of days of training. Made a backlog, estimated, planned a sprint and brought new team members on board. We didn’t have a scrum master but we ‘were’ looking for one. Can’t have everything at once we told ourselves.

Yellow Flag 1: Starting a process by ignoring pieces of the process before trying them is folly.

We start this grand adventure with a meeting where it was deemed, the biggest oldest crappiest hard-to-read, difficult-to-modify code would be the starting point for the new hotness. Not lets rewrite it, but take the ‘scroll of insanity’ as it was called, and use it to build on for the new hotness. But we sallied-forth, new process, one bad decision won’t kill us, and we plodded along for a couple of months. 
Then the QA manager had a life event and never recovered, he left the company and the rest of this story is sans QA having a voice. We told ourselves we can work without a manager ‘for the team’.

Yellow Flag 2: Working without a direct supervisor for more then 3 months. 

Don't get me wrong I don't want to work for just any manager, and neither should you. But if a decent (notice NOT GOOD) manager doesn't show up within 3 months, I can think of 2 things. The company doesn't attract quality people. Or they aren't making it their highest priority.

Soon our VP was walking through the building explaining to other people / board members the agile process in exquisite detail. Basically lying to their face, in front of ours.

Red Flag 1: When management tells everyone including you, we are Agile. Then proceeds to prevent you from being a self-directed team, doing backlog estimations, working on a single sprint at a time or defining done. Now that I see this, I wonder if it was a sadistic way of daring us to challenge him.

At this point we had a new development architect team. They had a working pre-alpha. As a concurrent B2B web application the VP decides we should load test this. I can handle a new challenge and was excited to get started.
First round of testing showed two concurrent users crash the new hotness. Spent a week tweaking on load testing cause clearly I’m missing something with the load test. After showing the development manager and VP the results. I was called a liar, and the load part of testing was given to someone else. Two weeks later he has the same results I do, and the development manager admits maybe there is an issue in our session management.
Meanwhile I'm off to functional automation. Automation is brittle and the 3rd party GUI libraries aren't helping. No hooks, dynamic id’s, I can’t work with this.  I of course had no manager to explain this to, but was informed, “It helps the developers move faster.” Which just puts QA, without a voice, further behind…

At this point the only thing keeping me at the company is the Warhammer Fantasy board between a developer and myself.
A co-worker and I start to share any job openings we find in an effort to get out, but due to the recession, there isn’t much.

As all this is happening, we move to production with two active sprints. In and of itself wasn’t insane until you note that shortly we had 3 active sprints in production with only two sprint teams. At one point we had three active sprints in production being worked on with bugs, an active sprint in stage, one in QA and a sixth in development.

Yellow Flag 3: Is having multiple sprints active per sprint team. 
Red Flag 2: Is having six different sprints being actively worked on, by two sprint teams.

We had a sprint board which was viewed as sacred ground, only the PO could touch, move, write, etc. on or with it. At one point one of the testers called the board the joke that it was, and said that without the ability to move, edit or touch things, it was stupid. He said this to the POs face. Within 10 minutes he was in the VP’s office being told to apologize or he was fired.

Red Flag 3: Anyone who criticizes the process, being threatened with their job. 

So, we were fast approaching an imaginary deadline of an alpha version of our software, and we were behind, working OT and generally trying to get 'er done. This particular company prided itself on the 2 weeks over christmas only requiring a skeleton crew, so most of us didn’t mind the OT, in a couple of weeks most of us would get a respite and then come back refreshed. When the VP came out and announced, “ very one will be working over christmas, no PTO except on the holidays themselves, and mandatory 50 hours every other week“, yes they had canceled christmas. For a date that only they believed in, never mind that most people were doing 60-80’s.

Yellow Flag 4:(approaching red) Is when management changes work schedules for you.  

Projects dead. QA ignored. Automation maintenance nightmare. VP unwilling to compromise. 5 active sprints. 

The day I gave my two week notice I had to write a letter to quit, cause I couldn’t find the VP (who was my manager) for the entire day. He never did show up.

So he politely asked me to not inform people of my two week deadline, he’d like to handle it. Of course I was more them willing to allow him to  handle it in his own way.  A week rolled by, with no announcement, so I was unable to tell anyone. As a result no one thought it important to learn and take over the tasks that I was doing. Week two started with a scum where people started to assign me work, and at this point I was tired of waiting and kinda upset. I told the entire team that most of that work needed to be someone else’s cause Friday was my last day. Everyone was upset, and yet officially my leaving still wasn’t announced till one day before my last.

To top off a wonderful last week of trying hard to make sure people are prepped for me being gone. My exit interview consisted of HR telling me over the phone, “No, that’s not why you’re leaving.” What do you say to that?

Red Flag 4: The process of handing off your knowledge to someone is stymied, and no one in management cares why you are leaving. (Albeit this flag is more for people still there.)

Monday, November 10, 2014

James Bach and Dr. James Whittaker: An attempt to understand viewpoints


Introduction, Intentions and Limits


I don't have the political ties to many of the big thinkers of the industry.  I have seen James Bach speak, had him address my blog, addressed his blog, read his book and I have even asked him a question at CAST, but I don't know him personally.  I have attempted to address Dr. Whittaker in his blog, but have never got a response and I have never meet him.  I have read Dr. Whittaker's (in)famous exploratory testing book, which is currently the second most popular posts I have written.  I take that to mean I am not the only one who has interest in Whittaker's views and I suspect I'm not the only one to never have met him.  In my consideration of both these individuals, I am not attempting to make it personal.  As you read this, please keep in mind that this is my attempt to understand their viewpoints.  This is an attempt to parse the words of each of them to gain insights.

I came from CAST a few months ago and while I'm not going to directly talk about CAST I want to talk about a subject near and dear to my heart. In listening to Jame Bach's talk, most of which was around the process of testing, but one piece was clearly controversial. This was the Testing vs Checking and the ideas around it. In talking with Bach, I now know I hold disagreements with some of his model. I mostly kept silent regarding this, except for my consideration of Heuristics and Algorithms where I pondered if what humans and computers do is equal, it just happens human are complex and unknown currently compared to that of a computer. This leads down the free will question that is unanswerable, which is why I didn't deep dive into that subject. I still am resisting that particular topic for now, but I may return to it on a later date.

Bach has interesting ways of subdividing his world, with the terms Testing and Checking, with the word Test meaning something different in the past, even for Bach. Some day I will do a word of the week on Test, but that isn't today's subject either. In fact I'm still dodging giving my detailed, full opinion on checking vs testing, but be assured it will come up. Today's topic actually is about my studies of James Whittaker and what appears in my view as James Bach's answer to that viewpoint. I choose these two in part because I was interested in why Whittaker left testing and for Bach's part, he is a prolific exemplar of a testing professional for me to consider.  Bach also has address some areas that Whittaker has talked about, making him a easy to cite exemplar as well. This is not meant to be name calling, but rather I am trying to describe how I see these thinkers in test hold positions and viewpoints that are deeply held by many individuals. Certainly other people have variations of these viewpoints and certainly I maybe miss-reading parts of these viewpoints. I struggle not to put words in people's mouths, but I have to admit some of this is speculation based upon the comments I've found that Whittaker has made between 2008ish and 2014. Those are limited and I have studied them over months, so my sourcing is limited. James Bach, on the other hand, is prolific, which means I might be missing data. With that said, on with the show!

Warning: This is a long winded personal discovery post, you have been warned.

 What Dr. Whittaker has said on Testing


In .NET Rocks 408 from 2009, Whittaker complained about only having guiding practices, with no solid knowledge, saying,
“I'll just be happy when testing works... because they have guiding practices, guiding theory....”   
Later he noted how more testers means, in his words, lazy developers:
“We claim in many groups a 1 : 1 dev/testing ratio [in Microsoft]... Google [has]... 1:10 tester ratio... 1:1, you think oh wow, that is a lot of testing, also a lot of lazy developers to be perfectly honest... At Google they have no choice but to be a little more careful in development.”
Whittaker wants to remove as much inventive thinking from the process as possible, saying things like “In Visual Studio 2010... much of that [change analysis] we will automate completely..." and
“What can you tell me about the new tools... we think you will be able to reproduce every bug.”  That is a fairly wide and amazing claim.  He even starts talking about intelligent code that will find bugs all by itself,  “Only if it is the right sort of unit testing. If it isn't finding actionable bugs... throw it out. If the test cases themselves were feature aware, they know, they know what bugs they are good at finding... this is the future I want.”  Why do you need to inject intelligent reasoning when all problems have been solved before, but are hard-coded to one particular program?  “I'm convinced, on this planet, everything that can be tested, has been tested.”  In fact, why not remove testing as an activity all together?  Just have the machine figure out what needs to be tested, even before you hit compile!  “We[test] need to be invited in to be a full fledged member... we have to get to the point where we [test] are contributing more. Can we get software testing to be more like spell check is in word.... this needs to be the focus of software testers. These late cycle heroics and these testers who are married to late cycle heroes are Christmas Tester Past. ”

Dr. Whittaker imagines a future where testers are more-or-less developers. They might have a few test skills but basically once the magic tools exist, then developers should be solving these problems.  But of course he might have changed his mind since then?  I mean that was in 2009!  What about modern times?

Quality is no less important, of course, but achieving it requires a different focus than in the past. Hiring a bunch of testers will ensure that you need those testers. Testers are a crutch. A self-fulfilling prophesy. The more you hire the more you will need. Testing is much less an actual role now as it is an activity that has blended into the other activities developers perform every day. You can continue to test like its 1999, but why would you? - March 25, 2014, http://www.stickyminds.com/interview/leadership-and-career-success-purpose-interview-james-whittaker
Nope, Dr. Whittaker's viewpoint has gotten even more extreme. Dr. Whittaker might say I am a developer given the 'advanced' automation I write, but he certainly thinks most testers should just go away. He would rather have testers all fired, because [manual?] testers just enable developers to write bad code.  Why does he feel this way? Well first of all, he's in a super-large company that hires a great deal of the best talent there is.  Microsoft has some amazing developers and clearly they can write some amazing code, so maybe he sees them and assumes everyone else is like that.  Before he was at Microsoft, he worked for Google, so his perspective might be skewed.  Perhaps Microsoft and Google can get away with fewer or no testers.  Facebook seems to do it. He certainly seems to hint at it with his most recent blog post on the subject.  His infamous 'traditional test is dead' talk also seems to indicate that you need enough people to dog food your product, which requires a large community of people willing to file bugs for you. That does not apply to everyone, but he might not realize that?  Dr. Whittaker's history has, as best I can tell, been mostly about bigger companies and teaching. Perhaps there are other reasons I have not been able to discover.  I'd love to hear from Dr. Whittaker!  Clearly James Bach, has a reply...

James Bach and Automation


James Bach, a man who consults with and trains hundreds (or more?) of testers each year also has an interesting vantage point.  He mostly works on testing from an intelligent gathering of information through investigation.  He argues, in essence that any investigation made by a programmer in order to write automated tests (i.e. "checks") or executing the automated tests is in fact not equal to that of a human being.  All of Dr. Whittaker's magic tools make less sense in the mind of Bach.  Not that Bach doesn't want these tools, but he would argue without an intelligent human, the tool provides no additional value.  He has gone so far as to start labeling automated testing as checking. Not saying they have no value, but saying that their scope is limited compared to a human. I think Bach would even argue an AI-oriented test method, load testing, model driven testing and other "beyond human" testing could be done by a tester, given infinite amounts of time and/or humans. To be clear, James Bach seems to not view it as possible that there are tests that are beyond human, but I will address this later.  Therefore, I could simplify the argument to this:

if(Human.Testing > Computer.Testing) then throw new DoNotOverloadWord("Computer Testing and Human Testing are not the same!");

Ironically, Dr. Whittaker might agree with that simplification if only you change the ">" to "<".  While I might agree with James Bach in his basic point, I think that there are a few concerns.  One is that trying to change a word used in many different cultures, not just software testing to something new just makes another standard.  He's just added another definition to an already complicated word, making it even harder to understand. Worse yet, it seems like in defining the word that way, Bach has accidentally made his own standard for the word test, which will compete with other standards. I don't know if Bach wants to be a sort of defacto-ISO standard or not, but that's how it might become.  I certainly sometimes see how his passionate arguments could be perceived as an attempt to be sort of standard.  On the other hand, I also hate the answer 'it depends on context' (without something to back it up), so at least when I read Bach's blogs, I know what he means when he says checking.  Dr. Whittaker doesn't seem to put in nearly as much rigor in his words, making his ideas a little more slippery.  I don't know that there is an ideal way of handling this problem.  Perhaps that is part of why schools of testing showed up, so you know whose standard definitions you use.

When you say "Test" which definition are you using?  The Bachian 2014 definition.  Oh, you've not moved on to the 2015 definition?  No, it's out now?

Back to the question of computers and if they can go beyond humans in testing/checking.  I take it that Bach feels that computer's ability to test/check cannot go beyond people from this graphic:


Machine Checking is inside Testing and James Bach defines Testing as:
Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc. 
So if you run a load test, the load that the machine generates allows you to observe the system under load, using machine generated metrics like page load times or database CPU usage. The question is where does the load test code belong?  Is it 'checking' in that it navigates the pages?  I suppose it does blow up if it can't load the page, which is a check, but the primary point of load tests is not about the functions of the page as much as the non-functional requirements. Is the data gathered by the load test a model?  If I have a graph of users to page load times, isn't that a model... created by a machine?  Bach does not seem to have a reply to this, other than to say that a machine embodies its script and thus is separate from humans. Whittaker on the other hand does not care as much about the distinction as he does about getting the technology good enough to get the data into engineers hands without a middleman tester writing a specialized load test script. Better yet, in Whittaker's mind, get your users to do the testing by deploying the code.




Another issue is that while the test executes, it is limited to the data and code written, even if you have a genetic algorithm for testing and use Google as a datasource.  These are powerful algorithms and Google as a datasource outmatches my own knowledge base in a sense. While I agree this is perhaps different than a human who is perhaps in some way more advanced than said algorithms, it can also be beyond human.  Perhaps because humans can reason and the algorithms are limited to the code written there is a distinction to be made, which is fair enough, even if I still am not 100% convinced.  Is reasoning just another term for 'a process we don't fully understand' while code is something we can?  Is 'tacit knowledge' and 'cultural understand' really limited to just humans or will computers with enough data be able to generate similar models some day?  I admit human brains seem different, maybe we have a soul, maybe we have free will, maybe....  But to be fair, even with my imagined super-advanced testing system, I know 98% of automated testing systems are not attempting to approach these sorts of advancements.  Even if our automation systems aren't nearly as good as I think they can be, does that mean that testing vs checking will always be a dichotomy or will programs eventually get good enough that we can't tell the difference?  And what if humans are always special and better, what about diminishing returns? Hand-washed clothing maybe cleaner and possibly more water efficient, but a washing machine is preferred by nearly everyone in the modern world.  Perhaps all of this doesn't matter right now, but Dr. Whittaker seems to think we're there.  Maybe places like Microsoft are there?  Maybe "Tester" as a role makes less sense as more of that role is automated away, and maybe when you throw enough developers to create the automation, you can have most of your testing automated?

So then there is another question that comes to mind, if executing a "automated test" is checking, while the human is writing the algorithm to do the 'checking', is that testing?  In speaking with Bach, it is not in his mind.  My brain, thinking about the problem, writing code, creating models in my head about the system under 'test' (system under check?) is not testing according to Bach.  Very interesting viewpoint, making my claim of about this site being 98% about testing untrue in Bach's viewpoint.  Dr. Whittaker would seem to say that of course that is testing, even if I personally shouldn't be doing it or at least I should also be writing production code.

To be fair, Dr. Whittaker also argues that users are better than machines or testers.  This is a point both Whittaker and Bach might agree on, even if Bach might say only in some context.  But, in saying that I think Dr. Whittaker loses the point that users of a product can change products quickly and are fickle.  The mighty IE 6 failed and clearly IE is no longer top dog.  Listening to users might have been part of that, but an insider advocating for the users also might have been helpful! Users often don't know what they want and what they want changes quickly. This has known this for years. What makes Dr. Whittaker think that all products have a single user base that only wants one set of features? One man's bug is another man's feature.  For that matter, what about products that don't have a exploitable user base?  How do you dog food software for shipping products (OMS) when it costs money every minute the software is broken?  How about we dog food a pace maker; that will work, right?  James Bach would note that quality is not a thing you can build into software but rather a sort of relationship.  I suspect that if you break the software too often, you harm the user's interest in that software and they will move on.

What can I see from here?


I very much appreciate that these giants of the industry allow me to stand on their shoulders and look out upon the sea of possibilities.  I don't know exactly where I stand in an 'official' sense, but I am sure that both of them hold good ideas, ideas that should be investigated... nay, should be tested.  To his credit, James Bach agrees and wants to continue to make his ideas better.  The problem is that with different context there are different needs.  I have pondered from time to time if we could lock down context (e.g. E-Commerce website, X-Y income, X-Y products sold, Z types of browsers), could we say something about which style of testing would make more sense?  In considering both arguments, I have to say that in my experience, automation can go beyond what a human can test, but the value provided often is different than that provided by manual testing.  On the other hand, often the results require human analysis, meaning that human judgment is still required. Like a load test that knocks over a system after throwing 500% of normal load, a human is required to do the analysis and render a value judgment.  I having also seen some development shops that have gone without testers. At least in one case, they seemed to have almost pulled it off in a superficial view, but once you dig in, I think it became clear that some people or places do need testers.  I don't think there is a way of giving detailed advice that is universal, and without some well controlled studies, it maybe impossible to tell when a particular methodology is superior in a given context.

While I can't speak to what method is the best or which are superior to others, I can speak to what worked and what didn't for me in my context. Using only my own contexts and what I have seen I can make a few broad statements.  Few people do automation well.  The most successful automation does not involve a UI or when it does, it is in a manumated process. Those who do automation well are treasures.  Few, but more people do manual testing well.  These people are also treasures.  If you have only bench testing by the developers, adding either automated or manual testing (or checking) is likely to increase the quality for the product in small to medium size organizations.  I am not sure how to logically organize test/development for large organizations (more than 500 people in software development).

Not everything is easy to be tested by code.  Not everything is easy to test by people.  If you can't get people to do the testing, either it might not matter enough or you should automate it and accept the limits of checking.  If your automation keeps breaking either your software is flaky or perhaps it shouldn't be automating (yet, if ever).  Testing is hard and you need people who understand testing, be they developers or testers.  For that matter, you also need people to understand software development, the business, the customers and probably the accounting.  How you label these people may affect how they do the job, and maybe it matters, but what matters most is that the job is done as much as it needs to be done.

Ultimately, software development is hard.  Dr. Whittaker and James Bach come at the problems from very different places, but both want to improve the quality of software.  Even with their extremely different views both have contributed towards their goals.  These extremes do cause some polarization and sadly this post might sound negative towards both their views, but it is hard to find common ground between two extreme views. On the other hand, Whittaker helped make Visual Studio 2008 and 2010 better at testing while Bach has been working on perfecting RST!  Both these men have worked towards their visions and have created useful artifacts from their near disparate visions. While I have disagreements with both sides, I am glad they were around making my job of testing better and easier.

So if I haven't said it yet, thank you.

One last thing.  If Dr. James Whittaker or James Bach respond to this with any corrections, I will update this post.  I maybe challenging my understanding of their ideas but I am not trying to put words in their mouths.

Tuesday, November 4, 2014

Consultant's War

My co-contributor Isaac has been pondering the fact that we see more consultants blogging regularly, speaking their mind and controlling the things we talk about.  I don't care if it is  Dorathy "Dot" Graham, Markus Gärtner, Matt Heusser,  Michael Bolton, Michael Larsen (Correction: As Matt pointed out, I got my facts mixed up and Micheal is not a consultant), Karen Nicole Johnson,  James Bach or Rex Black.  There are dozens of consultants I could have listed (I mean, how could I have missed Doug Hoffman!), some of whom I have personally worked with.  So this is not meant to be a personal attack on any of them.  I realize I list more CDT folk than others, and while that is not intentional, I read less of the thought leaders from the other schools.  If you look, I bet most of the people you read either are or were consultants [In interest of full disclosure, I have written a hand full of articles and have been paid, but I am not nor have I ever been a consultant, and with you reading this, it is at least one counter-argument, but we'll get to that in a bit.].  If you made it here, there is little doubt you've read at least of those author's works at least once.  They are stars!  I mean that literally.  I found this image of James Bach just by searching his name:
From: http://qainsight.net/2008/04/08/james-bach-the-qa-hero/
I didn't even use image search to find that, it came up on the first main page of my Google search results!  I respect James, he writes well and has interesting ideas, but I have no opinion on his actual testing skills, as using his own measure, I have not seen him perform.  Keeping that in mind, I have not seen any of the above consultants do any extended performances in testing, even if I have listened to many of them talk and gained insights from them.  They know how to communicate and they are often awesome writers.  Much better than I am.  They all are senior in the sense that they have done testing for years.  Some of them disagree with others, making it often a judgment call of their written works on who to listen to.  Some of it is the author's voice.  In my opinion, Karen Johnson is a much softer and gentler voice than James Bach.  But some of it is factual.  I have documented several debates between Rex Black and various other people I have listed.  Rex has substantially different ideas on how the world should work regarding testing.

 ISO 29119


James Christie, another consultant brought up a standards in CAST 2014.  It was a good talk and clearly he had some valid concerns about the standards that have been created.  I have talked about that earlier (I am not going to include as many 'justifying' links as all my points and links are in the earlier article).  I am actually not terribly interested in the arguments and in reality, ISO 29119 is more of a placeholder than the actual point of interest.  I am more interested in discussing who is creating these debates.  In part we are in a war between consultants and in part we are in a war of 'bed making'. Clearly the loudest voices are from those who have free time and have money at stake. The pro-ISO side has take the stance of doing as little communication as possible, because it is hard to have a debate when one side won't talk. It is a tactical choice, and it is very difficult to compromise when one side doesn't speak. Particularly when the standard is already in place, the pro-standards side doesn't have much to gain from the messy-bed side who wants the standard withdrawn.

Both groups of consultants have money to gain from it.  A defacto-RST-standard would make Bach a richer man that he already is.  Matt Heusser's LST does give at least some competition of ideas, but still that isn't much more diversity. The standards would probably make the pro-standards consultants more money as they can say, "as the creator of the standard I know how to implement it."  Even if they made no more money in so far as making the standard, they will be better known for having made the standard. Even if I assume that no one was doing this out of self-interest, the people best represented are the consultants and to a lesser degree, the academics, not the practitioners who may feel the most impact.  Those leading the charge both for and against the standard are primarily consultants or recent ex-consultants. Clearly this is a war for which the people with the most time are waging, and those are the consultants. Ask yourself, of those you have heard debate the issue, what percentage are just working for a company?

Granted, I am not a consultant, but it takes a LOT of effort to write these posts.  It isn't marketing for me, except possibly for future job hunting, and the hope that I will help other people in the profession.  I know non-consultants are talking about it, but we don't have a lot of champions who aren't consultants.  Maybe most senior level testers become consultants, perhaps due to disillusionment of testing at their companies.  Maybe that is why consultants fight so bitterly hard for and against things like standards.  Perhaps my assertion that money at the table is a part of it is just idle speculation not really fit for print.  I can honestly believe it to be that is possibly the large majority of the consultants involved.

 Bed: Do you make yours?


Then what is it that causes these differences of view?  Well let me go back to the bed making.  To quote Steve Yegge:
I had a high school English teacher who, on the last day of the semester, much to everyone's surprise, claimed she should tell just by looking at each of us whether we're a slob or a neat-freak. She also claimed (and I think I agree with this) that the dividing line between slob and neat-freak, the sure-fire indicator, is whether you make your bed each morning. Then she pointed at each each of us in turn and announced "slob" or "neat-freak". Everyone agreed she had us pegged. (And most of us were slobs. Making beds is for chumps.)
That seems like a pretty good heuristic and I think it is also a good analogy.  Often those who want more documentation, with things well understood before starting the work and more feeling of being in control via documentation are those who feel standards are a good idea.  They want their metaphorical beds made.  They like having lots of details written down, they like check lists and would rather make the bed than have a ‘mess’ left all day. A nice neat made bed feels good to them. Then there are the people who see that neat bed and think it is a waste of time at best. At worst, they think that someone will start making them make their bed too. Personally, I think "Making beds is for chumps", just like Steve Yegge. Jeff Atwood would go even further and call it a religious viewpoint:
But software development is, and has always been, a religion. We band together into groups of people who believe the same things, with very little basis for proving any of those beliefs. Java versus .NET. Microsoft versus Google. Static languages versus Dynamic languages. We may kid ourselves into believing we're "computer scientists", but when was the last time you used a hypothesis and a control to prove anything? We're too busy solving customer problems in the chosen tool, unbeliever!
Clearly the consultants, many of whom I take valuable bits of data from care about what they do and they tried to lock down their empirical knowledge into demonstrable truths.   But that doesn't mean they have 'the truth'. I know as testers we try to have controls, but I am not so convinced we have testing down to a science.  It is why I feel we aren't ready for standards, but I also recognize the limits of my own knowledge.

For what it is worth, I tend to be against bed making, I think having all this formal work and making checklists is rather pointless unless they fit what I am doing. Having one checklist to rule them all with a disclaimer that YMMV and do the bits you want doesn’t sound much like a standard, but those whom like their bed nice and neat probably feel very happy when they come home at night. The ‘truth’ about the value of making your bed, the evidence that it is better is less than clear. Maybe one day we will have solid evidence in a particular area that a particular method is better than another, but we aren’t there yet in my opinion. But still I don’t make my bed.  In case you are wondering, I think this is probably one of the hardest questions we have in the industry:  What methods work best given a particular context and what parts of the context matter most?