Belief: Absolute Conviction or Probability?

I have long been thinking about the nature of the world, the nature of belief, knowledge, faith, etc. I have come to a conclusion that is both obvious and perhaps to some, scary.  This is one of those personal adventures, and it will take a little bit to explain.  It's also rather abstractly connected to testing, so if you are looking for how-to articles on testing,  I'd go looking here or even here instead.  You've been warned.

By modifying the definition of belief ever so slightly, my language in fact sounds completely reasonable, and I suspect that my version of belief is much closer to the reality of what a belief really is in a psychological sense.  In particular, I can say I have observed it with people who do scientific-like work.  Let's first start with dictionary definition of belief.

According to Merriam-Webster, a belief is:

1. a state or habit of mind in which trust or confidence is placed in some person or thing

2. something believed; especially : a tenet or body of tenets held by a group

3. conviction of the truth of some statement or the reality of some being or phenomenon especially when based on examination of evidence

Knowledge in epistemology is well known to be a "True Justified Belief."  Effectively, the difference between belief and knowledge is that a belief may not be true and it might not be justified. You can only claim to know something when it is true and justified (although how much justification is still in question).  I have also done some informal consideration of justification, which can be valuable in justifying a bug.

In a World...

Imagine, if you will, a world where beliefs were held so tightly that they were in fact the way you ran your life with exact and precise calculations.  You might be like some of the Vulcans who dedicate themselves to pure logic, as described in Star Trek, except your dedicate would be to your own beliefs, not logic per-se.  If, for example, you believed in a set of development principles, you would never break any of these principles.  Assuming your principles were designed correctly to never have functional bugs, a functional bug would not exist in a code base you exclusively created because your beliefs would be personal law, with you never straying. The only possible remaining set of functional bugs are those which are not covered by your beliefs or areas where your personal mental model of your belief was not exactly in congruity with the actual statements of the guiding principle, which is just a form of misunderstanding.

For example, if you stated your belief was "No text box I create will have an injection attack possible.", but your mental model was only based upon SQL injection and did not consider command injections.  Then if you had a vulnerability to a command injection attack in your text box, is it the fault of your beliefs that no text box should be vulnerable to injection attacks?  I think not, because you either had a limit to your understanding of your philosophy (what you believe an injection attack to be) or the world (what injection attacks are known to the world).  However, a more subtle question and interesting question is, did you really hold the belief?  I think the answer is complicated.  Internally, the answer is yes, of course you believed.  However, from an external view did you hold the true belief?  Yes, you held the belief in your own way, however, it might not meet some external evaluator's view of how that belief should be held.  This is a communication problem people run into all the time when they define words differently.

Thus, your personal reality would appear to succumb to your beliefs, or your beliefs would say nothing about reality. That is to say you would ignore your own senses when you observed the natural world contradict your beliefs. This occasionally happens at a developer's desk when they say, "That isn't possible." as I demonstrate a bug to them.  Do they see the world shifting out from under them or are they ignoring it?  Do they see their belief ruined and their world is crashing down?  For some people, this Vulcan-like demand for the perfect belief, the idea of the world crashing down might come close to their view of life.  Then there are others who choose to doubt only the facts they don't like, so that their belief is maintained.  But do these groups go from the point where they say “I believe” and run their life exactly according to their proclaimed beliefs?

Perhaps a few do run their life exactly according to their beliefs, but I suspect that it is more likely most have a journey where you slowly reform both themselves and their beliefs.  Even with all the defenses around their beliefs, and all the denial for the facts contradicting, I think given time to consider, people can migrate beliefs.  Like the developer who claims the bug is not possible, they adjust their beliefs over time. So, either most of us are able to hold two opposing beliefs at the same time ("That isn't possible" and "That just happened") as we reform our beliefs, or are able to ignore those former beliefs somehow, thus truth in the mind and actions are not strongly related.  The latter option is sometimes referred to as rationalization.

For the sake of argument, I am going to assume that in fact people can hold two opposing beliefs at the same time. The reasoning I am assuming this is that if belief of a truth and how one acts have limited relations, then we are creatures whose entire rationale does not matter and thus the entire nature of belief doesn’t matter in this context. Another reason for my assumption is that in working with software testers, I have observed this ability to weight two opposing beliefs.  In my observations, it wasn't that they had ignored their belief and just done something else but rather taken context into consideration.

50% Chance of Opposite Day

If we can in fact hold two opposing “beliefs” then those are not beliefs in the traditional sense.  One is not in fact completely confident in their belief.  Instead, what they almost become is an internal struggle for which belief is more accurate in the modeling of the world.  Since different people model the world differently, a realization occurs that multiple models can both be correct.  In a real sense, the person with said beliefs is weighing non-mathematical probabilities.  Now, if that is what we are doing, trying to decide which version of belief is more likely, and we have the ability to replace one belief with another based upon this evaluation, beliefs are just things we attach a probability on.  Another way of saying this is that they are context driven.  The probability that a particular belief will satisfice is determined through life-experienced context and those probabilities are constantly being re-evaluated as more of life is experienced.

I try very hard to avoid politics in this blog, but I wanted to address a good example of differences in mental models where it is not clear what the meaning or intent is.  Recently, a woman claimed that she is black when her parents disagree with her and claim she is white.  The interesting question here is, what is it to be "black"?  Is it a life style?  Is it a social upbringing?  Is it a description of particular genes?  Is it a color?  At what time and under what conditions?  Is it an origin?  How many generations back?  Is it a description of being disadvantaged? Is it an artificial classification?  How many of these checkboxes do you have to have to be considered black and which oracle(s) do you listen to?  Is this another true Scotsman problem?

If beliefs are actually statements we find likely to be true, we are attaching what I will refer to as a probability. It may not be numerically calculated, but a consideration of one’s experiences, perceptions, internal mental structure, and numerous other factors. Ultimately we come to some sorts of conclusions. "I write great code", "I’m in love", "JCD writes good blog posts", ad nauseum. Can you in fact believe both “men are evil” and that “evil does not exist”? If those are just probabilities, then yes, those can in fact both be beliefs at the same time for one person. Keeping in mind that knowledge is a true justified belief; you can’t “know” both of these things at once, because only one can logically exist at once, but which one? Even with all the brains we have, this appears to be a Gordian knot of a problem. Cutting the knot using things like Occam’s razor only goes so far (It is only a razor!).  Worse yet, the question of context pops its head in here.  "In the past year JCD has written good blog posts" adds context that the first statement did not have.  A developer's defense of "No one would do that" is true in the context of "that I like and hasn't made a mistake and ...."  Perhaps with enough context one could claim knowledge, but you know the saying about building an idiot proof box... I suspect if you could build enough context, someone would just come up with new unconsidered situations.

Why not accept our nature and in fact embrace it? If someone says that “men are evil” and provides limited evidence, you can assign a probability to that (let us assign an arbitrary tag “somewhat likely” to this). Then someone else comes along with highly convincing evidence that in fact evil can’t exist. You weigh the factors you now have and change the scales, applying a “very likely” for "evil can’t exist" and downgrade the “men are evil” viewpoint to “fairly unlikely.” You need not deny either one of those items and yet you can still hold a legitimate opinion that "According to my present data, evil is unlikely to exist."  The tricky thing is, English does not make it easy to give these sorts of answers.

"Nonsense is so good only because common sense is so limited." - George Santayana

In a brief change of topic, I want to stop and address what some readers might be thinking at this point.  I am sure some people see this as non-sense.  So I want to take a brief aside to consider the possibility that this idea doesn't matter.

"In the West, we take the “law of the excluded middle” as self-evident  and  as fundamental  to  logic.  Under  this  rule,  a proposition can have two states: something “is X” or “is not X”. There is no middle ground. In contrast, several Indian logicians assumed four states, “is X”, “is not X”, “is both X and not X” and “is neither X nor not X”. I puzzled over this for many years. I am not confident that my understanding of how this could be meaningful is the same as their understanding. But in human affairs, in interpreting  people,  how  they  interact,  what  situations  they  are  in,  and  how  to negotiate  with  them,  I  came  to  view  the exclusion of the middle as a heuristic rather than a law or a rule. It is often useful to assume that “X or not X” are the only two possibilities, but when that is not productive, it is time to look for a third way." - Dr. Cem Kaner, Tea-time with Testers, June 2015, Pg 61

I could go on, as Dr. Kaner certainly has more around this.  The idea that perception is in fact reality is often skewed.  In the same magazine issue, Jerry Weinberg tells a story in which the three team members whom did all the talking were perceived as the leaders performing actions but the the one with an effective action was not the team member being defined as the leader.  The team member with the affective action was another employee whom only did one thing, silently solve the problem.  Perceiving motion for action is not an uncommon occurrence.

The point I'm trying to make is those whom see belief as a solid, absolute thing probably have a more difficult time seeing that 'reality' is more difficult to pin down.  It seems likely to me, that for most people, as they gain more experience, they too will reform their view of belief into something more generalized, such as a probability.

"The unexamined life is not worth living." - Socrates

Briefly, I want to address another sticky subject, faith.  Faith is a belief with only external sources for justification. You might be able to cite examples where the external source was previously correct, but the external source is still your only justification. Faith is often replaceable with “trust” or “hope”. You can say I have faith that login is broken because my manager is telling me it is so. The justification is based upon hope or trust in some external or internal force that your belief is true. If instead we consider faith a probability based upon your level of trust in a given oracle, you can in fact see faith is just a specialized description of a belief.

One problem with this is our language does not naturally imply that our statements are guess work, nor easily allow one to explain what the most likely evaluation you have thus far computed. People say “I know ...”, or “It must be ...”. We assert our knowledge because people are not actually comfortable with “I don’t know, but I think ...” It is in our nature to say we are right. We even prefer people in which we perceive confidence, even though it maybe a lie.  It is also convenient:

"A little inaccuracy sometimes saves a ton of explanation." - Saki

Compare "According to my present data, evil is unlikely to exist." and the shorten "evil does not exist."  In my view, these absolutes often don't represent what people really think, but I admit it's only a probability.

Another problem is that people will claim this is a flip-flop philosophy that does not in fact hold a person’s “feet to the fire”. While this is true to a certain extent, it is designed to actually be more in line with how people and our estimation of reality actually work. People are rarely able to be self-consistent in what they say, much less what they do. How often do people say “Do as I say, not as I do.”?

In recent years, people have been able to pull out long and specific quotes from others because of how our technology records everything.  We are very much able to examine other people's lives.  Yet, in looking for this consistency, we fail to examine our own lives and the changes we make.  If other people are often unable to be consistent, it's likely you too will likely fail at consistency (Consider: did you know every time you recall something, the memory is re-written and thus modified?).  Yet we like consistency.  There are still two obvious options.  Either we are really failures in consistency but pretend we are not or we are constantly evolving our understanding and are designed to evolve our thinking.  I'd give it about a 20% / 80% probability for anyone individual in circumstances similar to mine when I consider....Oh you get the idea.

Ultimately this is a small chunk of analysis attempting to describe my view of how people work.  It is hard, if not impossible, to write out a full view due to the fact no one else has experienced my life and the context that brings.  I also recognize other views exist and I think they have some probability of them being correct and am open to replacing my own view with a more probable one.  So please feel free to share any insights you have, even if they contradict my own view.  I encourage you to sit down and attempt to write out your views on how people work.  It can be a useful exercise, as it will give your opinions a more solid foundation.  But if you have done all that hard work, why not post it as a comment?

Expectations...again...

This was written a long time ago...but finally edited to readability.

So I just finished the AST BBST Instructors course. In it, we started by introducing ourselves. I of course summarized my introduction from this blog cause I was being lazy, and it said what I thought I wanted to say.

In response to that I was asked:

"It sounds like you have the confidence and experience to be a manager, but I wonder if this might make it hard for you to teach? What do you think?" - Another Student

I responded with:

"Yes I find that my mental model has a hard time bonding with people who aren't also self-starters and/or newbies. However I've been specifically working on how do we 'as a testing community' get the first level of people into testing. I still have a problem dealing with people who just want to be spoon-feed information.
My experience is mainly in taking people with 3-5 years of experience and tuning (yes tuning) them into the next skill they need to be the best at their current position. However so far this has been limited to where I work, as I usually understand the context enough to say "if you could do this" it's the next skill you should learn.
So part of my goal is to be able to teach less experienced people. But I still struggle with if I should TRY to deal with non-self-starters…"

I’d like to take a more detailed look at my personal thoughts on the matter. Specifically around, “Does your own attitude and personality cause you difficulty in teaching people?” I’m going to try and detail an internal dialogue I’ve been having about this lately. I first tried to sum up some feelings in an older blog about Expectations.

This is what I mean when I say, "not-self starters". There have always been people at places I've worked that are comfortable doing just the basics, show up at work, do their job, go home. These are the people that when asked in an interview "What do you do to improve your skills", have an answer similar to "On the job training". These are the people that don't seem to want to take time outside of work to improve their skills (which to me translates to "I don't consider this job a career"). Some of these people seem bright, others are merely content at getting along and that's enough for them.

Then there are the self-starter people who will go out of their way to learn new things. These people ask you questions, and when you say, "tell me what you know about it so far", they blow your mind with the research they have done. Or alter your understanding of the subject with some new pieces that you haven't heard or seen of yet. Sometimes at a minimum they have clearly researched and/or understood the material, but don't change your level of knowledge.

I know mentors / leaders want to use terms like taught, educated or instructed. But when you have self-starters, that isn't the appropriate wording. The closest I've come is tuning them. When I chat with a self-starter cause they are asking "What should I learn next?" (and them asking is one of the key points). There are a couple of ways to encourage them. And generally that is all you can do. Point in a direction, say 'that way', and then get out of their way.

• Have them learn a new skill. Sometimes these are easy for beginning self-starters. Don't know SQL, yep I have yet to meet any tester that doesn't benefit from knowing it. As the self-starters get more and more experience, this can get harder and harder to find for each individual. Currently I've had serious success with just allowing them the freedom to find new things. This is the real power of the self-starter, they aren't okay with sitting idly by and surfing reddit, they WANT to provide value, they WANT to solve problems.
• Have them level up a skill they are already strong in. This helps those who have just finished something rigorous, demanding or seriously mentally intensive. (This can be work related like finishing a project or a mentally challenging class, or a non work related life changing event.) It allows them to learn something, but with the permission to not be as intensive with it. An example would be having someone learn the singleton pattern, instead of a whole new programming paradigm like LISP.
• Have them up a skill they are weak in. I don't normally recommend this for very many people. I tend to follow the Good to Great idea, that weakness isn't inherently bad. But there are genuinely some weaknesses you need to either compensate for, or bring up to a minimum bar.

Sometimes even that isn't enough. I recall one situation where I was attempting to tune someone I classify as a full-blown self-starter. We were going over the OSI model, and I was attempting to explain how each layer could be an attack point for testing. Unfortunately we were not speaking the same language, after about 30 minutes, they were frustrated and on the verge of crying. I was frustrated and not understanding why they couldn't get the 'simple concepts'. We parted that day, and I don’t recall us ever trying to train together again. I took this as a personal failure…how could I not explain such simple concepts (which OSI is and is not) to someone who tries really hard when they self-educate and generally succeeds? It took me a long time to understand that I had a base of knowledge about computers that they had never had exposure to. Bad on me for assuming what they knew. The problem was that I know stuff they didn't. They also know stuff I don't. It takes time and effort to truly get to know people. I hadn't taken the time or maybe didn't have the perception to understand that they had no idea what I was talking about. They didn't have the trust to tell me 'What the hell are you talking about'.

S'long as you learn to temper what you tell people you want, with realistic acceptance of what people can really accomplish. It all comes down to do you trust your people to work hard, do they trust you to not hose them with unreasonable expectations. It takes some serious time to build that trust. It's why you see people follow a strong leader around to companies.

If you don't fall into the category of self-starter as I've laid out here. There is still hope, you can become one. Start today, motivation breeds motivation, find something interesting, and learn it. When you're done learning that, find another. Then repeat ad nauseum.

I've going to wrap up and caveat this entire article with a thinking exercise for you.
Can your expectations be too high? Is this a bad thing?
How else would we ever achieve something new?

Resumes, Lying and Embellishment

I was reading how Raji Bhamidipati had noted how many people with little experience were using fake experience on their CVs (I will use the term resume).  They had several questions on why people choose to be unethical and fraudulent on their resume.  Furthermore, she states that many of these people end up being hired using their fake resume.  These hollow resumes, were not caught, but leave the person in question of being caught for the rest of their time at their job.  This entire article is speculative based upon my experience and research, but should not be taken as a sure thing.

History and Brains

I won't pretend this is a new issue, or something that 'our generation' started, but I suspect it has gotten worse over time.  First of all, economics and social history is a part of this.  Speaking specifically of the United States, after the 1950s, most of Europe was in a economic ditch due to the destruction from World War 2.  People could easily get jobs and brought up the 'baby boomer' generation.  The US invested a lot into tech, and made a large number of advances.  Without digging deep into the history, the boomer generation slowly amassed wealth and was able to assist their children to go to college and unlike any other time in history, the US population mostly has college education.  Unlike many areas in the world, in the United States it is often the case that you take out what is effectively a long-term loan for education.  While this is overly broad, many people excepted quick success or ignore the bill until after they graduated. The US slowly moved away from manufacturing jobs, leaving us with many 'service' jobs, like changing oil and child care.  One of the few other sectors with significant growth was tech.  Also part of the cultural zeitgeist was that the boomer generation encouraged a belief in their children of being special.  With large debts from school, few places to work (that pay well) and unreasonably high expectations in life, much of the younger generation has been forced to compete in an environment they were not prepared for.  Add in pressure from social networks where your successful friends are highly visible, and lying to get a job might seem like a way out and up.

There is another real set of reasons involving the brain.  The first one is, many of those who lie on their resume are young.  They have little life experience and don't have a sense of the possible consequences.  They also don't have the brain to handle the risks.  Literally, those who are young enough don't have as many connections to the executive part of the brain and make dumber choices.  It is also why crime trends towards youth as well.  Notice how the drop off age is about 25.  About the same time the brain develops.

Another factor involves the Dunning-Kruger effect, people don't know when they aren't very good and when they are very good they think less of themselves.  Those who don't know there lack of skill think they are amazing, so they tend to inflate their resume with less truthful statements.  Worse yet, since they are lying to themselves, they are in some ways more capable of lying to others.  Those who are more knowledgeable tend to know how little they really know.  They therefore seem less capable, which helps bolster the case of the less skilled.

Employers Worst Enemy: Themselves

The final factor is the automated checking of resumes that looks for keywords in a resume.  Sometimes playing alphabet soup can help you get a job.  If you have little to no experience, alphabet soup is about the best bet you can make.  If you can back up the claims even a little it might make sense to do so.  Sometimes this is referred to as 'Embellishment'.  However, that can become a slippery slope.  In what might have been the worst interview I ever observed, in looking to find an upside, my coworker choose a technology the candidate had listed and asked him to talk about it.  The resume had categories, like 'familiar' and 'expert', with this technology being marked as expert.  I believe it was Tomcat servers, something I think most of us were not expert in and was not used in that company, so we were not evaluating his actual knowledge.  His first comment was that he only used it in a testing context, and that was how he was an expert.  So we asked how he'd find a bug in said system, or how he'd get updates deployed.  The best answer we eventually got after digging into his background was other people did everything for him.

But what did we do?  If he was smart, we taught him perhaps how to answer that form of question better without ever learning any of the tech he claimed.  In essence, even though we didn't hire him, he could learn the wrong things from that interview.  Worse yet, while he didn't get our job, we had no way of notifying other employers of his behavior.  We couldn't say how we caught him in several lies because his former employers included employers we had, so we knew some of the work was not his.

From an employers point of view, when they are first hiring, it is not unlikely they don't know what a good tester looks like.  They do some web research and come up with a list of properties that sound reasonable to them and come up with a job description.  If you follow the CDT view where things depend on context, it will be hard for an employer to ask smart questions.  Worse yet, ISTQB is about the only thing an employer might be able to use to filter with.  Ignoring my personal opinion on the value of such a cert, since an employer probably doesn't know much about testing or  ISTQB, they are not likely to go doing the validation of such a certificate.  If someone feels ISTQB is of little value, they might justify in their head, "My employer won't know the difference."

The employer is more likely to offer on the low end of the pay scale, even if they want experienced people.  Sometimes this is because they can't afford anyone more advanced, else they would already have a test team.  Other times it is because they are small and thus don't have enough cash to afford senior people.  Even if they have a test team, not everyone treats this as more than a job, and might not take their interviewing seriously.  This allows beginners in, as well as those whom lie on their resume.  In my experience, often these sorts of companies also have death-march-like development processes because the company has not matured.  This makes it so that many newer recruits end up leaving.  Another area where this happens is in contracting.  When you need 30 testers in seats and you are contracted to have them, or your pay is tied to the number of people, you have a conflict of interest between hiring just anyone and keeping the customer happy.  You might hire a few 'iffy' people because you can hide them in the middle of some better employees.

 What Do We Do?

The truth of the matter is I don't know that we can as a society eliminate this sort of low-level corruption.  I worry as automation gets better and less high paying jobs come available what sort of world we will live in.  One likely result is with increased competition we will see an upswing in this sort of falsification.

In trying to evaluate my own predictions, I tried to find any data around trending, to see if this problem is in fact getting worse.  I couldn't find anything substantial, but a variety of different percentages were thrown around and nothing that felt authoritative.  The one thing I did notice was the numbers were typically large, like 40% of resumes have some form of fraud in them.  Okay, you caught me, it was 39% with other numbers on that page lower.  Oh, and that was percentage of organizations, not resumes.  Darn, you're reading up on my data now?  I just wanted to make the number seem more round and dramatic.  What's wrong with that?  The point is, we all have to be careful with our language and clear on what we expect.  I imagine you expect that I try to get my facts straight and while some rounding might be implied, that the unit of measure would remain consistent.  For example, culturally, Latin Americas commonly include pictures with their resumes, while in the United States, that is unusual.  So if you send a resume to Latin America, you better include a picture, as that is the norm.  I know no one of us can set such expectations, but you can set them for yourself and ask about unclear parts of resumes that don't seem to follow your standard.  We have to make honesty the norm by being honest ourselves.

Obviously if you are having people ask you about advice regarding lying on your resume, the answer should be no.  I have given recommendations for people, and I have had one friend list work he did for me in his resume when I was about 16-18 years old.  He actually did do some testing, but it was not for a formal company.  I said he could because he did the work, formal organization or not.  Some people might see that sort of buddy system as flawed because you can use contacts to get jobs, but those without contacts are disadvantaged.  I have even heard claims that this can be accidentally racist/sexist, based upon who you know and trust.  The fact of the matter is, that hiring people you know and trust has always been a thing.  The reality is that hiring based upon personal knowledge of someone else's previous work is one of the best filtering mechanisms.  Hiring based in part upon the recommendation of someone you know is probably helpful.  Dr. Kaner suggested a entire packet of stuff be used to track someone's career, but that doesn't help much for those starting off.  Perhaps a broader network, like that of Dr. Kaner or LinkedIn is also of value, but that only helps vet people you know.  I recognize that some people may disagree with the idea of hiring people you have formerly worked with, and I welcome discussion around that.  I think it is better to have an open discussion than for me to stand on my soap box and expect you to just accept what I say.

There are two things we can do today for every candidate.  The one easy thing we can do is be careful with our hiring.  Get to know the candidate, ask lots of questions, and in the end, if you aren't sure, don't hire them.  I know that can hurt candidates, but I think it is important to choose the right person rather than getting the wrong person but moving faster.  The second piece is to investigate the claims of a candidate.  While I think it is better to do so in an interview, even after an interview, if a candidate makes a claim, it isn't too late to go look into it.  Better to spend ten minutes now than waste 6 months with them and another 3 months trying to get rid of them.  If that means calling the candidate to ask them a question, get on the phone and call.

While most of this has been about the interviewer insuring they are not being lied to, the candidate too should be concerned about the company.  From a candidate side, ask detailed questions, and dig into the details.  What is your bug count?  How much does it vary?  What flaws are there at the company?  What strengths?  And again, don't assume the interviewer isn't embellishing in their answers.  I don't think any interviewer would say "It's hell here.  Go somewhere else."  So investigate the company and the workplace environment.

Ultimately, you have to work with either the candidate or the people interviewing you for 8 hours a day, for a long time, so choose wisely.

Words of the Week: Critique and Criticism

Preamble / Ramble

This is a first for me and word of the week, as I typically consider a word without much personal context, but this week I am going to include a context. While at CAST this last week, I heard some valuable input regarding my blog. No matter how I get feedback, I try to take it and make it useful. I tend to be direct with my feedback, but for some, that can sound harsh, so this week I’m going to consider two different words. If you don’t want to hear the backstory, just skip down to the next section. However, for those that care, I do want to give some context and clarity in my own words. I have now heard from multiple sources that they felt my writing reflected a negative viewpoint towards the Software Testing World Cup 2014. I think that to be an inaccurate statement. I did have some issues with it, and I felt I was doing a deep dive analysis of both the pros and the cons of the contest, up to the point I had gotten. In assuming that I was writing a report about my experience with the contest, going into both the good and the bad. I also wanted to talk about what I could gain from the contest and what could be done better. In my second part I wanted to consider what the judges had to say, so I could learn where to improve as well as note what additional feedback would be nice to have. I understand that this is a contest, but I think the more important piece is what can be learned from it. To be fair, perhaps the organizers didn’t want what I had to say, as they actually asked for:

“At the moment we are very curious about your personal STWC experience. We would love to read blog posts or any other kind of written reports about your participation. It doesn’t matter if you write from your personal point of view or from the perspective of your whole team. We are interested in hearing how much fun you had during the competition, what you have learned from this experimental challenge, and what the difference is between testing while having fun together with friends and testing at work. We are also interested in how you communicated with your team members and what your team’s strategy was. Did you start testing right away or did your team follow a strategy? How did you create that strategy? What was your biggest challenge within your team, what kind of bugs were you able to detect and how did you like working with the Agile Manager tool?”

I took this to mean they were excited to receive any feedback regarding the design of their contest. They also wanted to hear what we did and why we did it. I heard that they were open for a critique of the process so that they could attempt to improve it. Furthermore, we were not told not to post scores, as we felt that it was important that those who participated could compare scores and learn where they could improve.  In fact I encouraged others to post scores, but no one actually did so. We also thought it might add value to future judging to see a relatively unemotional reaction to the scores given. We made the judge’s scores more anonymous as it felt important not to provide any identifiable information. However, it seems some saw what I thought was a critique others felt was criticism. This leads me to the two words, critique and criticism. Having re-read my content I can see that it could be read as criticism, and while I won’t take back my words, I will keep that in mind for the future. So on to the words of the week.

Back to our Regularly Scheduled Program

First allow me to define these two words in the way I think of them, without doing much research nor using Google to define the words. I think of criticism and critique as similar, but with a difference in tone and intent. To criticize someone is to give back negative statements, with either the intent to hurt/harm, with no positive points and to lack any possible ways to improve, be they implicit or explicit. Calling someone stupid is to be critical of someone’s intellectual factuality. It is likely intended to harm, has no positive points and does not hint at a way of improving. To critique is to provide feedback with the intent to improve a person or person’s work in some way. There is a mix of some positive points in the comments as well as negative comments. Gushing praise or all negative points is probably not a critique. Furthermore, at least some of the statements need to be actionable. Critique also feels very student/teacher-y. Criticism on the other hand seems like it refers to bullies and angry people shouting at each other. I want to capture this in a readable way for later usage:

Criticism	Critique	Gushing Praise
Negative comments Intended for harm/hurt No method for improvement provided.	Negative and positive comment mix Intended to improve the content, person(s) or future Provides methods for improvement.	Positive comments Maybe intended for manipulation (even if meant to be ‘improve another person’s day’)

Now let’s see how I did. There are a lot of different people’s views on these words, so let me try to capture just a few of them.

Various versions of Critique and Criticism

According to the presently [August 20th, 2014] most popularly rated stack exchange view the difference is zero. They point out how in 1960’s academics started to try to use critique as a form of analysis rather than meant to censure, but that didn't stick.  They cite several modern examples of mixed usage of the terms. In comparison, Paul Brians, the author of Common Errors in English Usage, says:

“Josh critiqued my backhand” means Josh evaluated your tennis technique but not necessarily that he found it lacking. “Josh criticized my backhand” means that he had a low opinion of it.

Clearly their is some difference of opinion on the meaning of the word. In Writing Alone, Writing Together; A Guide for Writers and Writing Groups by Judy Reeves, there is a interesting comparison of criticism and critique:

“[1]Criticism finds fault/Critique looks at structure

[2]Criticism looks for what's lacking/Critique finds what's working

[3]Criticism condemns what it doesn't understand/Critique asks for clarification

[4]Criticism is spoken with a cruel wit and sarcastic tongue/Critique's voice is kind, honest, and objective

[5]Criticism is negative/Critique is positive (even about what isn't working)

[6]Criticism is vague and general/Critique is concrete and specific

[7]Criticism has no sense of humor/Critique insists on laughter, too

[8]Criticism looks for flaws in the writer as well as the writing/Critique addresses only what is on the page”

(NOTE: I numbered the following quote for later convenience)

It is not completely clear some of the differences between criticism and critique, as I can find fault in the structure of a work, which leaves me in no mans land. Wait… wait…

I want to analyze my last sentence, using the rules listed above. In the second rule, I clearly have a criticism of the writing. The the third rule says if I had asked in the form of a question, if I had said “It appears not completely clear some of the differences…” and perhaps ended with a question mark it would have been a critique rather than a criticism. That appears to me to mean the syntax matters more than the semantic content in the author’s opinion?* Oh, this gets us to rule four. While I made my statement objective and honest, it might not be considered kind. I can’t tell if my statement could be seen as negative or positive. It certainly isn’t positive, so I’m going to read between the lines and call it criticism. I think my statement seems concrete and specific, even if no exact example was given. However, it isn’t full of humor, nor is most of my writing.** The final dimension is about the author. With no reference to the author, I think I am addressing only what is on the page. Ultimately it feels that this set of tools has limited value as it takes too much analysis and when you have 5 / 8 on the criticism side, is it criticism? To the definition's benefit, it does try to capture the concepts I stated early about providing some positive points and providing negative points without beating the subject up. It adds to my definition attempt that you should be addressing the content rather than external influences and adds the idea that the critical comments be concrete. However, it fails to address the possibility of simply gushing rather than capturing the areas of improvement.

Obviously my example is contrived, but look at the above paragraph as whole (yes, I’m getting meta) I would call it a criticism using the rules Judy Reeves developed, yet it wasn’t intended to be. So it feels wrong in some way or perhaps I’m simply wrong. In my mind, she is looking for more positive views rather then stressing the analysis. That is to say the message is less important than the presentation. In my opinion, I’m critiquing the content, because I follow my own rules as well as the new rules I have developed.


* Yes, that was intentionally sarcastic. I hope you can appreciate the joke.
** Ignoring the above *’d item. It wasn’t that funny anyway.*

To Sum Things Up

Some people don’t see a difference between criticism and critique. In more academic circles, it appears that a divide is seen, even if the divide is a little fuzz. Some see it as a difference of purpose, some see it as a difference of the message and some see it as a difference of the content.  How do I see it?  Well, let me capture the attributes I think matter:

Criticism	Critique
Negative comments Intended for harm/hurt No method for improvement provided May focus more on the authors over the content "Censure, attack, abuse and name call"	Negative and positive comment mix Intended to improve the content, person(s) or future Provides methods for improvement Address the content rather than the authors Concrete examples "Deep analysis of other people's work"

In reviewing my own work, I think it falls under the label critique, but I can see how others might read something into it.  Written critiques are hard to convey personality.  You can see that with my footnote joke above.  In going so meta, I was intending on injecting some levity into an otherwise highly intellectual process.  But others might see it as 'unprofessional' or they might read my words as snarky.  In getting my article reviewed, Isaac said he thought it was a little on the snarky side, but that in his view it was just right to make the point.  Outside of getting my content reviewed, I am not sure I can defend against hurting people with a wide audience and perhaps that is Judy's point.  By being kind, you avoid the question completely.  Do you have a critique or feedback for me?  Feel free to post in the comments.

As a sort of post script, I happened upon this site after I wrote this article.  As I didn't want to shoe horn it in, yet felt it was of some value, so I put it here.

A Personal Failure Analysis: Why it appears CDT has failed

Talks & Failure Analysis

For the last few weeks I have been doing one presentation a week to various monthly groups and to a yearly event.  I have one more planned presentation in June, but this really isn't about my presentations, but rather the audience and fellow speakers.  

The first presentation was a meetup group Isaac and I helped form up in March.  Isaac and I gave the talk and it went really well.  Our audience was maybe 40% high-level QA with a few junior and the rest mid levels.  The talk was about testing an object in black box style.  We gave this talk 3 times total, and each time it seemed to go a little worse.  

When we did it with a group of developers, we moved to more white box/boundary testing style.  We had one person refuse to test out of principle.  In part I think us changing the style was at fault, but two sharper developers really got it, so I don't think it was totally our fault.  Two people really didn't get it until I changed boundary testing into a UI exercise.  This tells me even (junior) developers have a hard time visualizing what code does in their head.  

The final audience was a mixture of testers and developers.  The developers were often dis-interested in our talk and several of them left to go see a different track.  Either they felt applying black box techniques was not part of their domain or that they already knew this and wanted something new.  The irony of that is that 60% of the sessions were marked beginner, so learning something 'new' for a mid/senior developer is challenging at best.

I had a second talk which was a shorter version of my intended lecture I will be giving out of the country.  It was around reflections, something I think I have some level of mastery around, so I felt very prepared for my talk.  I will admit my cadence was a bit off, I probably rushed a little too much and perhaps didn't give enough audience breaks, but it was a code oriented talk.  Given the complexity, that might have been a small part of the problem but in reality, most of the audience were either junior developers or students who in their 2nd-4th year of college.  I had marked my talk as advanced but I couldn't control the people who attended.  This caused me to wonder something.  Why is it we are still talking about TDD, Intro to jquery, etc.?  I don't mean to say we shouldn't teach new people, but why are we mostly limited to beginners?

Why do we retrain people all the time?

In talking with 'experts' in the business, it becomes clear that my question of why do we repeat ourselves is because in some communities, such as testing, people don't stick with it in the long run.  Roughly 5 years appears to be the mean time between starting a career in testing and going to something else.  The second issue is that many people are not particularly interested in expanding their knowledge.  They just want to go to work, do their job and go home.  Don't get me wrong, I don't mean to say you should work 90 hour weeks, but many people have little passion for their work or improving their craft.  

Sometimes managers make these engineers look into 'new' development techniques, which create such a long tail into training that something like TDD continues to need talks 10 years later.  

With that analysis, I want to talk about it with an 'expert', and so I found the keynote speaker whom I spoke to at length.  I don't want to cite his name as I have not asked his permission nor do I have an easily accessible method to communicate with him (I don't have his email address and don't use twitter).  

He is a well-known developer who has a pod cast program with hundreds of different interviews with various people within the software industry.  In our discussion, I asked him if he knew Matt Heusser and he said he didn't.  I asked if he knew of James Bach, Cem Kaner, Rex Black...  He knew of no one I asked of, although having reviewed his interviews, the only tester I recognized that he interviewed was James Whittaker, which was many years ago.  I had not asked if he knew Whittaker.  I did ask if he knew any US experts/thinkers in test, but he declined to answer my question, saying he didn't think that way.  Perhaps my question was somehow confusing.

In speaking with this gentleman, a man who clearly had a wide variety of experience, and seemed to subscribe to the Analytical School of Testing, it became clear to me.  One of the reasons we end up not keeping testers around of long is because we as a group are not well known.  James Bach, in spite or perhaps because of his controversy, has one of the biggest names in testing.  Sadly, managers aren't demanding people start doing 'Bachian testing' (even though I suspect Bach himself would object to that name and concept).  We don't have a glamorous single one size fits all solution, like Scrum or TDD.  CDT is so vague no one but its practitioners knows exactly what it is.  It isn't easily measurable like Scrum or TDD which I can say if we are doing it or not (Are there sprints?  Are there unit tests?).  It makes me wonder if CDT has lost its war when managers can't easily measure if there CDT testers are even doing CDT testing, much less if CDT is working for them.  The metrics (be it sprints or number of unit tests) might ultimately not matter, but managers still at least one security blanket before they are likely to invest in structural change.

Solutions?

I'm not sure I have any useful solutions other than to say this appears to be a community issue.  I have tried doing mentor-ships with mild success.  It however doesn't scale beyond 1-2 people at a time.  I have tried doing lectures, but as soon as I stray past beginner style thinking or just talking about the concepts without talking about implementation, it seems to be clunky at best.  I have started a group with Isaac to see if a more personal monthly audience will help.  Maybe there we can make headway, but I don't know.  One nice thing with it being monthly and having multiple speakers, I might find out if the issue is in fact somehow with me and how I am presenting.  Even outside of presenting, maybe the way I communicate poorly and that just needs improvement?  For that matter, maybe I'm using the wrong language (English) to reach my audience of highly-motivate testers and software developers?  Perhaps there are more motivated testers outside of the US and my issues only revolve around where I live?

Perhaps I need to accept that I can only speak to a handful of experts, but how do I find and group them together?  How can the experts then package this stuff up together in such a way that they might learn from my knowledge?

Outside of myself, I know Kaner is moving to a more teaching oriented approach generating classes in which the students come to him.  Having taken a BBST course and talking to others who have, I know only about 1 in 3 (or less) of those students seem to both try and have the skills to succeed.  Some choose only to pass by the skin of their teeth and others simply don't get it.  Maybe some of it is because the course is in English, but it certainly isn't the only reason.  However, 1 in 3 might be as good as it ever gets, perhaps my expectations are just too high.  Kaner appears to have given up on a education only system by adding in certifications.  It appears to me the ultimate answer maybe that one has to market one's ideas more in order to convince people to change.  Perhaps the reason Kaner is talking about moving to such a path is because he failed to market his solution well enough.  Going back the keynote speaker I spoke to after my talk, he suggested that if he didn't know about the thinkers/experts of test or the ideas of CDT, they must not be that important.  Because if you don't hear about it, it can't matter.

Being a Fraud and a Failure

Often I feel like a fraud.  This comes in multiple sizes and shapes and I think it is often ignored in creative industries such as software because we liken ourselves more to scientists rather than artists.  Artists already know the feeling I am thinking off.  Art & Copy had some of the best Marketer's saying things like:

The frightening and most difficult thing about being what somebody calls a creative person is that you have absolutely no idea where any of your thoughts come from, really. And especially, you don't have any idea about where they're going to come from tomorrow. - Hal Riney

and

I think most of the creative people are so damn insecure that they want to think they know everything, but they know deep in their hearts they're just in deep trouble from the minute they get up in the morning. So if you can tell them "that's what you're supposed to be", that's kind of liberating. - Dan Wieden

When I go to test something, I start out with panic for a second or two, wondering what the hell I am going to do.  Then I draw from my experience and start developing tests.  But that panic for someone with less experience might last a good deal longer.  In CDT, there is a sort of freedom given, because 'it depends' on what your context requires.  I sometimes wonder if it isn't that there is a science, a way of telling what is best to do, not because context gets in the way but because what testing requires is as much art as science.  I may not be a pretend expert, but I know much of my creativity in testing comes from some place that is not just from my smarts.  Sure I have science to back up some of my testing, but I also have 'hunches.'

Hunches are not always easy to swallow when I know they could be wrong.  If I only have time for one feature but two features need testing, I could test feature X on a hunch and fail the entire company when feature Y is broken in a way I had not predicted.  Elizabeth Gilbert suggests we simply shift blame from the personal 'I failed' to a third party entity such as a muse, in which a muse hinted to me, giving me a hunch.  I don't find this a particularly satisfactory way of dealing with my feelings of being a fraud, failing at my job.  I don't like putting credit and blame onto a external entity, so I need another way of dealing with my feelings.

Alternatively, I can read testing blogs where we all collectively struggle to find better ways to improve our chances for success.  This can go all the way into the more scientific side to testing.  If only we had enough data, we could kill off the hunch completely.  The Factory School seems to see it as a problem of Taylorism, if only we could just lock down all the changes.  Document everything, know everything, then hunches aren't needed.

The more I research, the more I think our industry is full of fraudulent feelings with different people attempting to resolve them by adopting 'proven' systems and techniques.  They aren't the fraud, considering their sincere attempts to learn, but they have to find a way to limit failing.   So they turn to these 'experts' to solve your problems.  They have the answers, and you should just trust them.  They provide systems and will teach them to you, often for a fee.  However, if you fail, you aren't doing their system right, not that their system is broken.  You are forced to feel like you failed to heed the words taught, and only if you had worked harder, followed the rules closer, etc. you'd have succeeded.  These tools make us feel the failure and not that the tool failed.  Who is to blame, particularly if the tool often succeeds?

Then we move to team oriented solutions such as Agile.  Agile says it is a team effort, so if a bug is found in production by customers, the team is to blame.  Moving from an "I" to a "We", much like the muse.  This is a paradigm shift, even if not a perfect one.  You now can know if you are a fraud or not based upon the feedback from the team.  The problem is you cannot know if the team is filled with frauds, if the team is just being polite or if they are simply more experienced then you are.  To assume the group will be wiser than yourself, and able to judge your success has its limits based upon who they are.  To assume you are able to judge the team also maybe flawed.  To quote Robert Heinlein,

Democracy is based on the assumption that a million men are wiser than one man. How's that again? I missed something. Autocracy is based on the assumption that one man is wiser than a million men. Let's play that over again, too. Who decides?

Every method is flawed.  I personally like the method Art & Copy suggests.  Rather than blaming a third party or putting your faith in someone else's system, I think it is healthy knowing that sometimes you will fail.  That the feelings of being a fraud comes, but with as many wins, as many bugs as you've found, you aren't just pretending to test, you aren't a fraud.  You really are helping the customers, finding bugs early on, helping resolve issues.  Maybe you do fail, but pick yourself back up and do it again.  However, that is personal preference.  I don't think there is an answer.  Some will like science.  Some will like the sales pitches.  Some will like the art.  Some will like the team.  Some like having no right answer.  They are all fine and all flawed in their own way.