My Many Years in Testing and Automation Development

I have heard the question expressed in multiple different ways.  "How do I get into QA?"  "How do I progress my career?"  "How do I level up my skill?"  "You read whitepapers?!?"  "What type of automation should I write?"  I have written various articles around this broad range of topics, but I wanted to take a slightly different tact around one of the implied questions.  The implied question I'm considering is how do I continue down the path from where I am to become a better tester and what skills should I spend time working on improving?  Instead of trying to explore the topic in a general sense, I am going to go through a few interesting experiences I have had in both "Testing" and "Automation Development".  While I will be talking about automation, I do not mean to imply that is where you should spend your time learning, just that I will discuss how I made the choice.  I will try to concentrate on interesting stories and lessons learned.

I have been writing code for nearly half my life, but I started working in the development of test automation somewhere between 2004 and 2005.  I started professionally testing in 2002, although I had been writing my own code and testing it earlier.  I also did some testing for others before 2002, but I was not paid for that work.  I will try to describe the companies and context a little, but I will not use company names, to protect the innocent.  But first, a small introduction to perhaps explain how early on I was interested in QA.

When I started my life in QA, I was roughly 6 years old and curious about this clock in a video game called "The Last Ninja" for the NES.  It kept a timer of how long you had been playing the game.  It gave hours, minutes and seconds.  There was a place where no bad guys were with a stream and a park bench.  It was perfect for leaving my ninja to sit and just enjoy the scenery.  In only 99 hours 59 minutes and 59 seconds I could find out what happened when the clock ran out.  I knew Sonic would run off the screen after 3 minutes, so I conducted some smaller tests.  In my smaller tests, I discovered, unlike Sonic, The Last Ninja did not seem to mind waiting for short periods of time, so it seemed easy enough to test the longer case.  The only problem was I didn't want to lose my Nintendo for a week while I waited for the result.  So when I went on vacation, I left my system running.  When I came back, what I found was "0A" and the game was frozen up.  Now, many years later, I know 0A is hex for 10, likely meaning the clock had run over to hex and then the game had crashed.  I didn't understand the result, but I was curious about it.  I was later grounded for leaving my system on for that long.  You've been warned kids!

Years later, I was looking to get a job out of High School.  I had been programming text adventures for some years, but never had a 'real' programming job.  I enjoyed programming, I enjoyed the experience of creating, but still didn't feel like I knew that much.  Still I kept at it.  I had worked at the Post Office for 3 weeks in a Christmas rush season, but had little other professional work under my belt.  I was going to school and studying Computer Science, but that was going to take years and I needed money sooner.  So I started looking for a computer job.  I interviewed once, but didn't know what I was doing and did not get the job.  College and High School don't seriously prepare you for interviewing.  In my second job interview, I took a computerized test and passed all but one question (I tried typing in a search but they expected me to use the dropdown...  No, I'm not still bitter.).

I got the job and started working as a managed contractor for a overly large company who worked on specialized hardware.  I was a tester who did everything from competitive intel to stress testing to environment testing to configuration testing.  However, I had no training and really no idea that testing was a general skill set.  I incidentally learned a few simple QA skills, but nearly all of the learning I did was around social skills in my first year.  Perhaps one the most important skills I learned was to apologize is important and doubly so when you are a manager and have made a mistake.  I moved to a more complex set of testing, but it was a script based.  I learned more about networking and security testing.  I almost got a job in a team that was inspired by James Bach's work around exploratory testing (in one of the places that James and his brother did some of the development around the idea of exploratory testing), but I missed out to someone with a lot more experience.

Then, in a sad set of events a former lead who changed shifts passed away suddenly.  I took his place, an awkward thing for sure.  I learned about this idea that you could 'automate' tests.  I began to apply my existing programming skills in order to test various UI elements.  It worked well in some cases, but we struggled to handle descriptions (a QTP phrase for describing a way of IDing elements in a UI system).  A co-worker of mine had more difficulty picking up the programming as his specialty was in hardware, not software.  He came into work more than once complaining of nightmares around programming.  My coworker did eventually get better at automation, but that was never his strength (Aside: He's now a big-wig manager, likely making way more than me!).

I bounced around from place to place, with a rough total of nine different jobs (not all jobs were title changes).  One thing looking back is how I was always trying to go meta, not just writing good automation but improving the entire process.  In a large company, that can be very difficult.  I ultimately had a conflict with one of the managers and felt the need to leave.  Before I left, I was told to quit trying to improve the organization (by providing useful tools) but instead work on personal projects!  I did and learned SQL and web development while looking for a job.  I also had multiple open source projects under my belt by this point.

If a big company didn't work, I reasoned that a smaller company would be better!  I applied to one company and studied up.  Rather than applying to many companies, I reasoned it was better to be prepared.  This from having done so poorly years ago seemed to work reasonably well.  I got a job offer. I tried to negotiate for more money, but was declined.  I declined the job but was later called back and given a slightly higher offer.  I concluded that because of the recession that a much bigger offer was not going to happen.

I took a job at a 50 person company, and within 2 weeks of me joining the QA manager had quit.  This left the QA group answering to the VP of Engineering and it remained that way for nearly 2 years.  I worked for longer hours at this company than anywhere else.  I lost another co-worker who I was training, and whom I may have been the last person he talked to.  He was hit by a bus.  I learned the 'hit by a bus' scenario is both real and the worry about how the business knowledge loss will be much less impactful than the emotional impact from the loss.  I learned what the term 'death march' meant as I worked 80+ hour work weeks.  I learned how a recession can affect you in spite of the fact that your field is in demand.  I learned the dangers of saying that you won't ship with any [known] bugs.  I learned how working more hours did not equal more productivity.  I wrote nearly 1300 automated UI tests.  I found the most bugs at the company, in part because of the automation I wrote, and because all bugs must be fixed, it must logically be concluded I found the most important ones.  (Yes, that is sarcasm.)  Yet, I was not promoted nor paid more for nearly the entire 3 years I was there.  Only after 50% turn over and the firing of upper management did things change even a little, and by then I was too tired to care.  I left.  In case you want more details, Isaac also detailed his own personal journey at this same company.

Now we are getting into more recent events.  To be perfectly honest, I don't plan on telling anything more about the various company cultures because of how it might affect me.  However, I still learned social and technical lessons.

I learned that 'manual' testers could appreciate automation and automation results.  I, for the first time, got to work with other people with equal or better coding skills than myself as we developed automation.  I got to learn how to play a technical leader, even when I was not promoted into a leadership position nor had any authority over others.  While I had dealt with databases and did a little database testing before, I got to do development in which nearly all the code was  around database usage, including communication between database systems.  For the first time I had to deal with the integration of multiple team's automation efforts which were written in different languages.  By this point in my career I had written test code in C, Java, JavaScript, SQL (various flavors), C#, J#, VB.NET and VB Script.

On the social front, I got to work with both one of the best teams and one of the most difficult teams I ever worked with.  In dealing with the difficult team, I got to see how personality clashes worked and how to work around them.  To be fair, I have my own personality quirks too, but when these hit extremes it is imperative that you figure out how to be a positive force for the team rather than yet another point of contention.  Perhaps one of the biggest lessons I learned was that it is important to have a solid social connection and understanding of an individual before giving a completely honest evaluation of someone's skill set.  In fact, I took the lesson so much to heart that I try to be more careful with my day-to-day speech as a whole.

Finally, I want to describe a little bit of my attempts at professional growth.  I spent a good amount of time embracing various avenues for learning.  I took BBST, and while I only took the class for the first BBST course, I watched all the videos and read all the material.  I have written on this blog for a few years now.  I have spent time giving talks at various venues.  I have written letters, written for other professional venues, testing non-work apps and taken serious amounts of time trying to understand the systems under which work is performed.

So to sum this all up, I think I have just two simple suggestion for you:

Learn

Making mistakes is fine.  Just learn from them.  Learn how you learn.

Don't just do work

We all are making a journey, and at some point it will stop.  Making an impact is important.  But don't make work the only impact in life.

Autonomation: Old is New Again; Toyota and Lean

I came upon the word Autonomation recently and felt it was interesting enough to bring it up.  The concept comes from Toyota as they were developing Lean Manufacturing in the 1990s.  The primary goal for Lean Manufacturing is to eliminate waste and thus improve production and quality. Autonomation is also referred to as jidoka in Toyota's TPS process.  Autonomation or jidoka is one of the layers in Lean Manufacturing, meant to trap failures rather than produce results.  The earliest example Toyota notes is from 1924.

In 1896, Sakichi Toyoda invented Japan's first self-powered loom called the "Toyoda Power Loom." Subsequently, he incorporated numerous revolutionary inventions into his looms, including the weft-breakage automatic stopping device (which automatically stopped the loom when a thread breakage was detected), the warp supply device and the automatic shuttle changer. Then, in 1924, Sakichi invented the world's first automatic loom, called the "Type-G Toyoda Automatic Loom (with non-stop shuttle-change motion)" which could change shuttles without stopping operation. The Toyota term "jido" is applied to a machine with a built-in device for making judgments, whereas the regular Japanese term "jido" (automation) is simply applied to a machine that moves on its own. Jidoka refers to "automation with a human touch," as opposed to a machine that simply moves under the monitoring and supervision of an operator. Since the loom stopped when a problem arose, no defective products were produced. This meant that a single operator could be put in charge of numerous looms, resulting in a tremendous improvement in productivity. - http://www.toyota-global.com/company/vision_philosophy/toyota_production_system/jidoka.html

The term Autonomation feels similar to the term I coined perviously, Manumation.  In the wiki article on Autonomation, I found it interesting that Shigeo Shingo claimed there were 23 stages between manual and fully automated processes.  Unfortunately, there is no citation for where that claim was made and while I saw others repeat the claim, no one had any citation nor data on the stages that I could find.  In my mind, Autonomation is just one form of Manumation.  However, it is also an attitude.  You don't have to try to fully automate something on your first attempt to create automation.  The idea that you set your automation up knowing it will fail, that it will need humans but don't bother the humans until it fails and that the failure is easily traceable and fixable.  Also, it means attempting to fail quickly rather than generating a bunch of waste work.

Ultimately, automation of any sort is meant to help people.  If it helps you get work done, even if it requires a human touch, it is worth considering.  What sort of autonomation have you done?

Happiness: On the Job and in Life

In this post, Matt Heusser does a great job describing some of the problems around hiring older workers and some of the reasons older workers become less valued.  Obviously, this is a concern for all technology workers in the field, regardless if you are in test, operations or development.  I have not had much experience with it, but I do feel some of the call that those who are in the 40s and 50s start to feel.  I don't want to invest hours upon hours doing development outside of work while working 40 hours a week.  This blog, which I write for often, requires time which I could use to get money from somewhere else.  I don't get paid to do this.  Sometimes I clarify my own thoughts, or use this to prompt me to learn, but more often than not I am actually trying to teach others the hopefully valuable information I have.

However, there is a underlying subject I think is more important than just ageism in the work place.  I want to talk about happiness.

First, a few words, I am not a doctor of any sort, I am not giving advice for depression and if you are experiencing depression you should see a doctor.  One other thing, I'm going to ask you to resist clicking all the exciting links until after you've finished reading this article, as there is a metric ton of data on the subject, but interrupting the narrative will likely distract you.  Now on with the show...

Were We Happy In The Past?

In the past, almost everyone did farming.  There were a few other jobs, like tailor, doctor and merchant, but most people were farmers.  These jobs were fairly well known and a town depended on one (hopefully) competent doctor, one tailor, etc.  A large town might have a few of each, but only in the cities were there really any large quantities of any specialties.  These specialties were all relied upon and considered vital.  While there was anger around the usage of power and prestige these ranks sometimes bestowed, in general they were good jobs.  Life could be brutish and short.  However, the idea of manifest destiny  and excitement of the renaissance show that life in the past could be seen as meaningful.  Is being a farmer happy making?  Or being a doctor or ...?  Sadly, there is little documentation I could find about happiness in the distant past.  However, I did find lots on various problems we see an experience today.  It seems that the general fighting we have today was going on in the past.

It is strange that there should be so little reading in the world, and so much writing. People in general do not willingly read, if they can have any thing else to amuse them. -  Richard Burke to Samuel Johnson, 1783.

The number of technical students has declined constantly... IF THE SAME TEACHER CAN INSTRUCT 500,000 OR MORE [students] SIMULTANEOUSLY? - Radio Electronics, 1956, http://www.americanradiohistory.com/Archive-Radio-Electronics/50s/1956/Radio-Electronics-1956-05.pdf

So while I have been unable to find anything quantitatively correlated to happiness in the distant past (and only a few data point in the recent past), it does seem that the same challenges facing us have been unchanged in the past few hundred years.  We are constantly looking for productivity changes while desiring to keep our hard gotten gains.  Furthermore, while I have yet to read this tome, Steven Pinker suggests that we were actually more violent in the past.  So if violence decreases happiness, which I have found lots of evidence for, it would appear there is little basis for the assumption we were happier in the past.  In fact, it appears arguable that collectively we might be over all happier now then in the past!

Do Jobs Affect Happiness?

The next question is, do we know if farmers of today are happier compared to the general population?
In a word, no.  Happiness and job are nearly unrelated, however there are a few key factors that matter.  One is that you have enough money to survive and not worry about where your next meal will come from.  Another is having meaningful work and yet another is having autonomy.  There are also some negative correlations with jobs.  For example, not getting enough sleep or too much mental stress (some physical stress is good for you).  Lacking creativity can be an indirect form of mental stress as well.  Often this comes in a form known as boredom.  Basically, I think I can sum it all up in a single sentence:

You want a job that pays a living wage with difficult but surmountable and clear goals in which you have enough control to get it done and in which you will get back useful unambiguous feedback and time to recharge.

Great, we're done, right?  Well, I want to explore some more topics around this and how I think the reason our job does not seem highly connected to happiness.  The first is that we don't know how to practice being happy, so why would ones choice of job have an affect?  That is to say, if happiness is fairly mysterious then choice of job will appear to be a random or not related to happiness.  So how do you practice happiness?  For that matter, how can we discuss happiness when there are so many varying view on how to create it.  Is it something that comes without you noticing it or is it something you can actively seek?  If significant increases in wealth doesn't change happiness but for a small and short period of time, what does?

Rather than going on with so many links, I'm going to stop citing so many sources.  The purpose of this essay is not to do research for you but rather to explore the subject and not just trends I have seen in the research.  Furthermore, I wish to explore how to have a happy life, not just being happy on the job.

How We "Need" Society for Happiness

Helping others can, in and of itself hurt those we hope to help, without regards to the permission they give. You might have heard the phrase "What doesn't kill you makes you stronger." The correlation to that would be, "What makes life easier makes you weaker."  Obviously things like chronic pain and long term mental stress eventually do damage to a human being, but in general these two ideas have some truth to them.  Consider the case of the blind, a blind person might appreciate being given an 'extra hand' but at the same time, that might also make them less able to do things on their own when that helping hand isn't around.  Married couples are known to have slightly lower IQ than the unmarried because of dependencies on each other.

In a sense, as John Donne said, no man is an island. That does not equal a learned dependency in my book. Babies have real dependencies. We are dependent on doctors during surgery. We depend on lots of things. However, learned dependencies are not things we are incapable of doing. Maybe we don't want to do them or maybe it would cost too much of our time, but a learned dependency is when you quit trying because you're not expected to try. I suppose you could say I quit trying to change my oil because Jiffy Lube says I'm not expected to try. However, I would argue that my dependency on oil changes is more a matter of time, energy and my personal interest. That being said, it is a fine line.

In regards to happiness, is happiness truly just an internal thing or does it have external dependencies?  Can you live and enjoy pure silence, or do we really need music?  It is an interesting question, but even if you depend on external things, they may not really require other humans.  Enjoying waterfalls or existing books for example require no (new) human work.

Learned dependency, be it having someone else get you around, using a calculator to do math or depending on others for happiness is a problem because now you need something else in order to do the things that you want. I don't know how to give an algorithmic way of judging when it is enabling and undermining independence and when it is a genuine hand up.  Perhaps it is even in part personal taste, as the hermit choose no or few dependencies while the city dweller depends on many people for city services.  The hermit might even see those city services as enabling the city dweller, as the city dweller needs the bus in order to go see the opera they desire to see.  That is always a struggle, and has been one for ages.

If our dependencies on others needs to be managed, it seems only reasonable to ask about the other way around.  How do I prevent asserting myself into other people's need for genuine challenge? The Bible famously asks the question:

Am I my brother's keeper? - Genesis 4:9

In that case it was a way of ignoring responsibility for the murder of his brother, but the question of responsibility and social value for watching out for others has been around for centuries.  I don't pretend I will answer the question today, but I think it is a balancing act.  Rarely do we know what we really want. Our simulators for happiness are poor at best. Dan Gilbert talked about that in his TED talk.  So if we have a hard time knowing what will make us happy, pretending we know what other people want is silly.  This can become a wicked problem when what you want to do is help others but worry about foisting your culture, will or opinions on others without giving them room to grow.  Teachers suffer this question all the time.

 Can Happiness be Treated as a Duty?

The Stoic's claimed virtue is sufficient for happiness and that virtue is created by a will that is aligned with nature.  Kant's Categorical Imperative, in a very rough and simplified form, linked duty to what you imagine a reasonable person would do.  So unless you imagine a reasonable person would choose to be unhappy most of the time, it would appear happiness might in fact be a duty.  There are other philosophical theories like Hedonism which say that you should look to maximize long term happiness.  That is to say, have the day to day discipline to make your life better in the long term.  All of these various ideas about optimizing life with consideration for happiness seem to have something of a pattern to them.  They all push us to grow.  Be it via virtue, by considering how a reasonable person would live or look at the long term and attempting to minimize pain.

So how do you generate growth?  Well one aspect is to not be in fear.  Fear often is the opposite of growth.  You need to feel safe enough that you can work.  Even when in fear, such as when in a castle surrounded by an army, you have enough pause to feel safe enough to gather your wits and think.  Safety by itself isn't enough.  While safety is important, you have to be safe to try experiments, pushing yourself beyond what you have done before or in new ways allows for the growth, which leads towards happiness.  The feeling of growth is in my view, at least one of the causes of happiness.  However, if one only attempts to grow in one area, it can become an addiction.  Growth also has to relate to goals, and in particular unending goals.  Goals you can meet can later feel unsatisfying.  For example,  when you think of buying something, you feel more satisfaction then after you have purchased the item.  Since having meetable goals is less likely to create happiness, how do you create a unmeetable goal?  You could say, "I will weigh 150 lbs in 3 months," but this is an unsatisfying goal.  If you make it, maintaining it is much more difficult.  Instead, I will go to the gym 3 days a week remains satisfying because there is no 'end point'.

In addition to safety, you also need time to develop happiness.  You need to have time to work on your goals, or else you stagnate into surviving life.  Time can mean exact and dedicated time, such as a day of the week or hour in the day, or it can mean just having open variable spots.  Time is perhaps one of the trickier things to give detail on because different people like doing things in different ways.  Some people like things well planned out while others like to be more sporadic.  I have not seen evidence demonstrating that one is superior to the other, but personally I rather have some vague planning rather than chaos or an exact fixed schedule.

While the goals don't change often, they should be allowed some variety.  If you always go to the gym on Monday, Wednesday and Friday, you may end up burnt out.  Instead, you probably should have some time allocated to 'breaking' the goal.  Going to the library once a month instead of your Wednesday workout.  You should feel the pull to get back into the routine which helps make the routine more satisfying.  If you don't feel excited to go back into the routine, it demonstrates something is wrong with your goal.

Finally, you need to be able to see that these goals are working.  Without being able to check up on your goals, without being able to study the results and see if they match expectations you get little satisfaction from 'achieving' them.  While weighing 150 lbs might be a very watchable (E.G. an easy metric), you don't need that to see if going to the gym is helping you or not.  Since using easy metrics tend to make goals that end, those are likely not the first type of metric choices to make.  Saving enough money to buy a house is an admirable goal, but learning the habit of saving should be the real goal.  So using 'enough money to buy a house' as the measurement means you're probably watching the wrong thing.

Now if you have been reading this far, you'll probably notice something interesting.  I'm going to break a general rule I have of not copying and pasting and repeat myself from above.  Basically, I think I can sum it all up in a single sentence:

You want a job that pays a living wage with difficult but surmountable and clear goals in which you have enough control to get it done and in which you will get back useful unambiguous feedback and time to recharge.

If you noticed what I wrote in the job section and in the life section, you can see the life section is very close to a repeat.  In fact I would use the job section as a sort of check sum from what was discussed in the broader life category.

The idea that you can treat happiness as a duty is not exactly correct.  Instead, just like with getting a job that creates happiness, you need to do similar things in life.  It maybe formulaic sounding, but it is more complicated in one's actual life than any author can generalize.  A duty implies it is required, but this is more of a method, backed by out current science and history.  Can other methods work?  Maybe.  Is what I have written 100% correct for your life.  Very likely not.  All situations vary.  But the general ideas seem to remain consistent throughout time.

Disagree with me?  Well, my goal is to engage my readers, so please leave a comment.  I'll be happy to reply.  Oh yes, and now feel free to go click all those links and enjoy all that awesome data.

Book Consideration: Tools of Critical Thinking: Metathoughts for Psychology by David A Levy

“Cogito ero[sic] sum” – Rene Descartes.* Those were the first words I saw in the book that got me excited to read it, although disappointed at the typographical error. As I made it through Tools of Critical Thinking, I found much of what the book describes I already understood, and much of it came from a class I took years ago in philosophy and logic. That being said, it did give me a little bit of thought in regards to psychology. For example, the book talks about how words have implied value. Other concepts include how concepts are not things, there are different levels of an idea and naming something does not imply you understand it. These ideas are all things I understood reasonably well. Then the book seemed to have started pulling quotes from Slashdot, with the whole correlation is not causation. Literally five chapters were dedicated to this one concept, and I think that is perhaps not excessive, but for me, it felt like an “I already know that, teach me something new”.

As I continued my personal journey through the book, I found one very interesting new concept I had not heard of before. Fundamental Attribution Error. Sounds a little complex, but it really isn’t that hard to understand. Roughly, it means that we tend to overestimate a person’s personality and underestimate their situation. Strangely enough, the reverse seems to occur frequently when you are looking at yourself.

The author also talks of other subjects like extraordinary events and how given enough events, you would expect some to occur. One case I experienced involved me leaving a parking lot following a person and then, about an hour later, on the way back from lunch on that same day, being followed back to the same parking lot by that exact same person. I found it really funny, because of the impossibility to it, yet it happened. That being said, I don’t seriously take it as anything but a fun and funny event. At the time this type of event occurs, perhaps it is reasonable to have a little “brain breakage” and see it is something more, but one should understand that random event occur throughout our lives and within chaos patterns emerge (see fractals).

Mr. Levy continues on about deductive and inductive reasoning and what poor conclusions can be made from what appears to be reasonable reasoning, which again is covered by logic 101. My favorite flaw goes something like this: Spaghetti is a food therefore I am right. Basically, if you have a true premise (Spaghetti is a food) you can conclude anything and claim your logic is infallible. There is also inductive logic where you base your conclusion on a single specific type of evidence. This has its own set of flaws, like a low sample size creating false new knowledge.

Other topics of interest include the idea that an observer affects the observed (E.G. To Grok; See Stranger in a Strange Land). We apply schemas to all sorts of data, from gender roles to mental illness to height to various other items. Most people have a personal investment in their beliefs (strangely, belief isn’t defined). To know and label something is not to solve the problem (which should be obvious in the world of QA, but think about it with how you interact with people).

Now I want to circle back to my own job, testing. Is this a testing book, does this book have value in testing? I think the answer is yes, but only if you don’t already have these logical concepts already well understood. Not to say a review doesn’t help, but honestly I think one could get much out of reading a logic book. That being said, I suggest one read the chapter on schemas, confirmation bias, deductive logic, as they apply very nicely for much of what we do. Let me give an example of how this applies to my job using the chapter on representative bias. This is one chapter I have some issue with, not because it is wrong, but because I think it is not a complete picture. The book talks of assuming things based upon what you are observing (for example a person you are observing). As a real life example, if you have found one developer typically does good work, you might choose to do less QA for work they do, but is that because of personal feelings or hard data? I don’t have hard data, but I can say I never regretted choosing to do less QA on one particular former developer I did work with and instead spent more time focused on other developers. Is that in fact a confirmation bias on my part? I think not*.

One last subject I would like to talk of is schemas and how we develop them as people. I have had a long running set of conversations with a friend on the value/cost of applying a “Schema” (labels) to people, ideas, groups, etc. The obvious pro in my mind is that the schema provides a way of connecting us together. That is to say, we can learn new data by applying old schemas to new situations. For example, I could say that males tend to be more aggressive and more aggressive people tend to play more sports**. From this you might apply a schema pattern match on a male athlete and “guess” he is more likely to be aggressive. Now, you have no evidence for this particular case, which is where testing comes in (in my mind) to confirm or deny that in a particular case. The trick is to keep an open mind and understand that no matter what your judgment shows, it could be wrong and to accept it.  Even when you have a well developed schema, there is likely to be some outliers which you need to expect and be flexible enough to believe exist. If you don’t keep an open mind, you can end up dividing people into us vs. them, which does not provide real long term value to humanity.

The other approach is to say, “I simply don’t know without direct observation. Even then, my observations might be flawed, and without reproducibility, I know nothing.” This method (to me) is also reasonable, as it has no value-judgment and thus requires no schema. In other words, every situation is unique, so a schema is too rigid for the real world data. The problem I personally have with it is it also provides no structure, possibly no value, any way of generically apply new knowledge and no hope to really learn. In a very real sense, it is giving up on gaining structured knowledge, because your knowledge will always be limited, flawed and low resolution. To be clear, and fair, I think that it takes an amazing amount of will to refuse to judge and simply keep an open mind since anything is possible. Just the pure discipline might make this method worth committing to.

I could go on for pages on schemas, but I think that if you don’t understand what you are doing when you sort types of knowledge, you really need to read this book.  Also, if you exchange the word “schema” with “database design”, it also comes out as an interesting talk on SQL vs. No-SQL solutions.

In summing it up, I think it is a handy book for a QA/Test professional, handy enough I bought my own copy of the book.

* This reminds me of a joke I couldn’t resist not including. Rene Descartes walks into a bar and the bar tender asks if he would like a beer. Descartes says, “I think not!” and disappears in a puff of logic.

** This statement maybe untrue.  I ask you to apply your own reasoning and conclusions around the subject.

Expectations...again...

This was written a long time ago...but finally edited to readability.

So I just finished the AST BBST Instructors course. In it, we started by introducing ourselves. I of course summarized my introduction from this blog cause I was being lazy, and it said what I thought I wanted to say.

In response to that I was asked:

"It sounds like you have the confidence and experience to be a manager, but I wonder if this might make it hard for you to teach? What do you think?" - Another Student

I responded with:

"Yes I find that my mental model has a hard time bonding with people who aren't also self-starters and/or newbies. However I've been specifically working on how do we 'as a testing community' get the first level of people into testing. I still have a problem dealing with people who just want to be spoon-feed information.
My experience is mainly in taking people with 3-5 years of experience and tuning (yes tuning) them into the next skill they need to be the best at their current position. However so far this has been limited to where I work, as I usually understand the context enough to say "if you could do this" it's the next skill you should learn.
So part of my goal is to be able to teach less experienced people. But I still struggle with if I should TRY to deal with non-self-starters…"

I’d like to take a more detailed look at my personal thoughts on the matter. Specifically around, “Does your own attitude and personality cause you difficulty in teaching people?” I’m going to try and detail an internal dialogue I’ve been having about this lately. I first tried to sum up some feelings in an older blog about Expectations.

This is what I mean when I say, "not-self starters". There have always been people at places I've worked that are comfortable doing just the basics, show up at work, do their job, go home. These are the people that when asked in an interview "What do you do to improve your skills", have an answer similar to "On the job training". These are the people that don't seem to want to take time outside of work to improve their skills (which to me translates to "I don't consider this job a career"). Some of these people seem bright, others are merely content at getting along and that's enough for them.

Then there are the self-starter people who will go out of their way to learn new things. These people ask you questions, and when you say, "tell me what you know about it so far", they blow your mind with the research they have done. Or alter your understanding of the subject with some new pieces that you haven't heard or seen of yet. Sometimes at a minimum they have clearly researched and/or understood the material, but don't change your level of knowledge.

I know mentors / leaders want to use terms like taught, educated or instructed. But when you have self-starters, that isn't the appropriate wording. The closest I've come is tuning them. When I chat with a self-starter cause they are asking "What should I learn next?" (and them asking is one of the key points). There are a couple of ways to encourage them. And generally that is all you can do. Point in a direction, say 'that way', and then get out of their way.

• Have them learn a new skill. Sometimes these are easy for beginning self-starters. Don't know SQL, yep I have yet to meet any tester that doesn't benefit from knowing it. As the self-starters get more and more experience, this can get harder and harder to find for each individual. Currently I've had serious success with just allowing them the freedom to find new things. This is the real power of the self-starter, they aren't okay with sitting idly by and surfing reddit, they WANT to provide value, they WANT to solve problems.
• Have them level up a skill they are already strong in. This helps those who have just finished something rigorous, demanding or seriously mentally intensive. (This can be work related like finishing a project or a mentally challenging class, or a non work related life changing event.) It allows them to learn something, but with the permission to not be as intensive with it. An example would be having someone learn the singleton pattern, instead of a whole new programming paradigm like LISP.
• Have them up a skill they are weak in. I don't normally recommend this for very many people. I tend to follow the Good to Great idea, that weakness isn't inherently bad. But there are genuinely some weaknesses you need to either compensate for, or bring up to a minimum bar.

Sometimes even that isn't enough. I recall one situation where I was attempting to tune someone I classify as a full-blown self-starter. We were going over the OSI model, and I was attempting to explain how each layer could be an attack point for testing. Unfortunately we were not speaking the same language, after about 30 minutes, they were frustrated and on the verge of crying. I was frustrated and not understanding why they couldn't get the 'simple concepts'. We parted that day, and I don’t recall us ever trying to train together again. I took this as a personal failure…how could I not explain such simple concepts (which OSI is and is not) to someone who tries really hard when they self-educate and generally succeeds? It took me a long time to understand that I had a base of knowledge about computers that they had never had exposure to. Bad on me for assuming what they knew. The problem was that I know stuff they didn't. They also know stuff I don't. It takes time and effort to truly get to know people. I hadn't taken the time or maybe didn't have the perception to understand that they had no idea what I was talking about. They didn't have the trust to tell me 'What the hell are you talking about'.

S'long as you learn to temper what you tell people you want, with realistic acceptance of what people can really accomplish. It all comes down to do you trust your people to work hard, do they trust you to not hose them with unreasonable expectations. It takes some serious time to build that trust. It's why you see people follow a strong leader around to companies.

If you don't fall into the category of self-starter as I've laid out here. There is still hope, you can become one. Start today, motivation breeds motivation, find something interesting, and learn it. When you're done learning that, find another. Then repeat ad nauseum.

I've going to wrap up and caveat this entire article with a thinking exercise for you.
Can your expectations be too high? Is this a bad thing?
How else would we ever achieve something new?

The Future of Load Testing

First of all, I won't pretend I actually know what the future of Load Testing will look like, but I want to describe some of the different ideas I have seen and done.  Some of these things I have not seen or heard of by anyone else, certainly not on the internet.  So hopefully these will expand your thinking around Load Testing.

What is the Purpose of Load Testing?

Functional testing is designed to exercise the functionality of the system.  Frequently people talk about using Selenium or QTP to exercise a particular piece of functionality, often in the UI.  With the Test Pyramid it is suggested that these sorts of tests should try to hit below the UI level for various reasons.  No matter, if you test an API, a UI, Console App or some other hook below the UI, if your concern is around the Testing Pyramid, it is likely you are trying to test the functionality.  You are often interested in large sets of behaviors and the way the system responds.  When you do Load Testing, in most cases, few of the broad tests are as important.  You are not trying to see if the system will handle all the corner cases and often Load Tests don't even check anything besides a response code.  The raison d'etre for a Load Test on the other hand often has less to do with how the system functions but in ways the system reacts to many different inputs occurring in a relatively quick fashion.  Granted some Load Tests are less about the number of inputs and more about the style of the input (E.G. large files) or other types of constraints (e.g. with less RAM).  To sum it up in a general statement, Wikipedia describes it this way:

Load testing is the process of putting demand on a system or device and measuring its response. - http://en.wikipedia.org/wiki/Load_testing

However, the majority of Load Tests simply want to understand how many inputs a system can handle given a certain profile of inputs.

All of that sounds rather abstract, but if you go reading Microsoft's very hand guide on types of performance tests, you will see that the underlying purpose of this sort of testing varies.  They use terms like load test, performance test and stress test for different meanings.  I think all of this data is really useful and valid.  I however, am going to use Load Test (in capitals) to describe any number of different purposes.  Instead, what I want to look at is what sorts of ways we can organize our testing to make for a better long term experience.  These ideas could be applied to many of the various purposes of Load Testing, so assuming you understand your Load Testing goals you can tailor these ideas to your organization.

35-50% of the Internet Traffic

The first idea I think worth exploring is the Netflix model of Load Testing.  Effectively, trying to have a production-like QA system is silly for Netflix because that would be like having a second set of the internet for QA.  In fact, if you were like Netflix, you would then have to have an insane number of additional systems to generate a load anything like your customers...  or you could just have production traffic mirrored between the two systems.  Having a second prod-like environment is of course not going to work.  So they came up with a radical set of strategies, but I think this sums it up nicely:

We have found that the best defense against major unexpected failures is to fail often. By frequently causing failures, we force our services to be built in a way that is more resilient.  - http://techblog.netflix.com/2012/07/chaos-monkey-released-into-wild.html

The best method they found was basically to attack production and see how it would respond.  This however still leaves the question of how do they specifically test their code will handle new loads.  While they do some Load Testing, their biggest defenses are the scaled down production traffic mirrors they run, the ability to go back to old code and the fact that AWS allows you to spin up new instances all the time.  In effect, they turn the problem on its head, but this really only is helpful if you can use AWS to grow quickly and if your traffic is relatively uniform.  Also, at Netflix scale, you can hire a lot of engineering talent to build this up.

The Limits of Load Test Systems

When trying to leverage user traffic doesn't work, you have to start looking for other options.  Using something like JMeter is interesting.  I call out JMeter because that is what Netflix uses above and beyond their production traffic mirror.  A tool like JMeter takes traffic from a proxy and feeds it into a script (I guess you could hand code it if you are crazy).  Then you edit the script and parameterize it.  You run the script over and over again using multiple threads and try to create load.   These tools might instrument the systems under test or you might have to do that yourself.  In either case, the data gathered is outputted and left for some poor soul to try to understand.  After having been in this position several times, let me say that it truly is difficult to understand these results, in particular because you had to use record-playback from a proxy.  Just as it is a bad idea to use record-playback in automating your functional tests, I think it is a bad idea to do so with such tools.

In one of my former companies, they had one specialist whose only job was the deal with these proxy-recorded scripts.  They are a mess and I'm not going to pretend I know how to fix them.  However, I do have a few ideas and all of them involve your already created friends, the functional tests.

Functional Tests == Load Tests: With or Without UI

When you have a functional test, you might have a complex setup and tear down.  However, the test itself is often fairly simple.  There are two major ways you can create functional tests, one is using a UI and the other is to hit just below the UI, perhaps at an API level.  So logically you can do two things to create load.  One is to scale up your UI tests.  I have seen this done and know of others who have tried to do it.  It is a fairly big engineering feat to create a Load Test using Selenium, but I know it can be done.  Be warned, this can be very expensive, as it requires one OS per hand full of threads you want to run plus overhead for selenium hub nodes.  The other option is to use your existing API tests and create a Load Test overtop of that.  You might have to simplify the data creation and you might have to remove the validation if those take too long, but this is a very easy method of Load Testing the system.  I have personally built several Load Tests around this idea.

Now we have talked for years about how functional tests should be run in a CI environment.  If you are writing your Load Tests like you write your functional tests, using the same systems, then why not run your Load Tests nightly?  Obviously there are some questions you want to ask up front, like if someone will be alerted because of it or what impact that has on your functional tests.  Another question you might consider is what sort of load do you want?  If you need to actively watch to make sure the system stays up, a nightly Load Test would not make much sense.  On the other hand, what if you did a small load for a short period of time?  You could capture the resulting data and plot it on a chart.  This way as you gathered more nightly data, you would have a rough understanding of what you expect.  Now, you aren't running a Load Test once a sprint with little idea of what changes might cause impact, with specialized scripts that take a lot of effort to maintain.  Instead, you have a trend line and will notice changes.  Now this won't tell you when you will fall over or some other data points, but it does give you a change detector.  Furthermore, when you have to fix your functional test, your fix automatically goes into the Load Test.

The next piece is you can run multiple 'threads' at once, not all of which are load related.  If your Load Test can't do validation, while running the Load Test you can run some functional tests to see if the system still appears to function.  Since this is a job in your CI manager, it should be easy to kick off.  You can even manually test while your Load Test runs.

Eventually, you might realize that your functional and load tests only vary in how much load there is and how complex the setup/validation is.  You might realize, like the Load Test, you can metrickitize the functional tests so that you see if a functional test start taking longer.

Load Test?  What is that?

If you notice my description, my efforts are to make the code base easier to maintain and have as much data as possible.  This ultimately makes your Load Testing efforts and your functional efforts look very similar.  They live in the same code base, call the same functions, effectively the concepts merge.  The only differences are in the design, setup, cleanup and how heavy the validation is.  I do think there is value in having different words to describe the intent, but merging the code allows you to get more done with less resources.  One of the biggest advantages I have personally seen is that I actually understand what the test does, where as when I was using tools like JMeter, I often had no idea how it worked.  I have learned a ton about HTTP and HTTPS just by building my own tools.  Now not everyone has time for that and I think ultimately we will want tooling to help make this easier.  However, I am not sure if the costs of having a different tech stack and code base is worth the value the current tools provide, so you might have to make your own for now.

If you have found your Load Testing tools are working well today, then feel free to ignore this.  I know not everyone has the same needs as we have.  I know that the tools we have today do serve some purposes, but my experience hints that they frequently are as much trouble as the value they add.

"In the Year 2000"

- Conan O'Brien, et al.

In trying to predict the future it is really difficult to say what will or will not happen.  Conan O'Brien has been predicting what will happen in the year 2000 for more than 15 years, but unlike him, I have not had the benefit of seeing the future.  With that said, I suspect that in the future we will see more machine learning style systems that take our Load Test data and create load profiles based upon real time data.  Such a system would be able to adaptively adjust based upon metrics in the systems under test and would also detect what changes happened and what code appears to have caused a particular slowdown.  It will correlate this and help find what is causing systems to fail.

I also suspect that we will have a better set of load profiles that can push or break systems.  Load Test systems of the future might go through those profiles on a daily or build-basis and inform you when a build/day has uncharacteristic results, again based upon some form of machine learning.  It will start looking a lot more like functional tests which you only examine when something strange occurs or when you are auditing your tests.

Obviously all of this takes some fair amount of work and effort to produce.  We presently don't have the tooling to do this and while bits and pieces have been worked at, I have heard of no one actually doing this.

What sorts of things would you like to see in future load testing systems?  What areas have you struggled with?  I have found very little experiential data around load testing.  I'd love to hear from others on this topic, even if you just dump a link to your blog post.

Tainter's Composite

Introduction

First let me start by defining my terms.  Dr. Joseph Tainter, is an anthropologist who looked into the question of why societies collapse.  A composite is combining multiple things into a single image.  I believe using Tainer's mode of thinking, one can create a model or system including for software and organizations.  This model, using a lot of systems thinking includes the work of the Dreyfus model, The Gervais Principle and general economics, but the framework all rests on is from Tainter.  I will explain Tainer's ideas, however, if you don't know about the Dreyfus model or Gervais Principle, you will need to read up on those before continuing further.  In fact, without those you will be lost.  I realize it is a huge amount of reading just to read this post, but I promise you it is worth your while if you want to understand the inner workings of business culture.

Tainter asserts that societies become more complex as the societies needs for solutions to problems increase.  That is to say, in order to solve a problem, the society adds complexity.  For instance, when a tax loophole is discovered, society might create a rule around that loophole to end it.  This complexity means that someone must create the law, the people who are affected must learn of it, there must be a means of enforcing the law, someone has to interpret the law and when violated someone must enforce the penalty.  Most laws might add minor complexity, but at some point the laws become uncountable, making it very difficult to follow the law.  Now this by itself might not be a problem, unless the value received from additional complexity declines compared to the cost.  When enough of these sorts of mounting complexities cost more to the society than the society produces, eventually the society will fall.

A Corporate Example

Let me consider a different form of this, one that is very easy to understand.  Let us suppose that you have a piece of software that requires 9 engineers but has an income stream sufficient to pay for 10 engineers.  Customers keep demanding new features, and the complexity of the system rises.  Each new features has both cross-cutting concerns and interlaces with existing features.  More bugs are found, but few new customers are added.  Eventually, those whom knew the project will leave, but the complexity of the project does not.  The company finds itself needing more engineers in order to support the product.  The new engineers conclude that the best thing to do is to rewrite the application.  Even if you ignore Joel's words (from the last link about refactoring vs rewrite), you know that then they had to hire even more engineers to support the old application while the rewrite was in progress.  The company had not been making that much money, didn't appear to be able to make much more and so the obvious thing to do is the close shop.  A sociopath from The Gervais Principle would clearly do so.  They would not feel bad for the engineers who lost their jobs, for the customers who lost the product or even the history of the company.

This scenario has not happened to me personally, but I know that at least twice my entire automation code set was abandoned because the person whom replaced me was either not an expert or was at least less experienced compared to me in their programming skills and the company had no one else beside myself who knew the automation code base.  Instead of trying to learn the code base, the less knowledgeable people threw it away and started over.  Why do that?  My guess is the person doesn't understand the complexity in the system and decide that the last guy or gal must have been an idiot.  I too have inherited code bases, and not once have I started over from scratch.  I have thrown away pieces and parts, but never the whole, so I can't directly answer it.  I suppose, I once discover there was former automation scripts about 6 months after I started a project written in a different programming language.  In that particular case, the code developed had already outstripped all the functionality from the existing discovered code.  Even in this case, I still reviewed the work to ensure we had capture the same set of solutions.  This leads into an interesting question of losing complexity by forgetting, however, I am not going to cover that topic.

Company Growth & Avoid Responsibility

This also applies to big companies and as I alluded to earlier, governments.  According to Tainer, complexity growth without equal or greater forms of production end up with an eventual collapse.  Consider HR policies of different organizations.  With self-employment, there is no HR, thus zero complexity.  With a company of just a few people there is likely still no HR, but rather just a group of people all working towards a unified goal.  Often these people are very close.  Eventually there is a need for someone to manage the complexity and you hire someone into HR.  As you keep growing, someone does a boneheaded thing, like not taking time off and not call in sick for a week and then show up expecting that they still have a job.  There is no policy saying they will be fired and so they demand that they either keep their job or get unemployment.  So a policy is added, no big deal...

Then you get to be a mega corp with 100,000 employees.  You have divisions bigger than most medium sized companies and you have a handbook big enough to use as a deadly weapon.  Big organizations are often not run nearly as kindly towards employees because (in part) no one is empowered enough to go do that or if they are, it is spread unevenly causing jealousy, which causes things to then be 'not allowed'.  Having a pizza party for success now makes 500 others unhappy as they smell your reward.  You made 50 people happy and 500 grumpy.  An expert would have seen the problem ahead of time and planned on making it outdoors or away from the other employees, but the person who had the party didn't know better. So the company decides to make a rule saying you can't do that.  Eventually all these rules add up.  They hurt morale because the Dreyfus expert knows its stupid but has to think about how to get around the rule, making their 'expert' skill less valuable (as the point of the expert is they don't use rules).  They, like most Gervais losers (economic losers) flee, making such companies lose their best employees. It also encourages people to not try to make work better as that would break norms and likely added more rules to the already overly large set of rules they have to deal with.  Finally it encourages people to quit thinking because there is a rule, making them even less effective.

There is yet another reason that rules are created.  As The Gervais Principle describes, the sociopaths are always interested in getting the clueless to accept as much responsibility as possible while giving limited credit or authority.  The clueless are a buffer for the sociopaths so that they get as much value as possible from the losers without having to deal with the losers.  This is a form of protection that a zealous clueless person is willing to do, however, the clueless often don't know how to actually run an organization.  The clueless are in a position in which they use baby talk to pretend they know what they are doing.  Since they don't actually know what they are doing, the sociopaths use rules to force the clueless to do what the sociopath thinks is right.  These types of rules create a different type of complexity.  This is a social complexity, where the sociopath must juggle personalities to get the most production.  The clueless is actually a cost, but a small cost considering they are a sort of hedge that if things go wrong, the clueless can be blamed.  In a larger sense, this means that the person with bad ideas but who is good at manipulation can maintain power for a long time in spite of making bad choices because those choices aren't attributed to that person.  This can eat away at an organization or society, building up costs from complex institutions with mild production value, further corrupting the society.

Cat & Mouse

Capitalism is an attempt to alleviate the ineffective manager who somehow remains employed via competition.  The problem with capitalism, is like any system, the complexity builds up to protect the system from being hacked.  This can be anything from monopoly laws to defenses from regulatory capture.  However, this sort of question is mostly capture by Tainter himself, so I won't dive into it.

Ultimately the Tainter Composite describes how complexity not only infects society but the institutions around society.  The final piece I want to describe is how this leads to a cat and mouse set of activities.  Often rules are made because there is a real and legitimate concern.  Some of my examples might seem silly but are actually true (I have experienced some of them).  However, even 'smart' rules can get you into trouble.

Consider black hat crackers and the white hat hackers.  Code is in fact just a form of rules, often very useful rules.  You are currently reading this using a system with more lines of rules than one person could memorize.  The trouble is, there is always a group who wants to enjoy finding ways around these rules for their own purposes, just like some of the troubles in capitalism.  It might be someone who just adds everyone to your friends list or it might be to take all the money out of your bank account.  As security gets better, more rules have to be applied to protect the system.  Will code ever get to the point where maintaining and adding rules are too expensive for one person ?  Will it ever get to the point where building up all the rules is more costly than accepting the attacks?  To some degree that has happened where most people have traded up freedom (complexity) for a walled garden (less value).  Once again, this gets into reducing complexity, which is a topic for another day.

Rather than telling you how this applies to testing, effectively creating rules for your brain, I am going to try not to add complexity and let you interpret it for yourself.  If you found this useful, please write a comment and I will write more on the topic, otherwise, I will leave this dense, difficult topic alone for a while.

Learning: Twitter, LinkedIn and Other Communities

I find it fascinating how many little testing communities there are.  There are those in who love to tweet to each other, those who like to use LinkedIn either to simply create networks of testers or the forums.  There is Matt Heusser's community, James Bach has a following and does Skype coaching sessions.  There is the offer Isaac received to write for TestHuddle.  There is StickyMinds, STP, Professional Tester, Tea Time with Testers.  There is the top 100+ blog list.  This is not nearly all the data being generated about testing but it is a start.

The offer from TestHuddle brought up a discussion between Isaac and I.  Isaac has received credit for a few blog posts I have done, mostly likely due to the fact that Isaac's name can be found in Twitter where as I don't have a Twitter account.  I am not angry or bitter over that.  I hold Isaac in high regard and even though he posts less often, he often helps edit my blog posts.  It is a true collaboration, so credit isn't at all a concern.  However, I do find it interesting that testers often miss the detail of the "Posted By" at the end of each post.  The fact that these folks don't notice who is doing the posting suggests that they don't filter by who did the writing.

So What?

Everyone gathers information in different ways, but I suspect many people don't seriously consider the community and how they choose their community. Different people have different amounts of effort they wish to put into their profession, even when they do more than on the job training.  Even as prolific a reader as I am, I have not read even 1% of that content on testing.  In my writing on this blog, I try to reach a particular audience, those who have some advanced knowledge and particularly with interest in automation.  However, I also spend a good third of my posts to think about the community at large, that is to say either the CDT community or the entire testing community.  What I want to examine isn't just my own views, but to help get you on the road to thinking about the communities you help create.  To help you see that knowing who wrote something is important in deciding whether to spend your valuable time reading it or not.

For me, I am not well enough known to continuously hold a place in the entire testing community's consciousness, but occasionally I write something that gets some attention.  I sometimes write comments on other people's blogs.  I write for StickyMinds on occasion if I feel the content makes sense for that community. I co-run a user group.  However, I refuse to get on Twitter or accept LinkedIn invites from people I don't know.  In fact, this and email are basically the only reliably consistent ways I communicate to the outside testing community.

Why?

What I want to address is the reasoning behind this and what strategies you can use to filter the data you get.  First of all, using the Dreyfus model of skills, I would label myself as an expert within automation/tools and proficient with in the world of manual/exploratory testing.  I am only competent with handling social situations outside of my interactions with technical people.  This is of course just a self evaluation with approximate labels, but you too can evaluate yourself (but beware the Dunning-Kruger effect).  What is it you bring to the table, what is it you value?  What gives you energy and what annoys you?  As a quick example, if you read my writing often, you might notice I don't speak much of my personal life.  I prefer being private in that regard.  I don't enjoy registering my email over and over again.  I often use mailinator to avoid registering with my real email.  These are both for personal privacy reasons and because I hate being spammed even more, zapping even more life out of me.

With this in mind, many communities feel invasive to me.  They want to force me to sign up.  Twitter is pretty bad about this.  I do have a LinkedIn account, but I use that to maintain contact with people I know, not as a social network.  I have limited time and energy, and while I will respond to reasonable questions to my work, I can't be everything to everyone.  I don't want a huge set of people I don't know giving me endorsements when they can't truly know my skill set. For me, I want to get deep into a topic and I want to hear from other people who will deep dive into interesting and hard topics.  I don't care about the fluffery of social networks and I dislike the corruption LinkedIn encourages.  Why spend my time annoyed at people for claiming I have skills that they can't possibly know?

There is one other thing that I mentioned earlier that I think is of value.  I write both here on this blog and on StickyMinds.  In part because my work gets more exposure, but more importantly, the people who read StickyMinds are a different audience.  The level of knowledge and expectations on what they have come for is different.  If I started writing about national politics in this blog, I would lose audience I have and perhaps would get a different audience.  Knowing how the communities you 'live in' are seen by the outside can help tell you what sort of content the outside world is likely not discussing in the community you are in.  For example, I would not talk in detail about my multi-threaded load test implementations in StickyMinds.  Having that level of source code would not likely be published, much less be helpful in that community.

What you can learn!

So what do you value and how can you improve your signal to noise ratio?  I have found very few blogs that are always useful.  Instead I look to see if at least one in the last few posts has been useful.  Those that I personally don't find useful, I will ignore.  I would define useful for me in this context as something that is challenging.  I don't mind if I ultimately disagree with the conclusion.  In this post, I hope not to convince you that my opinions are correct but rather that you should develop your own.  I sometimes listen to youtube videos to see if the speaker can provide me with insights I would otherwise not have.  If it seems like that might be the case, I then will actually watch the video.  I find that the firehose of data can sometimes introduce me to new bloggers who I get value from.  The way I do that is by looking to see if there is a topic I want to read about.  Then I glance at who wrote it and if I don't know them, I will read it.

I wrote previously wrote about leveling up your skills in which I suggested nearly a dozen different ways to gain new skills or level up your existing skills.  The question is, what do you value and thus what skills do you want to work on?  Go find the sort of blog that will help you.  Or maybe you want more videos.  Go look at youtube.  There is plenty to learn there.  Or use pod casts.  That is partly about presentation and how it fits into your life (and how you learn).  The other question is what sort of content do you want. It might ultimately become clear that our blog is not the sort of blog you want to read.  That's okay.  Perhaps you need more data around a wiki based acceptance testing framework like FitNesse.  You probably won't get that from here, so find a blog or community that will help you.  Sometimes learning should not have a goal, and if you only do goal-based research, it might be valuable to try looking at something you wouldn't normally read about.

One of the major goals I have in teaching others is getting them to think for themselves.  I may use myself as an example to consider, but I am just an example, not necessarily the model you should follow.  In a sense, this post is about getting people, such as yourself, to be more reflective in your learning.  If you learn to learn just a little better, that will be a gift that will keep on giving for the rest of your life.  So spend a little time and reflect on how you gather information and learn.

Talking Topics

So we had a Lean Coffee about 4 or 5 months ago, where everyone wrote down what they would like to hear at our local Testing meeting group. BoiseOnTest

Over that time, JCD and I have had those cards on my desk, and randomly gone through them and tried to find people to talk about the topics rated the highest (don't comprehend, see http://leancoffee.org/)

I've grown tired of these sitting on my desk, and I think other people might benefit from having a list of topics to talk about for test meetings, blogs or use them as you see fit.

I have purposely NOT provided context or organization for these, as I think that stimulates more creativity.

Future of Testing / Ways to Evolve
Bridging the gap between manual and automation testing
Automation: What it can and shouldn't do
Mythbusting as part of testing
Ambition, Drive and Contribution
Change (Company, Position, Attitude)
Where do experienced tested provide value (vs monkeys)
Test Design
Test Strategies for continuous integration / deployment
Risk Management
How to find great testers
Test automation classification (unit, acceptance, BVT, smoke, etc)
Training resources for testing (recommended courses, books, blogs, etc)
Effecting change
Job satisfaction
SDET or Not
Advancement / Promotion in a Test organization
Planning
Time management
Speaking / Writing
Social Skills
Communication
What's hidden in the remaining 10%
Test technology (tools, trends, etc)
Web Security Testing
Testing in different states, conditions and combinations
Gathering useful information (logs, databases, etc)
Have speakers give talks on what they do on a day-to-day basis
Experience reports
Modeling
Strategies to influence development teams to test more
Data Driven Testing
Session Based Test Management
Testing Approach / Strategies
Waltzing with Bears - Communication up (specifically talking with authority, management, C levels)
Load testing techniques
The Scientific Method
Risk Analysis
Logic and Rational Thought
Math
Equivalence Classing
Deciding what to test when you're "Testing Everything"
Grey box vs White box testing


And now my desk is slightly cleaner.

The World Wants to Replace You With Something Better

In a world where technology is constantly getting better, with fewer jobs and more automation (speaking generally, not 'test automation'), the world appears threatening.  However, the world has always been this way, just less extreme.  Politicians have always looked for better funding, new ways of communicating and ways to look better.  They slowly edged each other out, but in doing so changed the world, each forging more and more ways of getting funding/communication channels/etc.  The world doesn't want you, it wants the next guy's better idea.  I speak of politicians because it is an easy way to see how people 'vote' for what they like (not what they need or should get, but that could be a whole blog post of its own).  Darwin speaks of this in terms of evolution.  Markets speak of this in economic terms using money as a voting mechanism.  One general term that is used is progress.

The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man. ― George Bernard Shaw

Automation of all stripes appears inevitable.  Will all jobs go away?  I doubt that, at least not in my life time, but what will the world look like?  Have you read and considered The New Luddite Challenge?  Like Bill Joy, I have and I still don't have a lot of answers.  I want to automate things because I hate wasting time doing the same boring things over and over again, but I don't think all work is a waste per-se.  Some people really like cooking, and so they are not wasting their time cooking, even if they cook similar dishes over and over again. On the other hand, just surveying some of my co-workers, I am sure cooking is not a pleasure for everyone. Even worse, waste in cooking is not just time, but also in burnt meals and trashed leftovers no one wanted.   In fact, in the last 50 or so years, we have started seeing food automated be it via fast food or food in a freezer.  While we are not forced to have those sorts of meals, automation has left many of us without particularly strong cooking skills.  The world has replaced what might have been superior meals for meals that require little-to-no washing up or cooking at all.

If you look at recent history, we had different people doing whatever it was that they enjoyed.  They would then hire other people to do the things they didn't want to do.  For example, I test software, but I don't much care to do my own garage repair, so I hire someone to do that.  If we automate some jobs or even just sub-tasks in jobs away, what do we do with those who want to still do the job that was then automated?  One of the easy examples is what happens to truckers when we automate driving?  Obviously children that don't know the 'joys of truck driving' won't dream of being truckers, so maybe this is a self-correcting problem, but I don't like saying 'wait until all those old school folks die off'.  I am not here to give answers to this topic, certainly not for such a complex set of questions, but I think if you write automation, you should seriously explore the effects and ethics of the profession and how it changes the world, before you create a world you don't like.

In fact, I would go so far as to say you should frequently examine what you are doing to see if it is what you want to see done.  That is to say, if you work for a company that gets cat pictures to people, is that something you care about?  Do you feel that another cat picture platform is what matters?  Perhaps you'd rather hack together a bunch of low-value apps and make a few hundred per app.

A Real World Example

Ultimately, our automation of simple tasks might eventually cost society some of our jobs.  Yes, test-related jobs can be lost by automation.  To get the regression I have produced with automation would require more people than we could hire, but to even reach something near that level, my company would have to hire 3-5 people with low levels of knowledge ("Beginners").  My wage might meet these theoretical low-level beginners, but they would be simple button pushers.  While there are pros and cons to automation (E.G. Testing vs Checking), I think the coverage could be argued as less for the human side, but to be fair, given enough training, the quality of the testing might go up.

Let us speculate in my circumstances.  There are 50+/- fields that have to be filled out, with different types of allowed values based upon different modes across 12 different version of the product.  These are all API-based, and with very simple text-feedback.  It took me about 6 to 12 months to automate to the level I expect these low-level testers would work at, and maybe 3 of those months were around discovery of how the system worked.  It was all regression work with only 2-3 new features over that time, all of which were covered by a different tester.  In my speculation, I would imagine these beginner testers would only do the regression, not the new features.

So let us consider what would happen if we hired those testers.  The testers would work for ~6 hours a day, and would be typing in input and checking the output.  It would increase headcount, and they might need another manager or at least a lead.  The other pieces of automation wouldn't have been able to leverage the work and we would have had to hire even more people.  Our organization would have ballooned into a factory school testing organization.  It would have been ineffective, but would have had more human-coverage.  I have worked in such an organization 10 or so years ago, we had ~100-150 people banging away at 20+k worth of written tests.

I believe automation in this case was the right choice.  I think it adds serious benefits to the organization and those benefits are long lasting.  I think it is best we never hired those fictional testers.  Yes, I recognize that I have created a false-dichotomy in that there could have been other possible choices in the hiring process, such as 2 mid-level exploratory testers.  I still think automation was a better choice.  Perhaps not testing this system could have been an option, but I think automation was a good idea... or using unit tests... or ....  Perhaps there was a better option, but they got me and my automation.

The Future Is Now

As I have documented before, many organizations are moving towards a No Tester view of the world.  In a very real possibility, some software may not need to be tested in the traditional manner.  The future may not need you. In some ways, I think that is reassuring, but for others it is indeed sad.  Charles de Gaulle popularized the quote, "The graveyards are full of indispensable men."  My efforts, as valuable as they are, are only temporary until the next innovation, but also my mistakes are only temporary until they are patched.

Everyone must leave something behind when he dies ...  It doesn't matter what you do, so long as you change something from the way it was before you touched it into something that's like you after you take your hands away. ― Ray Bradbury, Fahrenheit 451

As we keep moving forward, change will continue to come.  The lesson in this is not to give up.  It is not that you shouldn't go for a testing job.  It is not that you shouldn't bother coding.  Rather, go learn a new skill and tomorrow learn another.  The future wants a better world, and while none of us know exactly what that will ultimately mean, it certainly does not mean you should stagnate and quit learning.  Go learn.  Then find hard problems and work on them.