Book Consideration: Tools of Critical Thinking: Metathoughts for Psychology by David A Levy

“Cogito ero[sic] sum” – Rene Descartes.* Those were the first words I saw in the book that got me excited to read it, although disappointed at the typographical error. As I made it through Tools of Critical Thinking, I found much of what the book describes I already understood, and much of it came from a class I took years ago in philosophy and logic. That being said, it did give me a little bit of thought in regards to psychology. For example, the book talks about how words have implied value. Other concepts include how concepts are not things, there are different levels of an idea and naming something does not imply you understand it. These ideas are all things I understood reasonably well. Then the book seemed to have started pulling quotes from Slashdot, with the whole correlation is not causation. Literally five chapters were dedicated to this one concept, and I think that is perhaps not excessive, but for me, it felt like an “I already know that, teach me something new”.

As I continued my personal journey through the book, I found one very interesting new concept I had not heard of before. Fundamental Attribution Error. Sounds a little complex, but it really isn’t that hard to understand. Roughly, it means that we tend to overestimate a person’s personality and underestimate their situation. Strangely enough, the reverse seems to occur frequently when you are looking at yourself.

The author also talks of other subjects like extraordinary events and how given enough events, you would expect some to occur. One case I experienced involved me leaving a parking lot following a person and then, about an hour later, on the way back from lunch on that same day, being followed back to the same parking lot by that exact same person. I found it really funny, because of the impossibility to it, yet it happened. That being said, I don’t seriously take it as anything but a fun and funny event. At the time this type of event occurs, perhaps it is reasonable to have a little “brain breakage” and see it is something more, but one should understand that random event occur throughout our lives and within chaos patterns emerge (see fractals).

Mr. Levy continues on about deductive and inductive reasoning and what poor conclusions can be made from what appears to be reasonable reasoning, which again is covered by logic 101. My favorite flaw goes something like this: Spaghetti is a food therefore I am right. Basically, if you have a true premise (Spaghetti is a food) you can conclude anything and claim your logic is infallible. There is also inductive logic where you base your conclusion on a single specific type of evidence. This has its own set of flaws, like a low sample size creating false new knowledge.

Other topics of interest include the idea that an observer affects the observed (E.G. To Grok; See Stranger in a Strange Land). We apply schemas to all sorts of data, from gender roles to mental illness to height to various other items. Most people have a personal investment in their beliefs (strangely, belief isn’t defined). To know and label something is not to solve the problem (which should be obvious in the world of QA, but think about it with how you interact with people).

Now I want to circle back to my own job, testing. Is this a testing book, does this book have value in testing? I think the answer is yes, but only if you don’t already have these logical concepts already well understood. Not to say a review doesn’t help, but honestly I think one could get much out of reading a logic book. That being said, I suggest one read the chapter on schemas, confirmation bias, deductive logic, as they apply very nicely for much of what we do. Let me give an example of how this applies to my job using the chapter on representative bias. This is one chapter I have some issue with, not because it is wrong, but because I think it is not a complete picture. The book talks of assuming things based upon what you are observing (for example a person you are observing). As a real life example, if you have found one developer typically does good work, you might choose to do less QA for work they do, but is that because of personal feelings or hard data? I don’t have hard data, but I can say I never regretted choosing to do less QA on one particular former developer I did work with and instead spent more time focused on other developers. Is that in fact a confirmation bias on my part? I think not*.

One last subject I would like to talk of is schemas and how we develop them as people. I have had a long running set of conversations with a friend on the value/cost of applying a “Schema” (labels) to people, ideas, groups, etc. The obvious pro in my mind is that the schema provides a way of connecting us together. That is to say, we can learn new data by applying old schemas to new situations. For example, I could say that males tend to be more aggressive and more aggressive people tend to play more sports**. From this you might apply a schema pattern match on a male athlete and “guess” he is more likely to be aggressive. Now, you have no evidence for this particular case, which is where testing comes in (in my mind) to confirm or deny that in a particular case. The trick is to keep an open mind and understand that no matter what your judgment shows, it could be wrong and to accept it.  Even when you have a well developed schema, there is likely to be some outliers which you need to expect and be flexible enough to believe exist. If you don’t keep an open mind, you can end up dividing people into us vs. them, which does not provide real long term value to humanity.

The other approach is to say, “I simply don’t know without direct observation. Even then, my observations might be flawed, and without reproducibility, I know nothing.” This method (to me) is also reasonable, as it has no value-judgment and thus requires no schema. In other words, every situation is unique, so a schema is too rigid for the real world data. The problem I personally have with it is it also provides no structure, possibly no value, any way of generically apply new knowledge and no hope to really learn. In a very real sense, it is giving up on gaining structured knowledge, because your knowledge will always be limited, flawed and low resolution. To be clear, and fair, I think that it takes an amazing amount of will to refuse to judge and simply keep an open mind since anything is possible. Just the pure discipline might make this method worth committing to.

I could go on for pages on schemas, but I think that if you don’t understand what you are doing when you sort types of knowledge, you really need to read this book.  Also, if you exchange the word “schema” with “database design”, it also comes out as an interesting talk on SQL vs. No-SQL solutions.

In summing it up, I think it is a handy book for a QA/Test professional, handy enough I bought my own copy of the book.

* This reminds me of a joke I couldn’t resist not including. Rene Descartes walks into a bar and the bar tender asks if he would like a beer. Descartes says, “I think not!” and disappears in a puff of logic.

** This statement maybe untrue.  I ask you to apply your own reasoning and conclusions around the subject.