A Personal Failure Analysis: Why it appears CDT has failed

Talks & Failure Analysis

For the last few weeks I have been doing one presentation a week to various monthly groups and to a yearly event.  I have one more planned presentation in June, but this really isn't about my presentations, but rather the audience and fellow speakers.  

The first presentation was a meetup group Isaac and I helped form up in March.  Isaac and I gave the talk and it went really well.  Our audience was maybe 40% high-level QA with a few junior and the rest mid levels.  The talk was about testing an object in black box style.  We gave this talk 3 times total, and each time it seemed to go a little worse.  

When we did it with a group of developers, we moved to more white box/boundary testing style.  We had one person refuse to test out of principle.  In part I think us changing the style was at fault, but two sharper developers really got it, so I don't think it was totally our fault.  Two people really didn't get it until I changed boundary testing into a UI exercise.  This tells me even (junior) developers have a hard time visualizing what code does in their head.  

The final audience was a mixture of testers and developers.  The developers were often dis-interested in our talk and several of them left to go see a different track.  Either they felt applying black box techniques was not part of their domain or that they already knew this and wanted something new.  The irony of that is that 60% of the sessions were marked beginner, so learning something 'new' for a mid/senior developer is challenging at best.

I had a second talk which was a shorter version of my intended lecture I will be giving out of the country.  It was around reflections, something I think I have some level of mastery around, so I felt very prepared for my talk.  I will admit my cadence was a bit off, I probably rushed a little too much and perhaps didn't give enough audience breaks, but it was a code oriented talk.  Given the complexity, that might have been a small part of the problem but in reality, most of the audience were either junior developers or students who in their 2nd-4th year of college.  I had marked my talk as advanced but I couldn't control the people who attended.  This caused me to wonder something.  Why is it we are still talking about TDD, Intro to jquery, etc.?  I don't mean to say we shouldn't teach new people, but why are we mostly limited to beginners?

Why do we retrain people all the time?

In talking with 'experts' in the business, it becomes clear that my question of why do we repeat ourselves is because in some communities, such as testing, people don't stick with it in the long run.  Roughly 5 years appears to be the mean time between starting a career in testing and going to something else.  The second issue is that many people are not particularly interested in expanding their knowledge.  They just want to go to work, do their job and go home.  Don't get me wrong, I don't mean to say you should work 90 hour weeks, but many people have little passion for their work or improving their craft.  

Sometimes managers make these engineers look into 'new' development techniques, which create such a long tail into training that something like TDD continues to need talks 10 years later.  

With that analysis, I want to talk about it with an 'expert', and so I found the keynote speaker whom I spoke to at length.  I don't want to cite his name as I have not asked his permission nor do I have an easily accessible method to communicate with him (I don't have his email address and don't use twitter).  

He is a well-known developer who has a pod cast program with hundreds of different interviews with various people within the software industry.  In our discussion, I asked him if he knew Matt Heusser and he said he didn't.  I asked if he knew of James Bach, Cem Kaner, Rex Black...  He knew of no one I asked of, although having reviewed his interviews, the only tester I recognized that he interviewed was James Whittaker, which was many years ago.  I had not asked if he knew Whittaker.  I did ask if he knew any US experts/thinkers in test, but he declined to answer my question, saying he didn't think that way.  Perhaps my question was somehow confusing.

In speaking with this gentleman, a man who clearly had a wide variety of experience, and seemed to subscribe to the Analytical School of Testing, it became clear to me.  One of the reasons we end up not keeping testers around of long is because we as a group are not well known.  James Bach, in spite or perhaps because of his controversy, has one of the biggest names in testing.  Sadly, managers aren't demanding people start doing 'Bachian testing' (even though I suspect Bach himself would object to that name and concept).  We don't have a glamorous single one size fits all solution, like Scrum or TDD.  CDT is so vague no one but its practitioners knows exactly what it is.  It isn't easily measurable like Scrum or TDD which I can say if we are doing it or not (Are there sprints?  Are there unit tests?).  It makes me wonder if CDT has lost its war when managers can't easily measure if there CDT testers are even doing CDT testing, much less if CDT is working for them.  The metrics (be it sprints or number of unit tests) might ultimately not matter, but managers still at least one security blanket before they are likely to invest in structural change.

Solutions?

I'm not sure I have any useful solutions other than to say this appears to be a community issue.  I have tried doing mentor-ships with mild success.  It however doesn't scale beyond 1-2 people at a time.  I have tried doing lectures, but as soon as I stray past beginner style thinking or just talking about the concepts without talking about implementation, it seems to be clunky at best.  I have started a group with Isaac to see if a more personal monthly audience will help.  Maybe there we can make headway, but I don't know.  One nice thing with it being monthly and having multiple speakers, I might find out if the issue is in fact somehow with me and how I am presenting.  Even outside of presenting, maybe the way I communicate poorly and that just needs improvement?  For that matter, maybe I'm using the wrong language (English) to reach my audience of highly-motivate testers and software developers?  Perhaps there are more motivated testers outside of the US and my issues only revolve around where I live?

Perhaps I need to accept that I can only speak to a handful of experts, but how do I find and group them together?  How can the experts then package this stuff up together in such a way that they might learn from my knowledge?

Outside of myself, I know Kaner is moving to a more teaching oriented approach generating classes in which the students come to him.  Having taken a BBST course and talking to others who have, I know only about 1 in 3 (or less) of those students seem to both try and have the skills to succeed.  Some choose only to pass by the skin of their teeth and others simply don't get it.  Maybe some of it is because the course is in English, but it certainly isn't the only reason.  However, 1 in 3 might be as good as it ever gets, perhaps my expectations are just too high.  Kaner appears to have given up on a education only system by adding in certifications.  It appears to me the ultimate answer maybe that one has to market one's ideas more in order to convince people to change.  Perhaps the reason Kaner is talking about moving to such a path is because he failed to market his solution well enough.  Going back the keynote speaker I spoke to after my talk, he suggested that if he didn't know about the thinkers/experts of test or the ideas of CDT, they must not be that important.  Because if you don't hear about it, it can't matter.