Showing posts with label Introduction. Show all posts
Showing posts with label Introduction. Show all posts

Monday, November 9, 2015

New Adventures in Software Testing

I recently described my past years in testing, but this post is more about the future, using data to back it up.  However, we can only look into the future using the mirror of of the past.  In writing my last 70 or so articles, I have gather a good number of thoughts, and I have posted on average 1-2 in depth posts a month.  In using blogger's data, I had assumed that I was reaching an audience of substance.  However, in investigating my actual reach, it is substantially smaller than I thought.  I have two purposes for writing.  One is to collect my own thoughts, the other is to document my ideas for others to use.  Until today.

I am backing away from my editorial adventures for the time being in order to focus on another adventure.  But wait, don't leave yet.  You've not even asked what it is.  Well, here is the last article I plan to write in the near future.

Often times, I have been asked what sort of code someone should write when they want to learn to write code.  I often come back with the reply, well what sorts of problems do you want to solve?  Why write code at all?  If you have no vision for what to write, being motivated to write code can be incredibly hard.  Even if the answer is I want to write web applications to get a job, that at least provides a hint into the motivations and what sorts of study you need.  Some people have no focus at all and just want to do "something cool".  I have found these people rarely have the interest in completing projects, as cool moves so fast.

In learning to code, I started out with games, like most other people my age.  I built up a set of programs over the years, some of which I spent years developing.  Often times, these programs were designed for a particular need I experienced.  Like my advice, I work on what I find I need rather than what is cool.  Several years ago I became a land lord and found I needed a way to track my write offs.  I also had a job interview that involved PHP, so I wrote a simple tool to track write offs using PHP and Postgres.  I didn't get the job.  My PHP application was clunky and required me to record all the data by hand, yet my bank knew about these transactions.  Why couldn't I just automate the task?  I didn't enjoy working in PHP so I moved to .net and started building a windows application.

I worked for about 3 weeks and had a rough framework designed, primarily using reflections and a Excel-like control.  I started to refine it and within a month I had an application I was happy with.  I kept refining it over a few years, but it was always just meant for me.  I fixed big bugs, but it was meant to please me, so I didn't worry too much about it.

Then, just this last year, my boss, Isaac asked me a question.  He asked me if I was ever going to sell my project.  I thought a while around this and decided it would be a interesting effort and a good learning experience.  I have always wanted to go to work for myself, so why not?  I started in on my adventure.  My adventure included multiple iterations and efforts.  Many of these efforts have been running parallel over the last year.

What Features Do I Need?


I knew what I needed.  I needed one serious feature, the ability to mark items as write offs.  No one else except for the massive applications supported that sort of feature.  These mazes of confusion and madness were more complex than I wanted.  Most of the small open source applications either didn't support the feature or were equally complicated.  Furthermore, the thing I personally needed only a little is one of the big words in personal financing, budgeting.  So, like the heuristic of comparable products, I started examining other products.  I worked on developing features and creating test maps around what would need testing.  But more on that later.

This is a more complex question than you might otherwise think.  For example, what does your EULA look like?  How do you authenticate your key?  What other non-functional requirements am I missing?  I developed a large list of requirements and had to build them all out.

How Do I Start a Business?


I had to start investigating how to license software.  Even the word license is tricky, as I don't sell software, I license its usage.  That affects taxes.  You have to gather all sorts of documentation for the government to allow you to start a business.  Then there is the logo.  What about the store front?  The list goes on and on.  On the good side, I have no employees, so I that simplifies a great deal.

Marketing?


Let me be frank, to the few people who read this regularly.  I'm not good at marketing.  I choose privacy over announcing my existence over and over again.  To make even a personal blog big, it appears one must do lots of advertising.  Isaac suggested it takes 2 years of effort to maybe hit it big, ignoring Black Swans.  I still don't have a personal twitter handle, although I do have one for business to my chagrin.  This will likely be the hardest part for me.  I have a few tweets queued up, and several blog posts about personal finance written.  Hopefully my efforts will pay off, but one never knows.

I also am starting to learn about Google Ads and Bing Ads.  In reality, things are way more complicated than you might imagine.  Google's Ad system is crazy-complicated.  Campaigns connect to Ads which tie to keywords or to a screen scrapper Google uses.  Google Ads use a complicated bidding system where the winner doesn't pay what they bid but the max the last guy bid.  The more I learn the more I learn I know nothing.

Documentation?


I grabbed a free for a year AWS instance of a wiki system and started writing.  I kept writing and writing.  I have written documentation for several open source systems and have done some fairly extensive documentation in the past.  In this case, I tried tackling the problem from several different ways.  The first way I tried was to write out different features of the system.  Then I tried writing things from a different tact.  I wrote about how you can accomplish tasks.

Testing?


As I built out new features, I discovered how complicated my code base started to get.  I had never done any serious regression on my work since it was built just for me.  I had added features I thought might be useful, but it was still poorly tested.  So I built up a list of all the features and major feature interactions and kept a copy in google docs.  Once my program was in a mostly stable state I started rigorously testing it.  I wish I could claim I didn't find any bugs, but that would not be true.  Instead, I found more than two dozen bugs.  Many of them were relatively minor, others were usability.  My list of needed features grew even larger.  I spent 10-20 hours a week for several weeks testing while I spent my normal time on my day job.

One interesting thing I want to note to all my fellow testers is how hard it is to do both the software development and testing.  While I certainly did testing, I found giving some time between developing the code and doing in depth testing gave me some time to forget exactly how I coded something.  That way, my bias in expectations would be gone, allowing me to note usability issues much more clearly.  I also noticed how it was hard to remain motivated.  Testing for 40 hours a week and then going home and doing more of the same is a challenge.  If you're on a death march, imagine how unmotivated a developer is to do any testing of their software.

Shipping is a Feature


Ultimately, I got my application to a stable state.   While I do still occasionally find a bug, I have not found any big bugs in weeks.  I am still doing occasional testing in that I use it for myself.  I do find value in some of the new features I have added in the past 12 months, so I spend more time in my application than I use to.  It's still fairly trim compared to many of my competitors, but I think that is a feature.  Simpler system often have their own sort of value.  I think it is ready for release.  So, on Oct 31st, I released my program, Money Pig, into the world.




I'm also stepping away from our blog for a while, while I work on Money Pig.  I shall return!

- JCD

Wednesday, July 15, 2015

My Many Years in Testing and Automation Development

I have heard the question expressed in multiple different ways.  "How do I get into QA?"  "How do I progress my career?"  "How do I level up my skill?"  "You read whitepapers?!?"  "What type of automation should I write?"  I have written various articles around this broad range of topics, but I wanted to take a slightly different tact around one of the implied questions.  The implied question I'm considering is how do I continue down the path from where I am to become a better tester and what skills should I spend time working on improving?  Instead of trying to explore the topic in a general sense, I am going to go through a few interesting experiences I have had in both "Testing" and "Automation Development".  While I will be talking about automation, I do not mean to imply that is where you should spend your time learning, just that I will discuss how I made the choice.  I will try to concentrate on interesting stories and lessons learned.

I have been writing code for nearly half my life, but I started working in the development of test automation somewhere between 2004 and 2005.  I started professionally testing in 2002, although I had been writing my own code and testing it earlier.  I also did some testing for others before 2002, but I was not paid for that work.  I will try to describe the companies and context a little, but I will not use company names, to protect the innocent.  But first, a small introduction to perhaps explain how early on I was interested in QA.

When I started my life in QA, I was roughly 6 years old and curious about this clock in a video game called "The Last Ninja" for the NES.  It kept a timer of how long you had been playing the game.  It gave hours, minutes and seconds.  There was a place where no bad guys were with a stream and a park bench.  It was perfect for leaving my ninja to sit and just enjoy the scenery.  In only 99 hours 59 minutes and 59 seconds I could find out what happened when the clock ran out.  I knew Sonic would run off the screen after 3 minutes, so I conducted some smaller tests.  In my smaller tests, I discovered, unlike Sonic, The Last Ninja did not seem to mind waiting for short periods of time, so it seemed easy enough to test the longer case.  The only problem was I didn't want to lose my Nintendo for a week while I waited for the result.  So when I went on vacation, I left my system running.  When I came back, what I found was "0A" and the game was frozen up.  Now, many years later, I know 0A is hex for 10, likely meaning the clock had run over to hex and then the game had crashed.  I didn't understand the result, but I was curious about it.  I was later grounded for leaving my system on for that long.  You've been warned kids!

Years later, I was looking to get a job out of High School.  I had been programming text adventures for some years, but never had a 'real' programming job.  I enjoyed programming, I enjoyed the experience of creating, but still didn't feel like I knew that much.  Still I kept at it.  I had worked at the Post Office for 3 weeks in a Christmas rush season, but had little other professional work under my belt.  I was going to school and studying Computer Science, but that was going to take years and I needed money sooner.  So I started looking for a computer job.  I interviewed once, but didn't know what I was doing and did not get the job.  College and High School don't seriously prepare you for interviewing.  In my second job interview, I took a computerized test and passed all but one question (I tried typing in a search but they expected me to use the dropdown...  No, I'm not still bitter.).

I got the job and started working as a managed contractor for a overly large company who worked on specialized hardware.  I was a tester who did everything from competitive intel to stress testing to environment testing to configuration testing.  However, I had no training and really no idea that testing was a general skill set.  I incidentally learned a few simple QA skills, but nearly all of the learning I did was around social skills in my first year.  Perhaps one the most important skills I learned was to apologize is important and doubly so when you are a manager and have made a mistake.  I moved to a more complex set of testing, but it was a script based.  I learned more about networking and security testing.  I almost got a job in a team that was inspired by James Bach's work around exploratory testing (in one of the places that James and his brother did some of the development around the idea of exploratory testing), but I missed out to someone with a lot more experience.

Then, in a sad set of events a former lead who changed shifts passed away suddenly.  I took his place, an awkward thing for sure.  I learned about this idea that you could 'automate' tests.  I began to apply my existing programming skills in order to test various UI elements.  It worked well in some cases, but we struggled to handle descriptions (a QTP phrase for describing a way of IDing elements in a UI system).  A co-worker of mine had more difficulty picking up the programming as his specialty was in hardware, not software.  He came into work more than once complaining of nightmares around programming.  My coworker did eventually get better at automation, but that was never his strength (Aside: He's now a big-wig manager, likely making way more than me!).

I bounced around from place to place, with a rough total of nine different jobs (not all jobs were title changes).  One thing looking back is how I was always trying to go meta, not just writing good automation but improving the entire process.  In a large company, that can be very difficult.  I ultimately had a conflict with one of the managers and felt the need to leave.  Before I left, I was told to quit trying to improve the organization (by providing useful tools) but instead work on personal projects!  I did and learned SQL and web development while looking for a job.  I also had multiple open source projects under my belt by this point.

If a big company didn't work, I reasoned that a smaller company would be better!  I applied to one company and studied up.  Rather than applying to many companies, I reasoned it was better to be prepared.  This from having done so poorly years ago seemed to work reasonably well.  I got a job offer. I tried to negotiate for more money, but was declined.  I declined the job but was later called back and given a slightly higher offer.  I concluded that because of the recession that a much bigger offer was not going to happen.

I took a job at a 50 person company, and within 2 weeks of me joining the QA manager had quit.  This left the QA group answering to the VP of Engineering and it remained that way for nearly 2 years.  I worked for longer hours at this company than anywhere else.  I lost another co-worker who I was training, and whom I may have been the last person he talked to.  He was hit by a bus.  I learned the 'hit by a bus' scenario is both real and the worry about how the business knowledge loss will be much less impactful than the emotional impact from the loss.  I learned what the term 'death march' meant as I worked 80+ hour work weeks.  I learned how a recession can affect you in spite of the fact that your field is in demand.  I learned the dangers of saying that you won't ship with any [known] bugs.  I learned how working more hours did not equal more productivity.  I wrote nearly 1300 automated UI tests.  I found the most bugs at the company, in part because of the automation I wrote, and because all bugs must be fixed, it must logically be concluded I found the most important ones.  (Yes, that is sarcasm.)  Yet, I was not promoted nor paid more for nearly the entire 3 years I was there.  Only after 50% turn over and the firing of upper management did things change even a little, and by then I was too tired to care.  I left.  In case you want more details, Isaac also detailed his own personal journey at this same company.

Now we are getting into more recent events.  To be perfectly honest, I don't plan on telling anything more about the various company cultures because of how it might affect me.  However, I still learned social and technical lessons.

I learned that 'manual' testers could appreciate automation and automation results.  I, for the first time, got to work with other people with equal or better coding skills than myself as we developed automation.  I got to learn how to play a technical leader, even when I was not promoted into a leadership position nor had any authority over others.  While I had dealt with databases and did a little database testing before, I got to do development in which nearly all the code was  around database usage, including communication between database systems.  For the first time I had to deal with the integration of multiple team's automation efforts which were written in different languages.  By this point in my career I had written test code in C, Java, JavaScript, SQL (various flavors), C#, J#, VB.NET and VB Script.

On the social front, I got to work with both one of the best teams and one of the most difficult teams I ever worked with.  In dealing with the difficult team, I got to see how personality clashes worked and how to work around them.  To be fair, I have my own personality quirks too, but when these hit extremes it is imperative that you figure out how to be a positive force for the team rather than yet another point of contention.  Perhaps one of the biggest lessons I learned was that it is important to have a solid social connection and understanding of an individual before giving a completely honest evaluation of someone's skill set.  In fact, I took the lesson so much to heart that I try to be more careful with my day-to-day speech as a whole.

Finally, I want to describe a little bit of my attempts at professional growth.  I spent a good amount of time embracing various avenues for learning.  I took BBST, and while I only took the class for the first BBST course, I watched all the videos and read all the material.  I have written on this blog for a few years now.  I have spent time giving talks at various venues.  I have written letters, written for other professional venues, testing non-work apps and taken serious amounts of time trying to understand the systems under which work is performed.

So to sum this all up, I think I have just two simple suggestion for you:

Learn

Making mistakes is fine.  Just learn from them.  Learn how you learn.

Don't just do work

We all are making a journey, and at some point it will stop.  Making an impact is important.  But don't make work the only impact in life.

Tuesday, September 17, 2013

Introduction…a little late.

tldr: Red hair, chops, boots, attitude…

Currently I am JCD's manager and previously managed Veronique.  I have built multiple testing organizations, as well as doing my own test work.  My focus is in being a jack of all trades.  I can write automation, deal with C-levels, influence developers, run manual regression, create load tests and lead intelligent people.  I do an awesome job on all, but I do depend on my team of rockstars to deal with whatever technical challenge we come across.

Part of the reason that I started blogging is to increase my proficiency at writing. I'd like to extend my sphere of influence from just merely the people that I've worked with or hired, to something more epic scale. I require large goals in order to satisfy my small ego. I'm constantly amazed at most people's willingness to do the bare minimum and not improve themselves. Internally, I'm driven to do things the best that I can at the moment.

I started testing cause coding 8 hours a day sucked. Having the ability to code / test / research / choose my own priorities is critical to my success. I believe there is no "one true way", and I value results over ceremony, metrics, paper shuffling and acting busy.

I have found value in context driven development and while I don't require my people to follow it, I do expect them to be able to talk about it with some intelligence.  I hold high standards and tend to be a little brash in my language.  I expect people to be intelligent enough to be able to research things they don't understand rather than spoon feeding you links.

I have lived in my little valley for my entire life and have slowly learned from our diverse community of development scenarios….You may see me around as Isderf, my long time handle.

BLAH
Like JCD, I need to say my views do not represent any company I have worked for, past present or future.
/BLAH

Thanks to JCD for ghost writing a lot of this, as I hate talking about myself without several scotches…

Tuesday, September 3, 2013

A Quick Introduction


I work with my fellow blogger Isaac (and possibly others if we can get them to join).  As Isaac's lackey, I primarily work on Automation, be it an API, a text based get/post system, a website or a database.  I do occasionally come out of my automation shell and do manual testing, but it is not my specialty.  

I believe both automation and manual testing are required and I do both even when I am automating a system.  At a high level all software is about communication, and testing is about communicating the state of a system.  I simply write software that communicates at a different level than a manual tester, both to the system under test and to the world at large.

You may see me speak to the entire software ecosystem, probably even a bit beyond what the typical tester considers.  I believe to some degree it is our responsibilities (collectively) to consider all the ramifications of our software, even possibly beyond the traditional bounds of our stakeholders.  I think testing should consider the global impact and if possible make the world a little better.  Be that making error messages easier to read to lower stress to making medical equipment as safe as humanly possible.

That all being said, my views do not represent any company I work for, past, present or future.  I also do not claim my views represent those of Isaac or any other fellow posters.  In fact I hope they argue with me in the comments! :)

EDIT: I want to note that I will be posting some of older (as in 2-3 weeks old) blogs I had done on my own before convincing others to join force with me.  This might make it seem like I blogged way more than I do.