Monday, December 15, 2014

More on No Testers

I sometimes look to see who is reading my articles, and if the incoming link looks interesting, I try to see if I can find the blog it came from, maybe even comment on it.  I particularly wanted to see if anyone had comments on my latest blog about No Testers.  Well I did that today.  I went to look at a Russian article (in Russian), and found a comment by Maxim Shulga saying (translated by Google):

Detailed and very adequate answer to the same question to my http: //about98percentdone.blog ... The only pity is that uncomfortable read: white on black. Burns my eyes :)

Now normally I would just reply to such a comment by filling out the form or using my gmail credentials and writing back.  I might say something silly about how I know not everyone likes the black on white style, as Jeff Atwood pointed out years ago.

While Google translate did translate the article, it didn't translate the comments nor the method to reply to a comment... using some developer tools I was able to establish this was from Disqus.  I am now going to chronicle my efforts to get an account and demonstrate how usability might have benefited from a tester, not to mention a bug tracking system.  Before I go on, I want to say I have no business relation with Disqus or any other discussion or forum/blog related software outside of this blog.  I am not trying to pick on them, it just happens I was trying to get something done and they are blocking me.  They also happen to have no testers, or at least no one with a label of "QA" or "Tester" or anything outside of 'engineering'.  This just happens to be an interesting example which relates to my previous post. I did not intentionally go looking for a company that has no testers and only discovered after I started writing this blog post that they have no testers.  I have contacted them about the issues I have noted here and none of the issues relate directly to security or should go unpublished for ethics reasons.

What I Found


Item 1:

Disqus did not give me English versions of their UI for replying even though my browser should be requesting English.  I tried several browsers, including IE which I checked to see it was set to en-US.  I am not sure if this could be detected via any sort of metrics to figure out they should be looking at the browser language rather than some user/blog setting.  To me this is the sort of choice that happens when you don't think long and hard about localization.  But perhaps they know and plan on changing it or maybe they think this is the right choice.

Item 2 & 3:

So I go to https://disqus.com/ and click sign up.  I am asked for a email AND a username.  I don't know what the username is used for in this context and there is no useful description or even an icon to click.  So I enter an email, username I enter JCD and a password and get told "Username already exists."  So I try JC-D.  Nope, "Letters and numbers only please."  Okay, what about JÇD.  It is all ASCII characters but I just get "Letters and numbers only please."  In fact, it thinks all sorts of things are not letters or numbers in their view but which I would consider letters.  I didn't bother with much Unicode but I imagine it would have the same sorts of issues.  Maybe this is intentional, but their error is not meaningful to me.  Worse yet, it excludes many people with names that are not just alpha characters.  My name has a hyphen in it, thus the JC-D.  The hyphen was excluded, excluding me using my real name.  Granted the math models would say I don't count as few people have hyphenated names.  Maybe I shouldn't care, maybe the name doesn't matter, but the username's usage is not clearly explained.  However, not including Markus Gärtner because his name has a non-English character seems really wrong.

Item 4:

I thought for a moment to use their gmail integration.  I have used it for Khan's Academy to log in and it worked fine.  So I tried but it looked like they wanted me to have a Google+ account, which I intentionally don't have.  I am a bit odd, wanting both privacy and a voice.  I don't like giving away personal details and being tracked, even if I have professional opinions that I wish to voice.  My professional and personal lives mix, but only a little.  So that was a no-go for me.  Worse yet, I get an warning saying that OpenID2, the method Disqus uses to sign up, will being no longer supported early next year (April I believe).  Clearly they have some updating to do.

Item 5:

I considered using a fake email address, but their terms of service were not on their sign up page.  In fact, most of the links on their page went away on the sign up page.  Maybe that is intentional.  Maybe it was A/B tested.  If so, awesome, but for me it was less than optimal.  I admit to not being the main use case, and perhaps that is one problem with testers.  We are not equal to users.

Item 6:

I go to report these issues and the best I can find is a contact support.  Not saying support is a bad place to start, but the form of input gives me about 3 sentences and a scroll bar.  If I wanted to tweet the error, that might work, but I had detailed points to give.  Only someone who is concerned with the customer would notice this, but I, a potential customer did.

Analysis of Why the Issues Exist


I suppose the question is, how do we capture this sort of data or if we care.  Maybe annoying your users is okay when you are a free product.  Maybe alienating users is fine when your metrics show few users try to use Unicode.  Perhaps that is what data scientists are useful for, deciding which problems matter?  Maybe having a functional tester would notice these issues and bring them up?  Having the customer deal with the problems until you figure out if it is a good idea is not an uncommon model, particularly if you own the market.  But keep in mind I never did get to make that comment.  Speaking in psychological terms, even with professional distancing, I will have a more negative view of their product and it will take effort on their part to turn me around.  Even if they magically changed it all tomorrow a potential customer like me might be long gone.  Perhaps with a billion users, it doesn't matter.

While I personally don't feel this way, but maybe the Disqus team is not the right team, which is the argument made for hiring 'the right team' that I have heard from the no tester camp.  If they were, I'd not be writing this post, with so many issues.  I think the language used by the no-tester side is unclear what the 'right team' is, and perhaps what they think testers do.  I am not sure that a mythical right team, with or without testers will ever produce bug-free code, but there are certainly good and bad team mixtures.  I feel that it is rather more difficult to evaluate their team dynamics having never met any of them.

Perhaps they are 'the right team', and I am just the wrong customer?  That is the other half of this particular no tester argument.  That testers are not like customers, so use customers.  As a customer who thinks like a tester, maybe I'm not representative?  Then again, if I pull out my heuristics, I can compare this to other products that don't require you to sign up at all to write a comment.  That may not be a complete defense in comparing myself to a real customer, but it certainly gives credence to these issues.

Finally, it could be they do have QA/Testers but they were renamed to some other title.  That just made it harder for me to figure out if they have testing and if what was built was what was intended.  These are design choices, but it is unclear if there was anyone questioning these design choices. Without someone in that role, the 'get it done' mentality comes into play, at least in some organizations.  Perhaps that happened here.

I am sure the reality of Disqus is way more complex than I have presented it, but I am an outsider.  I welcome any feedback from the company and will update this accordingly.  I also was not looking for this example case.  I was not planning on posting any more this year.  It just showed up and I thought it was interesting.  I'd love to hear from those who feel no testers is an appropriate choice and how they would interpret this.

To Maxim Shulga, I am sorry you don't like my black background with white text.  I will take your view under advisement if I ever try to re-theme this blog.  I hope at least the content is useful.  And next time, just leave a comment on my blog... this reply to your comment took way too long to write. :)

5 comments:

  1. :) Hello JC-D. Yes, black and white scheme is not good for me, but content is very useful. So I'll continue to read your posts :)
    And I didn't realize that my post may cause such deep test session for Disqus :)
    So do you think that I need to change my comment engine to G+?

    ReplyDelete
    Replies
    1. To be clear, I was not attempting any sort of comparison of the commenting engines except for the sign up process. I admit I tend to prefer Google's commenting in that it integrates with gmail and is easy to use, but I also know that other locales have other services they prefer. Sort of like how in the US ebay is popular(ish) but in China, the main service is Alibaba. For you and your readers, Disqus might be the best choice. I just had a sub-optimal experience, but it is odd for an English user to read a Russian site.

      A major problem with choice is often that it doesn't matter that much, but figuring out what is the best option takes too long (E.G. http://xkcd.com/1445/ ). It is why some products can get away with having lower quality. They can take advantage of Information Asymmetry (http://en.wikipedia.org/wiki/Information_asymmetry ) and the costs to gather that information.

      I have not gotten a response from the company yet, but I only notified them yesterday. Customer responsiveness can sometimes be a proxy measure for the quality of the company. It costs the company time and money to respond, and the quality of the response hints at where their priorities are. I would say if the choice of comment engine mattered little to you and you were interested in changing, I would look for how Disqus responds to my issues. At least then you would know how the company would handle an issue you did have that did matter to you in the future.

      Delete
  2. By the way - the main problem of blogspot comment engine is default deselected "Notify me". I always forget to select it. But definitely if I write comment - I want to get answer.

    ReplyDelete
    Replies
    1. On my own blog it is set to always notify me, so I don't have that problem.

      I once asked my friend who worked at Microsoft's Messenger team why MSN Messenger sorted their log of past messages from oldest to newest. He said that there was a 60/40 split, with 60% of users wanting it sorted oldest to newest and 40% wanting what I thought was the rational choice, to have the newest messages on the top of the log. I still think they are wrong, but I can't argue with the choice, assuming their data was right and that the 40% didn't care more than the 60%. That is to say, I felt passionate enough to write him about it, while the majority of 'oldest first' users might not have really cared. That is a tricky part of data science, it is hard to measure passion. Anyway, it might be that Google has studied if the checkbox should be checked or not and found more users wanted it unchecked than checked. I have no QA engineer friends at Google to answer that question, but Google is very data driven in their development, so it is plausible.

      Delete
  3. Here is the response I received from Disqus:

    Hi JCD,

    Thanks for reaching out with this feedback.

    To clarify, there are two kinds of names Disqus uses; a username, used for logging in and moderation (must be unique); and a Full Name which is the name that is displayed with your comments (does not need to be unique).

    As for usernames not allowing special characters, please note that display names can contain any special character and we recommend users from countries which use languages with special characters add those characters to their display name.

    Please note that using Google to authenticate login does not link you Google+ account to Disqus.

    To view our terms of service, please visit: https://help.disqus.com/customer/portal/articles/466260-terms-of-service – for reference, we do not prohibit signing up with a fake email (although you won’t be able to receive email notifications).


    We hope this helps and please let us know if you have any other questions about this topic by replying to this email.

    If there’s anything else we can help you with, please start a new case with us at https://disqus.com/support/
    //Qarly
    Product Support
    Disqus

    ReplyDelete