Manual User Testing Has Died. Long Live Selenium!
When developing web applications, one must be very careful with the quality of the tool that is going to be released. Most users will only give you one chance to introduce a new feature. If it doesn’t work well it will be very, very difficult to regain their trust. As such, when something new is done here at the Torrenegra Labs, we test it a lot. We also attempt to test all the other features that may be affected by it.
Here is how we do it: the developer tests in the development environment and then again in the staging environment. The product developer then tests in the staging environment. So far we have three tests. In average, every new feature requires three rounds of quality assurance. Thus, we have nine tests so far. When approved for release, the feature is then to be published and tested once again in the production environment. These final tests are done by the product developer and somebody else (maybe the developer or a super-user). That is eleven tests in total. Eleven times of doing, almost, the same thing over and over. Although some people may find this fun, most developers and product developers hate it. I am one of them.
For many years I looked for ways of automating those test. There were a few paid tools out there that could, in theory, do that. We didn’t find any of them good enough, in our opinion, to invest in. Last year, however, two members of our team (German being one of them) started to play with a new and free tool called Selenium. This year, when starting work on LetMeGo, I decided to investigate again and found the Selenium tool by my own means. Even though Selenium was still in beta, we saw maturity and potential in it and decided to use it.
Selenium started in 2004 at ThoughtWorks, a company I greatly admire. The project eventually became open source and many more people joined in to help. The full history of Selenium can be read here.
Getting started using Selenium is not easy. It takes a few days to learn the tricks required to properly record the automated tests. Recording the tests takes approximately twice the time it takes to perform one test. Henry was the first one recording tests. He did it while performing QA for the Hendrrix scope (the subsystem in charge of user account management in LetMeGo). The first test he recorded had to be re-recorded from scratch. This happened as we were finding the most appropriate way of matching the recorded Selenium user tests with the use cases of the requirements. All in all, implementing Selenium delayed Henry’s originally planned schedule around three weeks. Lucho and German also added automated user tests using Selenium, Lucho for the Morrison scope (the email messaging subsystem) and German for the Web scope. This also pushed back their schedules between one and two weeks.
Was the delay worth it? I would bet on it! I guesstimate that we will probably save around one month of QA work in the near future when we get ready to release the beta version of LetMeGo. By then, we won’t have to do any manual testing of user test cases as 99% of them will have already been recorded. It will be just a matter of running the automated scripts and watching the bar go green! The program will also be running in a Windows PC and a Mac every day performing integration tests. If errors are found, it will trigger alarms right away, the same way it happens with unit tests.
I think that Selenium is the best companion for unit testing (TTD). Together, they give the product developer and the developers the peace of mind they deserve and need to do a better job. I am happy!
PS. Congratulations to Henry for reaching the alpha milestone for Hendrrix! Great work 
– Congrats as well to German and Lucho for re-reaching the alpha milestones for Morrison and the Web Scope by adding automated Selenium tests to them.
PPS. The usage of Selenium as an automated user-testing tool has been already documented in the RRápido methodology.
Alexander Torrenegra
