GUI Test Automation: Squish Success at Mixed-Technology Online Poker Company
Jan 22nd, 17:04 UTC
A leading online poker company uses various editions of Squish, froglogic's popular GUI test automation tool, to test web pages and applications using different technologies and on different platforms. We can't tell you who they are or even show screenshots of their products, but they did allow Graham Abell, one of their QA Automation Engineers, to tell us about their experiences with Squish.
Graham has worked in test automation for around four years. Prior to
Squish, he used a variety of test automation tools including WinRunner,
QTP, and LoadRunner. We're grateful for him taking the time to
answer our questions so thoughtfully.
Graham's company develops two applications—a software installer for
Windows and Mac OS X, and a client application for Windows and
Mac OS X. The installer is downloaded from a web page. Graham's
company tests the web page with Squish/Web and the installer and client
applications using Squish/Qt and Squish/Mac—with both applications
being tested on both of their target platforms.
At present Graham's company has 3,000 tests, with plans to expand to
15,000 tests in their next phase of test development.
Why Squish?
Prior to using Squish, Graham's company only supported Windows for their
installer and client application. When they ported from .NET to Qt to
broaden their market to include Mac OS X users, they needed to
find a new testing tool, since what they used at that
time—QTP—did not support Qt or Mac OS X
applications. Graham told us:
We reviewed all the tools which supported Qt, and Squish appeared to be
the most mature product. Its support for cross platform testing also
appealed to us as we had previously been able to test only on Windows.
Graham's company conducted its search for a suitable tool in 2008. As part
of that exercise they contacted a number of companies that provide
testing automation products, but froglogic stood out, as Graham points
out:
We were in contact with several companies but froglogic was by far the
most responsive. This was something which we valued coming from pretty
slow customer service with our previous tools.
Graham's company got an evaluation copy of Squish and developed some proof
of concept tests against a very early release of their new client
application. Graham also noted that having access to Squish's source code
was a critical benefit since it gave them the flexibility they needed to
fully automate the testing of their client application. They were also
very happy with froglogic's licensing model, commenting:
It was refreshing to see a company address licensing in a manner which
makes sense for automation. A single developer will need to run tests
across various machines and so the Squish licensing model completely
matches that real life usage.
Graham also gave three specific technical reasons why his company had
chosen Squish over the competition:
- The option to use languages such as Python, which have a very
active community and a wealth of modules available makes test
development a lot easier—we don't have to reinvent the wheel at
every turn
- Squish's ability to access the Qt API allows us to interact with
the client application and complete tests that wouldn't otherwise be
possible
- Squish's cross-platform support of multiple technologies is
unique amongst test automation tools
In addition, Graham told us that he had found Squish's support for
running tests across multiple machines to be particularly useful.
Squish in Practice
Once Graham's company started seriously using Squish they were pleased to
discover that Squish has a low system overhead compared with the other
tools they had used. This meant that they were able to run tests on
lower specification machines, and in particular, on virtual machine
instances.
Graham's team makes use of the HP Quality Center™ in conjunction
with the Squish integration add-on. They have found this to be
particularly helpful in view of the large number of tests they are
developing and maintaining.
Apart from accessing Qt's API, Graham has found the Squish Spy—a
tool for identifying application objects and for viewing and verifying
their properties—to be the Squish feature that he makes the most
use of.
Graham's team are all experienced with test automation, so when they
adopted Squish they didn't encounter any significant problems learning
how to use it— they simply had to translate from what they had
done with their old tools to the Squish approach. It wasn't all plain
sailing however, and initially they had some difficulties in accessing
aspects of their client application's state. Once they'd learnt more
about Squish's facilities and with some advice from froglogic, they soon
resolved this problem.
One of the main reasons why Graham's company uses test automation is to
keep the amount of manual regression testing to the absolute minimum.
They also use Squish, insofar as possible, to support the testing of new
features, something done in conjunction with the company's separate
manual QA team.
Graham's company are unusual in that they don't make much use of the
Squish Object Map, but keep track of their object descriptions
externally. Fortunately, Squish's open design and use of open file
formats, means that it is almost always possible to integrate Squish into
existing QA practices and procedures.
When switching to Squish, Graham's team didn't encounter any significant
technical hurdles. However, they did have to rewrite all their old tests
as part of the changeover, but they have found the benefits have already
outweighed the costs of doing this. And they have now supplemented
Squish with some in-house tools developed in Python and .NET to produce
the exact testing environment and processes that suit their needs.
Conclusion
For Graham's company, Squish has made it possible for their QA team to
provide better test coverage and improved test reliability. One
consequence has been improved application quality. And they've also
managed to reduce the number of person-hours required for automated
testing.
We'll leave the final words to Graham:
Support from froglogic is unparalleled, I've never dealt with a
company who consistently respond so quickly to support tickets and
always do their best to resolve the issue as quickly as
possible.
froglogic's team would like to thank Graham for taking the time to
share his and his team and company's experience with Squish, and we
look forward to a continued successful relationship.
About froglogic
froglogic GmbH is a software company based in Hamburg,
Germany. Their flagship product is Squish, the market-leading
automated testing tool for GUI applications based on Qt, Java
AWT/Swing and SWT/RCP, Mac OS X Carbon/Cocoa, and for HTML/Ajax-based
web applications running in a variety of web browsers. froglogic also
offers services in the areas of QA/automated testing and C++/Qt
programming and consulting. For more about froglogic visit http://www.froglogic.com.
(Submitted by Reginald Stadlbauer of froglogic)
| 
