Automated GUI Testing: Squish Success at Weatherford Petroleum Consultants
Mar 17th, 19:28 UTC
Weatherford Petroleum Consultants (WPC) provides consulting services in such areas as geology, geophysics, petrophysics, reservoir modelling, and production technology. They also develop the Sendra core flow simulator which simulates SCAL experiments.
We were very pleased to speak with Arent Arntzen, Sendra's Scrum master (project manager) about their use of froglogic's Squish GUI Test Automation Tool. Arent has been doing software testing for about two years, and Squish is the first GUI testing tool he has used.
Introduction
Weatherford Petroleum
Consultants (WPC) provides consulting services in such areas as
geology, geophysics, petrophysics, reservoir modelling, and production
technology. They also develop the Sendra core flow
simulator which simulates SCAL experiments.
WPC employs about 80 people in Norway, while its parent company, Weatherford, employs around 50,000
people world-wide. Sendra is used by more than two dozen oil companies,
institutes, and universities around the world.
Although WPC develop a wide range of software, Sendra is the first
product that they have used Squish with. The Sendra simulator was
initially created in the early 1990s, but in 2009 a new version,
"Sendra 2009", brought the simulator into the twenty first century
with a modern GUI written in C++/Qt 4. Sendra is developed, tested,
and deployed on Windows.
We were very pleased to speak with Arent Arntzen, Sendra's Scrum master
(project manager). Arent has been doing software testing for about two
years, and Squish is the first GUI testing tool he has used.
Why Squish?
Arent told us that he wanted to use a GUI testing tool for Sendra to
avoid having to create C++ unit test harnesses that to some extent would
have had to reimplement much of the functionality handled by the GUI.
With Squish, none of that is necessary since Squish automatically runs
the entire application under test (AUT), so testing can be done against
a live running application rather than only with individual classes in
isolation. And, of course, Squish can also be used to access and exercise
individual classes as well as the whole AUT.
WPC discovered Squish through a web search, and also met some of the
Squish team at a trade show. They began by requesting an evaluation
version to see if Squish could meet their needs, and after satisfying
themselves of Squish's capabilities--and finding that it was a far
better match for their needs than a competing tool that they also
evaluated--they bought their licenses.
For Sendra testing, WPC have a toolchain consisting of three tools:
Squish, the QtTest module's QTest class, and CruiseControl. The Sendra
testers use CruiseControl to provide continuous integration and testing,
including the management of their Squish-based automated regression
tests. (Squish provides a specific integration for CruiseControl, as
well as integrations for various other test management tools. Squish
itself uses simple open file formats and provides command line tools
that are independent of the Squish IDE which makes it easy to integrate
Squish with just about any testing toolchain.) WPC use the QtTest module
for some unit testing, but almost all of the Sendra testing is done by
Squish, as Arent told us:
We did a long search for testing software and as far as I can see we are
covering all the testing we can think of using Squish. The fewer tools we
need the better--and so the potential coverage of Squish is one of
its truly excellent features.
Squish in Practice
Arent also mentioned that he found Squish very easy to get started with,
although it did take the Sendra testers a few weeks to really understand
all the Squish concepts. And now, having learned Squish, they make use
of almost all of its features on a regular basis.
Arent's team write their tests in Python, and are very happy that they
are able to use an open non-proprietary testing language for their
tests.
At the time of this writing, WPC have developed over thirty Sendra test
cases. They expect to have several hundred test cases by the time they have
finished creating their complete set of test cases for the software as
it stands today. And of course, this doesn't include the additional
tests that will be created to verify bug fixes and new features as part
of Sendra's on-going development and testing process.
One area that was initially a struggle for the Sendra testing team,
concerned the robustness of their test scripts.
They found that
application changes could sometimes lead to test scripts breaking.
Fortunately, by making use of Squish's Object Map, problems that arise
because of changes to application objects can be minimized, or sometimes
eliminated entirely. Arent also mentioned that his team make extensive
use the Squish Spy which shows application objects, including their
names (useful for identifying them in manually written tests), and their
properties at the time of spying (useful for inserting verification
points, either using the Squish IDE or in manually written test code).
The Sendra testers do two separate sets of testing. One set of tests is
short and fast to execute--these are designed to quickly verify
that a binary ready for putting in their ftp download area is sound. The
other set involves more involved and long-running tests and is used to
provide deeper coverage and to keep track of the application's
performance as features are added and bugs fixed.
Arent told us that Squish had brought benefits in many areas, including,
better test coverage, improved application quality, improved release
predictability, and improved test reliability. He also said that using
Squish had reduced the number of person hours required to perform
testing, and has made it possible to have shorter testing cycles. A less
tangible, but nontheless important benefit, is that Squish has improved
the Sendra team's peace of mind. This is because they add tests for
fixed bugs, so their use of Squish is reducing the chances of
regressions, or as Arent put it:
Bugs reported are squished forever so to speak!
The Sendra testers did not do a formal return on investment (ROI) study
on Squish--the time savings, quality, and reliablity improvements
that Squish helped them achieve, have already more than paid for the
initial investment in licenses and learning time.
Conclusion
After the initial few weeks learning period, Sendra's testers mastered
the art of GUI application testing with Squish. Nonetheless,
from time to time, as with any users of a technical product, they needed
help. Here's how Arent summarized Squish's techical support:
Completely timely, precise, helpful, diplomatic, and in short: outstanding!
Squish has made a demonstrable contribution to making Senda into a
better product--and to keeping it reliable, even in the face of bug
fixes and the addition of new features.
froglogic's team would like to thank Arent for taking the time to
share his and his team and company's experience with Squish, and we
look forward to a continued successful relationship.
Get a free and supported evaluation copy of Squish at froglogic.com.
(Submitted by Reginald Stadlbauer of froglogic)
| 
