Continuous integration (CI) has emerged as one of the most efficient ways to develop code. But testing has not always been a major part of the CI conversation.
In some respects, that’s not surprising. Traditionally, CI has been all about speeding up the coding, building, and release process. Instead of having each programmer write code separately, integrate it manually, and then wait until the next daily or weekly build to see if the changes broke anything, CI lets developers code and compile on a virtually continuous basis. It also means developers and admins can work together seamlessly, since the programming and build processes are always in sync.
Unfortunately, the quality assurance team does not necessarily reap the same benefits. While CI assures that your app keeps building successfully as the code is continually updated, it doesn’t automatically test how new builds behave within different types of environments. An otherwise well-run CI operation might require app testing to be done separately, on a non-continuous basis, instead of building it into the rest of the process.
This poses real problems for an organization. Unless you add automated testing to your CI mix, you could end up with an app that users can download and install properly, but which suffers from critical usability issues in certain browsers or operating systems. Arguably, an app that installs successfully but frustrates users due to lack of testing is worse than one that doesn’t install at all.
To make matters worse, there’s a sort of paradox that affects CI and testing. It goes like this: The better you are at CI, the more quickly your app gets updated. But as you make changes faster, you also increase the possibility that environment-specific bugs have been introduced, but have not yet discovered through tests. That’s not good, either.
Testing and Continuous Integration: Squaring the Circle
So far, this post might sound like a rant against CI. But I’m not actually here to tell you to revert to the slow, siloed development processes of yore (waterfall, anyone?) Yes, that would be one (silly) way to simplify your testing needs. But you can do much better by integrating testing directly into your CI process.
If you want to have your CI and eat it, too (or something like that), you need a testing solution that can deliver the following features:
- Scalability: You need to be able to run as many or as few tests as your ever-changing codebase requires.
- Automation: Without it, the whole CI process breaks down.
- Parallel testing: This speeds up testing so that your tests can be as fast as the rest of your CI process.
These features give your testing process the ability to be continuous.
Sauce Labs and CI Integration
If you’re reading this post, you may already be familiar with the ways Sauce Labs is designed to facilitate CI. But how do you actually take advantage of Sauce’s CI-friendly features?
The answer is that you integrate Sauce directly into your CI environment. And thanks to Sauce’s pre-built integration tools, doing this is a lot easier on all of the major CI platforms than it might seem.
The specific process for integrating Sauce with your CI platform of choice varies according to the platform you use. The processes are minutely detailed on the Sauce wiki, so I won’t repeat them here. But it is worth summarizing how the process works for the CI platforms you’re most likely to be using, which include:
- Visual Studio Team Services: Integration here is very simple and straightforward. Basically, you enter your Sauce Labs account credentials through the VSTS dashboard, and you’re off to the races. Firewalled development environments can be supported through Sauce Connect.
- Atlassian Bamboo: For this CI solution, Sauce offers a simple integration plugin, which you can configure easily through environment variables.
- Jenkins: Same story here as Bamboo: Simply install a plugin, configure as needed and start testing.
- TeamCity: A plugin is also available for CI through this platform. The only quirk is associating test results with Sauce jobs properly, which you can do easily enough by writing the session ID to stdout.
If your favorite CI platform is not on the list, fret not. Sauce Labs is also well supported on CircleCI, Solano CI and Travis CI by the developers of those platforms.
So, if you’re not already integrating testing with CI, it’s high time you should.
Source: Pixabay.com
Hemant Jain is the founder and owner of Rapidera Technologies, a full service software development shop. He and his team focus a lot on modern software delivery techniques and tools. Prior to Rapidera he managed large scale enterprise development projects at Autodesk and Deloitte.
