In part 1 of this series, “Supply Chain Agility Through Unattended Testing,” we established that unattended, automated testing is the juice that fuels high-powered organizations to achieve more. Swapping out traditional manual testing for unattended testing enables businesses to...
In part 1 of this series, “Supply Chain Agility Through Unattended Testing,” we established that unattended, automated testing is the juice that fuels high-powered organizations to achieve more. Swapping out traditional manual testing for unattended testing enables businesses to keep pace with increasing amounts of change while remaining competitive, embracing change as desirable, and fostering a culture of positive, strengthening, continuous testing.
The continuous testing mindset is a paradigm shift for how businesses operate. As technology evolves and user expectations soar, automation is indispensable. The continuous delivery pipeline is the implementation of this paradigm shift. It’s a series of steps that, taken as a whole, considers automated builds, tests, and deployments as one release workflow.
Organizations looking to take these steps towards continuous and unattended testing may find it somewhat daunting, but it doesn’t necessarily have to be all-or-nothing. After all, the best testing regimen isn’t just frequent, but also manageable and iterative.
But what different technology components do businesses require to successfully implement unattended testing? In this second part of the series, we’ll delve into how each of the five pillars of continuous testing contributes to a single, ultimate goal: efficient, timely code deployment, empowering businesses to go live with confidence.
The 5 Pillars of Continuous Testing
It helps to view the five pillars of continuous testing as individual items on a conveyor belt — in this case, an analogy for the pipeline.
1. Pipeline Platform
The pipeline platform tool helps teams facilitate running pipelines — the same pipelines that run automated, unattended tests.
Pipeline files are the code that drives the “conveyor belt” of the pipeline. Meanwhile, a pipeline agent — like Jenkins — is a service that runs the “jobs” in a given continuous delivery pipeline. Those jobs, like items on a conveyor belt, are picked up and completed by the agent.
Jenkins is a great choice because it’s an open-source tool, with solid support for both Azure and Amazon Web Services (AWS). It doesn’t come with licensing costs or other barriers to entry, and many enterprise customers are already familiar with Jenkins — making integration with leading testing solutions straightforward. Additionally, it comes with some plugin tools which are very useful for unattended testing.
When a pipeline agent like Jenkins is running, test data are streamed to Jenkins’ “manager,” which allows users to look at the test steps and output line by line.
2. Repository
Continuous testing requires integration from source control management. The source code repository is therefore the second pillar of continuous testing. Meanwhile, version control tools enable multiple users to simultaneously revise, add to, and test the same code base. It’s also considered the “single source of truth” because it tracks each and every code base change.
In most cases, repositories are Git-based, which includes GitHub and GitHub Enterprise, as well as Azure Repos. For continuous testing, any Git-based repository should work as long as it can connect to the customer’s pipeline platform of choice.
The repository platform itself is typically a platform as a service (PaaS). Popular choices include GitHub, BitBucket, and Azure DevOps (ADO). They all offer similar features, but ADO — a Microsoft product — has additional tools.
It’s easy to hook into the chosen source code repo: Cycle Feature Files, testing artifacts, and supplemental files for running the pipeline file all “live” in the repository.
3. Defect/Task Management Tool?
Once tests are complete and the results are in, another round of work begins. When tests pinpoint bugs in the code, they also identify the need for fixes.
Any unattended testing platform should integrate with defect (or bug-tracking) task managers like JIRA, Zoho Projects, and Bugzilla. Testing platforms should also seamlessly communicate information to Slack, Teams, or another instant messaging tool. Integrations like these are powerful because they connect automated testing to an automated to-do list for developers. They can take action directly from Slack and Teams instead of continuously looking at an interface and monitoring for change. And if teams want to save information, this can be done via cloud, or zip or PDF file, with very little work.
There are different ways to trigger runs for specific jobs: manually by launching pipelines themselves, using cron triggers — such as for regression testing — or based on code commits and pull requests.
4. Testing Agents
With pipelines scripted, the next phase is to understand where the pipelines are going to run. The pipeline platform is the tool that drives the pipeline to run, but pipelines need agents in order to actually run.
In the case of unattended testing, pipeline agents are virtual machines — such as AWS EC2 instances or Azure Virtual Machines (VMs), depending on which cloud an organization is running on — generally created dynamically, but in some cases manually.
Pipeline agents are testing agents or nodes created on-demand using tools that exist within pipeline platforms. If Jenkins runs with Azure, Microsoft creates the Jenkins pipeline to directly support Azure integrating with Jenkins: the Azure VM Agent plugin. These plugins allow the Jenkins manager to communicate with the cloud, essentially saying, I need a Windows Server virtual machine. Can you build me one?
The cloud builds the VM, either from an image with the testing solution (and any other necessary tools) already loaded on it or automatically, based on configuration during the check-in process. The VM is dynamically provisioned, configured, and then checked into Jenkins as a testing agent.
A continuous testing mindset means pipelines should be running on fresh testing agents. Consistency and predictability are key: No enterprise should run them on machines with different tools and configurations. Code is pulled directly from the repo — meaning it stays true to what’s in the repo — and the pipeline itself runs in the same sequence every single time. If the resulting test fails even with correct test variables, developers know to investigate more closely.
5. Test Engine
The last pillar is the testing engine itself which will be used to execute tests against the technologies and the different interfaces of the system under test.
With automated test scripts built and a stable test suite, teams need to integrate the command line version of the automated testing tool to the pipeline and testing agent. Cycle-CLI triggers interactive desktop tests across interfaces including browser drivers, Windows Application Drivers (WinAppDrivers), and radio frequency terminals.
To Cloud or Not To Cloud
Small businesses have traditionally used on-premise hardware. But the cloud is so accessible now that running infrastructure on it reduces upfront capital expenditure for deployment, shifting it into an operating expense model.
Ultimately, any business can spin up necessary workloads and delete them when no longer needed — only paying for them while they’re in use. As such, teams with medium- to large-scale testing needs, frequently-run tests, or concurrent users should consider moving their testing operations to cloud infrastructure.
A pipeline that’s truly continuous from start to finish uses workloads for tests, and then deprovisions them from the cloud when finished. This way, you only have paid for the agent while it was online, and the second it is deallocated, the cost goes away. There is tooling available for Jenkins to help dynamically provision testing agents to execute tests on cloud resources.
It’s Time To Accelerate Business Value
Some customers have ample engineering teams that take Cycle and run with it, integrating it with their existing testing pipelines. Others are only taking their first steps towards repeatable and scalable test automation.
The cloud engineering team at Cycle Labs has developed pre-built assets to help accelerate time-to-value for customers who are just making this transition, and are always happy to assist with custom pipeline deployment needs.
One of Cycle’s most successful customers utilized automation testing by setting up a testing pipeline infrastructure that allowed them to scale sustainably upwards of 20,000 test runs over five months. As a result, they continuously delivered risk-mitigated improvements, enhancements, and updates to their system.
This is unattended testing in action. Changing systems to meet increased demand is difficult — transforming businesses from top to bottom even harder. But with unattended testing, the near-impossible becomes a reality for organizations to seize and make work.
Are you interested in learning more about implementing test automation in your warehouse system implementation? Read our customer success stories, check out additional blog posts, or learn more about the Cycle platform.
The post The 5 Pillars of Continuous Testing: The Roadmap for Deploying Unattended Test Solutions appeared first on Cycle Labs.
