Dmitry Vinnik explores how the Execution Cloud and its self-healing capabilities can be used to run functional test coverage. The post Functional Testing’s New Friend: Applitools Execution Cloud appeared first on Automated Visual Testing | Applitools.
In the fast-paced and competitive landscape of software development, ensuring the quality of applications is of utmost importance. Functional testing plays a vital role in verifying the robustness and reliability of software products. With the increasing complexity of applications with a long list of use cases and the need for faster release cycles, organizations are challenged to conduct thorough functional testing across different platforms, devices, and screen resolutions.
This path to a better quality of software products is where Applitools, a leading provider of functional testing solutions, becomes a must-have tool with its innovative offering, the Execution Cloud.
Applitools’ Execution Cloud is a game-changing platform that revolutionizes functional testing practices. By harnessing the power of cloud computing, the Execution Cloud eliminates the need for resource-heavy local infrastructure, providing organizations with enhanced efficiency, scalability, and reliability in their testing efforts. The cloud-based architecture integrates with existing testing frameworks and tools, empowering development teams to execute tests across various environments effortlessly.
This article explores how the Execution Cloud and its self-healing capabilities can be used to run our functional test coverage. We demonstrate this cloud platform’s features, like auto-fixing selectors caused by a change in the production code.
Why Execution Cloud
As discussed, the Applitools Execution Cloud is a great tool to enhance any team’s quality pipeline.
One of the main features of this cloud platform is that it can “self-heal” our tests using AI. For example, if, during refactoring or debugging, one of the web elements had its selectors changed and we forgot to update related test coverage, the Execution Cloud would automatically fix our tests. This cloud platform would use one of the previous runs to deduce another relevant selector and let our tests continue running.
This self-healing capability of the Execution Cloud allows us to focus on actual production issues without getting distracted by outdated tests.
Functional Testing and Execution Cloud
It’s fair to say that Applitools has been one of the leading innovators and pioneers in visual testing with its Eyes platform. However, with the Execution Cloud in place, Applitools offers its users broader, more scalable test capabilities. This cloud platform lets us focus on all types of functional testing, including non-Visual testing.
One of the best features of the Execution Cloud is that it’s effortless to integrate into any test case with just one line. There is also no requirement to use the Applitools Eyes framework. In other words, we can run any functional test without creating screenshots for visual validation while utilizing the self-healing capability of the Execution Cloud.
Writing Test Suite
As we mentioned earlier, the Execution Cloud can be integrated with most test cases we already have in place! The only consideration is at the time of writing this post, the current version of the Execution Cloud only supports Selenium WebDriver across all languages (Java, JavaScript, Python, C#, and Ruby), WebdriverIO, and any other WebDriver-based framework. However, more test frameworks will be supported in the near future.
Fortunately, Selenium is a highly used testing framework, giving us plenty of room to demonstrate the power of the Execution Cloud and functional testing.
Setting Up Demo App
Our demo application will be a documentation site built using the Vercel Documentation template. It’s a simple app that uses Next.js, a React framework created by Vercel, a cloud platform that lets us deploy web apps quickly and easily.
To note, all the code for our version of the application is available here.
First, we need to clone the demo app’s repository:
git clone git@github.com:dmitryvinn/docs-demo-app.gitWe will need Node.js of version 10.13 to work with this demo app, which can be installed by following the steps here.
After we set up Node.js, we should open a terminal and run the following command to install the necessary dependencies:
npm installThe next step is to navigate into the project’s directory and start the app locally:
cd docs-demo-app npm run devNow our demo app is accessible at ‘http://localhost:3000/’ and ready to be tested.
Docs Demo App
Deploying Demo App
While the Execution Cloud allows us to run the tests against a local deployment, we will simulate the production use case by running our demo app on Vercel. The steps for deploying a basic app are very well outlined here, so we won’t spend time reviewing them.
After we deploy our demo app, it will appear as running on the Vercel Dashboard:
Demo App Deployed on Vercel
Now, we can write our tests for a production URL of our demo application available at `https://docs-demo-app.vercel.app/`.
Setting Up Test Automation
Execution Cloud offers great flexibility when it comes to working with our tests. Rather than re-writing our test suites to run against this self-healing cloud platform, we simply need to update a few lines of code in the setup part of our tests, and we can use the Execution Cloud.
For our article, our test case will validate navigating to a specific page and pressing a counter button.
To make our work even more effortless, Applitools offers a great set of quickstart examples that were recently updated to support the Execution Cloud. We will start with one of these samples using JavaScript with Selenium WebDriver and Jest as our baseline.
We can use any Integrated Development Environment (IDE) to write tests like IntelliJ IDEA or Visual Studio Code. Since we use JavaScript as our programming language, we will rely on NPM for the build system and our test runner.
Our tests will use Jest as its primary testing framework, so we must add a particular configuration file called `jest.config.js`. We can copy-paste a basic setup from here, but in its shortest form, the required configurations are the following.
module.exports = { clearMocks: true, coverageProvider: "v8", };Our tests will require a `package.json` file which should include Jest, Selenium WebDriver, and Applitools packages. Our dependencies’ part of the `package.json` file should eventually look like the one below:
"dependencies": { "@applitools/eyes-selenium": "^4.66.0", "jest": "^29.5.0", "selenium-webdriver": "^4.9.2" },After we install the above dependencies, we are ready to write and execute our tests.
Writing the Tests
Since we are running a purely functional Applitools test with its Eyes disabled (meaning we do not have a visual component), we will need to initialize the test and have a proper wrap-up for it.
In `beforeAll()`, we can set our test batching and naming along with configuring an Applitools API key.
To enable Execution Cloud for our tests, we need to ensure that we activate this cloud platform on the account level. After that’s done, in our tests’ setup, we will need to initialize the WebDriver using the following code:
let url = await Eyes.getExecutionCloudUrl(); driver = new Builder().usingServer(url).withCapabilities(capabilities).build();For our test case, we will open a demo app, navigate to another page, press a counter button, and validate that the click incremented the value of clicks by one.
describe('Documentation Demo App', () => { … test('should navigate to another page and increment its counter', async () => { // Arrange - go to the home page await driver.get('https://docs-demo-app.vercel.app/'); // Act - go to another page and click a counter button await driver.findElement(By.xpath("//*[text() = 'Another Page']")).click(); await driver.findElement(By.className('button-counter')).click(); // Assert - validate that the counter was clicked const finalClickCount = await driver.findElement(By.className('button-counter')).getText(); await expect(finalClickCount).toContain('Clicked 1 times'); } …Another critical aspect of running our test is that it’s a non-Eyes test. Since we are not taking screenshots, we need to tell the Execution Cloud when a test begins and ends.
To start the test, we should add the following snippet inside the `beforeEach()` that will name the test and assign it to a proper test batch:
await driver.executeScript( 'applitools:startTest', { 'testName': expect.getState().currentTestName, 'appName': APP_NAME, 'batch': { "id": batch.getId() } } )Lastly, we need to tell our automation when the test is done and what were its results. We will add the following code that sets the status of our test in the `afterEach()` hook:
await driver.executeScript('applitools:endTest', { 'status': testStatus })Now, our test is ready to be run on the Execution Cloud.
Running test
To run our test, we need to set the Applitools API key. We can do it in a terminal or have it set as a global variable:
export APPLITOOLS_API_KEY=[API_KEY]In the above command, we need to replace [API_KEY] with the API key for our account. The key can be found in the Applitools Dashboard, as shown in this FAQ article.
Now, we need to navigate to the location where our tests are located and run the following npm test command in the terminal:
npm testIt will trigger the test suite that can be seen on the Applitools Dashboard:
Applitools Dashboard with Execution Cloud enabled
Execution Cloud in Action
It’s a well-known fact that apps go through a lifecycle. They get created, get bugs, change, and ultimately shut down. This ever-changing lifecycle of any app is what causes our tests to break. Whether it’s due to a bug or an accidental regression, it’s widespread for a test to fail after a change in an app.
Let’s say a developer working on a counter button component changes its class name to `button-count` from the original `button-counter`. There could be many reasons this change could happen, but nevertheless, these modifications to the production code are extremely common.
What’s even more common is that the developer who made the change might forget or not find all the tests using the original class name, `button-counter`, to validate this component. As a result, these outdated tests would start failing, distracting us from investigating real production issues, which could significantly impact our users.
Execution Cloud and its self-healing capabilities were built specifically to address this problem. This cloud platform would be able to “self-heal” our tests that were previously running against a class name `button-counter`, and rather than failing these tests, the Execution Cloud would find another selector that hasn’t changed. With this highly scalable solution, our test coverage would remain the same and let us focus on correcting issues that are actually causing a regression in production.
Although we are running non-Eyes tests, the Applitools Dashboard still allows us to see several valuable materials, like a video recording of our test or to export WebDriver commands!
Want to see more? Request a free trial of Applitools Execution Cloud.
Conclusion
Whether you are a small startup that prioritizes quick iterations, or a large organization that focuses on scale, Applitools Execution Cloud is a perfect choice for any scenario. It offers a reliable way for tests to become what they should be – the first line of defense in ensuring the best customer experience for our users.
With the self-healing capabilities of the Execution Cloud, we get to focus on real production issues that actively affect our customers. With this cloud platform, we are moving towards a space where tests don’t become something we accept as constantly failing or a detriment to our developer velocity. Instead, we treat our test coverage as a trusted companion that raises problems before our users do.
With these functionalities, Applitools and its Execution Cloud quickly become a must-have for any developer workflow that can supercharge the productivity and efficiency of every engineering team.
The post Functional Testing’s New Friend: Applitools Execution Cloud appeared first on Automated Visual Testing | Applitools.