In a DevOps or continuous delivery context, regression testing must be automated to reduce the risk of deploying poor quality software into production. Good test automation should include testing at the API level, the user interface level, and the overall system level. The more tests that are automated, and then run early and often, the better.
There are tools to help achieve these goals of testing within the DevOps lifecycle. For the purposes of integration testing, functional testing, performance testing, and service virtualization, IBM offers the following testing tools, which are part of the Rational Test Workbench family.
Integration testing is about validating the integration interfaces, or points between individual functions or components. It costs more to test if you wait for the user interface to be in place on your system under test (SUT), and testing earlier in the lifecycle means that defects are less expensive to resolve. Additionally, as more applications move into the cloud, they will expose various APIs; integration testing tools help you test these APIs, which may not have their own user interface.
Rational Integration Tester helps you to create, modify, and run these integration tests, as well as performance tests, against the integration interfaces. You can build tests by recording existing system behaviors or from requirements, by entering the data to send and the data that is expected in return. You can also create virtual services to remove test dependencies and enable earlier testing. These techniques can be built into a continuous delivery pipeline to ensure each component is tested fully prior to delivery.
Rational Integration Tester Starter Edition is free to use and provides tools for recording and testing the APIs that your systems use every day.
IBM API Test can help you deliver quality software by helping you use the APIs themselves to test your software. With API Test you can visualize a (SUT) and gain insight about test systems and assets.
Functional testing is about validating the functionality of the application under test by exercising its user interface to emulate business scenarios. A functional test records all the user interactions, sensor inputs, and application behavior, and then creates a robust test. You can automate the execution of the tests to verify that the application works as expected.
Rational Test Workbench Web UI Tester provides the capability to do functional testing for web applications. You can execute data-driven tests for the web applications that are based on HTML5 and JQuery technologies. You can also combine the tests with Rational Functional Tester and Selenium test scripts in a compound test to automate end-to-end test flow.
To perform small-scale load testing using actual browsers against web applications, you can add load to your Rational Test Workbench Web UI Tester script by shell-sharing with Rational Performance Tester.
Rational Test Workbench Web UI Tester Starter Edition is free to use and provides capabilities for testing the user interfaces of modern web applications.
Rational Test Workbench Mobile Test Edition enables you to test native, web, and hybrid applications on Android and iOS platforms. In addition to testing the functionality of the applications, you can collect the performance data of the device’s resources such as CPU, memory, network in and out, and battery level. You can also automate the test execution to run the test on multiple devices in parallel.
Rational Functional Tester helps you automate functional tests of HTML, Java, Windows, .NET, Mainframe, Visual Basic, Silverlight, Eclipse, SAP, Siebel, Flex, Ajax, Dojo, GEF, and PowerBuilder applications. You can record user actions to create simplified test scripts that are easy to understand.
The goal of performance testing is to determine the system performance under a defined workload using different types of performance testing such as load, stress, and soak testing. Identifying the presence and cause of system and application bottlenecks can help reduce the risk of the application failing and the system not scaling when many users access the application.
Rational Performance Tester helps you automate load and scalability testing of web, ERP, Citrix, and server-based software applications. You can capture performance requirements (such as load, contention and volume) in your test plans and have Rational Performance Tester measure the results under varying loads. Tests are generated automatically by recording the network traffic that is generated when the client interacts with the application under test running on the server. This network traffic is then emulated on multiple virtual users to create realistic workloads that exercise the key business transactions. Test assets are automatically deployed to as many load generation agents as are needed to achieve the required scalability.
Rational Performance Tester Starter Edition is free to use and provides the ability to examine how your systems perform under load.
IBM Rational Performance Tester on Cloud helps you leverage the cloud to enable large-scale and globally distributed performance testing. This offering uses the SoftLayer public cloud to extend IBM Rational Performance Tester load generation capacity for HTTP, SOA, and socket tests on an as-needed basis employing a pay-as-you-go model. You can also use the VMWare private cloud for load testing. Currently, the VMWare private cloud capability is available on a trial basis.
Testing environments are often expensive and can have limited availability. For example, some components might not be available, others might require payment, and others might already be in use. Service virtualization addresses this problem effectively by providing a substitute for the parts that are missing.
Together with Rational Integration Tester, Rational Test Virtualization Server enables you to virtualize (or stub out) an application or database, or just part of an application or database, removing application test dependencies and reducing setup time and infrastructure costs.
Rational Test Virtualization Server provides a centralized repository for your virtual services so they can be deployed and shared.
Integration with continuous integration and deployment tools
Continuous integration and deployment tools help by providing updated builds for testing and, more importantly, updated test environments.
With continuous integration (CI), changes from each developer are integrated, built, packaged, and deployed to a test environment. You can rely upon this process to identify code compile and integration problems immediately after the code changes are checked into the version control system. It is easier to identify and debug issues from a single developer’s changeset than to sort through many changes checked in from a group of developers.
Using continuous deployment (CD), you can automatically build, package, and deploy changes as they become available. CD requires mature automated build, package and deployment procedures, which are often referred to as a deployment pipeline. With CD, the deployment process is a routine event, enabling development to deliver features more frequently. CD can improve quality because it is highly automated and repeatable, and usually results in smaller deployments. Risk is reduced and, if there are problems, they are easier to address.
The IBM Rational Test Workbench family supports integration with continuous deployment automation tools such as IBM UrbanCode Deploy. Rational Integration Tester supports continuous integration tools such as Jenkins.
Integration with quality and change management tools
Implementing effective quality assurance and testing processes requires a comprehensive framework that starts with a clear understanding of what needs to be tested.
Change management is all about the planning, tracking and controlling of project schedules and resources, change requests, and software source versions. Code changes are easier to manage with a single tool to plan across teams, code, run daily standup meetings, plan sprints, and track work.
The Rational Test Workbench family supports integration with various tools for quality management such as Rational Quality Manager and change management such as Rational Team Concert.
Get started with Rational Test Workbench Starter Editions
The Rational Test Workbench starter editions let you get up and running quickly with the Rational Test Workbench suite, at no cost. From the user interface level down to the APIs that power your systems, along with the performance delivered by those systems, the starter editions allow you to test your applications from top to bottom.
Rational Performance Tester on Cloud
- Using Rational Performance Tester on Cloud
- Rational Performance Tester cloud schedule overview
- How to do more timely and easy performance testing: cloud
Integration with IBM UrbanCode Deploy
- Running functional tests from IBM UrbanCode Deploy
- Running performance tests from IBM UrbanCode Deploy
- Running test workbench tests from IBM UrbanCode Deploy