Inside UiPath Test Manager

In this blog post we'll explore UiPath Test Manager and its connectors, while discussing crucial implementation details that can propel test strategies towards success.

It's essential for a Test Management product to ensure a smooth data flow between Requirement Management tools and Defect Management tools. The inclusion of context and the ability to track test results are essential components of a comprehensive test inventory. With a multitude of Requirement Management and Defect Management tools available, customers may choose different options based on their needs. Therefore, providing seamless integration with these tools is vital for the success of any Test Management product, including UiPath Test Suite. To meet this user requirement, connectors are used.

In some instances, UiPath Test Suite is utilized solely for test automation purposes. In these situations, users may employ a third-party Test Management tool to keep track of their tests, while leveraging the features of UiPath to automate and execute them. In such scenarios, it's important to ensure that the test results are synchronized with the external Test Management tool, which serves as a second use case for connectors. These connectors facilitate the smooth transfer of results from UiPath Robot's automated test executions to the external system.

Connectors 

UiPath Test Manager uses connectors to establish connectivity between external ALM tools and itself. These connectors have been created with a generic interface that outlines how an external system should be seamlessly integrated into its structure. Once implemented, these pre-packaged connectors provide specific capabilities required for integrating particular ALM systems into Test Manager efficiently.

Users have complete control over configuring these connectors on an individual project basis within Test Manager itself. The platform allows users to seamlessly connect unlimited external systems and projects with ease through this approach.

When establishing connectivity between Test Manager and various third-party Requirement Management/ Defect Management (or Test Management) tools, each Connection represents an individual instance that's been meticulously configured based on specific requirements and guidelines. This configuration includes crucial information like URL of RM/DM tool, selection of relevant projects for data transfer purposes, authentication credentials among other vital aspects necessary for establishing connectivity competence effectively at all times across different contexts within the UiPath framework.

Below you can observe a screenshot of a configured connection.

And by using this information, Test Manager facilitates the seamless data flow between Test Manager and the third party tool.

Two way communication between Test Manager and third Party System in UiPath Cloud offering

The diagram above displays the various communications taking place between the Test Manager and the third party system. Additionally, it illustrates the type of data being exchanged between these systems. Additional information can be found in the following sections.

Connector Types

Below are the several ALM systems that Test Manager currently supports.

1. Jira Server/Data Center/Cloud

2. Azure DevOps

3. SAP Solution Manager

4. ServiceNow

5. Xray app for Jira

6. Redmine

Moreover, Test Manager includes a convenient webhook connector feature. Through this functionality, users are able to effortlessly establish and maintain webhooks for specific events within Test Manager. Once these webhooks have been registered, they'll be promptly activated whenever the corresponding operation occurs.

Implementation Details

In this context, we're technically faced with just two methods of communication.

1. Test Manager to the third party system

2. third party system to Test Manager.

1. Test Manager - Third party system

Two essential actions fall under this category: introducing flaws into a third-party defect management tool and updating the test execution details in an external Test Management tool.

Test Manager synchronizing defects and test execution to third party system

Third-party tools typically offer REST APIs for their business entities. Test Manager utilizes these REST APIs to generate defects or record test execution results in these tools. Moreover, these communications occur asynchronously as Background Tasks. As an example, when a user submits a defect creation request, a background task is registered, and its execution is scheduled for later.

Authenticating APIs of third party systems

The REST APIs of third-party tools are always authenticated, using one of several available authentication mechanisms. Therefore, Test Manager must select the appropriate authentication mechanism based on the connector type when making REST API calls.

Usage of background tasks

The data which needs to be uploaded during creation of defect or test execution can be huge. Uploading the necessary data, including screenshots and attachments, during defect or test execution creation can be a significant task. Additionally, if the third-party tool used is hosted in a different location and is dealing with high volumes of traffic, Test Manager may not have access to this information. These factors can result in long blocking operations for defect and test execution creation if the process is handled synchronously.

To address this issue, background tasks are employed to make the process asynchronous.

2. Third party system - Test Manager

There's a collection of operations that are categorized in this group.

1. Synchronizing requirements from any Requirement Management tool.

2. Synchronizing Test Cases from any Test Management tool.

In most cases, both requirements and test cases in the third party tool are treated as the same entity, but with some slight differences. This means that Test Manager can usually use the same approach to synchronize them. However, it's important to note that this approach may vary depending on the ALM system being used.Test Manager typically employs two different approaches to synchronize these details.

1. Registering a webhook in ALM system

2. Polling

1. Registering a webhook in ALM system

Most ‌ALM tools come equipped with the capability to register webhooks for specific events. Once an event takes place within the ALM tool, the registered webhook URL will be activated.

third Party System invoking Test Manager APIs on Requirement and TestCase changes

How webhooks are registered in third party tool?

ALM systems typically offer the capability to enhance their functionality by incorporating plugins into their ecosystem. To this end, Test Manager develops plugins and distributes them through the appropriate third-party tool marketplace. This enables customers to integrate the plugin into their third-party system and customize the Test Manager instance they wish to connect with. The attached screenshot displays the Test Manager plugin designed for Jira Cloud.

Test Manager plugin in Jira Cloud

To ensure synchronization, users are required to configure the issue types and provide the Test Manager URL. Additionally, an API key must be supplied. It's important to note that not all ALM tools allow for the addition of plugins to their ecosystem. In these instances, Test Manager assumes responsibility for registering webhooks in the ALM tool. This is accomplished by making the necessary API calls during connection creation.

Webhook URL and authenticating API requests from third party tool

Listing the URLs registered in Azure DevOps for creation of requirements/testcases.

Test Manager swagger

Furthermore, Test Manager exposes a range of APIs for each connector that support webhooks in a comparable fashion. It's worth noting the inclusion of a token placeholder, which serves as authentication for requests originating from the third-party system. Naturally, one may inquire about the source of this token. The operational sequence follows suit.

1. Test Manager: project admin or organization admins have to create a connection in Test Manager project.

2. Test Manager: once a connection is created, Test Manager creates a guid(token/API Key)

3. third party tool: User have to provide this token with the plugin configuration.

4. third party tool: creates webhook URLs with this token, and these URLs will be registered against events.

5. third party tool: when any subscribed event occurs, the URL will be invoked.

6. Test Manager: extracts token from the API and validates it.

7. Test Manger: if valid token, continues the creation/updation of requirement/testcase

2. Polling

Unfortunately, many third party systems don't offer support for webhooks. This can be quite limiting when it comes to syncing data from these systems. In such cases, our only option is to use polling.

To give users more control over the syncing process, we allow them to specify the polling interval. This way, they can determine when they'll see the synced data in Test Manager. When creating a connection, users will be able to configure the polling interval through a designated field. Test Manager will then pull data from the third party system at intervals determined by this user-provided value. To retrieve the necessary data, we rely on REST APIs provided by the third party tool.

This approach is currently being used for Redmine's requirement sync purposes.

Frequently asked questions

We consistently monitor the usage patterns of our users and search for opportunities to enhance their experience. In fact, we have already identified crucial points in the connector area and are currently working on implementing them.

Adding Test Manager IPs to allowed list

Jira systems allow users to exert control over API requests by establishing rules based on the IP address of origin. Users can choose to set up a list of approved IPs, which means that only requests originating from these designated addresses will be accepted. Any requests coming from other IPs will be rejected. Currently, Test Manager doesn't have any set static IPs for outgoing traffic.

This poses a challenge for customers who are subject to inbound network restrictions. However, efforts are underway to address this issue by implementing support for static IP ranges.

Validation errors during the connection creation

During the process of creating a connection, it's important to consider some specific details related to the third party system. The Test Manager has implemented validations to make sure that all the entered fields are valid. These validations primarily occur on the server side. When a user saves the connection, the provided data is sent to the server side and validated by establishing a connection with the third party system. Occasionally, users may need multiple attempts to input accurate data. We have recognized this as an area that requires improvement, and we're actively working to enhance this aspect in order to provide a more seamless experience for users.

Obstacles

When the Test Manager needs to support a new connector, there's a significant learning curve involved. The first step is to understand the domain objects of the new tool that needs to be integrated. Additionally, we must familiarize ourselves with the APIs that have already been exposed by these objects. It's important to note that domain objects can't be directly mapped between Test Manager and the third party system. Therefore, we must find ways to effectively map these objects between systems.

On top of that, there are even greater challenges in the plugin area. Each tool uses its own technology for plugin development. For example, the Jira Server plugin uses Java, while Jira Cloud plugin uses React and Node. The Redmine plugin uses Ruby on Rails, Azure DevOps uses React, and Service Now maintains its own ecosystem with a framework and development kit. As you can observe, there are numerous technologies involved.

Considering all these factors, careful consideration has been given to establish a mechanism that can handle all concerns related to plugins effectively. It's possible that in the near future, an innovative and sophisticated method for syncing data between systems will be introduced alongside connectors.

Our team contributing to UiPath Test Manager has made a significant contribution to our world of automation. If you are interested in joining them, be sure to check our career website for available roles.