LoadGen Terminology

Updated September 23, 2024 17:52

To get familiar with the different names and terminology we use in the LoadGen products we have created this article as an overview.

Target Technology

LoadGen supports testing on multiple different technological platforms:

Citrix Cloud, Virtual Apps and Desktops, XenApp, XenDesktop, and Presentation Server. In this article, we help you to choose between the LoadGen - Citrix Basic and Enhanced - Load Profiles.
Microsoft Azure RemoteApp, and Remote Desktop Services.
Windows Virtual Desktop.
VMware Horizon.
Fat Client (Windows 7 and later, and Windows Server 2008 R2 and higher).

Target Server

If you want to test a Citrix or a Microsoft Remote Desktop Services environment, LoadGen needs to know where the target server is so we can connect these target environments and retrieve resources (Published Applications). If you want to use a Fat Client setup, the wizard will skip this step.

For Citrix Virtual Apps and Desktops environments, LoadGen supports Storefront and PNAgent technologies to connect to the target environment. You can also use an ICA file. If you choose to use a Storefront or PNAgent connection, this wizard also needs to know which Published Application to connect to. You can use any of the test users' accounts to connect to the target environment and retrieve a list of available Published Applications to choose from. From LoadGen Q3 2020 release it's also possible to use the Citrix External Logon option. With this option, LoadGen supports authenticating Citrix sessions via a NetScaler, Azure AD, or an Active Directory Federation Server (Azure AD via ADFS).

For Microsoft Remote Desktop Services environments, LoadGen can use an RDS Gateway server to connect. You can configure the connection parameters for the RDS Gateway server in this wizard or connect to the RDS server directly.

Fat Clients

Our software (Load & Performance, End-to-End Monitoring, and LoadGen Functional & Automated testing) support Fat Clients in Windows 7 and higher and Windows Server 2008 R2 and higher environments. If you want to test or monitor your Windows environment you can test your Windows desktops, the way users would.

Testusers, test users or vUsers

To run a test, you need test user accounts (Testusers or vUsers) that will be used to set up sessions and perform the actions in your test (see Flow). These accounts are regular users who are created in Windows Server Active Directory for either Citrix, Remote Desktop Services, VMware, or Windows Fat Client (traditional server-desktop) environments. In the case of the use of a Fat Client desktop, you can also choose a local user account. Please make sure these user accounts are created before you run a test. We advise that these accounts do not have any run-once settings and have a password that does not expire.

For LoadGen WebTesting these users are not actual end-users, but rather, they are placeholder users created for the purpose of the load test. The test users simulate real user behavior and interactions with your web application, allowing you to measure system performance under different load conditions.

Workloads, Flows, test script, or vUser simulation

In the domain of performance testing, a 'workload' is a key term that signifies the collection of tasks and processes that a system or application is subjected to during the testing process. LoadGen, a leading performance testing tool, adopts a structured approach to defining workloads. Here, we delve into the process of defining workloads within LoadGen, focusing on the core components: Flow/Workload, Usertype, Application, Application Blocks, and Actions. The LoadGen Workload contains all the useractions that you want your test users to perform while running a test or monitoring session, and the measurements you want to take. It includes validations (either on window captions, controls, images, or pixels), mouse clicks, checks, and many, many other actions that you can use to build a robust test script or vUser simulation for testing.

You use the LoadGen Studio to build and test your flows.

Flow / Workload: The Flow, also known as the Workload, forms the backbone of the testing process. It represents the sequence of actions that simulated users will undertake during the test. It's the main item that encapsulates everything that will happen in a given test scenario, from the applications that will be used to the types of users that will be simulated.
Usertype: The Usertype serves as a placeholder to define specific sets of users who will run a particular application. The Usertype can be determined based on a percentage of total test users, specific test usernames, or those defined in the LoadGen Director active canvas (test users). Notably, definitions in the LoadGen Director supersede the others. This enables a more granified control over the distribution of user types in the testing process.
(LoadGen) Application: This is the specific application you want to test. In the context of web testing, it's the placeholder for testing a specific site or test case. You must choose the specific browser you want to test with and can add multiple applications (therefore, multiple different browsers in one usertype). Furthermore, you can override the proxy settings on an application as well. These LoadGen applications are interchangeable, providing flexibility in how you structure your testing scenarios.
Application Blocks: Application blocks are used to split applications into manageable sections for better debugging. These blocks are often used for different actions such as starting an application, entering text, or saving a document. Application blocks can also be used for defining essential measurements you want to take, such as the time it takes to load a program, a webpage, or open a specific file. This division into blocks facilitates a more structured and organized approach to testing.
Actions: Actions represent the specific tasks that the simulated users will perform during the test. They aim to replicate real user interactions with the system, providing an accurate picture of how the system will perform under realistic conditions.

Iterations

When running a load test, it's crucial to understand the dynamics of user interaction with the application. The application's load is not static; it fluctuates as users start and finish their interactions. The first user might still be generating load when the last user begins their test, creating a simultaneous load that the application needs to handle.

One way to ensure that all users contribute to the nominal load during the test period is by setting a high enough iteration for the flow. The iteration is essentially the number of times the applications in the flow are repeated. You have a couple of options when setting this:

Specific Number of Times: If you know the exact number of repetitions you want to simulate, you can set a specific number of iterations.
Until a File is Removed from a Shared Location: If you want the load test to run until a specific condition is met - such as the removal of a file from a shared location - this option would be appropriate. The test will continue to repeat until the specified file is no longer present.
Specific Time: If you prefer to run the load test for a certain period, you can set a specific time duration for the iteration. The applications in the flow will continue to repeat for the duration set.

One important aspect to consider when setting up the iterations is the duration of the load test. For instance, the first user might still be generating a load when the last user starts their test. Thus, it's crucial to ensure that the iteration count or duration is high enough to allow all users to generate a nominal load. This ensures that the load test accurately represents the system performance under maximum load.

Transaction or Application

With LoadGen we use transaction as the full time for an application to run in a Workload. Applications are created in the LoadGen Studio and represent all of the actions performed by a test user in a test or End-to-End Monitoring session.

Selecting a specific application will result in the screen here below. An application consists of Application Blocks, which are placeholders to group your actions (i.e. a folder in the Windows Explorer).

Double-clicking an Application Block will open the LoadGen Studio Recorder. Here you can create your user workload.

Sub-transaction or Measurement

The sub-transaction or measurement time is the time it took for a particular action: i.e. waiting on Outlook to open completely or sending a mail, or opening a file. These sub-transactions can be dynamically created as Validations in the LoadGen Studio. In the example below, we start the "stopwatch" and time how long it takes before the New Email button in Outlook is visible for the test user:

Validations in LoadGen

Validations are the most critical components in LoadGen test actions, ensuring that a specific condition or state is met during a test. These validations allow you to verify the expected behavior of the application or system under test. We offer several types of validations that can be used depending on the application environment, including OCR and QR validations. Below are the supported validation types:

1. Caption of Foreground Window

Description: This validation checks the title of the active (foreground) window. It is useful when the window's caption, such as "Notepad" or a part of it like "Word – Document", needs to be validated to ensure the correct application window is in focus.

2. Control in Foreground Window

Description: Some applications like Chrome, SAP, Java, and even browsers like Microsoft Edge and Internet Explorer expose their controls to the operating system. This validation helps in web testing by allowing the LoadGen test to interact directly with and validate controls (buttons, text fields, etc.) from a user perspective.

3. Image Validation

Description: This is a powerful and fast method to find and validate a specific image on the desktop or within a particular area of the desktop. When a control or text isn’t available or cannot be validated via traditional methods, image validation ensures that an expected visual element is present.

4. Single Pixel Validation

Description: Similar to image validation, but focuses on validating a single pixel at a specific location on the desktop. This can be used to check if a UI element is in the correct position, or if a specific color is present in a particular spot.

5. OCR (Optical Character Recognition) Validation

Description: OCR validation allows for the detection and validation of text displayed on the screen, even if it’s part of an image. This is useful in scenarios where the text cannot be selected or exposed as a control. The OCR engine reads the text directly from the screen and compares it to the expected result. This is especially beneficial when dealing with non-selectable text in virtualized environments like Citrix.
Use Case: For example, if an error message or confirmation text is displayed within an image, the OCR validation can extract and verify the text, ensuring the correct information is displayed to the user.

6. QR Code Validation

Description: QR validation allows LoadGen to read and validate QR codes that appear on the screen. This feature can be used in environments where QR codes are generated as part of the application workflow, such as for login processes, two-factor authentication, or any other situation where QR codes are used.
Use Case: In scenarios where a QR code is presented for scanning, the validation ensures the correct QR code is displayed by comparing the extracted data to the expected content. This can be useful for validating secure logins or application workflows involving QR codes.

Systems under Test

You can monitor the performance of a machine in your environment during a test or End-to-End monitoring. This will give you diagnostic information and insight into how your environment is performing in correlation to application performance, read more here.

LoadGen Load & Performance Testing

Load Scenario

A Load Scenario holds up 1 or more phases (Load Ramp-Up) where vUsers are triggered to start their session and thus place load or stress on the back-end infrastructure. In the full version, you also can use Custom Actions where you can send emails or start processes on multiple events during a load test. Read more about Load Scenarios in this article.

Load Ramp-Up

LoadGen will log on testusers in batches. You can configure the ramp-up of the load by setting the number of testusers to log in simultaneously in a single batch (until all users are logged on), and how long to wait between two consecutive batches. Read more about Load Scenarios in this article.

LoadGen End-to-End Monitoring

Monitoring schedule

The monitoring schedule allows you to define how often users are logged on during the monitoring period. By default, LoadGen will launch the testusers every 15 minutes, but this is a variable setting.

Other options include the daily start and end times and which days of the week to monitor.

Microsoft SQL Server instance

LoadGen needs a pre-existing Microsoft SQL Server instance to store all test results data.

Please ask your administrator to provide you with a Microsoft SQL Server instance. Microsoft offers a free Express edition of SQL Server that you can use (it has a size limit of 4GB).

You can create new databases in your SQL Server instance from this wizard if your SQL Server account has the privileges to do so.

LoadGen Agents

Please read our article about LoadGen Agents here.