Performance testing comprises the measurement and/or calculation of one or more predefined performance parameters. Such testing is carried out to obtain meaningful results for informed, accurate decision-making on, for example, the ability of a particular 5G system to execute a defined task or provide a defined service.
Reference interfaces are where performance test measurements are made. They must be selected from the available, i.e. exposed, interfaces. A measurement requires at least one reference interface.
Once performance parameters and baseline parameters for the system under test (SUT) have been defined, it is necessary to establish the testing system for measuring and/or calculating corresponding performance values. The testing system needs to accurately measure the performance parameters and accurately reproduce or maintain the baseline parameters, see Figure 1. A system under test may comprise one or more independent wireless systems.
The required values for performance parameters, and the values for baseline parameters influencing the wireless communication system(s), are defined in accordance with the real-world production system and/or the automation system controlling that production system.
Performance testing can be carried out in three differing testing environments:
• Target real-world environment
• Reference environment
• Laboratory environment.
A given wireless industrial automation system will always comprise three systems: physical production system, distributed automation system and wireless communication system, shown in Figure 2.
The testing system is introduced at the interface between the automation and wireless communication systems, which is called the reference interface.
The result of any measurement applies to the performance of one or multiple logical links, consisting of one or multiple physical links. Wherever an automation device is connected to a wireless communication module, a corresponding reference interface exists. This is required to perform tests.
Figure 4 shows two reference interfaces between local application functions and the wireless communication functions, with a total of six endpoints.
The two devices with their local application functions exchange messages via their wireless communication functions. To test performance, e.g. message loss ratio or transmission time, messages are sent from the application function as part of the testing system to the communication function via the reference interfaces. A performance parameter measurement between logical source end point (LSEP) and logical target end point (LTEP) is event-triggered by message send time and message receive time at the endpoints. The testing system is connected to the reference interface hardware, i.e. the testing system takes the place of the automation application.
Examples of performance testing can be found in the subtopics “Examples of Round-Trip Performance Testing for Devices and Networks” {link to Examples of round-trip performance testing for devices and networks here} and examples of one-way logical link testing in the subtopic “Examples of One-Way Logical Link Performance Testing.” {link to Examples of one-way logical link performance testing here}
[1] L. Underberg and L. Rauchhaupt, “Performance testing of novel wireless communication networks for industrial automation on the example of 5G,” Conference on Communication in Automation (KommA), 2019.
