There are two types of parameters to be considered in performance testing. Firstly, performance parameters, also known as characteristic parameters, are those to be measured and/or calculated. They allow the assessment of a system’s performance under certain conditions. These conditions have to be precisely described and controlled, i.e. by means of the second type of parameters, the baseline parameters, which influence performance but are not themselves the subject of testing.
Table 1 contains a selection of performance parameters that are relevant to industrial automation. For more in-depth descriptions of the terms and a full list of performance parameters, please see the 5G-ACIA white paper “Performance Testing of 5G Systems for Industrial Automation”.
Table 1. Selected performance parameters relevant to industrial automation.
| Transmission time
|
The interval between the moment of delivery of the first user data bit, or byte, of a message to the source reference interface and the moment of delivery of the final user data byte of the same message to the target reference interface. Also known as “one-way latency” in communication systems. |
| Message reception interval | The interval between two consecutive messages received. It depends on the application and its transfer interval. The variation in packet delay (or latency jitter) could be a suitable performance parameter. |
| Response time | The interval between the moment of delivery of the first user data bit, or byte, of a request message to the source reference interface, and the moment the final bit, or byte, of the response message is delivered to the same source reference interface. Also known as “round-trip latency” in communication systems. |
| Data throughput | The number of user data bytes, or user data bits, transferred at the target reference interface per unit of time. |
| Message loss ratio | The ratio of the number of lost messages to the total number of messages sent. |
There are two types of baseline parameters: those related to the application itself, e.g., the user data length, the transmission interval, and the type of communication service; and those related to the environment in which the application operates during testing, e.g., the type of propagation environment, the spatial extent of the industrial facility, and the positions of wireless devices. Both parameter types are defined by 5G-ACIA on the basis of various standards [1, 2, 3, 4] and other publications [5, 6].
A more complete list of baseline parameters of relevance to performance testing of industrial applications with their descriptions can be found in the 5G ACIA white paper “Performance Testing of 5G Systems for Industrial Automation”.
This subtopic is based on information from the 5G ACIA white paper “Performance Testing of 5G Systems for Industrial Automation”. For more information, please refer to the white paper.
[1] 3GPP TS 22.104: “Service requirements for cyber-physical control applications in vertical domains,” V16.5.0, 2020.
[2] VDI/VDE, VDI/VDE 2185-4: “Radio-based communication in industrial automation – Metrological performance rating of wireless solutions for industrial automation applications,” Part 4, 2019.
[3] 3GPP TS 22.261: “Service requirements for the 5G system,” V16.13.0, 2020.
[4] IEC 62657-4: “Industrial communication networks – Wireless communication networks – Part 4: Coexistence management with central coordination of wireless applications,” IEC, 2019.
[5] S. Willmann, Aspects of Dependability, Magdeburg, 2017.
[6] 5G-ACIA, Use Cases and Requirements, 2019.
