Introduction

After a decade of progress in Industry 4.0, the European Commission introduced the concept of Industry 5.0 to address emerging societal challenges [1]. Industry 5.0 emphasises the collaboration between humans and machines, which operate within the same environment. Consequently, achieving a deep integration of machine intelligence and human capabilities becomes essential for reaching higher levels of efficiency [2].

The increasing interest in robotics, AI, and human-centred technologies is creating new opportunities to improve industrial processes and operator well-being. Many researchers are working to increase human–robot collaboration [3–5].

Previous studies involved the presence of robots to aid human workers during hazardous tasks. The presence of robots for heavy lifting and other ergonomically taxing activities significantly improved the ergonomic risks for the operators [6, 7].

In such contexts, the integration of robotic or automated solutions into existing workflows may remain challenging, mainly due to the complexity and unpredictability of human actions. In recent years, supporting manual operations from an ergonomic perspective has emerged as a promising avenue to enhance operator safety, well-being, and overall productivity [8].

An ergonomic workstation reduces worker fatigue, helps prevent musculoskeletal disorders, decreases absenteeism, and improves overall worker performance [9]. Ergonomic assessment typically relies on the identification of hazardous or non-optimal movements, which, if monitored in real time, could be mitigated or delegated to automated systems. However, developing such support tools requires the ability to accurately recognise and classify human movements as they occur in industrial settings.

To address this challenge, this work investigates the use of motion capture (MoCap) data to classify operator movements with the ultimate aim of identifying non-ergonomic tasks that may be suitable for automation. We construct a dedicated dataset by recording MoCap data from multiple individuals performing predefined movements, and we use this dataset to train a Machine Learning (ML) classifier. The resulting system is then evaluated both on the collected dataset and in a real industrial use-case scenario to assess its robustness and practical applicability.

Moreover, this work aims to highlight the ergonomic data with the movements that are potentially risky. The MoCap data are used then as an input for IPS IMMA (Intelligently Moving MAnikins) [10]. This software allows the digitisation of the manikins, and it can automatically compute the Rapid Entire Body Assessment (REBA) of the input data. The objective is to obtain a REBA report from IPS IMMA and to automatically analyse it, ultimately generating a report that highlights risky movements based on the REBA scores. The human digital twins in IPS IMMA are created from and move with the MoCap recordings. Thus, each joint angle can be measured directly in the software, eliminating the observer variability inherent in traditional observational ergonomic assessments. This ensures that the REBA scores obtained are objective and reproducible, removing the need for manual validation of the index.

The term ergonomics first appeared in an article published in a Polish journal by Wojciech Jastrzębowski [11] in 1857. The International Association of Ergonomics [12] defines ergonomics, also known as human factors engineering, as a scientific discipline focused on the analysis of the interactions between humans and machines, robots, or other components of a given system, aimed at improving human activity and enhancing overall system performance [13]. It is possible to apply ergonomics wherever there is human–machine interaction. Different examples are available in the literature, such as the healthcare sector [14, 15], agriculture [16], manufacturing, logistics [17], etc. In an industrial scenario, the application of Human Factors Engineering also seeks to enhance the performance of manufacturing processes by improving the usability of systems for operators, thereby reducing the likelihood of errors, increasing efficiency, and supporting both the physical and the mental well-being of users [8]. Various methods are used in ergonomic assessments. These can be broadly divided into techniques based on the employee’s physiological effort (e.g., electromyography or heart rate monitoring) and those based on observation. The latter are the most widely used, as they do not require specialised equipment and allow assessments to be conducted over longer periods [18]. Observation-based methods can be further categorised into four groups:

• Methods that assess the posture of body segments during task execution;

• Methods that evaluate an employee’s workload;

• Methods focusing on specific body segments;

• Questionnaire-based methods.

There are different postural analysis tools used in ergonomic analysis, and one of these is the REBA. REBA is an ergonomic assessment tool designed to evaluate the risk of work-related musculoskeletal disorders associated with postural load [19]. It systematically analyses body postures, force, repetition, and coupling to generate a risk score, supporting the identification of tasks that require ergonomic intervention.

In the present work, we use MoCap to record the motion and assign it to a human digital twin. MoCap is a technology used to digitally record human movement with high spatial precision and high temporal resolution. MoCap systems can be classified into three main categories based on the underlying sensor technology:

• 1.

  Optical marker-based systems: These systems use high-speed cameras and reflective markers placed on the subject’s body. By tracking and interpolating the position of each marker, the system reconstructs their absolute coordinates within a 3D environment [20].

• 2.

  Inertial sensor systems: These rely on Inertial Measurement Units (IMUs) attached to the body. IMU-based systems are relatively simple to operate, cost-effective, and minimally intrusive in industrial workflows [21].

• 3.

  Markerless systems: Modern markerless approaches leverage artificial intelligence and computer vision to capture movement without the need for physical markers. While they offer minimal intrusiveness within the working environment, their accuracy is generally lower than that of marker-based or inertial systems [22].

The scientific literature shows that MoCap technology is applied across a wide range of domains, including gaming, industrial settings, sports, healthcare, and rehabilitation. In the context of industrial applications, ergonomics represents a particularly relevant area in which MoCap systems have gained increasing importance [23].

Nowadays, thanks to the advent of Industry 5.0, the scientific literature shows a growing interest in the use of MoCap for ergonomic evaluation, as demonstrated in [23–25]. Butlewski et al. [18] mentioned four key advantages of using MoCap in ergonomic assessments:

• Improved speed and accuracy in movement data collection;

• The ability to capture conditions that closely resemble real-life scenarios;

• Simultaneous processing of large volumes of data;

• The possibility of translating MoCap data into standard ergonomic assessment methods.

Bortolini et al. [20] present an original motion analysis system based on MoCap data, designed to facilitate the calculation of common ergonomic indices typically used in manufacturing environments. Building on this approach, Bortolini et al. [26] introduce a framework based on Microsoft Kinect that enables the simultaneous assessment of production performance and operator ergonomics during manufacturing activities. Another Kinect-based application is presented in [8], where MoCap is used to dynamically adjust workstation configurations in real time, improving ergonomics and reducing the risk of biomechanical overload and awkward postures during assembly tasks. Battini et al. [27] propose an in-house ergonomic platform capable of evaluating four ergonomic indices and providing real-time feedback to workers using Xsens MoCap technology. Later, Berti et al. [28] demonstrates that the WEM-Platform can monitor multiple workers simultaneously, delivering real-time ergonomic index progression during collaborative tasks. MoCap can also support ergonomic assessment by analysing human–environment interactions within design processes.

For instance, Keshvarparast et al. [29] presents a mathematical model for ergonomically designing a human–robot collaborative workstation in the pre-deployment phase using MoCap data. Kačerová et al. [30] investigates whether Virtual Reality (VR) combined with MoCap can support ergonomic improvements in working postures, showing that this integration is highly effective. Berti et al. [22] discuss an architecture that leverages MoCap data to monitor human operators in real time, detecting awkward postures and unbalanced workloads.

Finally, Brosche et al. [31] highlight a key limitation of traditional ergonomic methods: they often identify hazardous postures or workplaces without providing practical guidance for improvement. To address this gap, the author proposes a structured methodology that analyses a workstation, identifies ergonomically risky postures, and suggests improvements through targeted automation that replaces operators in hazardous tasks. Ceylan et al. [32] propose an automatic algorithm that processes MoCap data to generate the design of a mechanical figure capable of reproducing the recorded motion. Simonetto et al. [33] introduce a methodological framework to support assembly workstation designers, integrating MoCap and VR technologies with a particular focus on operator well-being.

Bertoli et al. [6] present a methodology for automating an existing manual workstation by leveraging MoCap and Digital Twin technologies. In previous work, task classification was performed manually; this paper aims to classify the operator’s movements automatically.

An important research direction concerns the extraction of operators’ movements directly from MoCap acquisitions, to identify all the tasks required to complete the analysed process. Vantigodi and Venkatesh Babu [34] propose a method for human action recognition from MoCap data based on the variance of skeletal joints. Similarly, Ijjina and Mohan [35] develop a human action recognition approach using a convolutional neural network trained on MoCap data.

Cho and Chen [36] introduce a system for action recognition from skeletal data using deep neural networks, with features derived from relative joint positions, temporal differences, and normalised trajectories. Cai et al. [37] implemented a hybrid random forest (RF) algorithm for the recognition of seven human motion patterns. Iriondo Pascual et al. [38] use the RF algorithm to classify hand postures of grip types. Jaén-Vargas et al. [39] evaluate various deep learning techniques by processing raw MoCap data as time series through sliding windows, successfully predicting human activities such as walking, sit-to-stand transitions, and squatting; deep learning is employed due to its ability to model raw sequential data effectively.

Igelmo et al. [40] present an AI-based approach capable of predicting both human movement and its ergonomic evaluation based on activity classification. Barnachon et al. [41] propose a method using histograms of 3D MoCap data for action recognition, enabling early prediction of the ongoing activity. Finally, Kadu and Kuo [42] introduce an automated MoCap data classification technique based on a tree-structured vector quantisation method.

An emerging line of work applies these classification techniques directly to standardized ergonomic scoring tools rather than to generic action labels. Li et al. [43] propose an automated Rapid Upper Limb Assessment (RULA) system utilizing a two-stage deep neural network framework, where the first network extracts 2D skeletal key points and the second estimates the risk action level, under the assumption of uniformity regarding muscle use and external loads during data labelling. Wang et al. [44] transition from static evaluations to a dynamic task assessment specifically tailored for construction workers, utilizing a Probabilistic Neural Network to enhance skeletal detection under harsh site conditions. Han et al. [45] introduce a Spatio-Temporal Graph Convolutional Network to automate REBA scores within complex nursing environments. Rodrigues et al. [46] address severe joint occlusions, such as hidden wrists, by shifting the algorithmic paradigm toward traditional RF models, demonstrating that consistent RULA scores could be derived solely from highly visible joint angles. Jiang et al. [47] offer a wide-spectrum technological overview that contrasts vision-based approaches with wearable sensors and radio-frequency signals, providing technical insight into traditional feature extraction methods alongside lightweight neural network architectures optimized for mobile deployment. Qureshi et al. [48] systematically review Human Activity Recognition via smartphones and smartwatches, evaluating edge computing as a means to preserve user privacy and system portability. Gorce et al. [49] conduct a cross-sectoral statistical meta-analysis comparing deep learning against traditional ML across multiple occupational domains. Taken together, these studies confirm growing momentum toward automating standardized ergonomic indices, but also reveal that no existing approach couples movement classification with REBA scoring on real industrial MoCap data, the gap this work addresses.

The present work is part of a broader, long-term framework aimed at addressing gaps in the literature regarding automation in both ergonomic evaluation and physical workstation design. This activity represents the first step towards this goal, focusing on movement classification coupled with trunk position assessment based on the REBA methodology. Future developments will extend the ergonomic evaluation to the complete REBA score and, depending on the operational scenario, to additional ergonomic assessment methodologies.

In this research, we present the movement classification using the RF classifier, implementing a sliding-window approach, to maintain the data temporal information.

The contributions of this paper are:

• 1.

  The creation of a labelled MoCap dataset specifically designed for classifying industrial operator movements;

• 2.

  The development of a ML pipeline based on this dataset;

• 3.

  The automation of REBA extraction in movements that are critical from the ergonomic perspective;

• 4.

  The evaluation of the proposed approach in both controlled and real-world industrial settings.

The results demonstrate the potential of combining MoCap data with ML techniques to support ergonomically informed decision-making and to lay the groundwork for future human–robot collaborative systems.

Materials and methods

Use case description and MoCap system selection

To evaluate the proposed system in a real industrial scenario, we selected two different workstations in CHIMAR S.p.A. [50], which is a logistics company that operates by managing the reception, classification, packaging, and transportation of goods, and is a leading company in this field, specializing in integrated solutions for packaging and logistics.

The use case consists of the assembly of a customised wooden box for shipping purposes using nail guns, visible in Figure 1. In Figure 1a, the operator picks some wooden planks and places them on a table to create a wooden container. The operator walks around the table to assemble one side of the container and uses a nail gun to fasten it (Figure 1b). After assembling the base and the walls of the wooden container, the operator moves the pieces to the ground with the help of a forklift and then assembles the container (Figure 1c). Finally, the operator composes the lid of the wooden container and places it on the box (Figure 1d).

Display full size

Figure 1. Wooden box workstation. (a) Placement of the wooden planks. (b) Operator using the nail gun. (c) Assembly of the container with a nail gun. (d) Closing the container with its lid.

The RF classifier takes as input the kinematic data acquired from a MoCap system. Specifically, the Xsens Awinda Starter, an IMU-based wearable system, was selected to overcome the occlusion limitations inherent to camera-based systems. In the industrial environment of the use case, the operator moves across the plant floor, making full area coverage with fixed cameras impractical. Additionally, AGVs operating within the plant could intermittently obstruct the cameras’ field of view, resulting in missing frames and incomplete operator tracking.

Random forest

Random forest conceptualisation

A decision tree (DT) is an ML model that predicts an output by recursively splitting the input space based on feature values. DTs are a technique used in statistics, data mining, and ML, and fall within the category of supervised ML [51]. Each internal node represents a decision rule, each branch a possible outcome of that rule, and each leaf node a final prediction [52]. DTs are easy to interpret and computationally efficient.

An RF is an ensemble model that combines many DTs to improve predictive performance and reduce overfitting. Each tree is trained on a bootstrapped sample of the dataset and uses a random subset of features at each split. The final prediction is obtained by aggregating the outputs of all trees (majority vote for classification, average for regression). Compared to a single DT, an RF is more stable, more accurate, and significantly less prone to overfitting, at the cost of reduced interpretability.

To classify the operator’s movements when performing their tasks, the RF classifier is selected. The choice is motivated by the good performance of the RF compared to other ML algorithms in classifying movements [53]. We chose not to adopt a deep learning approach because, although such methods can achieve high accuracy and remove the need for explicit feature extraction, they do so at the expense of robustness and interpretability and require large amounts of training data. RFs are a multipurpose tool that can be applied to both classification and regression problems, including multiclass classification [54].

This is not the first study where an RF is used to classify movements from MoCap acquisition. Iriondo Pascual et al. [38] use RF classification to evaluate hand movements based on hand postures. Bargellesi et al. [55] presented an RF-based algorithm for hand-gesture recognition and showed that, when combined with an appropriate feature extractor, it offers an effective solution for smart-environment applications. In this work, we used the data captured through the Xsens Awinda Starter MoCap system as input for the RF model.

The MoCap acquisition produced frame-based measurements collected at a fixed sampling frequency, including joint angles, joint positions, joint velocities, joint accelerations, and other related kinematic variables. Since an RF cannot be directly trained on raw time-series data, which requires a set of fixed features as input, we transformed the temporal sequences into feature vectors. To achieve this, the data were segmented into discrete intervals, and for each interval, we computed a set of descriptive metrics, including:

• Mean;

• Variance;

• Minimum value;

• Maximum value;

• Fast Fourier Transform peak frequency;

• Fast Fourier Transform peak magnitude;

• Fast Fourier Transform total energy (sum of squares of the amplitudes).

This procedure allowed us to represent the temporal dynamics of the MoCap data in a format suitable for RF training while preserving the most relevant statistical and frequency-domain characteristics.

Additionally, we decided to implement the sliding windows approach during the feature extraction process, as it enables the extraction of fixed-length feature vectors from continuous signals, which is essential for models such as RF that cannot operate directly on variable-length sequences. Moreover, overlapping windows allow the method to capture local temporal dynamics while preserving continuity between adjacent segments, thus improving the model’s ability to detect subtle transitions or short-term patterns.

Furthermore, by summarising each window through statistical and frequency-domain features, the approach reduces noise and dimensionality, facilitating more robust learning. Overall, the sliding window technique provides an effective and computationally efficient strategy for transforming raw time-series data into a structured representation suitable for supervised learning. Figure 2 represents the methodology described.

Display full size

Figure 2. Feature extraction using the sliding window approach.

Random forest implementation

The MoCap system used in this project is Xsens Awinda Starter [56], an IMU-based MoCap system [57]. The Xsens system includes 17 IMUs, which need to be worn as seen in Figure 3a. Every Xsens acquisition was saved in an XLSX file. The sensors provide data up to 60 Hz wirelessly, and the indoor range is 20 meters [28]. This system is extremely accurate, and in Guidolin et al. [58] the average error reported is of 0.26 degrees when tracking motion in the range that characterises most human movements. The RF classifier is generated using the scikit-learn library [59] in Python version 3.13. Scikit-learn is an open-source Python library that provides efficient implementations of a wide range of ML algorithms [60], including RF [61].

Display full size

Figure 3. Xsens Mocap. (a) Placement of the Xsens sensors on the human body. (b) The eight joints selected in the manikin.

From the use case described below, the basic movements performed by the operators while executing their tasks are defined. The basic movements are:

• 1.

  Walking: the operator is walking in the scenario; for this first implementation, we do not consider whether the operator is walking carrying something or not.

• 2.

  Standing: the operator stands normally while working at the workstation table. The operator performs different tasks, such as applying labels, preparing boxes, closing the products inside the boxes, applying nails to wooden boxes, etc. Although the tasks vary, the operator’s lower limbs remain static while the upper body performs the required movements.

• 3.

  Bending: the operator bends to pick up an item from the ground and place it on the workstation table, or to place an item on the ground that was picked up from the workstation table.

After defining the basic movements to be performed, we recruited nine individuals to execute at least two of these movements while wearing the Xsens IMU sensors. Collecting data from multiple participants rather than a single individual helps reduce bias associated with a specific person’s movement patterns. Moreover, the participants differ in height, weight, sex, and age, providing a more diverse and representative dataset; Xsens manages to model different gender-specific models [62]. Specifically, the participants are four men and five women, the age range is between 24 and 43 years old, the height range is between 155 and 190 cm, and the weight is between 50 and 105 kg. We recorded several acquisitions in which each participant performed a specific action, allowing us to collect directly labelled data that will be used as training input for the RF model. During data acquisition for the training set, each participant performed the same movement continuously for at least one minute. For the bending movement, the entire cycle was recorded and labelled as bending, encompassing the lowering phase, the held bent position, and the rising phase back to an upright posture. For the walking movement, participants walked freely within the available space. For the standing movement, participants stood at a workstation table while performing manual tasks such as manipulating objects. The Xsens acquisition data provides 903 data points for each frame. The data consist of:

• Segment Orientation Quaternion: 92 data for each frame;

• Segment Orientation Euler: 69 data for each frame;

• Segment Position: 69 data for each frame;

• Segment Velocity: 69 data for each frame;

• Segment Acceleration: 69 data for each frame;

• Segment Angular Velocity: 69 data for each frame;

• Segment Angular Acceleration: 69 data for each frame;

• Center of Mass: 9 data for each frame;

• Joint Angles ZXY: 66 data for each frame;

• Join Angles XZY: 66 data for each frame;

• Ergonomic Joint Angles ZXY: 18 data for each frame;

• Ergonomic Joint Angles XZY: 18 data for each frame;

• Sensor Free Acceleration: 66 data for each frame;

• Sensor Orientation Quaternion: 88 data for each frame;

• Sensor Orientation Euler: 66 data for each frame.

To evaluate the body motion, we decided to focus on some specific parts of the human body, like the elbows, wrists, hips, and knees. For this reason, we select some of the data from the complete dataset, specifically the one collected from the joint angles XZY section.

Those data, visible in Figure 3b, are:

• 1.

  Right/Left Elbow Flexion/Extension;

• 2.

  Right/Left Wrist Flexion/Extension;

• 3.

  Right/Left Hip Flexion/Extension;

• 4.

  Right/Left Knee Flexion/Extension.

For this project, the RF algorithm has 10 DTs. The MoCap data is acquired with Xsens at a frame rate of 60 Hz. It is important to define the window size of the input data and the overlap of the windows. For this implementation, we set a window size of 120 frames, i.e., 2 seconds, and an overlap of 70 frames. The random seed is 42, and the splitting criterion is Gini impurity. To train and test the model, we divide the dataset into two parts: 80% of the dataset is the training set, and the remaining 20% of the dataset is the test set; the data are split randomly. The training set dimension is 4,612, while the test set dimension is 1,153. The model is trained using the training set and validated through the test set, so that the accuracy of the RF is evaluated based on the results given by the latter set. Figure 4 shows the confusion matrix.

Display full size

Figure 4. Confusion matrix of the test set.

We read the correct labels associated with the operator’s movements on the rows of the matrix, while on the columns the predicted ones. Given the structure of the confusion matrix, the values outside the main diagonal represent the incorrect predictions of the model. We observe in Figure 4 that the model reaches high accuracy in predicting the correct movement of the operator; the accuracy is 98.44%. The precision, recall and F1-scores obtained on the test set are reported in Table 1.

Table 1. Test set results.

Movement	Precision	Recall	F1-score	Support
Bending	0.98	0.97	0.98	327
Standing	0.97	0.98	0.97	318
Walking	1	0.99	1	508

Display full size

To assess the generalisation ability of the RF classifier to unseen subjects, a leave-one-subject-out cross-validation strategy was implemented and evaluated across all nine participants. For each fold, one participant was excluded from training and used exclusively for testing.

Table 2 reports the classification accuracy and the corresponding precision, recall, and F1-score for each movement class per participant. The accuracy consistently exceeds 91% across all folds. Entries marked as NA (not applicable) indicate that the corresponding movement was not performed by that participant during data acquisition.

Table 2. Test set leave-one-subject-out cross-validation results.

Removed participant	Accuracy	Movement	Precision	Recall	F1-score	Support
Male 1	96.50%	Bending	1	0.94	0.97	62
		Standing	0.91	1	0.95	72
		Walking	1	0.96	0.98	68
Female 1	91.24%	Bending	0.99	0.88	0.93	146
		Standing	0.8	0.99	0.89	71
		Walking	NA	NA	NA	NA
Female 2	94.44%	Bending	1	0.89	0.94	143
		Standing	0.83	1	0.9	71
		Walking	0.99	1	0.99	74
Male 2	98.09%	Bending	0.99	0.99	0.99	153
		Standing	0.99	0.93	0.96	71
		Walking	0.97	0.99	0.98	143
Male 3	91.70%	Bending	NA	NA	NA	NA
		Standing	1	0.76	0.86	294
		Walking	0.99	1	0.99	586
Female 3	99.61%	Bending	NA	NA	NA	NA
		Standing	1	0.99	0.99	294
		Walking	1	1	1	726
Female 4	98.69%	Bending	NA	NA	NA	NA
		Standing	1	0.96	0.98	326
		Walking	1	1	1	742
Male 4	95.15%	Bending	1	0.93	0.96	164
		Standing	0.83	1	0.91	72
		Walking	1	0.96	0.98	73
Female 5	99.08%	Bending	0.99	1	0.99	136
		Standing	0.99	1	0.99	69
		Walking	1	0.97	0.99	120

Display full size

NA: not applicable.

Before implementing the RF algorithm in a real industrial scenario, we decided to evaluate it in a controlled environment. We collected sequences in which the three participants performed all movements in a random order. This acquisition will be called, in the following, a random acquisition. Thanks to the Xsens software, it is possible to generate a manikin that reproduces the captured movements; later, this feature allows for the evaluation of the classification performance of the RF. To understand the system performance, we save the Xsens manikin’s video, and we save the RF result as an .srt file, to be able to add it as a subtitle in the manikin’s video. An example of an .srt file resulting from the RF is presented in Table 3.

Table 3. Result structure from the RF classifier.

No.	Start time	End time	Movement
0	00:00:00,000	00:00:05,000	Standing
1	00:00:05,000	00:00:07,000	Walking
2	00:00:07,000	00:00:11,000	Bending
3	00:00:11,000	00:00:14,000	Walking

Display full size

One may notice that it consists of several time intervals, each associated with the predicted movement classification. To use the RF algorithm with data without labels, we use the RF model obtained with the training dataset.

Figure 5 shows the three motions that we recorded and analysed with the RF. On the video recording of the digital twin, we superimpose the subtitles obtained with the RF. One may notice from the picture and from the previous confusion matrix that most of the classifications are correct.

Display full size

Figure 5. Recorded motions with labels from the RF. (a) Walking. (b) Bending. (c) Standing.

No additional noise filtering was implemented, as the Xsens MVN software automatically applies post-processing to reduce noise prior to exporting the data in XLSX format. Regarding computational cost, the total execution time for the training pipeline is 128.379 seconds, 126.661 seconds are required to load the raw data from the XLSX files exported by the MVN software, 1.272 seconds to train the RF model, and the remaining time (under 1 second) to initialise the software pipeline. The total size of the XLSX files is 3.07 GB. As is standard for supervised learning models, training is performed offline and is not intended for real-time execution. The prediction phase, however, is computationally lightweight and suitable for real-time deployment. All experiments were conducted on a laptop equipped with an Intel Core i7-10750H CPU @ 2.60 GHz, 16 GB of RAM, running Windows 10.

Results

RF implementation in the use case

The operator’s movements were recorded using the Xsens Awinda Starter system. Simultaneously, a video recording of the operator was collected to enable a qualitative visual comparison between the real operator and the corresponding Xsens digital manikin. Notice that the use case operator was not part of the training dataset, ensuring that the evaluation reflects the model’s ability to generalize to a completely unseen individual. All three movement classes, walking, standing, and bending, were observed during the use case recording. In Figures 6a and 6b, the walking movement of the operator is visible; the operator is walking to grab the second wall of the wooden container, which is placed on the table. In Figures 6c and 6d, the standing movement of the operator, the operator adds two stamps. In Figures 6e and 6f, the bending movement of the operator is visible; the operator bends to pick up the external wall of the wooden container, to later fix it with the nail gun machine.

Display full size

Figure 6. Operator in subsequent frames. On the left operator video, on the right Xsens manikin and RF label. Operator while walking (a), (b). Operator while standing (c), (d). Operator while bending (e), (f).

What we obtain from this data is that the standing movements are not only when the operator is standing up, but also when the operator maintains a posture with the legs for some time. A specific case where this occurred was when the operator was bent over while using the nail gun to assemble the wooden container. Because the legs remain stationary during this task, the RF algorithm correctly labels the transition into the bent posture as “bending.” However, while the operator remains bent over performing the assembly, the algorithm misclassifies the posture as “standing”.

Ergonomic evaluation

IPS IMMA data extraction

IPS IMMA is a Digital Human Modelling software that incorporates advanced path-planning techniques. The software is equipped with a comfort function that enables the positioning of the manikin in ergonomically optimal and reliable postures while satisfying predefined constraints, such as grasping and visibility requirements [63]. Currently, IPS IMMA implements several ergonomic assessment methods, including RULA, REBA, OWAS, and EAWS [10]. These methods allow users to account for ergonomic risks during the early phases of the design process, as well as in the evaluation of specific operational scenarios. In addition, IPS IMMA supports full-body MoCap through integration with the Xsens system, thereby leveraging all IPS IMMA functionalities, including the application of ergonomic assessments to recorded MoCap data. Ergonomic evaluations are a proactive strategy used to identify physical stressors, such as awkward postures, high force, or repetitive motions, that can lead to long-term musculoskeletal disorders. Their primary goal is to isolate the root cause of these risks, allowing for interventions that resolve discomfort before it evolves into chronic pain or permanent injury.

In this project, IPS IMMA was used as the primary tool to perform the ergonomic evaluation and to provide an initial indication of ergonomic risk. Two different methodologies were adopted for this purpose. The first methodology was designed to evaluate the risk associated with a specific posture. For this purpose, the REBA ergonomic assessment was adopted, as it enables screening the whole-body physical risks related to working postures. In this initial phase, the analysis focused exclusively on trunk posture, as this was the parameter that could be evaluated using the classified movements. Once the activity was recorded using the Xsens system, and in parallel with motion classification, the recorded data were provided as input to IPS IMMA. A human digital twin of the worker was then created using a manikin parameterized with the worker’s anthropometric measurements. The REBA ergonomic evaluation was performed on a frame-by-frame basis. The resulting scores for each REBA criterion were stored in a .csv file. Subsequently, these data were combined with the motion classification results to identify periods of trunk flexion and to determine the corresponding degree of flexion, including the presence of trunk twisting or lateral bending.

From the REBA data exported from IPS IMMA, we decided to evaluate only the trunk position when a bending movement is identified from the RF. We decided to concentrate on the bending movement and not to consider the mass of the object used during the task’s execution. As visible in Figure 7, REBA identified four possible values for trunk position, which are:

Display full size

Figure 7. REBA trunk position value.

• 1.

  When the trunk is 0°;

• 2.

  When the trunk is +/– 20°;

• 3.

  When the trunk is between 20° and 60°;

• 4.

  When the trunk is more than 60°.

At this point, the data selected to be computed are:

• 1.

  A report with all the movements classification from the RF;

• 2.

  The percentage of all the movement categories accounted for all the activity time;

• 3.

  A report that automatically extracts the REBA value for the trunk position when a bending movement is identified in the RF classification.

Verification of ergonomic integration in the industrial use cases

With reference to the industrial use case presented above, the results obtained in the previous section are now reported. The time required to construct a wooden container is 31 minutes and 30 seconds (Figure 8).

Display full size

Figure 8. Movement classification and trunk REBA analysis during wooden container construction. (a) Bar chart showing the sum of all the movements performed during the task execution. (b) Sum of the time for each REBA trunk position when the operator is bending.

• A report with all the movement classifications from the RF. In Table 3, the initial report of all the movement classifications is presented. As is visible, it is possible to see every movement classification with the time when the movement starts and when it ends.

• The percentage of all the movement categories accounted for all the activity time. In Figure 8a, a bar chart is reported with the sum of all the movements classified from the RF. It is possible to see that the entire pallet construction has a duration of 31 minutes and 30 seconds. In percentage, the operator is:

  • a.

    Bending accounts for 6.7% of all the wooden container construction time;

  • b.

    Standing accounts for 64.3% of all the wooden container construction time;

  • c.

    Walking accounts for 29% of all the wooden container construction time.

• A report that automatically extracts the REBA value for the trunk position when a bending movement is identified in the RF classification, with the analysis. Figure 8b shows the REBA trunk position scores summed for each zone whenever the RF classifies a bending movement. At the top of each column, there is the sum of time that the operator spent in each REBA score.

Making a comparison with all the REBA scores for trunk position when the operator is in bending, the operator states:

• 9.24% in zone 1 while bending;

• 15.99% in zone 2 while bending;

• 45.98% in zone 3 while bending;

• 28.79% in zone 4 while bending.

Making a comparison with all the REBA scores for trunk position for the entire cycle time, the operator states:

• 0.62% in zone 1 while bending;

• 1.07% in zone 2 while bending;

• 3.07% in zone 3 while bending;

• 1.92% in zone 4 while bending.

Discussion

Nowadays, there is an increasing attention on operators’ well-being, and the understanding of people’s movement to comprehend the ergonomics of them is becoming a central topic, thanks to Industry 5.0 growth. This paper presents the implementation of an RF classifier for the automated recognition of operator movements in an industrial environment from MoCap data, coupled with the extraction of the REBA trunk position score during identified bending movements. This work represents the first step towards a long-term framework aimed at automating ergonomic risk assessment and informing task delegation to collaborative robotic systems.

Because RF does not manage temporal series, a sliding-window approach with feature extraction is used to describe the temporal data. This approach is evaluated both in a controlled environment and in a real industrial use case. Furthermore, the integration with IPS IMMA enables automatic extraction of REBA trunk position scores frame-by-frame during bending movements, providing a quantitative characterization of trunk flexion in the analyzed task. Future developments will extend the ergonomic evaluation to the complete REBA score.

Several considerations can be made regarding potential improvements to this work. An interesting analysis would involve examining the input features provided to the RF algorithm. A comparison between the features selected in this study and additional available data should be conducted. Similarly, the choice of metrics used to characterise the time window should be evaluated to identify those that best describe the temporal dynamics, as well as to determine the optimal window size and overlap.

Regarding the motion analysis, although the results indicate good classification performance, it would be highly valuable to determine whether the operator is carrying an object or not. With the current dataset, this information cannot be inferred; however, a possible solution would be to incorporate an electromyography sensor. These sensors measure the electrical activity of muscles; when a muscle contracts, it generates an electrical signal that the devices record. Moreover, it is necessary to improve the correct detection of the posture when it is in a static posture. In other words, if the worker stayed in a bending posture for a while, the RF detects a bending posture for all that interval of time.

A recognised limitation of the current implementation is that the RF tends to misclassify sustained bending postures as standing. This behavior appears because when the operator remains bent with the lower limbs stationary, the classifier interprets the static leg posture as standing. Potential mitigation strategies include RF hyperparameter refinement, acquisition of additional training data covering static bending postures, and the application of post-processing techniques to smooth the classification output over time.

As this work represents the first step towards a broader framework, several directions are identified for future development. First, load handling, such as carrying objects or lifting materials, will be considered, as carried loads directly affect musculoskeletal risk and are ergonomically relevant. Since IMU sensors capture body kinematics but cannot measure the mass of handled objects, the incorporation of electromyography sensors could complement the MoCap data and enable the detection of load-bearing conditions.

Second, the ergonomic evaluation will be extended beyond the trunk position criterion to encompass the complete REBA score, providing a comprehensive assessment of whole-body ergonomic risk. Depending on the operational scenario, additional ergonomic assessment methodologies may also be considered.

Finally, introducing a post-processing step on the RF classification output could enable a more detailed activity breakdown, allowing the automatic generation of an activity-based time analysis in which each task is logged with its duration and frequency of occurrence within the work cycle. This would further enhance the granularity and practical utility of the framework.

Regarding the automation process, future work should focus on evaluating how to automate the most critical aspects of the workflow. In the presented use case, from a health and safety perspective, the most demanding activities for the operator are those that require bending. These actions could be replaced with an automated solution. A possible improvement would be to exploit the operator’s movements, collected through the MoCap sensors, to classify bending motions and automatically delegate these tasks to an automated solution that performs the same actions.