Impact of HNSCC in healthcare

Head and neck squamous cell carcinoma (HNSCC) rank as the 6th most common cancer worldwide, with approximately 890,000 new cases and 450,000 deaths each year recently [1]. In Hong Kong, HNSCC incidence has risen up to 16,000 new cases reported each year, mostly due to tobacco use, alcohol consumption, betel quid chewing, and HPV infection [2]. Furthermore, 5-year survival rate for late-stage HNSCC patients is less than 50%, primarily due to tumor recurrence, metastasis, and therapy resistance [3]. Radiotherapy combined with surgery, chemotherapy, or immunotherapy remain as main treatment methods, yet heterogeneous treatment responses demand more comprehensive personalized approaches [4]. In our study, we attempted to refocused the study on oropharyngeal squamous cell carcinoma (OPSCC) as a particular example of HNSCC, acknowledging this specificity as a strength for a homogeneous analysis.

Consideration of current prognostic methods

The Tumor, Node, Metastasis (TNM) staging system (AJCC 8th Edition) is the gold standard for HNSCC prognosis, it relates to tumor size (T), lymph node involvement (N), and distant metastasis (M) [5]. However, this system suffers from the following weakness:

• Biopsies may promote patient discomfort and risk [6], while it has irreplaceable role in the confirmation of diagnosis.

• Time delays in histopathological analysis deter immediate treatment decisions [7].

• Difficult to detect tumor heterogeneity, molecular subtypes, or dynamic treatment responses [8].

It appears that, other than biomarkers (e.g., PD-L1, HPV status), non-invasive, AI-enhanced radiomics may offer a promising alternative, yet predictive models, like other AI models, can only augment rather than replace AJCC staging.

Radiomics and AI in HNSCC prognostic prediction

Radiomics extracts high-dimensional quantitative features (e.g., texture, shape, wavelet transforms¹) from computed tomography (CT), MRI, and PET scans to identify tumour characteristics [9, 10].

The main advantages are:

Prognostic prediction modeling: Correlates imaging features with treatment response, survival, and recurrence [11, 12].

Recent studies have noted that machine learning (ML) algorithms have good performance (Table 1). However, single-model approaches suffer from overfitting and may lack generalizability [16].

Ensemble learning: an improvement

Ensemble methods combine multiple models to enhance prediction performance:

• 1.

  Bagging [e.g., Random forest (RF)]: Reduces variance via bootstrap aggregation [17].

• 2.

  Boosting (e.g., XGBoost): Iteratively corrects errors [18].

• 3.

  Voting ensembles: Achieves better accuracy in HNSCC prognosis [19, 20].

Research gaps and objectives

Despite progress, critical gaps persist, such as:

• 1.

  No stacking-based radiomics models for HNSCC prognosis exist.

• 2.

  Most studies use single-institution data, limiting generalizability [21].

• 3.

  Multi-modal data integration, that is, imaging with genomics, has more room for investigation [22].

Research workflow

The study followed the following steps (Figure 1):

Display full size

Figure 1. Research workflow.

• Performance was evaluated via ROC-AUC, accuracy, sensitivity, and specificity.

Patient data

TCIA serves as the data source for the current study. This publicly available repository is operated by the National Cancer Institute (NCI). Planning CT images of patients who received radical radiotherapy for HNSCC during the period from 2003 to 2013, with “HNSCC” labeled dataset was used. The research included GTV and PTV radiotherapy structures and clinical information about patient age, sex, and diagnosis, smoking habits, staging progression, and three- and five-year survival statistics. After quality assessment, finally, 164 valid cases out of the 215 confirmed collections were obtained.

ML models

Cancer prognosis is typically assessed based on specific time points, such as the five-year survival rate [23]. This allows for objective comparisons across different cancer studies, as patients who survive five years post-treatment are generally classified as "cancer survivors."

In this study, the five-year survival rate was selected as the treatment outcome to ensure comparability with other studies. Of the 164 cases, 118 patients survived, while 46 did not. An overfitting test was performed by randomly selecting 46 cases from the 117 patients who survived to balance the sample outcomes.

ML process

Two ML models were employed:

Two-layer stacking ensemble machine learning (SEML) model

In this model, the radiomics data were divided into two sets for data splitting: training (70%) and testing (30%). Four classifiers were utilized: decision trees (DT), RF, support vector machine (SVM), and generalized linear model (GLM). The training set was used to train the models, which were then validated with the validation set to generate predictions. Predicted outcomes were quantified numerically, with values of 0 representing survival beyond five years or death from other causes, and 1 indicating death within five years of diagnosis.

The SEML consists of heterogeneous classifiers organized in a two-layer structure. The base classifiers were initially trained with the radiomics data. The meta-learner was trained using the prediction results from the base classifiers, with XGBoost selected as the meta-classifier. The selection of an appropriate meta-classifier is crucial for model performance. Previous studies have indicated that XGBoost is optimal for recurrent HNSCC prognosis [24–26]. The final prediction outcome was the training result of the XGboost. Details of protocol were illustrated in Figure 2.

Display full size

Figure 2. Workflow for the adoption of a stacking ensemble machine learning model. DT: deep learning; RF: random forest; SVM: support vector machine; GLM: generalized linear model.

Data analysis

The predicted outcomes from the two-layer stacking model were compared with those from the base classifiers. Model performance was evaluated using the receiver operating characteristic (ROC) curve, with metrics including the area under the ROC curve (AUC), accuracy, specificity, sensitivity, and chi-square test, which were calculated using ROCkit from the University of Chicago (1995). SEML and DL models were tested on the same holdout dataset (30%) using Delong test by Matlab calculation.

Demographic cohort

The dataset comprised 215 patients, of which 51 were excluded due to missing data, resulting in 164 cases for analysis. Both PTV and GTV CT datasets were collected. Demographic details are summarized in Table 2.

Comparison of performance between individual model and stacked model

SEML models using SEML as classifier, consistently outperformed any of the individual ML models in ROC analysis.

Among 4 of the individual ML models using PTV features, SVM attained the best performance in prognostic prediction for HNC. Compared with the SVM model, the SEML model further enhanced the performance in both accuracy from 73.3% to 93.3% (AUC = 0.723 vs. AUC = 0.982) and sensitivity from 66.7% to 100%. The details refer to Table 3.

Prediction performance of SEML and DL models

The SEML model demonstrated exceptional performance, with AUC ranging from 0.82 to 0.982 across all target volumes. For PTV radiomic features, the AUC reached 0.982. The model exhibited sensitivity of 100%, specificity of 83%, and accuracy of 93%. In contrast, the AUC for GTV features was slightly lower at 0.82, with sensitivity, specificity, and accuracy at 62.5%, 85.7%, and 73.3%, respectively. In comparison, the DL model showed an AUC ranging from 0.605 to 0.774, with accuracy between 0.655 and 0.724, indicating superior performance of the SEML model (Table 4).

ROC analysis

Despite the PTV radiomics features yielding better predictions, ROC analysis indicated no significant difference between PTV and GTV features (chi-square test, p > 0.05) (Figure 3).

Display full size

Figure 3. ROC curve of 5-year survival using GTV and PTV radiomics features in the SEML model. GTV: gross tumor volume; PTV: planning target volume; SEML: stacking ensemble machine learning.

Comparison of SEML and DL models

The SEML model consistently outperformed the DL model in predicting 5-year survival for HNSCC patients. Sensitivity, specificity, accuracy, and AUC of the SEML model surpassed those of the DL model, although ROC analysis indicated no statistically significant differences between PTV and GTV radiomic features (chi-square test, p > 0.05, Figure 4).

Display full size

Figure 4. The ROC curve of GTV and PTV using deep learning model. GTV: gross tumor volume; PTV: planning target volume.

Test for SEML and DL performance

A DeLong test for AUCs is conducted to compare the performance of SEML and DL. It was noted that there is no significant difference for GTV, but there is a significant difference for PTV (Table 5).

Future research direction

Incorporating clinical and genomic data alongside radiomics features could enhance predictive capabilities. Recent studies have indicated that radiomics-clinical (RC) models yield higher accuracy compared to radiomics-only models [31, 32]. Similarly, radiomics-genomics (RG) models have shown promise in improving survival predictions [33]. This opens avenues for developing integrated models that utilize multi-modal data sources, enhancing the overall predictive power, such as:

• Improvement for a more advanced AI model:

• Hybrid frameworks that integrate DL with ensemble approaches will help overcome limitations in DL models seen in this study. While we agree that extended timeframe prediction could offer theoretical insights, we proposed multi-modal integration: adding genomic markers to improve long-term robustness of the AI model [31–33].

• More sample size and feature diversity:

• Enhance the SEML model using larger and more diverse cohorts, including different demographic and tumor characteristics, to ensure broad clinical applicability and robustness of the model.

• External validation/Generalizability

• To increase SEML model reliability and ability to apply wider, a multi-step validation approach is proposed. (1) Partnership with international institutions [e.g., TCIA, International Cancer Genome Consortium (ICGC)] to test performance based on a different demographic, and the imaging techniques; (2) Prospective studies undertaken in not less than 3 hospitals using a standard feature extraction of images; (3) Comparison with previous approaches like TNM staging, single-model radiomics; and (4) Enrichment of prediction model incorporating genomic and clinical data. It is expected that outcomes will include generalization confirmation, biases detection, and ways of FDA clearance. The timeline is 6 months for setup, 1 year for retrospective validation, and 2–3 years for prospective trials. This approach ensures rigorous, scalable validation for clinical adoption.

• Clinical Deployment Pathway

• Integration of the SEML model in clinical practice is likely to launch in multi-centers or institutions, followed by modifications and final implementation in the form of operations in hospital systems to support real-time prognoses. There are challenges that may involve Imaging protocol standardization, clinician training, and practices. One of them is obtaining regulatory approval (e.g., FDA/CE marking). Data privacy (e.g., GDPR/HIPAA), and more efficient computational resource use will also help make deployment easier. The pilot studies and implementation in phases are suggested to fine-tune the model to integrate it with the real-world workflows.

• Hyperparameter tuning: In our future study, we adopt a systematic approach to model development that includes an initial exploration of hyperparameter settings for our base learners and the meta-learner. While the specific tuning methods, such as grid search or random search, were not detailed in the manuscript, we will conduct preliminary experiments to identify optimal parameter ranges for each model. For each base learner (DT, RF, SVM, and GLM), we will utilize default settings that are commonly accepted in previous studies, ensuring a strong baseline performance while allowing for model interpretability.

Additionally, for the meta-learner (XGBoost), the parameters were chosen based on best practices and general guidelines from the literature, focusing on performance metrics rather than exhaustive tuning, given the constraints of our dataset size.

To ensure reproducibility, we plan to provide detailed hyperparameter settings and tuning methodologies. We believe that this balance between rigor in model setup and practical application provides a solid foundation for our research findings.

Potential applications

The SEML model can pave the way for personalized treatment strategies by:

• Predicting patient-specific survival outcomes to optimize treatment plans.

• Integrating into clinical decision support systems for better, but not limited to, head-and-neck cancer management.

• Enabling earlier interventions in high-risk cases based on model predictions.

• Integrating clinical/genomic data. A study [34] indicated that the integration of pre-treatment CT-derived radiomic biomarkers and TNM stage was predictive of 5-year progression-free survival post-chemoradiation in locally advanced HNSCC (LA-HNSCC) patients, suggesting its utility for clinical risk stratification.

Conclusions

This study is the first to employ a stacking ensemble learning approach in a predictive model for estimating cancer prognosis in HNSCC patients. The SEML model demonstrated high accuracy (93%), sensitivity (100%), and specificity (83%) in predicting five-year survival. The results affirm the effectiveness of the stacking ensemble approach in enhancing prognostic accuracy, laying a foundation for its clinical application and potential to facilitate personalized treatment for cancer patients.