The dataset used in this research article is ethically approved by the Institutional Ethics Committee, Madras Medical College, Chennai, “Ec.No.02122021” titled “An Automatic Nodule Point Detection and Classification of Lung Mass by HRCT”. The initial version 1 dataset of three classes is Benign, Malignant, and Normal lung CT images, is available online with the following link https://doi.org/10.34740/KAGGLE/DSV/8288306. The other cancer types will be hosted online in the next version release. The dataset description is shown in Figure 1.

This clinical study included CT imaging data from 388 individuals who underwent chest CT scans at the BIR, Madras Medical College, Chennai. The inclusion and exclusion criteria were meticulously established to ensure the accuracy and relevance of the study’s findings.

Patients with a CT diagnosis of lung pathologies referred for biopsy, with Lesion size ranging from 8 mm to 20 cm. Lesion characteristics: solid nodules, part-solid nodules, cavitatory lesions, non-resolving pneumonias. A pathological diagnosis of non-mucinous adenocarcinoma based on the 2015 World Health Organization (WHO) classification of lung tumors.

Patients allergic to contrast agents, patients who were uncooperative during the study, patients with contraindications to CT-guided biopsies, cases with inconclusive histopathological examination (HPE) results, patients without preoperative thin-section CT images or with images that could not be analyzed due to artifacts or image noise, patients with prior treatment to the lungs cases with pathological specimens deemed inadequate for diagnosis under the 2015 WHO classification, individuals under the age of 18, pregnant women, patients who don’t provide consent to be part of study.

A proprietary dataset, referred to as the BIR Lung Dataset, was developed using CT scans of 388 cases, resulting in a total of 16,172 images. Scans were pre-processed to anonymize patient information, and annotations were performed by a radiologist to ensure precise image training. Both plain and contrast-enhanced CT imaging were conducted using a Siemens 32-slice CT scanner. Imaging parameters included: KVp: 130, mAs: average (80), slice thickness: 5 mm, reconstruction interval: 1.5 mm.

Ethical approval for this study was obtained from the Institutional Ethics Committee Review Board of Madras Medical College, Chennai (Ec.No.02122021). The need for informed consent was waived for this retrospective review of patient records, imaging data, and biomaterials. All CT data and pathological specimens were provided by the host institution.

Two independent pathologists reviewed and diagnosed all the lung specimens stained with hematoxylin–eosin and/or elastic van Gieson stain according to the 2015 WHO classification of lung tumors. The histological diagnoses of adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA), or IVA were confirmed by consensus decisions. The data of the 388 patients used for 3D-CNN model construction comprised AIS (n = 248), MIA (n = 47), and IVA (n = 93).

The CT image data were acquired with three types of multidetector-row CT scanners: Discovery CT750 HD (GE Healthcare), Aquilion PRIME (Canon Medical Systems), and LightSpeed VCT (GE Healthcare). The protocols used with each of the three scanners are summarized in Table 1. All targeted lung CT images were reconstructed using a 200–230 mm field of view from thin-section CT images reconstructed with a high spatial-frequency algorithm.

First, without using the 3D-CNN model, three chest radiologists from our institute who have sub-specialization in chest radiology evaluated the cases, i.e., with the grade of junior level to senior level, were independently assessed the CT findings. Each radiologist’s findings were then pooled into a common finding and compared to the 3D-CNN. The results were expressed in terms of percentage.

The model was initially developed using the BIR Lung Dataset, which originates from a single medical center. This introduces potential dataset bias due to the lack of demographic diversity and institutional variability. Furthermore, the dataset lacks access to patient-specific metadata such as age, gender, ethnicity, and smoking history, making it difficult to assess and mitigate demographic and population-level biases.

To partially address this limitation, external validation was conducted using the publicly available LIDC-IDRI dataset, which includes scans from multiple centers. This step was taken to evaluate the model’s generalizability across different populations. Additionally, future work will focus on incorporating demographically diverse, multi-center datasets with annotated patient profiles to enable fairness-aware model training and evaluation.

The image enhancement is carried out to highlight the inner features of the lung region to identify the nodule. The original greyscale images are transformed to 32-bit color, quantize applied to segment the lung region. The preprocessed images are fed for training. The lung nodule segmentation using the U-Net architecture. First, the CT images are pre-processed to enhance image quality and normalize the intensity values, ensuring consistency across the dataset. The pre-processed images are then fed into the U-Net model, which consists of an encoder-decoder structure shown in Figure 2. The encoder extracts multi-scale features through successive convolution and pooling layers, capturing both local and global contextual information. Skip connections are used to transfer high-resolution features from the encoder to the decoder, enabling precise segmentation by combining spatial and semantic information. The decoder reconstructs the segmentation map by progressively upsampling the encoded features and merging them with corresponding features from the encoder. This step ensures that the model retains fine-grained details crucial for delineating small and irregularly shaped lung nodules. During training, the model is optimized using a loss function, typically a combination of cross-entropy or Dice loss, to maximize segmentation accuracy. Data augmentation techniques are applied to increase the variability of training data and improve model generalization.

Once trained, the model predicts binary segmentation masks for input CT images, highlighting the boundaries of the lung nodules. Post-processing steps, such as morphological operations, may be applied to refine the segmentation and reduce noise. The resulting segmentation masks are then used for quantitative analysis, including measuring nodule size, shape, and volume, and can aid in further diagnostic and treatment planning tasks. The segmented nodules are fed to a transfer learning model for lung cancer classification.

In this study, we limited the classification task to distinguishing between benign and malignant nodules due to the presence of suspicious cases in the dataset and the lack of sufficient labeled examples for specific cancer subtypes. These suspicious cases, based on histopathology outcomes, represent diagnostically uncertain scenarios and were treated with caution during model training and evaluation. As a result, the current model prioritizes robust binary classification to ensure clinical reliability. These specific cases are handled as a validation set to test the prediction accuracy.

The U-Net architecture is a deep learning model built for semantic segmentation tasks. It is especially useful in medical image analysis because of its ability to collect spatial and contextual elements. This design is made up of two major components: an encoder (contracting path) and a decoder (expanding path), which are linked by skip connections to retain spatial information shown in Figure 3.

The U-Net model is used to segment the lung nodules. The segmented lung region is further reduced to the lung nodule by applying a color threshold integrating the Gaussian and bilateral filters that enhance the image outcome and make the machine learning model better understand how to classify the cancer types, such as adenocarcinoma, squamous cell carcinoma, large cell carcinoma, and small cell carcinoma. The segmented nodule is processed with a Gaussian and bilateral filter. Figure 4 shows different sigma values and the outcome. Figure 4A original input image, Figure 4B sigma = 0.8, Figure 4C sigma = 0.7, Figure 4D sigma = 0.5, range of outcome. The various lung nodule obtained is stored as a feature subset to feed the input for the transfer learning model for classification.

The enhanced U-Net model in the proposed method attains 97% accuracy, as shown in Figure 5. Thus, the model is better at identifying nodules when compared with the traditional identification.

In this research article, the transfer learning model is employed for the classification of the segmented nodule. The models employed for the study are VGG-16, Inception v3, and Efficient B0 Net. The architecture and its features are shown in Table 1. The assessment of the deep learning model in this research used recognised performance criteria, including accuracy, precision, recall, and F1-score, to enable an extensive evaluation of the model’s predictive abilities. The proposed method demonstrated noteworthy classification accuracy, prevailing over the baseline model, Efficient B0 Net, with a significant score of 98.3%. The accuracy and recall scores highlight the ability of the model to accurately distinguish between cancerous and benign nodules while reducing false positives and negatives.

Cross-validation enhanced robustness and generalisability across varied datasets, whereas confusion matrix analysis offered insights into particular categorisation difficulties. The model’s exceptional performance compared to leading architectures, such as VGG-16, Inception v3, and EfficientNet, further confirms its effectiveness. The use of sophisticated methods like colour transformation and transfer learning improved feature extraction and classification accuracy, enhancing its applicability in lung cancer detection.

The experimental evaluation of VGG-16, Inception v3, and EfficientNet B0 for classifying lung nodules into benign and malignant categories reveals significant findings. The results, summarized in Tables 2 and 3, highlight the comparative efficacy of these models based on accuracy, precision, recall, and F1-score metrics. Figures 6 and 7 provide detailed visualizations of the models’ training accuracy, validation accuracy, training loss, and validation loss over 100 epochs. The model was trained using the Adam optimizer with a learning rate of 0.0001, batch size of 32, and a maximum of 100 epochs with early stopping (patience = 10). The dropout rate was set to 0.5, and L2 regularization (λ = 0.001) was applied to prevent overfitting.

EfficientNet B0 demonstrated superior performance, achieving ~99.3% training accuracy and ~97% validation accuracy. Its high accuracy and low loss metrics across both datasets underline its robustness and efficiency as a diagnostic tool. The minimal gap between training and validation metrics suggests effective generalization with negligible overfitting. In contrast, VGG-16 and Inception v3 displayed slightly lower accuracies and higher losses, emphasizing EfficientNet B0’s advanced feature extraction and architecture optimization.

The learning curves for all models showed consistent training loss reduction, with EfficientNet B0 converging near zero by the 100th epoch. Validation loss followed a similar trend, with minor fluctuations stabilizing at slightly higher values than the training loss. These fluctuations reflect potential variations in nodule characteristics across validation samples.

EfficientNet B0’s architecture enabled a balanced trade-off between model depth, width, and resolution, leading to superior diagnostic precision. The model consistently outperformed VGG-16 and Inception v3 in reducing overfitting and ensuring reliable predictions, as depicted in Figure 7 and Table 4. Despite high initial metrics raising concerns about overfitting, implementation of stratified 5-fold cross-validation and regularization techniques such as dropout, L2 weight decay, early stopping, and data augmentation helped mitigate these concerns.

We further addressed class imbalance through advanced strategies. Data augmentation was applied more extensively to the minority class (MIA), and class-weighted loss functions were used to penalize misclassification of the minority class. Monitoring per-class metrics throughout training ensured balanced model performance.

The validation performance comparison between the private and LIDC-IDRI datasets demonstrates strong generalizability and high classification accuracy of the proposed model. On the private dataset, the model achieved superior scores across all key metrics: accuracy of 0.983, precision of 0.981, recall of 0.985, F1-score of 0.983, and AUC of 0.990. When evaluated on the external LIDC-IDRI dataset, the performance remained robust with slightly lower but still impressive scores: accuracy of 0.957, precision of 0.948, recall of 0.962, F1-score of 0.955, and AUC of 0.970. These results suggest that while the model performs optimally on the internal data, it also maintains high predictive capability on diverse imaging protocols and patient populations, validating its potential for real-world deployment in clinical scenarios (see Figures 8 and 9).

In a 100-case classification challenge, the AI model outperformed three radiologists: AI Model: 72% accuracy in benign classification, 28% in malignant, 0% in suspicious cases. Radiologists (combined): 69.3% accuracy for benign, 22.3% for malignant, 8.4% for suspicious cases. This performance highlights the AI’s consistency and reduction in diagnostic ambiguity.