WDRIV-Net: a weighted ensemble transfer learning to improve automatic type stratification of lumbar intervertebral disc bulge, prolapse, and herniation

Abstract

The degeneration of the intervertebral discs in the lumbar spine is the common cause of neurological and physical dysfunctions and chronic disability of patients, which can be stratified into single—(e.g., disc herniation, prolapse, or bulge) and comorbidity-type degeneration (e.g., simultaneous presence of two or more conditions), respectively. A sample of lumbar magnetic resonance imaging (MRI) images from multiple clinical hospitals in China was collected and used in the proposal assessment. We devised a weighted transfer learning framework WDRIV-Net by ensembling four pre-trained models including Densenet169, ResNet101, InceptionV3, and VGG19. The proposed approach was applied to the clinical data and achieved 96.25% accuracy, surpassing the benchmark ResNet101 (87.5%), DenseNet169 (82.5%), VGG19 (88.75%), InceptionV3 (93.75%), and other state-of-the-art (SOTA) ensemble deep learning models. Furthermore, improved performance was observed as well for the metric of the area under the curve (AUC), producing a ≥ 7% increase versus other SOTA ensemble learning, a ≥ 6% increase versus most-studied models, and a ≥ 2% increase versus the baselines. WDRIV-Net can serve as a guide in the initial and efficient type screening of complex degeneration of lumbar intervertebral discs (LID) and assist in the early-stage selection of clinically differentiated treatment options.

Introduction

Lumbar intervertebral discs (LID) are essential for spinal motion, flexibility, and stability [1]. Lumbar disc herniation (LDH), lumbar disc prolapse (LDP), and lumbar disc bulge (LDB) are three prevailing types of lumbar spine degeneration observed in clinical practice worldwide [1,2,3,4,5]. Degenerative changes in the LID including LDH, LDP, and LDB can occur as early as the first decade of life and can be associated with symptoms including severe/chronic lower back pain, sciatica, muscle spasms, and disability impacting the quality of life [1,2,3]. For patients simultaneously experiencing multiple conditions (i.e., comorbidity), the disease may advance to a more severe or complicated stage, causing generally a situation where differentiated health care or urgent intervention/treatment is needed [6, 7]. These patients are more likely to bear more stress and costs if the decision-making is sub-optimal. Clinically, it is of importance to automatically identify the more complicated cases (e.g., comorbidity-type) from those less complicated counterparts (e.g., single-type) at an earlier stage effectively.

Radiography images using X-, gamma-ray techniques, computerized tomography scans (CT), and magnetic resonance imaging (MRI) are primarily exploited in detecting the disease of LID [8, 9]. However, MRI is the preferred imaging modality for LID-related diseases versus other imaging techniques [9] and is widely utilized in treatment planning [8,9,10,11]. Accurate interpretation of the principal traits of MRI of LID is crucial for the preprocedural assessment of potential interventions for the lumbar spine such as rehabilitation training, injection-based treatment, and surgery [12]. Presently manual inspection by medical professionals represents the most commonplace approach to extracting information from MRI images [8]. The visual examination carried out slide-by-slide by experts typically relies on the expertise of medical professionals and mostly is time-consuming and bias-prone [6, 12]. Challenges characterized by ambiguity, inconsistency, or conflict of diagnosis are not uncommon, which may induce unintended medical consequences [6]. Therefore, the challenge of automating the classification of disc degeneration remains a significant concern for both patients and physicians [6, 12].

The diagnosis of LID degeneration is also influenced by factors such as image quality and analysis techniques, beyond the inherent bias of manual observation [10, 11, 13]. These variables significantly impact the evaluation of LID degeneration type and progression, as well as subsequent treatment decisions [5, 11, 14]. Specifically, as shown in Fig. 1, aeolotropic dimensional resolution, inter-type similarity, and intra-type variety of MRI of LID may yield difficulty in identifying the type or severity of degeneration [15,16,17]. Therefore, designing an automatic classification framework for LID degeneration is meaningful to address the issue of sub-optimal image quality, enhance the efficiency of physician diagnosis, optimize the operability level of LID clinical applications, and alleviate the financial, physical as well as psychiatric burden on patients [10, 11, 18, 19].

Fig. 1

Aeolotropic dimensional resolution, inter-type similarity, and intra-type variety of lumbar spine MRI (a–d) induce challenges in detecting the degeneration type of LID including bulge, prolapse, and herniation

The major contributions of this research are outlined as follows:

• 1. The proposed framework employs a weighted ensemble of fine-tuned benchmark models using a deep transfer learning approach to improve performance and generalizability in medical image processing.

• 2. The proposed approach aims to precisely identify and stratify single- and comorbidity-type degeneration in the MRI images of LID at an early phase that has not been investigated in previous literature, addressing the problems in traditional methodologies such as false-positive rates, false-negative rates, data noise (e.g., aeolotropic dimensional resolution, inter-type similarity, and intra-type variety), accuracy, and area under the curve value (AUC) of a sub-optimal single classifier.

• 3. The proposed ensemble model employs pre-processing steps, including data augmentation and denoising, to counter the challenges of resolution variations and data noises, which potentially bias the accuracy of stratification. The knowledge learned from ImageNet is effectively transferred and guides the model to the correct targets by yielding improved outputs.

• 4. The effectiveness and robustness of the proposed approach are evaluated using a variety of metrics including AUC, accuracy, recall, precision, F1-score, and confusion matrix. Further, the performance of the proposed framework is compared with and superior to the outcomes of the benchmarks, recent most-studied models, and other SOTA deep learning.

The remaining sections of the study are organized as follows: Section Literature review delineates literature work investigating machine learning, deep learning, and ensemble learning with medical image tasks and convolution-network processing. Section Results depicts the experimental tests. Section Discussion and conclusions concludes the research with research limitations and future directions. And section Materials and methods explains the proposed methodology.

Literature review

Machine learning and deep neural networks have proven effective in addressing the complexity and variability of medical image analysis [20,21,22]. In the last decade, numerous semi- or full-supervised algorithms have been devised to analyze medical images and assist medical diagnosis driven by the advancement of imaging quality and analytical techniques [18, 19, 21, 23,24,25,26,27,28,29,30,31,32,33]. Full supervised learning generally uses labeled data, whereas the semi-supervised counterpart normally regulates the output of deep learning.

Ensemble learning and deep learning are the predominant machine learning approaches that have witnessed applications in medical image-related tasks during recent years, achieving benchmark performance across various disease-related tasks [16,17,18,19, 28, 32, 34,35,36,37,38,39,40]. Deep learning has made substantial advancements in image analysis during the past decade, excelling at feature extraction [16, 17, 21, 25,26,27]. These techniques, particularly when combined with convolutional neural networks (CNNs), have been instrumental in assisting medical professionals with diagnosis. CNNs, evolving over the past half a century, are crucial in image classification [37,38,39,40], object detection [15, 21, 41], and segmentation [34, 42,43,44,45,46].

On the other hand, challenges and prospects in medical image processing have been widely discussed [16,17,18,19, 47,48,49,50,51,52,53]. For example, Tavana et al. [16] demonstrated the potential of deep learning in classifying the type of spinal curvature. Pandi et al. [47] and Niu et al. [48] highlighted the use of evolutionary deep full CNNs in medical image segmentation. Tanveer et al. [20] exploited deep learning to analyze speech signal tasks. In addition, Niu et al. [48] explored application scenarios other than medical image processing using deep learning. Additionally, Zheng et al. [53] discussed the use of deep learning techniques to detect image-level classification for breast histopathology, and XGBoost feature selection has been used to improve protein–protein interaction prediction accuracy [18, 19]. Nevertheless, debates remain regarding the challenge of generalizability [16,17,18,19, 47,48,49,50,51,52,53]. To date, to our best knowledge, no research has addressed the complicated comorbidity degeneration of the lumbar spine in connection with automatic stratification using clinical observations. This underscores the need for a reliable classification algorithm. Developing a robust detection methodology could improve processing efficiency and reduce error rates, motivating our effort to design an approach for automatic stratification of LID degeneration using MRI images.

Results

The comparison of the confusion matrix output for each model, the proposal approach, and other SOTA ensemble learning is shown in Fig. 2. The matrix reveals true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN), verifying the effectiveness and accuracy of the approaches. Training and validation loss trends during training are illustrated in Fig. 3, while accuracy, precision, recall, F1 scores, and AUC of all frameworks, including the proposed WDRIV-Net, are detailed in Figs. 4 and 5.

Fig. 2

Confusion matrix for a pre-trained Densenet169, b pre-trained ResNet101, c pre-trained InceptionV3, d pre-trained VGG19, e Tavana et al. [16, 17], and f the proposed weighted ensemble deep transfer learning model WDRIV-Net. The axis label of 0 represents single-type degeneration of LID, and 1 represents comorbidity-type degeneration of LID

Fig. 3

Trend observation of training loss and validation loss during training for the four baseline models. a Densenet169, b ResNet101, c InceptionV3, d VGG19

Fig. 4

Evaluation of accuracy, precision, recall, F1-score, and AUC in detecting the single-type degeneration of LID

Fig. 5

Evaluation of accuracy, precision, recall, F1-score, and AUC in detecting the comorbidity-type degeneration of LID

The confusion matrix suggested that the proposed WDRIV-Net outperformed individual models and other SOTA ensemble frameworks, yielding an accuracy of 96.25% in total (77/80). This surpasses DenseNet169 (82.5%, 66/80), ResNet101 (87.5%, 70/80), VGG19 (88.75%, 71/80), InceptionV3 (93.75%, 75/80), and Tavana et al. (88.75%, 71/80) [16]. Notably, three MRI images of single-type degeneration were misclassified as comorbidity-type degeneration by WDRIV-Net, while no comorbidity-type images were misclassified. Other models, including DenseNet169, ResNet101, VGG19, InceptionV3, and the SOTA model [16], exhibited higher FP and FN rates and lower TP and TN rates compared to WDRIV-Net.

The results in Fig. 3 outlined the trend change in training and validation loss over epochs during training. Although partial of the baseline models (e.g., Densenet169 and ResNet101) observed greater fluctuations, the overall loss trends for all four models were consistent, decreasing with the increment of epochs and approaching zero by the end of training. The optimal validation loss varied among models: ResNet101 observed ~ 0.14 at epoch 17, DenseNet169 ~ 0.05 at epoch 24, InceptionV3 ~ 0.04 at epoch 23, and VGG19 ~ 0.30 at epoch 24.

Figures 4 and 5 delineate the accuracy, precision, recall, F1-score, and AUC for classifying LID degeneration using the proposed WDRIV-Net, benchmark models, and other ensemble methods. The results suggested that WDRIV-Net outperformed all models for both single- and comorbidity-type degeneration, achieving superior accuracy, F1-score, AUC, precision, and recall. Accuracy and AUC followed similar trends across models, with WDRIV-Net observing the highest values, outperforming pre-trained InceptionV3, ResNet101, DenseNet169, VGG16, and the SOTA model [16]. The weighted ensemble approach improved the performance of individual models and surpassed all single methods and other ensemble strategies. Notably, the ensemble mechanism in [16], which used an equal-weight average, performed worse than the non-equal-weight WDRIV-Net and some baseline models (e.g., InceptionV3).

Further, to evaluate the performance of the proposal WDRIV-Net versus other recent models and ensemble learning, we summarized the outcomes in Table 1 with references to NASNetMobile [52], MobileNetV2 [54], VGG19 [55], NasNetLarge [52], ResNet50 [39], ResNet152 [39], DenseNet121 [42], DenseNet201 [42], the four individual benchmark models integrated into the proposed approach, and the ensemble model by [16]. These reference models represented widely studied or high-performing frameworks in other diseases and ensemble techniques [16, 19, 28, 31, 34, 35, 37, 39, 40, 42,43,44,45, 52, 54, 56,57,58,59,60,61,62,63,64,65,66].

Table 1 A comparison between the proposed WDRIV-Net with other models

Table 2 indicates that the proposed ensemble model observed 96.25% accuracy, surpassing [16] (88.75%), DenseNet121 [42] and ResNet152 [39] (90%), InceptionResNetV2 [23] (85%), and all individual baseline models (< 94%). NASNetMobile [52] recorded the lowest accuracy. Similar trends were observed for precision, recall, F1-score, and AUC, with WDRIV-Net achieving the highest AUC. The proposed model demonstrated a ~ 7.5% AUC improvement over [16], a ≥ 6% increase over other top-performing models, and a ≥ 2% improvement over its benchmark models.

Table 2 Summary of the private LID MRI sagittal-view data set and parameters

Discussion

In this research, we proposed WDRIV-Net, a weighted ensemble-based deep transfer learning framework, to classify LID MRI sagittal-view images from clinical settings, observing 96.25% accuracy for single- and comorbidity-type disc degeneration. WDRIV-Net outperformed other models with a ≥ 7% increase in AUC compared to SOTA ensemble learning, a ≥ 6% improvement over widely studied models, and a ≥ 2% improvement over baseline models (DenseNet169, ResNet101, InceptionV3, and VGG19). Similar gains were observed in precision, recall, and F1-score.

The framework leveraged transfer learning on ImageNet and effectively applied it to a private clinical LID dataset, with pre-processing, data augmentation, and ensemble techniques addressing limited samples and noise. By combining the strengths of deep learning and weighted ensemble learning, WDRIV-Net potentially enhanced generalization. LID degeneration is a prevalent lumbar spinal condition impacting individuals globally, leading to reduced quality of life and significant socioeconomic burdens. Early diagnosis is crucial, as delayed intervention risks disease progression, complicating treatment and increasing burdens on patients and healthcare systems. MRI imaging is the standard diagnostic tool, distinguishing comorbidity- and single-type degeneration to guide timely and appropriate treatment.

LID degeneration can progress rapidly under certain conditions, and distinguishing degeneration types is challenging due to factors like MRI image quality (e.g., anisotropic resolution), inter-class similarity, intra-class variation, and observation bias. Additionally, a shortage of physicians and the time-intensive nature of medical image analysis complicate diagnosis, underscoring the need for automated diagnostic systems. To address data limitations, we applied data augmentation to expand the sample size. Ensemble transfer learning, which combines automatic feature extraction through CNNs, deep learning, and knowledge transfer, is a widely used framework for medical image analysis, aiding in decision-making systems and LID application development. Integrating weighted ensemble learning, data augmentation, and transfer learning enables efficient and timely evaluation of lumbar spine abnormalities.

The major contribution of this research lies in developing WDRIV-Net, a non-equal-weight ensemble deep transfer learning framework that combines four pre-trained models (DenseNet169, ResNet101, InceptionV3, and VGG19) to classify LID degeneration types (single-type and comorbidity-type). WDRIV-Net achieved superior performance across all evaluation metrics, including accuracy, AUC, precision, recall, and F1-score, as shown in Table 2. It outperformed the baseline models, other widely studied models, and SOTA ensemble methods, delivering the best results in most cases. Specifically, WDRIV-Net achieved higher accuracy and AUC on the private test dataset compared to VGG19, InceptionV3, DenseNet169, ResNet101, and other SOTA ensemble methods. The improved performance, with lower false-positive and false-negative rates, demonstrates its potential to complement clinical decision-making effectively. This study shows that WDRIV-Net can handle classification tasks with size-constrained datasets and provides insights for clinical interventions and treatments for lumbar spine intervertebral disc degeneration.

This work has several limitations as well. First, the dataset used for training and testing included only sagittal-view images, excluding axial- and coronal-view data commonly used by medical experts for comprehensive diagnosis. The high-accuracy results of the experiments implied that the bias could be negligible or maintained at a small level for this dataset. Second, the original dataset was relatively small for the standard practice of deep learning, a challenge noted in other studies [16, 17, 67]. Future research could explore integrating multi-view data or designing models to merge information from different planes. Additionally, performance comparisons between transfer-learning-based models and non-transfer-learning approaches, such as ROI segmentation or bounding box cropping, could be evaluated when more data are available [67]. Third, the model determines whether an MRI exam reveals single or multiple symptoms of disc bulge, prolapse, or herniation, yielding an overall classification across spinal levels. However, it does not distinguish these symptoms at individual intervertebral disc levels, nor does it offer detailed symptom descriptions. Although the traits of disc levels of LID where available were reported in the medical text records and might be exploited for diagnosis, the information was not annotated directly in the image data. Future research could address this limitation by designing more advanced models by fusing medical text records or using more detailed annotated image data.

Conclusion

In this study, we propose a non-equal-weighted transfer learning algorithm by ensembling four pre-trained baseline models for the automatic type classification of lumbar intervertebral disc bulge, prolapse, and herniation. The proposed model can be integrated into the medial assessment system for the initial screening of LID degeneration cases with differentiated symptoms where the priority of intervention/treatment is of concern. This potentially results in reduced medical costs, mitigates risks from diagnostic ambiguity, and accelerates treatment decisions for LID degeneration. The outcomes identified in this study will be beneficial for healthcare practitioners, physicians, patients, and researchers in medical image processing.

Methods

The experimental results are illustrated to test the efficiency and effectiveness of the proposed approach. A private MRI data set consisting of 1182 lumbar spine sagittal-view images was collected at the Fujian Medical University Union Hospital, China, and Pingtan Comprehensive Experimentation Area Hospital, Pingtan, China. We deployed TensorFlow and Keras as the Python toolkits in the conda environment to facilitate the training and testing procedures. A standard desktop computer using Windows 11 with 16 GB of RAM and an NVIDIA RTX A4000 graphical processor was used to evaluate the performance of models and the proposed ensemble approach.

MRI data set of lumbar spine degeneration

The private data set comprised 1182 MRIs of lumbar spine degeneration images in total collected at the Fujian Medical University Union Hospital, China, and Pingtan Comprehensive Experimentation Area Hospital, Pingtan, China, with 591 cases each of single- and comorbidity-type degeneration. We randomly divided the 591 cases into 551 cases of training data and 40 cases of testing data for both types (Table 1). The ratio of the training sample size to the verification sample size during training was set up to 80:20. The data were labeled and checked by three medical experts with more than 5 (expert #1), 10 (expert #2), and 20 (expert #3) years of clinical experience in medical image processing, respectively. The data were initially observed and labeled by expert #1 to identify the type of disc bulge, prolapse, or herniation by examining and recording the traits of the LID. Multi-view data (e.g., sagittal-, axial-, and coronal-view data) were merged comprehensively to facilitate the correct manual examination in cases where single- or partial-view data were not able to draw the final diagnosis. Further, in the inspection of sagittal-view data, the expert recorded the traits of disc levels of LID where available and used for subsequent diagnosis. In the output of manual inspection, the expert reported an additional flag showing the type information, and whether two or more of these symptoms presented. These first-stage results were cross-checked and corrected by expert #2 and thereafter by the expert #3. Potential inspection error outputs from expert #1 were corrected by expert #2, and the error outputs from expert #2 were updated by expert #3 if identified in the sequential chain process. The examination process was conducted in compliance with the AAOS [68,69,70].

The final annotated dataset consisted of 1,182 cases with equal numbers of single- and comorbidity-type LID degeneration, forming the raw data for analysis. The deep learning model used representative sagittal-view data for training and testing. A retrospective review of patient cases from Fujian Medical University Union Hospital and Pingtan Comprehensive Experimentation Area Hospital was conducted for data collected between January and December 2022. MRI scans were obtained using Siemens 3 T MRI scanners, with cases involving multiple inspections excluded. The study was approved by the Ethics Committee of Fujian Medical University Union Hospital (2020YF023-01). Examples of the dataset are shown in Fig. 6.

Fig. 6

a–c Examples of lumbar spine MRI images of the single- and comorbidity-type degeneration in the collected private data set

Data pre-processing and data augmentation

The original MRI data resolution ranged from 320 × 320 × 3 to 512 × 512 × 3. To standardize, the images were resized to 224 × 224 × 3 for input into DenseNet169, ResNet101, InceptionV3, and VGG19 [23, 24, 39, 40, 42, 55]. Images were categorized into single- or comorbidity-type groups. Pixel values were normalized to the range from 0 to 1, and resizing and shuffling generated standardized samples. The analysis was challenged by limited data. To partially solve this issue, we first exploited the publicly available larger dataset ImageNet to transfer learning the traits of the widely used data set and thereafter utilized a data-augmentation technique to enlarge the size of training data and ameliorate the disturbance from data noise [10, 11, 21]. This process generated approximately ≥ 12,000 augmented images, helping prevent overfitting. Figure 7 illustrates the data pre-processing and augmentation procedures.

Fig. 7

Data pre-processing flow and data augmentation

Fine-tuning parameters and settings

An approach classifying the type of LID degeneration was proposed by using private MRI images of the lumbar spine. We utilized the data-augmentation technique by rotating (i.e., 20 degrees), width shifting (i.e., 0.05), height shifting (i.e., 0.05), and horizontally flipping the lumbar spine MRI images to solve the challenge of a limited sample. Thereafter, a weighted ensemble-voting-based classification method was applied to predict the type of LID degeneration.

The proposed weighted ensemble deep transfer learning model was trained on the balanced and augmented dataset using categorical cross-entropy loss and the Adam optimizer (learning rate = 1e−4). Accuracy was estimated across both sub-categories, with training set for 25 epochs. An early stopping mechanism monitored performance, halting training after 10 epochs of no improvement to conserve computational resources. A dropout rate of 20% was used to mitigate overfitting. AUC values were also measured to evaluate the model's capacity to detect positive and negative cases.

Metrics for stratification

Accuracy, recall, precision, and F1-score

The benchmark models, the proposed approach, and other ensemble learning were examined using the test data set when the training procedure was completed by exploiting the methodology of data augmentation and transfer learning. We evaluated the metrics including accuracy, recall, precision, F1-score, and AUC. We defined that true positive (TP) denotes the statistics information of comorbidity-type degeneration in the lumbar spine images, contrastingly true negative (TN) delineates the statistics of single-type degeneration. Consequently, false negative (FN) represents the statistics of comorbidity-type images incorrectly classified as single-type counterparts, and false positive (FP) demonstrates the statistics of single-type images that are incorrectly identified as comorbidity-type (Hashmi et al., 2020). The metrics of accuracy, precision, recall (or equivalently sensitivity), and F1 are determined by the formulas defined from Eq. (1) to (4):

$$Accuracy = \frac{TP + TN}{{TP + TN + FN + FP}},$$

(1)

$$Precision = \frac{TP}{{TP + FP}},$$

(2)

$$Recall = \frac{TP}{{TP + FN}},$$

(3)

$$F1 = \frac{2 \times Precision \times Recall}{{Precision + Recall}},$$

(4)

ROC and AUC

Receiver operating characteristic (ROC) analysis is a standard methodology used to evaluate the performance of a binary classification system, which is applied extensively in clinical medicine analysis [43]. The ROC curve is a two-dimensional plotting that illustrates the relationship between the true and false-positive rates of a binary classifier.

AUC, which is identical to the probability that the decision value allocated to a randomly selected positive sample is greater than the value allocated to a negative sample (or equivalently, a randomly chosen negative case with a smaller estimated probability than a positive counterpart belonging to the positive type), is a univariate description of the ROC curve [43, 61]. The AUC curve is a performance metric sketching the extent to which a classifier divides types. Fundamentally, a classifier seeks the optimal formation of samples from a multi-dimensional feature space to a one-dimensional space during the training process. Hence, the range of AUC is typically larger than 0.5 and lower than 1, with greater values representing better performance as the area enclosed is greater. A proposed approach to calculate the AUC for the binary classification is illustrated in Eq. (5) [38]:

$$AUC = \frac{{S_{0} - \frac{{n_{0} \left( {n_{0} + 1} \right)}}{2}}}{{n_{0} n_{1} }},$$

(5)

where $n_{0}$ and $n_{1}$ denote, respectively, the statistics of positive and negative cases, and $S_{0} = \sum {r_{i} }$, where $r_{i}$ is the rank statistics of the ith positive case.

Metrics for weighted proposed ensemble model

The proposed ensemble model weighs each baseline model and the mathematical formulation is governed by the eq. from (6) to (7):

$$WPred = \prod \left( {\mathop \sum \limits_{i = 1}^{4} {\mathcal{W}}_{i} Pred_{i} } \right),$$

(6)

$$\mathop \sum \limits_{i = 1}^{4} {\mathcal{W}}_{i} = 1,$$

(7)

where ${Pred}_{i} (i=1, 2, 3, 4)$ denotes the predicted outcome of the benchmark models, and ${\mathcal{W}}_{i}$ sketches the weights for the baseline models. The summation of all weights is equal to one.$\prod \left(\cdot \right)$ represents the algorithm of weighted ensemble deep learning, and $WPred$ is the prediction outcome using the proposed weighted ensemble algorithm. In our analysis, ${\mathcal{W}}_{i}=\left\{0.2, 0.1, 0.3, 0.4\right\}$ for the benchmark resnet101, DenseNet169, InceptionV3, and VGG19, respectively. The weights for the models were calibrated and finally decided by partially referring to the performance and the loss observations of each model.

Benchmark models

An optimal approach to the automatic classifying of types of lumbar spinal complex degeneration was presented based on MRI images of LID. To solve the limited-data issue, we employed data augmentation and denoising techniques and exploited the ImageNet data set to pre-train four models and thereafter transferred the knowledge learned from ImageNet to the private MRI data set of LID based on weighted ensemble deep transfer learning, which enhanced the generalization of the models by integration of the merits of both deep learning, transfer learning, and weighted ensemble learning.

ImageNet data set

ImageNet is a data set accommodating more than 15 million high-resolution labeled images incorporating nearly 22,000 categories. The images in ImageNet are collected from the internet globally and labeled by professionals [62, 71]. Pre-trained models based on the ImageNet database are used to extract the features of medical images and have proven validity and effectiveness in predicting types, classification, segmentation, and other clinical outcomes [18, 32, 34,35,36,37, 62, 71].

Weighted transfer learning

Transfer learning is a widely used machine learning method in learning applicable knowledge from constrained data for prediction issues [16, 49, 51, 53, 64]. It has gained significant attention for its ability to transfer knowledge across different data distributions. The effectiveness of transfer learning has been validated in a variety of scenarios [16, 49, 51, 53, 63, 64], with four main models: instance-, feature-, parameter-, and relation-based [64].

A recent advancement, weighted ensemble deep transfer learning, combines deep learning, feature transfer learning, and weighted ensemble learning of individual models [16, 63]. This approach enhances machine learning and transfer learning, offering better generalization, accuracy, flexibility, and robustness [16, 18, 48]. In our experiments, we observed that the non-equal-weight ensemble method outperformed the equal-weight approach.

Data augmentation

Recent breakthroughs in image processing have been driven by deep learning techniques, including classification [37, 39], object recognition [15, 21, 41], and segmentation [45, 46]. However, deep learning faces challenges such as small sample sizes and data quality. To address these, data augmentation generates synthetic data with alternative features while preserving the major traits of the original data, improving the performance of classifications [65, 66]. Additionally, combining data augmentation with transfer learning enhances model performance [59]. In this study, we employed both techniques to overcome these challenges.

VGG19 model

VGG stands for Visual Geometry Group, which is a deep-learning architecture with CNNs structures [55, 71]. VGG19, a variant of the VGG model, consists of 19 layers including 16 layers of CNNs, 3 fully connected layers (i.e., FC), and a softmax/sigmoid layer for prediction. VGG is a benchmark that outranks other deep neural baseline models in multiple vision tasks [55, 71, 72]. Studies exploring the effect of the CNNs depth on the accuracy in the sizable image-detection setting have identified that increasing depth to ≥ 19 weight layers with minuscule convolution filters can considerably enhance the performance of localization and classification, respectively. And the outcomes generalize well to other settings as well [15, 55].

The representative input to VGG typically consists of 224 × 224 RGB images with a receptive size of 3 × 3 by default [71]. A representative VGG19 model can comprise five blocks using CNNs and pooling layers to extract features, followed by FC layers and a softmax or sigmoid output layer for multi- or binary classification [55, 71]. Figure 8 outlines the schema of a benchmark VGG19 model used in the proposed ensemble analysis [55].

Fig. 8

Schema of a benchmark VGG19 model, and the compositional architecture used in the proposal

Resnet101 model

Resnet represents the residual network that reduces the cost of training [39, 40]. The “101” denotes the structure of weight layers. Resnet rephrases the networks as residual framework relative to the source input signal [39, 40, 73,74,75]. The Resnet framework exploits the identity shortcuts to capture essential residual information. Past studies have unveiled that the referenced residual networks excel in optimization with admissible accuracy [39, 40]. Experiments on ImageNet data set with a depth of over one hundred layers have yielded desirable outcomes with reduced complexity [39, 40, 74].

A typical ResNet model includes five layers of CNNs, each consisting of convolution, batch normalization, and activation processes. Figure 9 illustrates the ResNet101 model framework used in the proposed ensemble approach, where element-wise computation between the residual R(x) and input x is conducted to better preserve features [39, 73, 74].

Fig. 9

Schema of a benchmark Resnet101 model, and the architecture used in the proposal analysis

DenseNet169 model

DenseNet (Densely Connected Convolutional Networks) connects each layer to all preceding layers, using their feature maps as inputs for subsequent layers [42, 76, 77]. This architecture reduces issues like vanishing and exploding gradients, improves feature propagation, and reuses features while minimizing parameter size. DenseNet has shown significant performance improvements over SOTA networks with fewer computational resources. Typically, it features more layer connections compared to traditional CNNs. Figure 10 illustrates the DenseNet169 model architecture used in this study [42, 77].

Fig. 10

Schema of a benchmark DenseNet169 model, and the compositional architecture used in the proposal appraisal

InceptionV3 model

The Inception architecture optimizes local structures in a network using available components. InceptionV3 improves on InceptionV1 and V2 by addressing issues like label smoothing regularization and normalization, reducing overfitting. In our experiments, we used InceptionV3 with 1 × 1, 3 × 3, and 5 × 5 convolutions. We also tested InceptionResNetV2 with residual structures, but the ensemble framework with InceptionV3 outperformed InceptionResNetV2 in our setup (results not shown for brevity). Figure 11 illustrates the InceptionV3 model and its architecture used in our approach [23, 24].

Fig. 11

Schema of a benchmark InceptionV3 model with 1 × 1, 3 × 3, and 5 × 5 convolutions, and the compositional architecture used in the analysis

The proposed weighted ensemble deep transfer learning framework

Ensemble deep transfer learning is a recent development that combines transfer learning and weighted ensemble learning techniques to generate a robust architecture yielding the consequential improvement of performance, as well as better generalization capacities over individual benchmark models [16].

The proposed WDRIV-Net framework uses non-equal-weight soft-ensemble voting to predict outcomes based on the probabilities from each baseline model. This approach combines the predictions of four pre-trained benchmark models (Densenet169, ResNet101, InceptionV3, VGG19) trained on the ImageNet dataset, improving overall performance and reliability while preventing gradient vanishing or exploding issues. The weighted soft-voting classifier stratifies predictions based on each model's probability, and an ensemble aggregator applies calibrated weights for final estimation. Figure 12 illustrates the WDRIV-Net framework and algorithm flow.

Fig. 12

The schema of the proposed ensemble deep transfer learning