Objectives

External validation of a previously proposed decision tree (DT) model for predicting positive post-treatment 131I whole-body scintigraphy (WBS) findings in patients with differentiated thyroid cancer (DTC).
Development of an internal model using a multivariable logistic regression (MLR) algorithm, incorporating T stage and radioactive iodine (RAI) activity in addition to N stage and thyroglobulin (Tg) levels, demonstrating improved predictive value.

Methodology

Statistical analysis using non-parametric tests (Wilcoxon signed-rank test or chi-square test) for group comparisons.
External validation of the DT model using the same software procedure (ctree function in R) and cut-off values as the original study.
Development of an internal model using a MLR algorithm with 10-fold cross-validation to minimize overfitting.
Model performance evaluation using accuracy, positive predictive value (PPV), negative predictive value (NPV), area under the receiver operating characteristic (ROC) curve, and Brier score.

External validation: Area under ROC curve = 0.60 (95% CI, 0.56–0.64), PPV = 58% (95% CI, 41–74%), NPV = 90% (95% CI, 88–92%).
Internal model: Area under ROC curve = 0.75 (95% CI, 0.69–0.81), PPV = 90% (95% CI, 68–99%), NPV = 90% (95% CI, 88–92%), Brier score = 0.0841.
The internal model, including T stage and RAI activity, showed significantly higher predictive value than the external validation of the previously proposed DT model.

The external validation of the DT model yielded limited predictive value, likely due to differences in patient characteristics between the original cohort and the validation cohort. Specifically, the prevalence of positive post-treatment WBS was lower in this study (11.8% vs. 15.2%).
The internal model demonstrated improved performance, but its generalizability is limited by the single-center nature of the study. Further external validation in diverse populations is necessary.
The study highlights the challenges of applying prediction models across different populations and the importance of considering factors beyond those included in the original model. The inclusion of T stage and RAI activity significantly improved the model's performance.
The authors appropriately used 10-fold cross-validation to mitigate overfitting in the internal model. The use of a nomogram is a strength, providing a user-friendly tool for risk calculation.