| SOCR ≫ | DSPA ≫ | Topics ≫ |
Data Science and Predictive Analytics (UMich HS650)
Assessment 20: Prediction and Internal Statistical Cross Validation
SOCR/MIDAS (Ivo Dinov)
January 2018
Using the following two case-studies independently:
- Example 1: ALS (Amyotrophic Lateral Sclerosis)
- Example 2: Quality of Live in Chronic Illness (Case06_QoL_Symptom_ChronicIllness.csv)
Construct and test the following protocol:
- Review each case-study
- Choose appropriate dichotomous, polytomous, or continuous outcome variables, e.g., use
ALSFRS_slopefor ALS,CHRONICDISEASESCOREfor case 06 and cast as an outcome dichotomous outcome - Apply appropriate data preprocessing
- Perform regression modeling for continuous outcomes
- Perform classification and prediction using various methods (LDA, QDA, AdaBoost, SVM, Neural Network, KNN) for discrete outcomes
- Apply cross-validation on these regression and classification methods, respectively
- Report standard error for regression approaches
- Report appropriate quality metrics that can be used to rank the forecasting approaches based on the predictive power of the corresponding prediction/classification results
- Compare the results of model-driven and data-driven (e.g., KNN) methods
- Compare sensitivity and specificity, respectively
- Use unsupervised clustering methods (e.g., k-Means) and spectral clustering
- Evaluate and justify k-Means model and detect how agreement of the clusters with labels
- Report the classification error of k-means and compare it against the result of k-means++.
