Classification Options in Azure Machine Learning

1. Log in and go to the Azure Machine Learning Studio workspace provided in the lab.

2. Create a Training Cluster of D2 instances. Set the max cluster nodes to 2. You will need a lot of compute in this lab.

3. Create a new blank Pipeline in the Azure Machine Learning Studio Designer.

1. The necessary data is in the CRM Dataset Shared and CRM Upselling Labels Shared dataset nodes.

2. Change all missing values in CRM Dataset Shared to zeroes.

3. Join the cleaned data with the labels found in CRM Upselling Labels Shared.

4. Split the data into training and testing datasets, with 70% of the data being used for training. Be sure to use a random seed for repeatable results.

5. Set up models using each of the following machine learning algorithms. Use the default algorithm options, except as noted:

   • Two-Class Support Vector Machine
   • Two-Class Logistic Regression
   • Two-Class Boosted Decision Tree
   • Two-Class Decision Forest
   • Two-Class Averaged Perceptron
     • Set Learning rate to 0.1
   • Two-Class Neural Network
     • Set Number of hidden nodes to 50
     • Set Number of learning iterations to 25

6. Train each of those models with Col1 (from the Upselling Labels) as the label. Make sure to use the training data, not the testing data.

7. Generate predictions on the testing data.

8. Generate statistics from the predictions.

9. Submit the pipeline.

Note: Due to the large number of operations in this pipeline, it will take 10-15 minutes to complete. Grab a coffee or watch another lecture video while you wait.**

1. Right-click the CRM Upselling Labels Shared module, and select Visualize.
2. Right-click the Apply SQL Transformation module, and select Visualize Result dataset.
3. Right-click the first Evaluate Model module, and select Visualize Evaluation results.
   • Take note of the AUC value.
4. Right-click the second Evaluate Model module, and select Visualize Evaluation results.
   • Take note of the AUC value. Is the value lower or higher than the first model?
5. Right-click the final Evaluate Model module, and select Visualize Evaluation results.
   • Take note of the AUC value. Is the value or higher than the other models?
6. Select the model with the highest AUC value.

1. Adjust the Threshold Value, and take note of the accuracy metrics.
   • Do the results become more, or less accurate, as the threshold value is raised? Lowered?
2. Adjust the threshold, so that the results provide the highest true positive value, while keeping both false positive, and false negative values low.