Cleanse Missing Data Using the pandas Python Package

The Scenario

You are working as a database admin, and while attempting to load a CSV data file, you discover it has several missing values. Thankfully, you have learned some methods to resolve missing data from the awesome courses on acloud.guru!

You will take the following steps to clean up the missing values:

• Load the CSV file into a DataFrame.
• Drop any columns that are missing all values.
• Replace any remaining missing values with the last valid observed value of that column.
  • Note: This will leave some missing values at the beginning, as there was no last valid observed value.
• Write the data to a new file.

Log in to the server over SSH using the credentials provided.

The missing-data.csv file is already available in the lab instance, but if you'd like to follow along on another machine, you may download it from here.

This data was sourced from the Center for Machine Learning and Intelligent Systems. Learn more here.