In this course, we will explore how to wrangle data from diverse sources and shape it to enable data-driven applications. Some data scientists spend the bulk of their time doing this!
Students will learn how to gather and extract data from widely used data formats. They will learn how to assess the quality of data and explore best practices for data cleaning. We will also introduce students to MongoDB, covering the essentials of storing data and the MongoDB query language together with exploratory analysis using the MongoDB aggregation framework.
This is a great course for those interested in entry-level data science positions as well as current business/data analysts looking to add big data to their repertoire, and managers working with data professionals or looking to leverage big data.
This course is also a part of our Data Analyst Nanodegree.
Why Take This Course?
At the end of the class, students should be able to:
Programmatically extract data stored in common formats such as csv, Microsoft Excel, JSON, XML and scrape web sites to parse data from HTML.
Audit data for quality (validity, accuracy, completeness, consistency, and uniformity) and critically assess options for cleaning data in different contexts.
Store, retrieve, and analyze data using MongoDB.
This course concludes with a final project where students incorporate what they have learned to address a real-world data analysis problem.
Prerequisites and Requirements
The ideal student should have the following skills:
Programming experience in Python or a willingness to read a little documentation to understand examples and exercises throughout the course.
The ability to perform rudimentary system administration on Windows or Unix
At least some experience using a unix shell or Windows PowerShell will be helpful, but is not required.
No prior experience with databases is needed.
This course is developed in conjunction with MongoDB, Inc., the originator and primary contributor to the open source database MongoDB. MongoDB is the leading NoSQL database. Designed for how we build and run applications today, MongoDB empowers organizations to be more agile and scalable. It enables new types of applications, better customer experience, faster time to market and lower costs.
See the Technology Requirements for using Udacity.
What Will I Learn?
Use important skills from data munging to improve OpenStreetMaps data for a part of the world that you care about and give back to the community.
Lesson 1: Data Extraction Fundamentals
Assessing the Quality of Data
Intro to Tabular Formats
Parsing XLS with XLRD
Intro to JSON
Using Web APIs
Lesson 2: Data in More Complex Formats
Intro to XML
XML Design Principles
Lesson 3: Data Quality
What is Data Cleaning?
Sources of Dirty Data
Measuring Data Quality
A Blueprint for Cleaning
Lesson 4: Working with MongoDB
Data Modelling in MongoDB
Introduction to PyMongo
Getting Data into MongoDB
Operators like $gt, $lt, $exists, $regex
Querying Arrays and using $in and $all Operators
Changing entries: $update, $set, $unset
Lesson 5: Analyzing Data
Examples of Aggregation Framework
The Aggregation Pipeline
Aggregation Operators: $match, $project, $unwind, $group
Multiple Stages Using a Given Operator
Lesson 6: Case Study - OpenStreetMap Data
Using iterative parsing for large datafiles
Open Street Map XML Overview
Exercises around OpenStreetMap data
Final Project Instructions
I recommends Buy premimum account for High speed+parallel downloads!