Tag Archives: machine learning

Reinforcement Learning

Simply stated: Reinforcement Learning deals with actions and rewards (positive or negative). The rewards help to dictate the future actions.

Many children learn via reinforcement learning. Here is a simple example from my childhood. As a child, I was told not to touch the hot stove.

My action: I touched the door of the hot stove when it was open.

My reward (negative): I burned my hand, and it hurt.

My future actions resulted in me staying away from the hot stove.

This was a very simplified example of reinforcement learning. Here are two more great introductory references:

  1. Simple Beginner’s guide to Reinforcement Learning
  2. Deep Learning Research Review: Reinforcement Learning

Conversations with future data scientists (YouTube Playlist)

Last week I spent some time chatting with future data scientists. I set up a camera to record some of the answers. Below are a few of the questions addressed.

  • How did I transition to data science?
  • Why start a data science project?
  • Should a new person focus on machine learning or deep learning?
  • What is an example data science project?
  • Why is real-time important?

Hopefully the videos and answers are helpful to others. Enjoy! And I kept most of the videos fairly short. If you enjoy the videos, please subscribe to the YouTube channel, Learn Data Science. Also, if you have a question you would like answered, please leave a comment below.

Data Science Live Book

Pablo Casas has published a book freely available online, Data Science Live Book. To quote from the book,

It is a book about data preparation, data analysis and machine learning.

The book is open source, and the code examples are written in R.

Columbia University Applied Machine Learning Online

Columbia University’s course Applied Machine Learning Spring 2018 by Andreas C. Müller has all the lecture notes, slides, homework, and videos posted online.

Andreas is also the author of the book Introduction to Machine Learning with Python.

DataCamp Community News Site

DataCamp recently launched a new community site, Data Science News, for sharing and discovering data science news. It is similar to Hacker News if you are familiar with that site.

The 3 Stages of Data Science

Businesses everywhere are racing to extract meaningful insight from their data. Many organizations are spinning up data science teams and attacking problems (some more successful than others). However, one of the challenges is determining the current stage of data science within the organization. Next is determining the desired stage of data science.

Below are 3 stages of a truly mature data science organization.

1. Dashboards

The beginning stage of data science is dashboards. It is all about answering “How much?” and “What happened?” by looking at reports of historical data. If done well, it might even help an organization answer “Why”. Many organizations will refer to this phase as Business Intelligence.

The dashboard stage can be very expensive for an organization, in terms of people-hours and money. It usually involves investments in:

  1. Data Warehouse or some other storage environment, for storing the data in a single location for easy reporting
  2. ETL (Extract Transform Load) Tools for manipulating, combining, and moving data to the data warehouse
  3. Reporting Tools for displaying the results and allowing users to “explore” the data

Here are some common questions that can be answered via traditional dashboards:

  • How many customers live in each region?
  • What were the sales on Black Friday?
  • How many patients visited the hospital last month?

As you can see, there are large amounts of value that can be gained by this phase alone. It is critical for a business to clearly understand past performance. Unfortunately, this phase is where many businesses stop.

2. Machine Learning

The real “science” of data science does not begin until the second stage which is machine learning. It focuses on estimating quantities that cannot be directly observed. This could be what movies a customer will like, the price of a company’s stock tomorrow, or the causal effect of a particular advertising campaign. Machine Learning uses the data from the first phase and applies statistical or other methods to gain additional insights.

Think of machine learning as answering the following:

  • When a customer moves, will he/she spend money at a hardware store?
  • When a credit card purchase is made, what is the probability the charge was fraudulent?
  • What is the expected lifetime value of a new customer?
  • If a hurricane is coming, what will people buy? (pop tarts? it is true).

Notice the connection between an event and some outcome. The value of machine learning comes from estimating the causal outcome of potential events. This phase is filled with terms such as: machine learning, data mining, and statistical modeling. The machine learning stage is all about looking into the future!

3. Actions

Determining the actions to perform, is the third and final phase. It tries to capitalize on the results of machine learning in order to take appropriate actions. The following actions might be suitable for the events identified in the predictive section above.

  • When a customer moves, send a “welcome to the neighborhood” packet with coupons to nearby hardware stores.
  • Decline the fraudulent charge or deactivate the credit card.
  • If the new customer has very high expected lifetime value, provide some special treatment or offers to ensure the customer becomes a customer for life.
  • When a hurricane is approaching, place Pop tarts near the front of the store.

As you can see, good machine learning from the second phase can lead to clear actions.


Claiming success in Data Science is all about conquering all three stages. Each stage builds upon the previous stage. If you have put in the effort to complete the first stage, why not continue to the second and third stages?

5 Data Science Research Papers to read in Summer 2017

In the past, the blog has included 7 Important Data Science Papers and 5 More Data Science Papers. Here is another list if you are looking for something to read over the summer.

Best Practices for Machine Learning Engineering

Martin Zinkevich, Research Scientist at Google, just compiled a large list (43 to be exact) of best practices for building machine learning systems.

Rules of Machine Learning:
Best Practices for ML Engineering

If you do data engineering or are involved with building data science systems, this document is worth a look.

Recent Free Online Books for Data Science

This is just a short list of a few books that I have have recently discovered online.

  • Model-Based Machine Learning – Chapters of this book become available as they are being written. It introduces machine learning via case studies instead of just focusing on the algorithms.
  • Foundations of Data Science – This is a much more academic-focused book which could be used at the undergraduate or graduate level. It covers many of the topics one would expect: machine learning, streaming, clustering and more.
  • Deep Learning Book – This book was previously available only in HTML form and not complete. Now, it is free and downloadable.

Machine Learning Yearning Book

Andrew Ng [Co-Founder of Coursera, Stanford Professor, Chief Scientist at Baidu, and All-Around Machine Learning Expert] is writing a book during the summer of 2016. The book is titled, Machine Learning Yearning. It you visit the site and signup quickly you can get draft copies of the chapters as they become available.

Andrew is an excellent teacher. His MOOCs are wildly successful, and I expect his book to be excellent as well.