You Want Data Science, Now What?

I am often confronted by people or organizations whom have heard about data science but don’t know where to start. It is a valid concern. Data science is a broad topic with different meanings to different people.

Here are the common questions I hear. Should I hire a data scientist? Should I hire some consultants? Should I build a data science team? There is no perfect answer for those questions because it depends upon your organization and situation. I would like to suggest a different approach. At first, don’t worry about the titles and organizational structure. Worry about the problems you want to solve. First, start out with 2 questions.

1. What is the goal (be specific)?

This question might seem obvious, but it is often overlooked. Don’t start with data science just because you have heard about others using it. A bad goal for data science is: be data-driven to increase profits. While that might be a high-level strategy, it is much too broad. Better goals are:

  • identify which customers are likely to leave
  • identify which products a customer might buy next
  • determine what cities would be best for expansion
  • find the most profitable type of marketing for your organization
  • predict if a person will get cancer in the next year

These are examples of specific goals that data science can help to address. Work hard to narrow your goals to something specific. If you can get enough specific goals, then you might be able to increase profits.

2. What action can be taken?

This is very important. All the predictions and fancy data science does you no good if your organization cannot take any action. For example, sticking with the previous examples. Suppose you can predict if a person will get cancer in the next year. What do you do with that information? Do you send the person an email? What if you are wrong? Do people really want to know that? That is a tricky situation to handle and any action you take has an ethics component.

Other situations have simpler actions, such as identifying the products a customer might be next. Common actions might be: sending a coupon, displaying an add, or suggesting the item be added to the cart.

Another factor to consider with the action is cost. How much will it cost to perform some action. In certain businesses, it might be more profitable to attract new customers than retain existing customers. Thus, there is little advantage to identifying which customers are likely leave.


Data science is very exciting, and it has many positives. However, when done with incorrect expectations, it can lead to nowhere but headaches. Thus, before you start building a team or hiring some consultants, make sure you are clear on your goals and actions.

TensorKart and Neural Networks for MarioKart

Kevin Hughes used TensorFlow to train a Neural Network to play MarioKart. He calls it TensorKart. See his post for more details. It is a nice blog post and sounds like a fun project.
Sorry, the video has no sound.

Julia for Data Science Book

Today brings us a very welcome guest post by Zacharias Voulgaris, author of Julia for Data Science. This is an excellent new book about the Julia language. By reading it you will learn about:

  • IDEs for using Julia
  • Basics of the Julia language
  • Accessing and exploring data
  • Machine learning
  • Advanced data science techniques with Julia (cross-validation, clustering, PCA, and more)

The book has a nice flow for someone starting out with Julia and the topics are well explained. Enjoy the post, and hopefully you get a chance to check out the book.

Introducing Julia for Data Science (Technics Publications), a Great Resource for Anyone Interested in Data Science.

Over the past couple of years, there have been several books on the Julia language, a relatively new and versatile tool for computationally-heavy applications. Julia has been adopted extensively by the scientific community as it provided a great alternative to MATLAB and R, while its high-level programming style made it easy for people who were not adept programmers. Also, lately it has attracted the attention of computer science professionals (including Python programmers) as well as data scientists. These people who were already very effective coders, decided to learn this language as well, since it provided undeniable benefits in terms of performance and rapid prototype development, esp. when it came to numeric applications. In addition, the fact that Julia was and is still being developed by a few top MIT graduates goes on to show that this is not a novelty doomed to fade away soon, but instead it is a serious effort that’s bound to linger for many years to come.

However, this post is not about Julia per se, since there are many other people who have made its many merits known to the world since the language was first released in 2012. Instead, we aim to talk about the lesser-known aspects of the language, namely its abundant applications in the fascinating field of data science. Although there are already some reliable resources out there pinpointing the fact that Julia is undoubtedly ready for data science, this book is the first and most complete resource on this topic. Without assuming any prior knowledge of the language, it guides you step-by-step to the mastery of the Julia essentials, helping you get comfortable enough to use it for a variety data science applications. It may not make you an expert in the language, but data scientists rarely care about the esoteric aspects of the programming tools they use, since this level of know-how is not required for getting stuff done. However, the reader is given enough information to be able to investigate those aspects on his own.

The Julia for Data Science book has been in development for about a year and is heavily focused on the applications part, with lots of code snippets, examples, and even questions and exercises, in every chapter. Also, it makes use of a couple of datasets that closely resemble the real-world ones that data scientists encounter in their everyday work. On top of that, it provides you with some theory on the data science process (there is a whole chapter of it dedicated to this, although other books usually devote a couple of pages to it). Although the book is not a complete guide to data science, it provides you with enough information to have a sense of perspective and understand how everything fits together. It is by no means a recipe book, though you can use it as reference one, once you have finished reading it.

The Julia for Data Science book is available at the publisher’s website, as well as on Amazon, in both paperback and eBook formats. We encourage you to give it a read and experience first-hand how Julia can enrich your data science toolbox!

Georgia Tech Masters in Analytics for Less than $10k

Georgia Tech University just announced a new online master’s degree in Analytics.

Georgia Tech Creates First Online Master of Science in Analytics Degree for Less Than $10,000

The degree will begin in August 2017 and will be fully online. It will offer 3 tracks:

  1. Big Data
  2. Analytical Tools
  3. Business Analytics (coming at a later date)

Deep Learning Summer School 2016 Videos

Deep Learning Summer School, Montreal 2016 is aimed at graduate students and industrial engineers and researchers who already have some basic knowledge of machine learning (and possibly but not necessarily of deep learning) and wish to learn more about this rapidly growing field of research. If that is you, there are plenty of videos to help you learn more.

ACM Townhall Webinar with Peter Norvig

The Association for Computing Machinery is hosting a Town Hall with Peter Norvig on A.I., Machine Learning, and More. Peter Norvig is the Research Director at Google, and a leader in the field of Artificial Intelligence (AI). Topics he might discuss:

  • deep learning
  • future of AI
  • teaching AI
  • academia vs industry
  • advice for grad students

The free webinar is Thursday, December 08, 2016 at noon Eastern time.

Recent Free Online Books for Data Science

This is just a short list of a few books that I have have recently discovered online.

  • Model-Based Machine Learning – Chapters of this book become available as they are being written. It introduces machine learning via case studies instead of just focusing on the algorithms.
  • Foundations of Data Science – This is a much more academic-focused book which could be used at the undergraduate or graduate level. It covers many of the topics one would expect: machine learning, streaming, clustering and more.
  • Deep Learning Book – This book was previously available only in HTML form and not complete. Now, it is free and downloadable.

Learn to Analyze Big Data with R – Free Course

R is a hugely popular language among data scientists and statisticians. One of the difficulties with open-source R is the memory constraint. All the data needs to be loaded into a data.frame. Microsoft solves this problem with the RevoScaleR package of the Microsoft R Server. Just launched this week is an EdX course on
Analyzing Big Data with Microsoft R Server.

According the syllabus:

Upon completion, you will know how to use R for big-data problems.

Full Disclosure: I work at Microsoft, and the course instructor, Seth Mottaghinejad, is one of my colleagues.

Data Scientists, Data Engineers, Software Engineers: The Difference According to LinkedIn

The differences between Data Scientists, Data Engineers, and Software engineers can get a little confusing at times. Thus, here is a guest post provided by Jake Stein, CEO at Stitch formerly RJ Metrics, which aims to clear up some of that confusion based upon LinkedIn data.

As data grows, so does the expertise needed to manage it. The past few years have seen an increasing distinction between the key roles tasked with managing data: software engineers, data engineers, and data scientists.

More and more we’re seeing data engineers emerge as a subset within the software engineering discipline, but this is still a relatively new trend. Plenty of software engineers are still tasked with moving and managing data.

Our team has released two reports over the past year, one focused on understanding the data science role, one on data engineering. Both of these reports are based on self-reported LinkedIn data. In this post, I’ll lay out the distinctions between these roles and software engineers, but first, here’s a diagram to show you (in very broad strokes) what we saw in the skills breakdown between these three roles:

Data Roles and Skill Sets
A comparison of software engineers vs data engineers vs data scientists

Software Engineer

A software engineer builds applications and systems. Developers will be involved through all stages of this process from design, to writing code, to testing and review. They are creating the products that create the data. Software engineering is the oldest of these three roles, and has established methodologies and tool sets.

Work includes:

  • Frontend and backend development
  • Web apps
  • Mobile apps
  • Operating system development
  • Software design

Data Engineer

A data engineer builds systems that consolidate, store, and retrieve data from the various applications and systems created by software engineers. Data engineering emerged as a niche skill set within software engineering. 40% of all data engineers were previously working as a software engineer, making this the most common career path for data engineers by far.

Work includes:

  • Advanced data structures
  • Distributed computing
  • Concurrent programming
  • Knowledge of new & emerging tools: Hadoop, Spark, Kafka, Hive, etc.
  • Building ETL/data pipelines

Data Scientist

A data scientist builds analysis on top of data. This may come in the form of a one-off analysis for a team trying to better understand customer behavior, or a machine learning algorithm that is then implemented into the code base by software engineers and data engineers.

Work includes:

  • Data modeling
  • Machine learning
  • Algorithms
  • Business Intelligence dashboards

Evolving Data Teams

These roles are still evolving. The process of ETL is getting much easier overall as new tools (like Stitch) enter the market, making it easy for software developers to set up and maintain data pipelines. Larger companies are pulling data engineers off the software engineering team entirely in lieu of forming a centralized data team where infrastructure and analysis sit together. In some scenarios data scientists are responsible for both data consolidation and analysis.

At this point, there is no single dominant path. But we expect this rapid evolution to continue, after all, data certainly isn’t getting any smaller.

Know Your Probability Distributions

In data science and statistics, probability distributions can be very important. I have been meaning to create a listing of them. However, I no longer need to since the fine folks at Cloudera have already created a list at Common Probability Distributions: The Data Scientist’s Crib Sheet.

Learn the distributions and pick a favorite. (My favorite of the common ones is the normal distribution. I also like the Cauchy distribution which is much less common.)

Learning To Be A Data Scientist