Another Big Data startup launches.
Hilary talks about data, datapeople, and the current momentum. She brings up some current challenges.
Challenges with Data
- Robust analysis on streams of data (in volume)
- Store data so that it can be processed in Real-time
- Better Education – Good news for this blog
- Imagination – Stop solving the same problems
- What to do with the data
Last week, Heroku announced a new feature to its PostgreSQL database service. The new feature is called Data Clip, and it allows users to share results of an SQL query. It has options to store the exact data from when the query was originally run or the query can be refreshed to return the current data. I can definitely see this being useful for debugging of code and troubleshooting, which may have been Heroku’s original intent.
I can also see the Data Clip being very useful for data science and quick sharing of relevant data. I doubt the Data clip can handle huge result sets, but huge data is not always necessary. Sometimes, being able to quickly share data results is just as important. Plus the Data Clip allows the results to be downloaded into Excel, csv, json, or yaml formats. Therefore the data can be easily manipulated from there.
I love infographics because they are a great way to convey information about data. They go well with the thought that Data Scientists need to also be good story tellers. Well Visual.ly is startup that is aimed at helping people create, share, and discover infographics. Here is a quick example I created about my twitter account.
My Twitter Infographic
This is a nice post by Socketware. It provides a nice overview of a few machine learning algorithms.
- Recommendation Mining
- Document Clustering
- Document Classification
- Frequent Itemset Mining
Another Big Data startup launches.
In a matter of days, Stanford will begin the second round of the free online machine learning course. I enrolled in the course last fall, and it exceded all expectations. Professor Andrew Ng is great. The prerequisites are minimal, so don’t worry if your math is a little rusty. Also, the videos are short (around 8 – 12 minutes). Therefore, you don’t need large blocks of time set aside. Just watch a video or two during your lunch and you should be able to keep up. There are programming assignments (optional) and review questions to go along with the videos.
Don’t worry if you fall behind. The videos will still be there. The material you learn is more important than the pace. If you don’t know machine learning, the Stanford class is a great opportunity to get started.
Here is Professor Ng’s introduction to the class.
A data scientist is someone who can obtain, scrub, explore, model and interpret data, blending hacking, statistics, and machine learning.
This definition is short and simple, but there are many more definitions out there. In fact CITO Research, a site for CIOs and CTOs, set out to define what a data scientist is. They interviewed six leaders in the data science community, and posted all of the interviews online. The interviews produced varied results, but focused on some main themes of what a data scientist should know.
After reading Hilary’s definition, the CITO Research interview’s, a great post at Quora, and numerous other articles, I created a list of data science skills:
- Machine Learning
- Story Telling (Communication)
- Big Data
I am sure this list will change and evolve over time, but that is where I am going to focus for now. If you have anything to add to the list, please leave a comment. If you are interested in gaining some data science skills, please follow along and let’s learn together.
A great article on data science appeared in the New York Times over the weekend. It is worth your time to read it.