Google has recently released a Jupyter Notebook platform called Google Colaboratory. You can run Python code in a browser, share results, and save your code for later. It currently does not support R code.
So, you have identified a fascinating new problem to solve with data. You correctly started with a problem and not the data. It seems both beneficial and interesting. Now where do you get the data? Here are 4 steps (in order) for how to find data.
1. Existing Data
The best place to start is the data you currently have. What data does your organization currently collect? How can you get access to that? Start there.
Then look for industry specific open data (data that is freely available). Many industries publish data monthly or yearly. Also, data is frequently available with government funded research. If industry specific data is not available, what other related data is openly available? It is often beneficial to augment your existing data with open data. Here are some lists of open data, Open Data, Part 1, Open Data, Part 2. There are also many others available.
Next, explore the opportunity of using an API to access data. Many application have existing API access. An API (Application Programming Interface) allows a person to write some computer code to pull machine-readable data from an existing system. Some are freely available, others have associated costs. Many allow the data to be available in near real-time. There are numerous API’s available where you can pull in data. Check with some of your existing applications. They are available for weather, stocks, news, social media, web analytics, and many more.
4. Create The Data
The last resort is to begin the creation of data. An obvious choice is to create a survey. Be careful because surveys can be trickier than initially thought. You often do not get good representation and the result is biased data. Another way to collect data is to change your application to begin collecting the desired information. You may even have to build a new application. Sometimes an entire process needs to be created or modified to include methods to collect the data. This last step usually takes the longest and costs the most money.
Awhile ago, I recorded a Facebook Live on Tips for Data Science Students. It goes along with the following post: Getting the most from your Data Science Masters Program.
Stick around for the Question and Answers at the end of the presentation where I talk about starting your own blog or entering Kaggle contests.
All of the other talks were great as well. You can register here Demystify Data Science Conference 2017 by Metis to get free access to all the other video presentations.
I have recently been exploring with creating YouTube Live Videos. Go to the Data Science 101 Facebook page to know more.
Enrolling in a master’s degree program in data science or business analytics is no small feat. It takes a lot of time, determination, and money. It can all be worth it as a more fulfilling and higher paying job might be in your future. However, just earning the degree does not guarantee a job in the future. Here are a few tips to maximize your master’s degree experience and enhance your chances of landing that great job.
Create a Project
This one is big because it helps with all the other tips. Pick a project that is unique to you. It should be interesting and fun. There are tons of open datasets available. The project can be any topic from something big like world education to something smaller like your own coffee consumption (for some of you that might not be small). All that matters is that it involves some data and you work on it. The project will help you learn new things and determine what is enjoyable. It will even give you a good discussion topic for future job interviews.
Determine the portion of data science you enjoy
Is it visualization, programming, modeling or something else (see Getting Started with Data Science Specialties for a list of specialties)? Then tailor as much of your program around that as you can. You will excel more at things you enjoy, and data science needs teams not individuals who think they can do everything.
Attend local meetups or conferences
Depending upon where you attend school, this might be easy or difficult. If your local area does not have a data science group, start one.
If you are ever offered the chance to speak to a group, take it. Whether it is a class, local club, church group, or a backyard barbecue; take advantage of the opportunity. Many people are not good at this skill, and practice will only make you better. Also, university settings are great places to practice. They are safe environments and the worst that is going to happen is a not perfect grade. Don’t wait until the stakes are high to begin your practice.
Make yourself visible to the data science world.
Share the slides from your presentations. Better yet, share the video if available. Make sure when a prospective employer searches for you online (and they will), they can easily see a trail of artifacts that demonstrate your interest in data science. You should probably have a presence on some of the following (you do not need them all): LinkedIn, Twitter, Instagram, Quora, Stack Overflow, GitHub, Youtube, Slideshare, Speakerdeck.
Find some local data science people in your area and connect. Offer to join them for coffee or lunch. Attend their presentations and get to know them. This can be others learning data science as well as more seasoned experts.
What others tips do you have for those currently enrolled in a data science masters degree program?
Andrew Ng, co-founder of Coursera and Deep Learning Expert, is launching a new specialization on Coursera. Details can be found at DeepLearning.ai or the Deep Learning Specialization Page. The specialization consists of 5 courses. They are free to audit and watch the videos. There is a fee to get graded assignments and receive a certificate of completion. The first course just started this week, so it is great time to start learning some deep learning.
Renowned data scientist, Kirk Borne will take viewers on a journey through his career in science and technology explaining how the industry-and himself have evolved over the last 4 decades. Starting with skipping lunches in high school to a systematic twitter obsession, Kirk will shed light on his road to success in the data science industry.
Kirk is universally considered one of the most (if not the most) influential voices in data science. If you are interested in a career in data science, this is a webinar you will not want to miss.
The webinar is 5:30 Eastern Time on August 29, 2017, and registrations are currently being accepted. It is free.
Businesses everywhere are racing to extract meaningful insight from their data. Many organizations are spinning up data science teams and attacking problems (some more successful than others). However, one of the challenges is determining the current stage of data science within the organization. Next is determining the desired stage of data science.
Below are 3 stages of a truly mature data science organization.
The beginning stage of data science is dashboards. It is all about answering “How much?” and “What happened?” by looking at reports of historical data. If done well, it might even help an organization answer “Why”. Many organizations will refer to this phase as Business Intelligence.
The dashboard stage can be very expensive for an organization, in terms of people-hours and money. It usually involves investments in:
- Data Warehouse or some other storage environment, for storing the data in a single location for easy reporting
- ETL (Extract Transform Load) Tools for manipulating, combining, and moving data to the data warehouse
- Reporting Tools for displaying the results and allowing users to “explore” the data
Here are some common questions that can be answered via traditional dashboards:
- How many customers live in each region?
- What were the sales on Black Friday?
- How many patients visited the hospital last month?
As you can see, there are large amounts of value that can be gained by this phase alone. It is critical for a business to clearly understand past performance. Unfortunately, this phase is where many businesses stop.
2. Machine Learning
The real “science” of data science does not begin until the second stage which is machine learning. It focuses on estimating quantities that cannot be directly observed. This could be what movies a customer will like, the price of a company’s stock tomorrow, or the causal effect of a particular advertising campaign. Machine Learning uses the data from the first phase and applies statistical or other methods to gain additional insights.
Think of machine learning as answering the following:
- When a customer moves, will he/she spend money at a hardware store?
- When a credit card purchase is made, what is the probability the charge was fraudulent?
- What is the expected lifetime value of a new customer?
- If a hurricane is coming, what will people buy? (pop tarts? it is true).
Notice the connection between an event and some outcome. The value of machine learning comes from estimating the causal outcome of potential events. This phase is filled with terms such as: machine learning, data mining, and statistical modeling. The machine learning stage is all about looking into the future!
Determining the actions to perform, is the third and final phase. It tries to capitalize on the results of machine learning in order to take appropriate actions. The following actions might be suitable for the events identified in the predictive section above.
- When a customer moves, send a “welcome to the neighborhood” packet with coupons to nearby hardware stores.
- Decline the fraudulent charge or deactivate the credit card.
- If the new customer has very high expected lifetime value, provide some special treatment or offers to ensure the customer becomes a customer for life.
- When a hurricane is approaching, place Pop tarts near the front of the store.
As you can see, good machine learning from the second phase can lead to clear actions.
Claiming success in Data Science is all about conquering all three stages. Each stage builds upon the previous stage. If you have put in the effort to complete the first stage, why not continue to the second and third stages?
The National Basketball Association is hosting an Analytics Hackathon.
Application are accepted until August 6, 2017 and the actual competition occurs on September 23-24, 2017. The competition has 2 focus areas:
- Basketball Analytics
- Business Analytics
The prizes for winning consist of:
- Lunch with NBA Commissioner
- A trip to the All-Star Game
- Tickets to a game of your choosing
To be eligible, you must be:
- At least 18 years old
- An undergraduate or graduate student