The Problem with Software Analytics

Software Analytics is the marriage of data science and software engineering.  It hopes to use data generated from software and software engineering processes to provide insights for creating better software.

The following is a quote from a software analytics round table discussion in 2013. All of the round table members are leading academics at prestigious universities.  Obviously, they were chosen because they are very accomplished and know the field.  Now, onto the quote.

Modern software services such as GitHub, BitBucket, Ohlol, Jira, FogBugz, and the like employ wide use of visualization and even bug effort estimation. We can pat ourselves on the backs even if those developers never read a single one of our papers.


Here is the source in IEEE Computer (which most likely you cannot access unless you are an academic): Roundtable: What’s Next in Software Analytics). For the non-academics an InfoQ reprint is available free online.

The academic research community cannot take credit for what Github, BitBucket, and others have done.  Yes, that academic research community is doing some excellent work, but most software practitioners are not seeing it because that research is being hidden in academic journals. The advancements might have occurred simultaneously and coincidentally, but there is not a clear causal relationship.  Unfortunately, the academic research is not getting into the hands of the software practitioners.

I would like to think the target audience of software engineering research would be software engineers, project managers, and developers. However, as this quote points out, those practitioners hardly ever see the research. If the research does not reach the intended audience, then there is a clear problem.  A problem that needs to be fixed.

Unfortunately, I do not yet know what the fix is. If you have any ideas, please leave a comment below.

If there is enough interest, maybe I will start something (just don’t know what that something is).

Spark Summit 2015 Livestream

Apache Spark is currently one of the hottest technologies in data science. That trend leads the Spark Summit 2015 to be one of the top conferences. Luckily, the conference organizers where kind enough to set up a free Spark Summit 2015 Livestream of the event. Here is a small glimpse of what will be covered:

  • Updates from Matei Zaharia, creator of Spark
  • Spark at NASA
  • Innovation with Spark
  • Three tracks of talks:
    1. Developer
    2. Data Science
    3. Applications
  • And much, much more

The livestream begins today, June 15, 2015, and continues through Wednesday. It appears keynotes and all three tracks will be available via livestream. If you cannot physically make the event, then this is probably the next best thing.