2015 was a tumultuous year for Big Data, with highs continuing to dazzle us with the potential world-changing power of data and analytics.
At the same time there were plenty of lows serving as ongoing warnings that much is still unknown about exactly how it will end up changing the world.
So here’s a quick review of the year, highlighting what I think were the most important or newsworthy stories in the world of Big Data and analytics over the last 12 months.
Source for picture: click here
January
2015 was the first year that started with more of us using our phones or mobile devices to connect to the Internet than computers or laptops. The number of mobile Internet users worldwide overtook the number of deskbound ones late in 2014, and this change in user behavior has huge implications for how app developers, corporations and even governments will collect data about us.
The year also started with the news that Microsoft was purchasing Revolution Analytics, developers of R. The open source programming language said to be the most popular in the world for statistical computing and predictive analytics.
Large scale data breaches – which have increased in size and frequency exponentially over the last decade – show no signs of abating. The US Government starts the year by announcing that details of four million of its employees have been stolen from its Office of Personnel Management.
February
A White House report into the opportunities and dangers created by Big Data concludes that although today’s technology offers unprecedented potential for growth and change, there are serious concerns over privacy and data protection that will have to be addressed with legislation.
Of paramount importance should be the ongoing progression of the Consumer Privacy Bill Of Rights – known informally as “Do Not Track” legislation, the report recommends. This means giving the legal right to a person to refuse another person, or organization, permission to track and record their personal data.
March
IBM announced that it is to invest $3 billion in Internet of Things technologies over the coming four years. The concept that all devices – not just computers – can talk and share information online, and interact with each other to make our lives easier, continues to be big business throughout 2015 and IBM say that their efforts will be focused on breaking new ground in real time analytics.
April
The long-awaited Apple Watch finally made it into stores this month, and was hotly tipped to become the world’s first “must have” Internet of Things gadget. Since the launch Apple has remained fairly tight lipped over how many it has sold, although it is known that one million orders were taken within six hours of being made available. We know that the iPad took a month to sell that many, and the iPhone took two months.
Several big fish of the Big Data pond unveiled new or expanded services. These included Amazon Web Services (AWS) which announced its customers would soon be able to take advantage of machine learning through Amazon Machine Learning, going up against Microsoft’s Azure and IBM Watson, as well as Google, which itself announced major updates to its Big Query technology.
May
Computing Research publishes its 2015 review of the Big Data market which finds that the number of businesses which have “no plans” to integrate Big Data and analytics in their operations has halved –from 33% to 16% in the past year. Of 400 “decision makers” across sectors including government, retail, financial services, technology and educational sectors, 76 per cent agreed that their organizational analytics were focussed on internal, operational data rather than external data.
June
German Chancellor Angela Merkel warned her citizens they would have to put aside their traditional fear of widespread public data collection, or risk the country being left behind in the global Big Data gold rush.
“Whoever sees data as a threat, whoever thinks about every piece of data in terms of what bad can be done with it, will not be able to take advantage of the opportunity of digitization” she said at a Berlin conference.
Reuters reported that her comments would be interpreted as being aimed at the “Mittelstand” group of family owned, traditionally managed German businesses.
July
Large scale data breaches continued to make headlines after 2014’s record breaking year for corporate data loss and theft. If not the largest, then certainly the one which filled the most column inches, took place in July, when dating site Ashley Maddison was attacked and pillaged. What made this hack so appealing to the press was the nature of the site, which heavily focused its marketing on the potential it offered for infidelity. The potential salaciousness of the stolen data made the story big news, but customers of mobile retailer Carphone Warehouse, crowdfunding site Patreon, telecoms operator T-Mobile, healthcare provider Anthem and credit reference agency Experian fell victim in their millions this year also.
August
Chinese retail and internet services giant Alibaba unveils Aliyun, its own cloud analytics engine designed to allow businesses to carry out machine learning-driven analysis on its own hardware. The move is in line with tech giants in the west which have all launched their own Big Data-As-A-Service infrastructure. Alibaba calls Aliyun “China’s first artificial intelligence platform” and says it can process 100 petabytes of data – the equivalent of 100 million HD movies – in six hours.
September
European privacy activists celebrated a victory this month when the EU Court of Justice declared the so called “safe harbor” agreement invalid. In a case brought against Facebook by Austrian privacy campaigner Max Schrem, it was claimed that, in light of Edward Snowden’s revelations, no guarantees could be given by US companies that they would be able to protect the privacy of EU citizen’s data. The case will have far reaching and legally binding consequences on any business which relies on transmitting data between the US and Europe.
October
The biggest tech deal in history took place this month, with hardware manufacturer Dell announcing that it would acquire data storage, cloud computing and analytics provider EMC along with its subsidiary, virtualization specialists VMWare, for $67 billion.
November
The UK’s Financial Conduct Authority (FCA) announced that it will investigate the use of Big Data by insurance companies. The agency said it will look into the growing use of large scale collection and analysis of customer data, particularly focusing on its growing use as an indicator for setting premiums. Issues which will be addressed include whether or not an increasingly automated and data-driven approach to setting premiums could lead to insurers adopting discriminatory practices.
December
Important steps towards codifying rules on how governments and corporations treat our personal data were taken in Europe. The General Data Protection Regulation (GDPR) was agreed, in draft form, following three years of discussion and research. The regulations will put stricter rules on what can be done with personal data and will apply to any business which operates in the EU or collects data on EU citizens, regardless of where they are based in the world. Crucially, the draft, which is expected to become law in 2017, states that companies can only collect personal data with the expressed permission of the people who it belongs to, and can only use it for the specific reason that permission has been gained for. Methods of “encouraging” users to give permission such as pre-ticking boxes on application forms or stating that taking a certain action implies consent will be outlawed, and firms which are in breach face fines of up to 4% of their global revenue.
DSC Resources
- Career: Training | Books | Cheat Sheet | Apprenticeship | Certification | Salary Surveys | Jobs
- Knowledge: Research | Competitions | Webinars | Our Book | Members Only | Search DSC
- Buzz: Business News | Announcements | Events | RSS Feeds
- Misc: Top Links | Code Snippets | External Resources | Best Blogs | Subscribe | For Bloggers
Additional Reading
- The 10 Best Books to Read Now on IoT
- 50 Articles about Hadoop and Related Topics
- 10 Modern Statistical Concepts Discovered by Data Scientists
- Top data science keywords on DSC
- 4 easy steps to becoming a data scientist
- 13 New Trends in Big Data and Data Science
- 22 tips for better data science
- Data Science Compared to 16 Analytic Disciplines
- How to detect spurious correlations, and how to find the real ones
- 17 short tutorials all data scientists should read (and practice)
- 10 types of data scientists
- 66 job interview questions for data scientists
- High versus low-level data science
Follow us on Twitter: @DataScienceCtrl | @AnalyticBridge
