The era of big data analysis is here to stay. Take your pick of 2014 proof points.
Tech watchers might cite the more than $200 million in venture capital raised by the top three NoSQL database vendors, or the $1 billion raised by the top-three Hadoop software distributors. Many took note of the recent declaration by Forrester Research that "Hadoop is no longer optional" for large enterprises, thanks to compelling "Hadooponomics" that make it a must for high-scale storage and data processing.
InformationWeek is more impressed by the testimonials of companies that are getting real value out of big data platforms and analysis techniques. Pfizer and Merck, for example, are developing more effective and affordable drugs thanks to big data techniques that are leading to more targeted treatments and more productive manufacturing processes. GE and others are demonstrating improvements in industrial equipment performance, uptime, and safety thanks to Internet of things-style applications.
And then there are the pioneers like The Weather Company and Facebook that say they just couldn't run their data-driven businesses without new platforms, even if they still have a place for more conventional tools like relational databases.
Here are five trends witnessed over the last year that point to progress in big data analysis:
1. SQL meets Hadoop
Hadoop is here to stay, so every data management vendor worth its salt must have a SQL-on-Hadoop or SQL-access-to-Hadoop option. Here are five of our most-read stories in the SQL-meets-Hadoop vein:
- Oracle Big Data SQL: 5 Key Points
- Cloudera Boosts Hadoop App Development On Impala
- Hortonworks Updates Hadoop Data Platform
- HP Taps Vertica For SQL On Hadoop
- MapR Ships Drill For SQL Analysis Of Big Data
Just remember that SQL is not designed to find correlations among variably structured data sets. Nor does it support machine learning, many advanced analytics techniques, or other approaches often associated with big data analysis. If SQL solved everything, we wouldn't need new platforms.
Read the full story on InformationWeek.
Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio