Best known as the team behind Cascading, the technology for Big Data applications averaging more than 130,000 downloads per month, Concurrent, Inc. is looking to address the application development and performance management pain points on Apache Hadoop. Called Driven, the new offering is a free cloud service that will provide ‘unprecedented visibility and control’ to Big Data applications developed, managed and deployed on Hadoop.
“Everybody says they’re Big Data but they’re not Big Data,” said Concurrent CEO Gary Nakamura.
“Our product will be the first to give you data insight running on top of Hadoop or any other fabric you’re running in.”
APM lowers the bar, and Driven lowers the bar even further, he said. Available as a public beta, Driven will be commercially available when it goes GA in Q2.
The timing would seem appropriate, according to a new survey of mainly application developers from Jaspersoft. It found that while 42% are still in tire-kicking mode, the number of companies that have a funded Big Data initiative has more than doubled (36%) from a similar survey 14 months ago (15%). Of the 56% of respondents with Big Data projects, 32% are in production or in development while 23% are in the planning stage. Of known deployments, 66% were on-premises with 34% were in the cloud.
“What we’re seeing from our community is a better understanding of Big Data and more willingness to commit to projects,” said Jaspersoft CEO Brian Gentile. “The survey suggests that experimental Big Data projects are on the rise while funded initiatives continue to increase as the tools and understanding of Big Data mature.”
Released in December, Gartner’s Magic Quadrant for Application Performance Monitoring reported that APM, which has been an implicit part of IT operations since the mid-1990s, and started seeing increasing focus since 2005, is now seeing shifts based on execution of differentiations and delivery models. It predicted the 2013 market would be worth $2.12 billion, up 5.4% from a year ago, but a slight drop from the 5.8% growth experienced between 2011 and 2012. The key players include AppDynamics, Riverbed Technology, Compuware, New Relic, Microsoft, AppNet, CA Technologies, HP, BMC Software, OpTier, Dell, IBM, and ManageEngine.
Although Concurrent may be dismissive of other vendors, Riverbed has called 2014 the year of Big Data APM. “Only big data APM provides the breadth and depth of application performance analysis to ensure that your increasingly complex application environment becomes and remains well tuned,” noted Tom Fisher in a blog in late December.
Another APM vendor, Compuware, just announced its APM for Big Data has been named a Trend-Setting Product in Data for 2014. “The data-driven demands on organizations have never been greater, and as a result, we’ve seen an explosion in data management interest and products,” said Thomas Hogan, Group Publisher of DBTA.
In his predictions for 2014 Nakamura stated every company is in the business of data, and this will drive the demand for cost effective and scalable Big Data platforms higher than ever before. “As the market starts to catch up to the Big Data hype, 2014 will prove that leveraging the right tools can make or break a project.”
He said more Hadoop projects will be swept under the rug as businesses devote major resources to their Big Data projects before doing their due diligence, which results in a costly, disillusioning project failure. However he cautioned that confusion will also be a big part of this movement and could mean the difference between the winners and losers. “Those businesses that can separate the wheat from the chaff, and execute upon this valuable knowledge, will be the ones to succeed.”
In an earlier briefing with IT Trends & Analysis Nakamura called Hadoop a total mess. “It’s whack-a-mole as far as solving problems.”
As Jaspersoft, Gartner and others show, despite the mess, real-time operational intelligence systems are moving from ‘nice to have’ to ‘must have for survival’. Gartner predicts that analytics will reach 50% of potential users by 2014, and by 2020, that figure will be 75%. While Hadoop may not be the only way to address Big Data, it is growing strongly, a compound annual growth rate of 55.63% between 2012-2016, with Hadoop-as-a-Service growing even faster, with a CAGR of 95.16% during this period. The Hadoop market is expected to reach $13.9 billion by 2017, with North America, which accounted for 53.85% of the overall market in 2012, leading the way.
Talking about Driven, Nakamura said the Hadoop and Big Data combination is runaway train. “They jump in with both feet without looking and once they discover the pool is empty at the bottom… the risk in the broader market is that if tools come don’t out fast enough the users will get disenchanted with Hadoop”.
Under The Hood
Called the industry’s first application performance management product for Big Data applications, Driven is a free cloud service and is an integral part of the Cascading community that enables Apache Hadoop developers, data analysts, data scientists and operations to see key application metrics in real-time, so that they can isolate and resolve problems immediately.
Driven details include:
-APM for Big Data
-significantly improves developer productivity and operations control by providing an
unprecedented level of insight into building and managing enterprise-grade data applications;
-facilitates and encourages user collaboration to build enterprise data applications;
-integrated with the Cascading open source community;
-free for development (cloud only) and licensable for production or on-premise deployments;
-deploy in the cloud or on-premise;
-February public beta (cloud service) free for development only; and,
-Q2 2014 Driven 1.0 General Availability