The Big Data Elephant in the Server Room - Let's Talk About It

Blog Post created by dowtr02 Employee on Nov 4, 2015


As Big Data continues to amplify the importance of data initiatives in big businesses – it also provides an opportunity for businesses to leverage data to make more informed decisions, gain a better understanding of customers' behaviors and needs, and tighten business operations.

But with that opportunity come the challenges associated with managing complex, heterogeneous Big Data environments. Success is bound with managing Big Data efficiently and economically. Inexpensive commodity servers running Open Source platforms have become the de facto standard for Big Data applications. Complexity can stand in the way of successful achievement. In a recent CA Technologies sponsored survey, two of the top five challenges in managing big data environments are because of complexity. 

This complexity is primarily due to the various Hadoop platform distributions, NoSQL data base technologies and range of open source tools that are used in Big Data application workflows. Data is constantly extracted from various source systems – both internal and external to the company – and processed (e.g. ETL) on Hadoop. Then data is either processed, or summary data is moved into various data stores (HBase, Hive or columnar databases) to then be queried for key insights – the product of your data.

Finding and isolating the root-cause of problems are increasingly more difficult given that the ever-growing application stack can make diagnostic information hard to interpret. Part of this problem is that enterprise-grade diagnostic tools are falling behind the complexity of your environment – a complex environment with a patchwork of vendors and partners. Subject Matter Experts are costly, and ones with the expertise to handle the entire data lake are becoming a rare commodity. It is difficult to hire Big Data talent. Often it is too expensive to take advantage of the value Big Data creates.

CA Technologies is taking a leadership role in transforming Big Data to address these challenges and simplify Big Data management. Get ahead of Big Data Management concerns, now. Join Sanjai Marimadaiah, VP of Product Management at CA Technologies, as he discusses how to simplify the Big Data chaos. Find out how to break down silos and gain maximum insight to help you make critical business decisions more quickly. Watch it here


[1] 2015 The State of Big Data Infrastructure: Benchmarking global Big Data users to drive future performance.