Big Data is everywhere. Predictive analytics and real time in-memory computing isn't everywhere.

This truth (if we can accept it to be so) represents something of an imbalance.

As a subset of data mining, predictive analytics driven by in-memory computing efficiencies now has an opportunity to bring real-time analysis and insight to fast-moving live transactional data flows. Or to put it another (rather shorter) way, we can now start to manage and understand Big Data better than ever.

Application use cases here might typically include:

• Meteorology
• Genetics
• Economics
• Climate simulation
• Oil exploration
• Financial analysis and scientific research
• Telecommunications, to name but seven good examples

If we combine contemporary approaches to predictive analytics with the newly arrived Intel Xeon Phi coprocessor that produces what is claimed to be over one teraflop per second in terms of workload computational power for highly-parallel workloads, then CIOs can start to think about what "50-processor core computing" will mean for us in the very near future.

Pushing forward in this space is SAP with its HANA in-memory computing appliance and platform. The firm is openly partnering with HP, Oracle, Cognizant and variety of other big (and smaller specialist) players to form strategic alliances that will help further the uptake of this kind of technology.

Why Is Analytics Working Better?
Part of the reason that in-memory predictive analytics intelligence is now becoming so important and much easier to bring to bear is hardware related and part of it is a software-related development.

On the hardware end, Intel has worked to sharpen data throughput between memory and processor cores. This means that work that goes on right in the heart of the machine that might have traveled at around five Gigabytes per second (Gbps) five years ago, now has a chance to move at 100 Gbps. On the software end, firms like HP and SAP have been working to produce what are typically referred to as "business process solutions" that can produce "context-aware experiences" to enable one sense-and-respond scenarios and, therefore, faster and more personalized interactions with customers.

As a matter of interest, HP also works to provide datacenter services to SAP that support its enterprise-wide hosting solutions for e-business applications - but that's another story.

In terms of actual application solutions running on SAP HANA, new products include the SAP Liquidity Risk Management application, the SAP Accelerated Trade Promotion Planning application and SAP Operational Process Intelligence software.

"These innovations show how SAP is rapidly delivering real-time, data-centric and industry-specific applications on the SAP HANA platform," said Dr Vishal Sikka, member of the SAP executive board for technology and innovation.

To take one example, SAP Liquidity Risk Management aims to provide banks with the ability to perform real-time, high-speed liquidity risk management and reporting on very large volumes of cash flows. In future then, banks will be able to instantly measure key liquidity risk ratios (such as the Basel III liquidity coverage ratio) and cash flow gaps to resolve potential liquidity bottlenecks. The application aims to allow banks to apply different stress scenarios, such as adjusted run-off rates and bond haircuts, to gain a deeper understanding of how market volatility can impact liquidity positions.

Lessons for CIOs
Now you don't have to be a bank CIO or financial analyst to understand the wider importance of this technology, i.e., this is the "harnessing of Big Data" catchline that you've already heard bandied about by countless IT firm's press departments, except now it's really happening.

The lesson for CIOs and the software application developers serving them is that we now have a route to predictive real-time analysis and the power to view billions of stored records and live transactional data at the same time. CIOs should look to their solution architects and business process experts to compose the analytical data models that will drive the next phase of their technology growth.

SAP is not the only firm driving this space, but its work to publicize its competencies in this arena is very prevalent. The fact that SAP pushes many of the interfaces for managing the result of its data analysis to both Apple iPad (and now Windows 8 format) mobile devices may have confounded Steve Jobs at the time, but this is the firm's proof point for showing off its Big Data number crunching applications. One day, quite soon, none of this will be a surprise.

This post first appeared on Enterprise CIO Forum.