Welcome!

@CloudExpo Authors: Elizabeth White, Pat Romanski, Yeshim Deniz, Zakia Bouachraoui, Liz McMillan

Related Topics: @CloudExpo, Microservices Expo, Containers Expo Blog

@CloudExpo: Article

The Growing Relevance of In-Memory Data Grids

Cloud is bringing IMDG technology back to the spotlight

The growing consumer affinity to cloud is spurring on various new technological trends. It's not all new technology mind you, but there seems to be a growing appetite for anything that can remotely be put into the context of cloud computing. In some ways, cloud has been good for bringing previously existing technologies back to the forefront and resulting in needed innovation in the area. Besides virtualization technologies, I believe that one of the best examples of this is in-memory data grid technology.

Despite what some may try to purport, in-memory data grids are not a by-product of the cloud computing revolution. The truth is they existed for a while before cloud, but to be fair, IMDGs probably owe a tip of the hat to the cloud computing craze for bringing them back into the spotlight. Increasingly, we are seeing workloads that are highly scalable, temporal, and elastic making their way into the cloud. These application characteristics often align nicely with the use of IMDGs, so we see renewed interest and quite a bit of innovative activity around these solutions.

I have been spending a lot of time lately working with users that are in the process of defining an evolution to their current enterprise middleware environments. In this work, cloud and IMDG technology has been front and center. My last few posts have been dedicated to talking about some of the cloud trends I have seen during the course of this work, but today I want to focus on IMDG solutions. Specifically, I want to share with you what I'm seeing in terms of how users are currently looking to use this technology, and provide my own thoughts about possible usage scenarios going forward. Let's start with the common, currently targeted usage scenarios:

1) IMDG as a database buffer: This is perhaps the single most common use case. Here users look to front traditional data stores with a distributed IMDG. This can serve to increase the performance of the application and thereby improve end-user experience by offering faster data access. It can also help to reduce the pressure on the backend database by making the IMDG instance the system of record and periodically batching changes to send off to the database. There are many different techniques for buffering an existing database with an IMDG, but the motivations (increase performance, decrease database reliance, decrease costs) are usually much the same.

2) IMDG as a simple cache: You may hear this referred to as the side cache scenario as well. This usage pattern is a little less intrusive to existing application architecture than #1 above. Here, applications receive an incoming request that requires data, and they first check in the simple cache instance to see if the data exists there. If not, the application proceeds to retrieve the data normally, typically from a relational database system, and then inserts this data into the simple cache. Obviously, when the application finds data in the simple cache, you reduce the path of the application and thus decrease overall response time. This is an especially prevalent pattern for storing conversational state (think HTTP sessions) for applications.

3) IMDG as a service request cache: This is really a variant of #2 above, but I call it out separately because users typically implement it at a different tier in the application architecture. Instead of updating the application to be aware of some IMDG for a simple cache, users insert this awareness further up the stream, often at the ESB tier. Requests come into the application environment, but before they hit the application, some component dissects the request to determine if it can fulfill it from the cache. If so, the entire path becomes significantly shorter, and if not the mediation component inserts the response in the simple cache on the way back out. Again, it is not very different than #2, but in my opinion it is worth distinguishing as it occurs at a completely different tier in the overall architecture.

Those are three common patterns that I run into quite frequently today with IMDG technology. There are a couple of more usage patterns that I have seen once or twice, but are not yet prevalent. ‘Yet' is the key word here as I tend to think we will be seeing more of these use cases in the near future:

1) IMDG as an event filter: To be clear here, I am not suggesting that IMDG instances would morph into event processors. There are completely separate solutions that do that very well. What I am talking about is using the distributed logic processing capabilities shipped with most IMDG solutions to quickly determine if a given event needs further processing. In this way, events that occur in the enterprise can flow through an easily scalable IMDG filter, and then only if necessary sent to an event processor for more expensive computation. In the increasingly event-driven architectures that are emerging today, I feel this could become a popular pattern.

2) IMDG as a map/reduce engine: Many IMDG solutions deliver the capability to distribute logic, as mentioned in #1 above, and many offer a map/reduce model as a means to do this. As skills on map/reduce programming start to permeate enterprise IT shops, I think users will see compelling use cases for IMDG built primarily on this methodology. The ability to quickly distribute logic, calculate results, and further refine those results all out in the grid is a powerful tool to have in your arsenal. It can in fact be quite liberating to leverage the processing power of a distributed grid to solve important yet complex business problems.

Now, I have no idea if the two use cases above will in fact pan out as mainstream in the near future. I see massive potential there because of their alignment with emerging architectures and their ability to deliver real business value.  However, I can also imagine IMDG technology taking off in entirely different directions. Whatever the near future holds though, I think one thing is certain: We are just beginning to explore the art of the possible when it comes to IMDG technology.

More Stories By Dustin Amrhein

Dustin Amrhein joined IBM as a member of the development team for WebSphere Application Server. While in that position, he worked on the development of Web services infrastructure and Web services programming models. In his current role, Dustin is a technical specialist for cloud, mobile, and data grid technology in IBM's WebSphere portfolio. He blogs at http://dustinamrhein.ulitzer.com. You can follow him on Twitter at http://twitter.com/damrhein.

CloudEXPO Stories
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In their Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, and Mark Lavi, a Nutanix DevOps Solution Architect, explored the ways that Nutanix technologies empower teams to react faster than ever before and connect teams in ways that were either too complex or simply impossible with traditional infrastructures.
According to the IDC InfoBrief, Sponsored by Nutanix, “Surviving and Thriving in a Multi-cloud World,” multicloud deployments are now the norm for enterprise organizations – less than 30% of customers report using single cloud environments. Most customers leverage different cloud platforms across multiple service providers. The interoperability of data and applications between these varied cloud environments is growing in importance and yet access to hybrid cloud capabilities where a single application runs across clouds remains elusive to most organizations. As companies eagerly seek out ways to make the multi cloud environment a reality, these new updates from Nutanix provide additional capabilities to streamline the implementation of their cloud services deployments.
@CloudEXPO and @ExpoDX, two of the most influential technology events in the world, have hosted hundreds of sponsors and exhibitors since our launch 10 years ago. @CloudEXPO and @ExpoDX New York and Silicon Valley provide a full year of face-to-face marketing opportunities for your company. Each sponsorship and exhibit package comes with pre and post-show marketing programs. By sponsoring and exhibiting in New York and Silicon Valley, you reach a full complement of decision makers and buyers in multiple vertical markets. Our delegate profiles can be located in our show prospectus.
In today's always-on world, customer expectations have changed. Competitive differentiation is delivered through rapid software innovations, the ability to respond to issues quickly and by releasing high-quality code with minimal interruptions. DevOps isn't some far off goal; it's methodologies and practices are a response to this demand. The demand to go faster. The demand for more uptime. The demand to innovate. In this keynote, we will cover the Nutanix Developer Stack. Built from the foundation of software-defined infrastructure, Nutanix has rapidly expanded into full application lifecycle management across any infrastructure or cloud .Join us as we delve into how the Nutanix Developer Stack makes it easy to build hybrid cloud applications by weaving DBaaS, micro segmentation, event driven lifecycle operations, and both financial and cloud governance together into a single unified st...
"NetApp's vision is how we help organizations manage data - delivering the right data in the right place, in the right time, to the people who need it, and doing it agnostic to what the platform is," explained Josh Atwell, Developer Advocate for NetApp, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.