@CloudExpo Authors: Liz McMillan, Zakia Bouachraoui, Dana Gardner, Yeshim Deniz, Elizabeth White

Blog Feed Post

If You Store My Tweets Today, I’ll Gladly Look at Them Next Tuesday

Cloud Event Processing – Where’s The Data?

There are several things left to cover in our #TwitYourl proejct.  One of the most glaring absences so far is storage – where do we put all of these Tweets?  What happens if we’d like to make changes to our RuleBots and test them with historical data?  Right now, we can’t.

In keeping with some of the rules of our implementation, an OnRamp is not allowed to update a database directly.  This would tie the OnRamp to the database implementation and make our architecture more brittle than it has to be.  Also, it would mean that the OnRamp had to have specific information about where it was sending a message – this is not allowed.

Remember, we are architecting for the cloud – the idea of baked in, point to point, connections inside of our cloud should have you recoiling in horror by now.  So how can we update databases easily within our Cloud Event Processing framework?

Well, it would be really easy if we had some metadata that described all of the data, or events, that the OnRamps were publishing on the bus.  And it would be even neater if all of that metadata was available in a central database or directory.  This way, when the infrastructure that #TwitYourl is built on top of is running, all one would need to do is go to some management console, bring up the information about how we’d like to handle a particular event, click the ‘Store in a Database” button and be done with it.  Click, click, click and we’re storing events – automagically; somewhere in the cloud!

So, let’s do this for #TwitYourl – I vote for MongoDB.  I know that Cassandra seems to be a fairly popular choice this past week, but we’re going to buck that trend and hook up MondoDB inside of our cloud and we’re going to feed Mongo some yummy Tweets!

Sorry, I couldn’t resist.

But before feeding Mongo, we’re going to work on some visualization this week.  We’re running a little behind schedule here and we need to get caught up!

Read the original blog entry...

More Stories By Colin Clark

Colin Clark is the CTO for Cloud Event Processing, Inc. and is widely regarded as a thought leader and pioneer in both Complex Event Processing and its application within Capital Markets.

Follow Colin on Twitter at http:\\twitter.com\EventCloudPro to learn more about cloud based event processing using map/reduce, complex event processing, and event driven pattern matching agents. You can also send topic suggestions or questions to [email protected]

CloudEXPO Stories
"There is a huge interest in Kubernetes. People are now starting to use Kubernetes and implement it," stated Sebastian Scheele, co-founder of Loodse, in this SYS-CON.tv interview at DevOps at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Is advanced scheduling in Kubernetes achievable?Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, answered these questions and demonstrated techniques for implementing advanced scheduling. For example, using spot instances and cost-effective resources on AWS, coupled with the ability to deliver a minimum set of functionalities that cover the majority of needs – without configuration complexity.
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes.
Here to help unpack insights into the new era of using containers to gain ease with multi-cloud deployments are our panelists: Matt Baldwin, Founder and CEO at StackPointCloud, based in Seattle; Nic Jackson, Developer Advocate at HashiCorp, based in San Francisco, and Reynold Harbin, Director of Product Marketing at DigitalOcean, based in New York. The discussion is moderated by Dana Gardner, principal analyst at Interarbor Solutions.
Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures. Offering private, hybrid, and public cloud solutions, Atmosera works closely with customers to engineer, deploy, and operate cloud architectures with advanced services that deliver strategic business outcomes. Atmosera's expertise simplifies the process of cloud transformation and our 20+ years of experience managing complex IT environments provides our customers with the confidence and trust that they are being taken care of.