|By Skytap Blog||
|January 25, 2015 12:45 PM EST||
Guest Post: Beyond Big Data and Benioff’s “AI Spring” to the Dawn of Dataware
Big Data, AI, Machine Learning, Hadoop, Predictive Analytics — we hear these terms every day from companies such as Cloudera, Trifacta and Dato (formerly GraphLab) that are securing many millions in financing. I believe that 2015 will be the year when the conversation moves from Big Data to the Dataware stack. Over the past twelve months we have seen a lot of companies across the big data spectrum emerge and while the language can be the same, there are clear product categories that have emerged which describe the market opportunity and future growth.
This is the Dataware stack. Dataware is the combination of infrastructure, data intelligence systems that apply algorithms and machine learning to the data, and the applications enabled by data intelligence that are changing how we do business and how we live our lives every day. And startups dominate the Dataware landscape.
We are at the very start of the data revolution. The consumer world got there first. Apps that know who we are and where we are and some other data points about us help us do a myriad of things every single day. On the business side, there has always been a lot of data but now there is not only an incredible growth in that data, there is an active appreciation beyond business analysts for using data in near real time to improve products and services.
What we are seeing now is a huge shift that is infusing data into every piece of our lives and making every app and service smarter. We have started to see this blending of data with what were rather static services with recent announcements from companies such as Salesforce.com and Workday.
The Dataware Framework is how I look at this new world of software in the age of big data. Dataware includes an Agile Data Stack of components for modern data applications and services and a Continuous Data Loop that brings usable data in and out of the stack. The Agile Data Stack has three layers including the underlying enabling infrastructure, the data intelligence layer, and the data-infused applications and services that benefit from those underlying components. The Continuous Data Loop is the representation of how data is continually being ingested, cleaned, visualized, recycled and refined and put back into the mix for future predictions so that modern applications can deliver intelligence in increasingly dynamic and personalized ways.
While most traditional IT customers and vendors are making moves to deal with the growth of big data — Dataware is largely the territory of startups and early adopters across every layer of the data stack.
At first, new sources of data and data systems will enhance and extend existing technologies such as databases, data warehouses and business intelligence tools. The new technologies will help unlock value in legacy systems and structured data silos along with new types and structures of data sources. But, as the Dataware infrastructure, intelligence and methodologies mature, data-infused applications and services will be built from scratch to disrupt industries and business processes.
Dataware will introduce net new processes and intelligence into the world’s oil exploration, research for cancer cures, advertising optimization, and yes, choosing movies and friends.
Here are four key principles fundamental to understanding the impact that Dataware will have on the technology industry.
- Big Data and traditional structured data work together as a “hybrid” of inputs to feed data-infused applications and services and will complement each other as these applications get built.
- Enabling infrastructure, including new types of databases (Cassandra, MongoDB, Hbase,) and data execution “engines” (Hadoop/Map Reduce, Spark), are primarily enablers and less likely to be where value is captured (when compared to the relational database era) in the Agile Data Stack.
- The Data Intelligence layer is where data, algorithms, data models and “pipelines” intersect to turn data into insights. More value will be delivered and captured in this layer than historic data “middleware” and BI tools have captured in the past. Companies focused on the enabling infrastructure today are likely to try and move up the stack into the data intelligence layer.
- Data driven applications and services are distinguished across two dimensions. First, are the data insights being delivered to a machine or a human? Second, are data insights/predictions being delivered in real-time or a batch/offline mode? Real-time insights delivered directly to a human end-user are the most challenging ones to run at scale. And, they are the most challenging systems to create a continuous feedback loop that delivers both instant gratification to the customer and compelling insights to the service provider.
It’s clear that not every company in the big data arena will succeed. There will be a lot of failures and there is already a lot of confusion around language and markets. The companies that will succeed will make themselves a core component of the Agile Data Stack or the Continuous Data Pipeline and will build their footprint from there.
Dataware is one way to frame the major areas of opportunity in what Mark Benioff recently called the “AI Spring” or what Microsoft is promoting with services like AzureML. But many questions remain including where the most promising markets exist, when those markets will be ready for rapid adoption and who amongst startup-ups and incumbents will emerge as winners and losers. What we do know is that Dataware will dramatically impact the technology industry over the next decade.
Matt McIlwain is an investor in Seattle with Madrona Venture Group who invests in enterprise, cloud and Dataware companies. Dato is one of his investments.
"I think that everyone recognizes that for IoT to really realize its full potential and value that it is about creating ecosystems and marketplaces and that no single vendor is able to support what is required," explained Esmeralda Swartz, VP, Marketing Enterprise and Cloud at Ericsson, in this SYS-CON.tv interview at @ThingsExpo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Mar. 30, 2017 08:00 AM EDT Reads: 4,579
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
Mar. 30, 2017 06:00 AM EDT Reads: 3,150
My team embarked on building a data lake for our sales and marketing data to better understand customer journeys. This required building a hybrid data pipeline to connect our cloud CRM with the new Hadoop Data Lake. One challenge is that IT was not in a position to provide support until we proved value and marketing did not have the experience, so we embarked on the journey ourselves within the product marketing team for our line of business within Progress. In his session at @BigDataExpo, Sum...
Mar. 30, 2017 04:45 AM EDT Reads: 3,373
MongoDB Atlas leverages VPC peering for AWS, a service that allows multiple VPC networks to interact. This includes VPCs that belong to other AWS account holders. By performing cross account VPC peering, users ensure networks that host and communicate their data are secure. In his session at 20th Cloud Expo, Jay Gordon, a Developer Advocate at MongoDB, will explain how to properly architect your VPC using existing AWS tools and then peer with your MongoDB Atlas cluster. He'll discuss the secur...
Mar. 30, 2017 04:15 AM EDT Reads: 1,013
Keeping pace with advancements in software delivery processes and tooling is taxing even for the most proficient organizations. Point tools, platforms, open source and the increasing adoption of private and public cloud services requires strong engineering rigor - all in the face of developer demands to use the tools of choice. As Agile has settled in as a mainstream practice, now DevOps has emerged as the next wave to improve software delivery speed and output. To make DevOps work, organization...
Mar. 30, 2017 04:15 AM EDT Reads: 2,279
Without a clear strategy for cost control and an architecture designed with cloud services in mind, costs and operational performance can quickly get out of control. To avoid multiple architectural redesigns requires extensive thought and planning. Boundary (now part of BMC) launched a new public-facing multi-tenant high resolution monitoring service on Amazon AWS two years ago, facing challenges and learning best practices in the early days of the new service.
Mar. 30, 2017 04:00 AM EDT Reads: 3,321
Niagara Networks exhibited at the 19th International Cloud Expo, which took place at the Santa Clara Convention Center in Santa Clara, CA, in November 2016. Niagara Networks offers the highest port-density systems, and the most complete Next-Generation Network Visibility systems including Network Packet Brokers, Bypass Switches, and Network TAPs.
Mar. 30, 2017 03:45 AM EDT Reads: 3,496
SYS-CON Events announced today that MobiDev, a client-oriented software development company, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place June 6-8, 2017, at the Javits Center in New York City, NY, and the 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex softw...
Mar. 30, 2017 02:15 AM EDT Reads: 4,166
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
Mar. 30, 2017 01:30 AM EDT Reads: 2,667
DevOps tends to focus on the relationship between Dev and Ops, putting an emphasis on the ops and application infrastructure. But that’s changing with microservices architectures. In her session at DevOps Summit, Lori MacVittie, Evangelist for F5 Networks, will focus on how microservices are changing the underlying architectures needed to scale, secure and deliver applications based on highly distributed (micro) services and why that means an expansion into “the network” for DevOps.
Mar. 30, 2017 01:00 AM EDT Reads: 8,360
In his session at Cloud Expo, Alan Winters, an entertainment executive/TV producer turned serial entrepreneur, will present a success story of an entrepreneur who has both suffered through and benefited from offshore development across multiple businesses: The smart choice, or how to select the right offshore development partner Warning signs, or how to minimize chances of making the wrong choice Collaboration, or how to establish the most effective work processes Budget control, or how to max...
Mar. 30, 2017 12:45 AM EDT Reads: 776
Interoute has announced the integration of its Global Cloud Infrastructure platform with Rancher Labs’ container management platform, Rancher. This approach enables enterprises to accelerate their digital transformation and infrastructure investments. Matthew Finnie, Interoute CTO commented “Enterprises developing and building apps in the cloud and those on a path to Digital Transformation need Digital ICT Infrastructure that allows them to build, test and deploy faster than ever before. The int...
Mar. 30, 2017 12:30 AM EDT Reads: 1,573
China Unicom exhibit at the 19th International Cloud Expo, which took place at the Santa Clara Convention Center in Santa Clara, CA, in November 2016. China United Network Communications Group Co. Ltd ("China Unicom") was officially established in 2009 on the basis of the merger of former China Netcom and former China Unicom. China Unicom mainly operates a full range of telecommunications services including mobile broadband (GSM, WCDMA, LTE FDD, TD-LTE), fixed-line broadband, ICT, data communica...
Mar. 30, 2017 12:15 AM EDT Reads: 3,655
SYS-CON Events announced today that Ocean9will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Ocean9 provides cloud services for Backup, Disaster Recovery (DRaaS) and instant Innovation, and redefines enterprise infrastructure with its cloud native subscription offerings for mission critical SAP workloads.
Mar. 29, 2017 09:30 PM EDT Reads: 2,505
Building a cross-cloud operational model can be a daunting task. Per-cloud silos are not the answer, but neither is a fully generic abstraction plane that strips out capabilities unique to a particular provider. In his session at 20th Cloud Expo, Chris Wolf, VP & Chief Technology Officer, Global Field & Industry at VMware, will discuss how successful organizations approach cloud operations and management, with insights into where operations should be centralized and when it’s best to decentraliz...
Mar. 29, 2017 05:15 PM EDT Reads: 3,895
SYS-CON Events announced today that Juniper Networks (NYSE: JNPR), an industry leader in automated, scalable and secure networks, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Juniper Networks challenges the status quo with products, solutions and services that transform the economics of networking. The company co-innovates with customers and partners to deliver automated, scalable and secure network...
Mar. 29, 2017 04:30 PM EDT Reads: 1,754
Deep learning has been very successful in social sciences and specially areas where there is a lot of data. Trading is another field that can be viewed as social science with a lot of data. With the advent of Deep Learning and Big Data technologies for efficient computation, we are finally able to use the same methods in investment management as we would in face recognition or in making chat-bots. In his session at 20th Cloud Expo, Gaurav Chakravorty, co-founder and Head of Strategy Development ...
Mar. 29, 2017 04:15 PM EDT Reads: 3,961
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
Mar. 29, 2017 04:00 PM EDT Reads: 3,298
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
Mar. 29, 2017 03:15 PM EDT Reads: 2,334
Imagine having the ability to leverage all of your current technology and to be able to compose it into one resource pool. Now imagine, as your business grows, not having to deploy a complete new appliance to scale your infrastructure. Also imagine a true multi-cloud capability that allows live migration without any modification between cloud environments regardless of whether that cloud is your private cloud or your public AWS, Azure or Google instance. Now think of a world that is not locked i...
Mar. 29, 2017 03:15 PM EDT Reads: 905