|By Roger Strukhoff||
|December 5, 2011 05:15 AM EST||
Hadoop and NoSQL are entering more conversations about enterprise IT and Cloud Computing. Mongo is now more than a guy who "like candy" and is "only pawn in game of life." The usual brilliant explosion of innovation is ongoing in this space, which is both a sympton and a driver of the new era of Big Data.
Facebook reports it now handles 30 petabytes of data (or 30 million gigabytes). Other numbers of that magnitude are appearing all over the place, along with estimates of daily data flows that seem to increase by a magnitude every few years.
All this should be good for manufacturers of chips, storage, and networks, even as the general principle of Moore's Law relentlessly drives per-unit prices down. It should be good for software developers as well, as multiple newish skills come to the fore.
The parent company of Cloud Computing Journal has announced it will be co-locating the First International Big Data Expo to Cloud Expo New York June 11-14. The announcement came before the doors even opened at the recent Cloud Expo in Silicon Valley, as the company's far-seeing chairman took in key input from his sponsors and advisors.
Big Data and Cloud Computing share some part of each other's circle in a Boolean diagram - specifically how much is not important, although Captain Obvious tells me that Cloud Computing is more important to Big Data than Big Data is to Cloud Computing.
In The Year -1 BCE...
Enterprise IT is really somewhere around Year Zero or Year One with Cloud Computing, even as all vendors are offering Cloud-something by now. We are thus at Year Minus-2 or thereabouts with the new conception of Big Data.
The coming year promises to be a boon for the private cloud, as companies seek to virtualize and optimize, track usage more effectively, and integrate proliferating mobile devices into the mix.
Meanwhile, simultaneously, those pesky mobile thingys - smartphones and tablets - are helping to create the modern-day Big Data monster. Yet they are not the real culprit per se. Rather, let's point our fingers at online transactions/interactions, point-of-purchase scans, and especially, all the fascistic monitoring software embedded in our browsers and phones.
The key differentiator is that cloud is something you do, but Big Data is something you have. The smartest technology companies such as Google, Amazon, and Facebook understood the implications of mass scaling from the get-go, and have had rare glitches in serving continuously expanding mass data to the masses. Walmart has also kept on top of things as the Big Data generated by about 1 million transactions per hour has evolved.
But what about everybody else? How prepared are most of the Fortune 1500 and the millions of SMBs for the Big Data that's flowing into their IT departments? Add social networking to the mix - the need to address, respond, and cultivate online communities who are blasting their informed and uninformed opinions your way every second - and Big Data emerges as a Big Problem. Or is that Big Opportunity?
Follow me on Twitter
In his session at 19th Cloud Expo, Claude Remillard, Principal Program Manager in Developer Division at Microsoft, contrasted how his team used config as code and immutable patterns for continuous delivery of microservices and apps to the cloud. He showed how the immutable patterns helps developers do away with most of the complexity of config as code-enabling scenarios such as rollback, zero downtime upgrades with far greater simplicity. He also demoed building immutable pipelines in the cloud ...
Jan. 21, 2017 05:30 AM EST Reads: 3,549
Traditional on-premises data centers have long been the domain of modern data platforms like Apache Hadoop, meaning companies who build their business on public cloud were challenged to run Big Data processing and analytics at scale. But recent advancements in Hadoop performance, security, and most importantly cloud-native integrations, are giving organizations the ability to truly gain value from all their data. In his session at 19th Cloud Expo, David Tishgart, Director of Product Marketing ...
Jan. 21, 2017 05:00 AM EST Reads: 2,993
Choosing the right cloud for your workloads is a balancing act that can cost your organization time, money and aggravation - unless you get it right the first time. Economics, speed, performance, accessibility, administrative needs and security all play a vital role in dictating your approach to the cloud. Without knowing the right questions to ask, you could wind up paying for capacity you'll never need or underestimating the resources required to run your applications.
Jan. 21, 2017 05:00 AM EST Reads: 4,006
Technology vendors and analysts are eager to paint a rosy picture of how wonderful IoT is and why your deployment will be great with the use of their products and services. While it is easy to showcase successful IoT solutions, identifying IoT systems that missed the mark or failed can often provide more in the way of key lessons learned. In his session at @ThingsExpo, Peter Vanderminden, Principal Industry Analyst for IoT & Digital Supply Chain to Flatiron Strategies, will focus on how IoT depl...
Jan. 21, 2017 03:45 AM EST Reads: 1,985
Adding public cloud resources to an existing application can be a daunting process. The tools that you currently use to manage the software and hardware outside the cloud aren’t always the best tools to efficiently grow into the cloud. All of the major configuration management tools have cloud orchestration plugins that can be leveraged, but there are also cloud-native tools that can dramatically improve the efficiency of managing your application lifecycle. In his session at 18th Cloud Expo, ...
Jan. 21, 2017 02:30 AM EST Reads: 6,109
The pace of innovation, vendor lock-in, production sustainability, cost-effectiveness, and managing risk… In his session at 18th Cloud Expo, Dan Choquette, Founder of RackN, discussed how CIOs are challenged finding the balance of finding the right tools, technology and operational model that serves the business the best. He also discussed how clouds, open source software and infrastructure solutions have benefits but also drawbacks and how workload and operational portability between vendors an...
Jan. 21, 2017 02:30 AM EST Reads: 5,041
With the proliferation of both SQL and NoSQL databases, organizations can now target specific fit-for-purpose database tools for their different application needs regarding scalability, ease of use, ACID support, etc. Platform as a Service offerings make this even easier now, enabling developers to roll out their own database infrastructure in minutes with minimal management overhead. However, this same amount of flexibility also comes with the challenges of picking the right tool, on the right ...
Jan. 21, 2017 02:15 AM EST Reads: 5,358
"Tintri was started in 2008 with the express purpose of building a storage appliance that is ideal for virtualized environments. We support a lot of different hypervisor platforms from VMware to OpenStack to Hyper-V," explained Dan Florea, Director of Product Management at Tintri, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Jan. 21, 2017 01:15 AM EST Reads: 4,900
There will be new vendors providing applications, middleware, and connected devices to support the thriving IoT ecosystem. This essentially means that electronic device manufacturers will also be in the software business. Many will be new to building embedded software or robust software. This creates an increased importance on software quality, particularly within the Industrial Internet of Things where business-critical applications are becoming dependent on products controlled by software. Qua...
Jan. 21, 2017 01:15 AM EST Reads: 5,052
The speed of software changes in growing and large scale rapid-paced DevOps environments presents a challenge for continuous testing. Many organizations struggle to get this right. Practices that work for small scale continuous testing may not be sufficient as the requirements grow. In his session at DevOps Summit, Marc Hornbeek, Sr. Solutions Architect of DevOps continuous test solutions at Spirent Communications, explained the best practices of continuous testing at high scale, which is rele...
Jan. 21, 2017 12:30 AM EST Reads: 4,570
A critical component of any IoT project is what to do with all the data being generated. This data needs to be captured, processed, structured, and stored in a way to facilitate different kinds of queries. Traditional data warehouse and analytical systems are mature technologies that can be used to handle certain kinds of queries, but they are not always well suited to many problems, particularly when there is a need for real-time insights.
Jan. 21, 2017 12:15 AM EST Reads: 6,375
In his General Session at 16th Cloud Expo, David Shacochis, host of The Hybrid IT Files podcast and Vice President at CenturyLink, investigated three key trends of the “gigabit economy" though the story of a Fortune 500 communications company in transformation. Narrating how multi-modal hybrid IT, service automation, and agile delivery all intersect, he will cover the role of storytelling and empathy in achieving strategic alignment between the enterprise and its information technology.
Jan. 21, 2017 12:00 AM EST Reads: 4,709
Containers have changed the mind of IT in DevOps. They enable developers to work with dev, test, stage and production environments identically. Containers provide the right abstraction for microservices and many cloud platforms have integrated them into deployment pipelines. DevOps and Containers together help companies to achieve their business goals faster and more effectively. In his session at DevOps Summit, Ruslan Synytsky, CEO and Co-founder of Jelastic, reviewed the current landscape of D...
Jan. 21, 2017 12:00 AM EST Reads: 5,108
IoT is at the core or many Digital Transformation initiatives with the goal of re-inventing a company's business model. We all agree that collecting relevant IoT data will result in massive amounts of data needing to be stored. However, with the rapid development of IoT devices and ongoing business model transformation, we are not able to predict the volume and growth of IoT data. And with the lack of IoT history, traditional methods of IT and infrastructure planning based on the past do not app...
Jan. 20, 2017 10:45 PM EST Reads: 929
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus o...
Jan. 20, 2017 08:45 PM EST Reads: 4,390
"We're bringing out a new application monitoring system to the DevOps space. It manages large enterprise applications that are distributed throughout a node in many enterprises and we manage them as one collective," explained Kevin Barnes, President of eCube Systems, in this SYS-CON.tv interview at DevOps at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Jan. 20, 2017 06:30 PM EST Reads: 5,465
The Internet of Things can drive efficiency for airlines and airports. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Sudip Majumder, senior director of development at Oracle, discussed the technical details of the connected airline baggage and related social media solutions. These IoT applications will enhance travelers' journey experience and drive efficiency for the airlines and the airports.
Jan. 20, 2017 05:45 PM EST Reads: 2,104
@DevOpsSummit at Cloud taking place June 6-8, 2017, at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long developm...
Jan. 20, 2017 05:15 PM EST Reads: 3,547
In his General Session at DevOps Summit, Asaf Yigal, Co-Founder & VP of Product at Logz.io, will explore the value of Kibana 4 for log analysis and will give a real live, hands-on tutorial on how to set up Kibana 4 and get the most out of Apache log files. He will examine three use cases: IT operations, business intelligence, and security and compliance. This is a hands-on session that will require participants to bring their own laptops, and we will provide the rest.
Jan. 20, 2017 05:15 PM EST Reads: 4,968
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
Jan. 20, 2017 05:15 PM EST Reads: 3,827