Welcome!

@CloudExpo Authors: Elizabeth White, Liz McMillan, Pat Romanski, David Paquette, Dana Gardner

Related Topics: @DevOpsSummit, Java IoT, Microservices Expo, Linux Containers, @CloudExpo, @BigDataExpo

@DevOpsSummit: Article

The DevOps Database | Part 3

Applying Feedback Loops to Database Change Management

In the third post in this series, I’d like to talk about the Second Way of DevOps: Amplifying Feedback Loops.  Here’s a refresher on The Second Way from my introductory post in this series:

The Second Way: Amplify Feedback Loops – This Way deals primarily with facilitating easier and faster communication between all individuals in a DevOps organization.  The goals of this step are to foster better understanding of all internal and external customers in the process and to develop an accessible body of knowledge to replace the dependence on expertise scattered across individuals.

I’ve stated before in this series that Database Change Management poses a unique challenge when your organization is shifting to an agile development methodology and implementing DevOps patterns.  Unlike other areas of your application stack, responsibility for managing application schema straddles two groups operating under somewhat opposed expectations. The development group is on the hook for producing more and more business critical features and releases at an ever increasing rate.  DBAs are tasked with providing a secure, highly available data platform and protecting the integrity of the organization’s priceless data.  The rate of schema change required by development to satisfy expectations can run head long into a database change process that is deliberate and metered by necessity to avoid downtime and data loss.  In organizations where these two groups are isolated from each other, you have the makings of a bottle neck in your release process.

The solution to this problem is embodied by The Second Way of DevOps. Communicate early, communicate often, communicate broadly, and prepare for what’s ahead. The tricky part is implementing the solution in a way that’s meaningful to every stakeholder in an organization’s application group.  At Datical, we’ve spent just as much time on how we organize and present the data associated with application schema changes as we have on automating the deployment of these changes.  We’ve rallied around the following key concepts to bring the The Second Way of DevOps to Database Change Management.

Proactive, Predictive Change Analysis
In an organization where development works independently of the database group, truly understanding the impact a stack of SQL scripts will have on downstream environments is a tedious and time consuming task.  Before these changes can be promoted, target environments must be meticulously evaluated for conflicts and dependencies that will impact the deployment process.  This often involves manual reviews and comparisons of diagrams and database dumps of complex environments. Achieving a high degree of confidence in the success of the proposed updates is difficult because it is so easy to overlook something.  Datical has developed a patent pending simulation feature called Forecast that automates this process.  The Forecast feature builds an in memory model of the target environment, simulates proposed changes on top of that model, and warns of potential error conditions, data loss and performance issues without touching the target database.  Because there is no impact to target environments, database administrators can Forecast changes several times during the development cycle to get ahead of issues that would normally be discovered much later in a pre-release review.  Development gets regular feedback on the changes they are proposing and can address issues that arise during the initial development phase when it is easier and safer to resolve them.   The two teams are working in unison to ensure a safe database deployment that works the first time without surprises.

Always Remember Where You Came From
Database changes are usually designed to address the immediate goals of an organization.  Once one set of requirements has been satisfied by a release, the motivations for the design decisions made for that release generally fades away as new requirements come along and new business initiatives take center stage.  Comments in SQL scripts and on the database objects themselves can be helpful in determining why things are the way they are, but these traces of the past are scattered everywhere. Making sense of the whole is an exercise in archaeology.   This was one of the driving forces behind our model based approach to database change management.  Our model is architected to provide a living history of your application schema.  Individual changes are tied to the specific requirement and release that necessitated them.  This data lives in the model so the information you need to make intelligent design decisions is right in front of you when you need it.

Know Where You Are
By tying the business reasons behind each schema change in the model, this information can be tracked in each database instance as it’s updated and included in Forecast, Deploy, and historical reports.  Tracking the changes in each instance and providing detailed reports allows you to easily disseminate information, effectively gate deployment steps, and quickly satisfy audit requirements. When everyone in your organization has access to thorough accounts of the Who, What, Where, When, and Why of any single database change in any environment, everyone is operating on the same level and can more effectively work towards a common goal.

Know Where You’re Headed
The model also facilitates concurrent development on multiple releases of a project.  By tracking changes made for several different releases in a single model, the development teams working on these releases are able to collaborate and stay ahead of changes made by other teams that may impact future releases.  Developers are able to unify redundant changes and eliminate conflicting changes as they implement instead of spending time on redesign later in the process when time is scarce and the cost of change is high.

More Stories By Pete Pickerill

Pete Pickerill is Vice President of Products and Co-founder of Datical. Pete is a software industry veteran who has built his career in Austin’s technology sector. Prior to co-founding Datical, he was employee number one at Phurnace Software and helped lead the company to a high profile acquisition by BMC Software, Inc. Pete has spent the majority of his career in successful startups and the companies that acquired them including Loop One (acquired by NeoPost Solutions), WholeSecurity (acquired by Symantec, Inc.) and Phurnace Software.

@CloudExpo Stories
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportuni...
Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability. He addres...
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
Aspose.Total for .NET is the most complete package of all file format APIs for .NET as offered by Aspose. It empowers developers to create, edit, render, print and convert between a wide range of popular document formats within any .NET, C#, ASP.NET and VB.NET applications. Aspose compiles all .NET APIs on a daily basis to ensure that it contains the most up to date versions of each of Aspose .NET APIs. If a new .NET API or a new version of existing APIs is released during the subscription peri...
SYS-CON Events announced today that 910Telecom will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Housed in the classic Denver Gas & Electric Building, 910 15th St., 910Telecom is a carrier-neutral telecom hotel located in the heart of Denver. Adjacent to CenturyLink, AT&T, and Denver Main, 910Telecom offers connectivity to all major carriers, Internet service providers, Internet backbones and ...
DevOps at Cloud Expo, taking place Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long dev...
Amazon has gradually rolled out parts of its IoT offerings in the last year, but these are just the tip of the iceberg. In addition to optimizing their back-end AWS offerings, Amazon is laying the ground work to be a major force in IoT – especially in the connected home and office. Amazon is extending its reach by building on its dominant Cloud IoT platform, its Dash Button strategy, recently announced Replenishment Services, the Echo/Alexa voice recognition control platform, the 6-7 strategic...
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Akana has announced the availability of version 8 of its API Management solution. The Akana Platform provides an end-to-end API Management solution for designing, implementing, securing, managing, monitoring, and publishing APIs. It is available as a SaaS platform, on-premises, and as a hybrid deployment. Version 8 introduces a lot of new functionality, all aimed at offering customers the richest API Management capabilities in a way that is easier than ever for API and app developers to use.
Personalization has long been the holy grail of marketing. Simply stated, communicate the most relevant offer to the right person and you will increase sales. To achieve this, you must understand the individual. Consequently, digital marketers developed many ways to gather and leverage customer information to deliver targeted experiences. In his session at @ThingsExpo, Lou Casal, Founder and Principal Consultant at Practicala, discussed how the Internet of Things (IoT) has accelerated our abil...
With so much going on in this space you could be forgiven for thinking you were always working with yesterday’s technologies. So much change, so quickly. What do you do if you have to build a solution from the ground up that is expected to live in the field for at least 5-10 years? This is the challenge we faced when we looked to refresh our existing 10-year-old custom hardware stack to measure the fullness of trash cans and compactors.
SYS-CON Events announced today that Isomorphic Software will exhibit at DevOps Summit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Isomorphic Software provides the SmartClient HTML5/AJAX platform, the most advanced technology for building rich, cutting-edge enterprise web applications for desktop and mobile. SmartClient combines the productivity and performance of traditional desktop software with the simp...
The emerging Internet of Everything creates tremendous new opportunities for customer engagement and business model innovation. However, enterprises must overcome a number of critical challenges to bring these new solutions to market. In his session at @ThingsExpo, Michael Martin, CTO/CIO at nfrastructure, outlined these key challenges and recommended approaches for overcoming them to achieve speed and agility in the design, development and implementation of Internet of Everything solutions wi...
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devices - comp...
Qosmos has announced new milestones in the detection of encrypted traffic and in protocol signature coverage. Qosmos latest software can accurately classify traffic encrypted with SSL/TLS (e.g., Google, Facebook, WhatsApp), P2P traffic (e.g., BitTorrent, MuTorrent, Vuze), and Skype, while preserving the privacy of communication content. These new classification techniques mean that traffic optimization, policy enforcement, and user experience are largely unaffected by encryption. In respect wit...
"My role is working with customers, helping them go through this digital transformation. I spend a lot of time talking to banks, big industries, manufacturers working through how they are integrating and transforming their IT platforms and moving them forward," explained William Morrish, General Manager Product Sales at Interoute, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
To leverage Continuous Delivery, enterprises must consider impacts that span functional silos, as well as applications that touch older, slower moving components. Managing the many dependencies can cause slowdowns. See how to achieve continuous delivery in the enterprise.
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...