Welcome!

@CloudExpo Authors: Pat Romanski, Jason Bloomberg, Yeshim Deniz, Elizabeth White, Karyn Jeffery

Related Topics: @DevOpsSummit, Java IoT, Microservices Expo, Linux Containers, @CloudExpo, @BigDataExpo

@DevOpsSummit: Article

The DevOps Database | Part 3

Applying Feedback Loops to Database Change Management

In the third post in this series, I’d like to talk about the Second Way of DevOps: Amplifying Feedback Loops.  Here’s a refresher on The Second Way from my introductory post in this series:

The Second Way: Amplify Feedback Loops – This Way deals primarily with facilitating easier and faster communication between all individuals in a DevOps organization.  The goals of this step are to foster better understanding of all internal and external customers in the process and to develop an accessible body of knowledge to replace the dependence on expertise scattered across individuals.

I’ve stated before in this series that Database Change Management poses a unique challenge when your organization is shifting to an agile development methodology and implementing DevOps patterns.  Unlike other areas of your application stack, responsibility for managing application schema straddles two groups operating under somewhat opposed expectations. The development group is on the hook for producing more and more business critical features and releases at an ever increasing rate.  DBAs are tasked with providing a secure, highly available data platform and protecting the integrity of the organization’s priceless data.  The rate of schema change required by development to satisfy expectations can run head long into a database change process that is deliberate and metered by necessity to avoid downtime and data loss.  In organizations where these two groups are isolated from each other, you have the makings of a bottle neck in your release process.

The solution to this problem is embodied by The Second Way of DevOps. Communicate early, communicate often, communicate broadly, and prepare for what’s ahead. The tricky part is implementing the solution in a way that’s meaningful to every stakeholder in an organization’s application group.  At Datical, we’ve spent just as much time on how we organize and present the data associated with application schema changes as we have on automating the deployment of these changes.  We’ve rallied around the following key concepts to bring the The Second Way of DevOps to Database Change Management.

Proactive, Predictive Change Analysis
In an organization where development works independently of the database group, truly understanding the impact a stack of SQL scripts will have on downstream environments is a tedious and time consuming task.  Before these changes can be promoted, target environments must be meticulously evaluated for conflicts and dependencies that will impact the deployment process.  This often involves manual reviews and comparisons of diagrams and database dumps of complex environments. Achieving a high degree of confidence in the success of the proposed updates is difficult because it is so easy to overlook something.  Datical has developed a patent pending simulation feature called Forecast that automates this process.  The Forecast feature builds an in memory model of the target environment, simulates proposed changes on top of that model, and warns of potential error conditions, data loss and performance issues without touching the target database.  Because there is no impact to target environments, database administrators can Forecast changes several times during the development cycle to get ahead of issues that would normally be discovered much later in a pre-release review.  Development gets regular feedback on the changes they are proposing and can address issues that arise during the initial development phase when it is easier and safer to resolve them.   The two teams are working in unison to ensure a safe database deployment that works the first time without surprises.

Always Remember Where You Came From
Database changes are usually designed to address the immediate goals of an organization.  Once one set of requirements has been satisfied by a release, the motivations for the design decisions made for that release generally fades away as new requirements come along and new business initiatives take center stage.  Comments in SQL scripts and on the database objects themselves can be helpful in determining why things are the way they are, but these traces of the past are scattered everywhere. Making sense of the whole is an exercise in archaeology.   This was one of the driving forces behind our model based approach to database change management.  Our model is architected to provide a living history of your application schema.  Individual changes are tied to the specific requirement and release that necessitated them.  This data lives in the model so the information you need to make intelligent design decisions is right in front of you when you need it.

Know Where You Are
By tying the business reasons behind each schema change in the model, this information can be tracked in each database instance as it’s updated and included in Forecast, Deploy, and historical reports.  Tracking the changes in each instance and providing detailed reports allows you to easily disseminate information, effectively gate deployment steps, and quickly satisfy audit requirements. When everyone in your organization has access to thorough accounts of the Who, What, Where, When, and Why of any single database change in any environment, everyone is operating on the same level and can more effectively work towards a common goal.

Know Where You’re Headed
The model also facilitates concurrent development on multiple releases of a project.  By tracking changes made for several different releases in a single model, the development teams working on these releases are able to collaborate and stay ahead of changes made by other teams that may impact future releases.  Developers are able to unify redundant changes and eliminate conflicting changes as they implement instead of spending time on redesign later in the process when time is scarce and the cost of change is high.

More Stories By Pete Pickerill

Pete Pickerill is Vice President of Products and Co-founder of Datical. Pete is a software industry veteran who has built his career in Austin’s technology sector. Prior to co-founding Datical, he was employee number one at Phurnace Software and helped lead the company to a high profile acquisition by BMC Software, Inc. Pete has spent the majority of his career in successful startups and the companies that acquired them including Loop One (acquired by NeoPost Solutions), WholeSecurity (acquired by Symantec, Inc.) and Phurnace Software.

@CloudExpo Stories
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
"The Striim platform is a full end-to-end streaming integration and analytics platform that is middleware that covers a lot of different use cases," explained Steve Wilkes, Founder and CTO at Striim, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
SYS-CON Events announced today that Calligo, an innovative cloud service provider offering mid-sized companies the highest levels of data privacy and security, has been named "Bronze Sponsor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalised support service from its globally located cloud plat...
SYS-CON Events announced today that Calligo has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo is an innovative cloud service provider offering mid-sized companies the highest levels of data privacy. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalized support service from its globally located cloud platfor...
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
SYS-CON Events announced today that DXWorldExpo has been named “Global Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Digital Transformation is the key issue driving the global enterprise IT business. Digital Transformation is most prominent among Global 2000 enterprises and government institutions.
SYS-CON Events announced today that Datera, that offers a radically new data management architecture, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera is transforming the traditional datacenter model through modern cloud simplicity. The technology industry is at another major inflection point. The rise of mobile, the Internet of Things, data storage and Big...
"DX encompasses the continuing technology revolution, and is addressing society's most important issues throughout the entire $78 trillion 21st-century global economy," said Roger Strukhoff, Conference Chair. "DX World Expo has organized these issues along 10 tracks with more than 150 of the world's top speakers coming to Istanbul to help change the world."
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"Outscale was founded in 2010, is based in France, is a strategic partner to Dassault Systémes and has done quite a bit of work with divisions of Dassault," explained Jackie Funk, Digital Marketing exec at Outscale, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...
"DivvyCloud as a company set out to help customers automate solutions to the most common cloud problems," noted Jeremy Snyder, VP of Business Development at DivvyCloud, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
While the focus and objectives of IoT initiatives are many and diverse, they all share a few common attributes, and one of those is the network. Commonly, that network includes the Internet, over which there isn't any real control for performance and availability. Or is there? The current state of the art for Big Data analytics, as applied to network telemetry, offers new opportunities for improving and assuring operational integrity. In his session at @ThingsExpo, Jim Frey, Vice President of S...
"We focus on SAP workloads because they are among the most powerful but somewhat challenging workloads out there to take into public cloud," explained Swen Conrad, CEO of Ocean9, Inc., in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"As we've gone out into the public cloud we've seen that over time we may have lost a few things - we've lost control, we've given up cost to a certain extent, and then security, flexibility," explained Steve Conner, VP of Sales at Cloudistics,in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
DX World EXPO, LLC., a Lighthouse Point, Florida-based startup trade show producer and the creator of "DXWorldEXPO® - Digital Transformation Conference & Expo" has announced its executive management team. The team is headed by Levent Selamoglu, who has been named CEO. "Now is the time for a truly global DX event, to bring together the leading minds from the technology world in a conversation about Digital Transformation," he said in making the announcement.
"We are focused on SAP running in the clouds, to make this super easy because we believe in the tremendous value of those powerful worlds - SAP and the cloud," explained Frank Stienhans, CTO of Ocean9, Inc., in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"Peak 10 is a hybrid infrastructure provider across the nation. We are in the thick of things when it comes to hybrid IT," explained , Chief Technology Officer at Peak 10, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We are still a relatively small software house and we are focusing on certain industries like FinTech, med tech, energy and utilities. We help our customers with their digital transformation," noted Piotr Stawinski, Founder and CEO of EARP Integration, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.