|By Pete Pickerill||
|February 11, 2014 10:00 AM EST||
In the third post in this series, I’d like to talk about the Second Way of DevOps: Amplifying Feedback Loops. Here’s a refresher on The Second Way from my introductory post in this series:
The Second Way: Amplify Feedback Loops – This Way deals primarily with facilitating easier and faster communication between all individuals in a DevOps organization. The goals of this step are to foster better understanding of all internal and external customers in the process and to develop an accessible body of knowledge to replace the dependence on expertise scattered across individuals.
I’ve stated before in this series that Database Change Management poses a unique challenge when your organization is shifting to an agile development methodology and implementing DevOps patterns. Unlike other areas of your application stack, responsibility for managing application schema straddles two groups operating under somewhat opposed expectations. The development group is on the hook for producing more and more business critical features and releases at an ever increasing rate. DBAs are tasked with providing a secure, highly available data platform and protecting the integrity of the organization’s priceless data. The rate of schema change required by development to satisfy expectations can run head long into a database change process that is deliberate and metered by necessity to avoid downtime and data loss. In organizations where these two groups are isolated from each other, you have the makings of a bottle neck in your release process.
The solution to this problem is embodied by The Second Way of DevOps. Communicate early, communicate often, communicate broadly, and prepare for what’s ahead. The tricky part is implementing the solution in a way that’s meaningful to every stakeholder in an organization’s application group. At Datical, we’ve spent just as much time on how we organize and present the data associated with application schema changes as we have on automating the deployment of these changes. We’ve rallied around the following key concepts to bring the The Second Way of DevOps to Database Change Management.
Proactive, Predictive Change Analysis
In an organization where development works independently of the database group, truly understanding the impact a stack of SQL scripts will have on downstream environments is a tedious and time consuming task. Before these changes can be promoted, target environments must be meticulously evaluated for conflicts and dependencies that will impact the deployment process. This often involves manual reviews and comparisons of diagrams and database dumps of complex environments. Achieving a high degree of confidence in the success of the proposed updates is difficult because it is so easy to overlook something. Datical has developed a patent pending simulation feature called Forecast that automates this process. The Forecast feature builds an in memory model of the target environment, simulates proposed changes on top of that model, and warns of potential error conditions, data loss and performance issues without touching the target database. Because there is no impact to target environments, database administrators can Forecast changes several times during the development cycle to get ahead of issues that would normally be discovered much later in a pre-release review. Development gets regular feedback on the changes they are proposing and can address issues that arise during the initial development phase when it is easier and safer to resolve them. The two teams are working in unison to ensure a safe database deployment that works the first time without surprises.
Always Remember Where You Came From
Database changes are usually designed to address the immediate goals of an organization. Once one set of requirements has been satisfied by a release, the motivations for the design decisions made for that release generally fades away as new requirements come along and new business initiatives take center stage. Comments in SQL scripts and on the database objects themselves can be helpful in determining why things are the way they are, but these traces of the past are scattered everywhere. Making sense of the whole is an exercise in archaeology. This was one of the driving forces behind our model based approach to database change management. Our model is architected to provide a living history of your application schema. Individual changes are tied to the specific requirement and release that necessitated them. This data lives in the model so the information you need to make intelligent design decisions is right in front of you when you need it.
Know Where You Are
By tying the business reasons behind each schema change in the model, this information can be tracked in each database instance as it’s updated and included in Forecast, Deploy, and historical reports. Tracking the changes in each instance and providing detailed reports allows you to easily disseminate information, effectively gate deployment steps, and quickly satisfy audit requirements. When everyone in your organization has access to thorough accounts of the Who, What, Where, When, and Why of any single database change in any environment, everyone is operating on the same level and can more effectively work towards a common goal.
Know Where You’re Headed
The model also facilitates concurrent development on multiple releases of a project. By tracking changes made for several different releases in a single model, the development teams working on these releases are able to collaborate and stay ahead of changes made by other teams that may impact future releases. Developers are able to unify redundant changes and eliminate conflicting changes as they implement instead of spending time on redesign later in the process when time is scarce and the cost of change is high.