SYS-CON MEDIA Authors: Liz McMillan, Elizabeth White, Pat Romanski, Gary Arora, Zakia Bouachraoui

Blog Feed Post

Expedite the Handoff from Dev to Test: An Interview with Gary Gruver (Part 2)

In the second part of our conversation with Gary Gruver (read part 1 here), we continue to discuss delivery pipeline inefficiencies of enterprise releases. As Gary describes in his book Starting and Scaling DevOps:

“The biggest inefficiencies in most large organizations exists in the large, complex, tightly coupled systems that require coordination across large numbers of people from the business all the way out to Operations.”

Quite a broad topic, so for our discussion we are drilling down on the dev to test hand off as well as test environments. We left off part 1 of the interview with this awesome soundbite from Gary: “Lots of orgs spend more time and effort getting the big, complex enterprise environments up and ready for testing than they do actually writing the code.”

In your book Starting and Scaling DevOps you state:

“For these orgs, it is much more important to push the majority of testing and defect fixing into smaller, less complex test environments with quality gates to keep defects out of the bigger more complex environments. This helps to reduce the cost and complexity of the testing and also helps with the triage process because the issues are localized to the subsystem or application that created them.”

Can you provide some additional insight on gating code?

GG: “For organizations with tightly coupled architectures, it’s important to build up stable enterprise systems using a well-structured deployment pipeline with good quality gates. Large test environments are incredibly expensive and hard to manage. As a result, they are not a very efficient approach for finding defects. You have to establish quality gates, where release teams are not allowed to move code further along the pipeline until they pass specific tests.”

What’s the process for introducing gated code into a delivery pipeline?

GG: “Find out what are the sources of the apps or subsystems that are breaking most frequently. You’ll want to start by gating those apps to find and fix defects before moving code onto the next, more complex test environment.

Create a paredo chart as to why tests are failing. When you have those metrics for each environment, use different tags for the various defects. Then you can see all the waste that can potentially come out of the system. This sounds easy, but it’s so rare that people step back and look at how their organization works in order to put metrics on it. There’s never time to step back and look at everything.”

How do you identify areas for improvement in the dev to test handoff?

GG: “First, I tell teams to answer this question: What percentage of defects are you finding in each of your environments? For example, if 90% of defects occur in the initial test environment, the system is good, meaning the dev team is receiving quality feedback. If you are finding a majority of your issues on the right side of the pipeline, closer to production, then feedback to dev is not that valuable as triage and root cause is much more complex. By the time you are in a complex environment, you should only be testing the interfaces.

The next question to ask is: What are the different types of issues found in the different test environments? Are you finding environment issues, deployment issues, problems with automated tests, the code…? We’ll look at test results from two perspectives to understand the best approach to triage. First, we look at whether the same test has passed from build to build, then we’ll look at how well the same build is performing as the test environment complexity increases from release phase to phase. To do this, each stage of the deployment pipeline needs a stable test environment for gating code.”

Besides incorrect configurations, what other problems have you run into with test environments?

GG: “I work with many organizations that are on a journey to test automation, meaning they still have manual testing left in the system. It takes work to make sure tests are triagable, maintainable, and can adapt as the app is changing. One organization I’m working with now runs 3000 automated tests on a weekly basis. Recently a release train was held up not because there was a problem with the tests, but it turns out a test environment didn’t have enough memory and CPU power to run the automated test.”

In your book, you state:

“For large tightly coupled systems, developers often don’t understand the complexities of the production environments. Additionally, the people that understand the production environments don’t understand well the impact of the changes that developers are making. There are also frequently different end points in different test environments at each stage of the deployment pipeline. No one person understands what needs to happen all the way down the deployment pipeline. Therefore, managing environments for complex systems requires close collaboration from every group between dev and ops.”

What is your advice for managing test environments in tightly coupled systems?

GG: “The DevOps journey towards efficiency must involve getting test environment configurations under version control, that way everyone can see exactly who changed what and when. Then there’s no need to hold big meetings like a Scrum of Scrums because everyone can see code progress down the pipeline to understand status. Also, developers need faster access to early stage test environments, so they can validate their changes and catch their own defects. Success really requires being able to provide environments with cloud-like efficiencies on demand.”

Stable test environments, under version control, and on demand. Sage advice.

Plutora Environments integrates with Jenkins to trigger builds and track component versions of test environments.

Please join us on Wednesday, January 24th for a webinar with Gary Gruver: Continuous Delivery Pipelines:  Metrics, Myths, and Milestones.

 

The post Expedite the Handoff from Dev to Test: An Interview with Gary Gruver (Part 2) appeared first on Plutora.

Read the original blog entry...

More Stories By Plutora Blog

Plutora provides Enterprise Release and Test Environment Management SaaS solutions aligning process, technology, and information to solve release orchestration challenges for the enterprise.

Plutora’s SaaS solution enables organizations to model release management and test environment management activities as a bridge between agile project teams and an enterprise’s ITSM initiatives. Using Plutora, you can orchestrate parallel releases from several independent DevOps groups all while giving your executives as well as change management specialists insight into overall risk.

Supporting the largest releases for the largest organizations throughout North America, EMEA, and Asia Pacific, Plutora provides proof that large companies can adopt DevOps while managing the risks that come with wider adoption of self-service and agile software development in the enterprise. Aligning process, technology, and information to solve increasingly complex release orchestration challenges, this Gartner “Cool Vendor in IT DevOps” upgrades the enterprise release management from spreadsheets, meetings, and email to an integrated dashboard giving release managers insight and control over large software releases.

Latest Stories
Alan Hase is Vice President of Engineering and Chief Development Officer at Big Switch. Alan has more than 20 years of experience in the networking industry and leading global engineering teams which have delivered industry leading innovation in high end routing, security, fabric and wireless technologies. Alan joined Big Switch from Extreme Networks where he was responsible for product strategy for its secure campus switching, intelligent mobility and campus orchestration products. Prior to Ext...
Isomorphic Software is the global leader in high-end, web-based business applications. We develop, market, and support the SmartClient & Smart GWT HTML5/Ajax platform, combining the productivity and performance of traditional desktop software with the simplicity and reach of the open web. With staff in 10 timezones, Isomorphic provides a global network of services related to our technology, with offerings ranging from turnkey application development to SLA-backed enterprise support. Leadin...
On-premise or off, you have powerful tools available to maximize the value of your infrastructure and you demand more visibility and operational control. Fortunately, data center management tools keep a vigil on memory contestation, power, thermal consumption, server health, and utilization, allowing better control no matter your cloud's shape. In this session, learn how Intel software tools enable real-time monitoring and precise management to lower operational costs and optimize infrastructure...
Public clouds dominate IT conversations but the next phase of cloud evolutions are "multi" hybrid cloud environments. The winners in the cloud services industry will be those organizations that understand how to leverage these technologies as complete service solutions for specific customer verticals. In turn, both business and IT actors throughout the enterprise will need to increase their engagement with multi-cloud deployments today while planning a technology strategy that will constitute a ...
Cloud-Native thinking and Serverless Computing are now the norm in financial services, manufacturing, telco, healthcare, transportation, energy, media, entertainment, retail and other consumer industries, as well as the public sector. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that pro...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Every organization is facing their own Digital Transformation as they attempt to stay ahead of the competition, or worse, just keep up. Each new opportunity, whether embracing machine learning, IoT, or a cloud migration, seems to bring new development, deployment, and management models. The results are more diverse and federated computing models than any time in our history.
Andrew Keys is co-founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereum.
Data center, on-premise, public-cloud, private-cloud, multi-cloud, hybrid-cloud, IoT, AI, edge, SaaS, PaaS... it's an availability, security, performance and integration nightmare even for the best of the best IT experts. Organizations realize the tremendous benefits of everything the digital transformation has to offer. Cloud adoption rates are increasing significantly, and IT budgets are morphing to follow suit. But distributing applications and infrastructure around increases risk, introdu...
DevOps has long focused on reinventing the SDLC (e.g. with CI/CD, ARA, pipeline automation etc.), while reinvention of IT Ops has lagged. However, new approaches like Site Reliability Engineering, Observability, Containerization, Operations Analytics, and ML/AI are driving a resurgence of IT Ops. In this session our expert panel will focus on how these new ideas are [putting the Ops back in DevOps orbringing modern IT Ops to DevOps].
Financial enterprises in New York City, London, Singapore, and other world financial capitals are embracing a new generation of smart, automated FinTech that eliminates many cumbersome, slow, and expensive intermediate processes from their businesses. Accordingly, attendees at the upcoming 23rd CloudEXPO, June 24-26, 2019 at Santa Clara Convention Center in Santa Clara, CA will find fresh new content in full new FinTech & Enterprise Blockchain track.
While a hybrid cloud can ease that transition, designing and deploy that hybrid cloud still offers challenges for organizations concerned about lack of available cloud skillsets within their organization. Managed service providers offer a unique opportunity to fill those gaps and get organizations of all sizes on a hybrid cloud that meets their comfort level, while delivering enhanced benefits for cost, efficiency, agility, mobility, and elasticity.
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science" is responsible for guiding the technology strategy within Hitachi Vantara for IoT and Analytics. Bill brings a balanced business-technology approach that focuses on business outcomes to drive data, analytics and technology decisions that underpin an organization's digital transformation strategy. Bill has a very impressive background which includes ...
On-premise or off, you have powerful tools available to maximize the value of your infrastructure and you demand more visibility and operational control. Fortunately, data center management tools keep a vigil on memory contestation, power, thermal consumption, server health, and utilization, allowing better control no matter your cloud's shape. In this session, learn how Intel software tools enable real-time monitoring and precise management to lower operational costs and optimize infrastructure...
Most organizations are awash today in data and IT systems, yet they're still struggling mightily to use these invaluable assets to meet the rising demand for new digital solutions and customer experiences that drive innovation and growth. What's lacking are potent and effective ways to rapidly combine together on-premises IT and the numerous commercial clouds that the average organization has in place today into effective new business solutions. New research shows that delivering on multicloud e...