SYS-CON MEDIA Authors: Pat Romanski, Zakia Bouachraoui, Liz McMillan, Elizabeth White, Yeshim Deniz

Article

Applying DevOps to Deliver Quality at Speed

Applying DevOps to Deliver Quality at Speed

Fannie Mae operates in the already heavily regulated financial services/mortgage markets. Add to that it is a government sponsored entity, which means it is laden with many of the policies and regulations of federal government agencies. Plus, they have 6,000 employees and annual revenue of $22 billion. So, they could be a poster child for, "You can't implement DevOps there."

Barry Snyder, DevOps Manager, Developer Platforms at Fannie Mae, begs to differ, which is what he did in his 2016 All Day DevOps talk, How Fannie Mae Applied DevOps to Deliver Quality at Speed. He looks at:

  • Agile and DevOps transformation
  • Doing more with less with greater quality
  • Lessons learned

Fannie Mae started implementing DevOps in 2014, and Barry was in the middle of it from the start. He gave his talk at the end of 2016, and, as he said, they still aren't fully adopted (are you ever), but he gave a status and a comparison between where they were and where they are now.

2012

2016

10 projects using modified-Agile. Essentially waterfall with terminology/concepts from Agile

155 using Scrum/SAFe and growing

9 to 18 months to release to production

Projects releasing every month

9 governance gates/inspect 100% before release to production

1 working group, 1 CAB, inspect 10% after release to production

2-4 months to provision dev and test servers

Minutes to provision dev and test servers

300-400 builds per month; 900-1200 deployments per month

3-4K builds per month; 12-15K deployments per month

Testing took 50% the time it took to construct the solution

80% savings in testing time and executed within the sprint

No means to measure and evaluate the productivity from release to release

Productivity increasing by an average of 30%-40% with costs going down by 30%

barry1.png

According Barry, these results are being sustained - they are not just a blip. To get there, they had to get an appetite for risk. Overcoming that hurdle required them to realize when you do 100% risk mitigation, you are going to spend more time avoiding risk than it would take to mitigate something.

They are also executing Application Lifecycle Management (ALM), a step up from CI/CD. They look at all of the tools and processes along the lifecycyle, peer reviews, automated building, testing, etc. They are implementing automated processes with a variety of tools to achieve this.

As an example, at Fannie Mae, it previously took 5-6 weeks to request a library, now it is immediate because it is automated through Sonatype's Nexus.

barry2.png

As we hear from all practitioners, it is a journey, not a sprint (pun intended). Fannie Mae started at the end of 2014, and they are still in progress. They went from 9-18 months per deployment to a month on a consistent basis. They are close to always deploying during sprints, and are working towards having an idea, writing code, and releasing it into production.

Barry presented lessons learned:

Leverage what they had while shifting left:

    • Release Management Infrastructure (evolved into our CI/CD)
    • VM provisioning infrastructure (evolved into our RapidIT)
    • Test Automation
    • Code Quality
  • Recognize it is a culture shift in how work is performed:
    • Agile Adoption "You do this for the people, not do the people.""We started on the IT side, then the business side saw the value add."
    • Automate Oversight/Transparency - You can't do this manually. With 120 apps, there were 8,000 libraries not accounted for.
    • Portfolio Analysis (Risk Reduction, Quality, ROI, Cost Reduction)
    • Application Quality is a Supply Chain
    • Trust developers and get out of the way
  • Measure Results
    • Shifts in quality
    • Adoption rate
    • Accelerated delivery
    • Benchmark productivity and associated productivity cost

You can watch Barry's entire talk online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year's event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It's all free and online on October 24th.

More Stories By Derek Weeks

In 2015, Derek Weeks led the largest and most comprehensive analysis of software supply chain practices to date across 160,000 development organizations. He is a huge advocate of applying proven supply chain management principles into DevOps practices to improve efficiencies, reduce costs, and sustain long-lasting competitive advantages.

As a 20+ year veteran of the software industry, he has advised leading businesses on IT performance improvement practices covering continuous delivery, business process management, systems and network operations, service management, capacity planning and storage management. As the VP and DevOps Advocate for Sonatype, he is passionate about changing the way people think about software supply chains and improving public safety through improved software integrity. Follow him here @weekstweets, find me here www.linkedin.com/in/derekeweeks, and read me here http://blog.sonatype.com/author/weeks/.

Latest Stories
Because Linkerd is a transparent proxy that runs alongside your application, there are no code changes required. It even comes with Prometheus to store the metrics for you and pre-built Grafana dashboards to show exactly what is important for your services - success rate, latency, and throughput. In this session, we'll explain what Linkerd provides for you, demo the installation of Linkerd on Kubernetes and debug a real world problem. We will also dig into what functionality you can build on ...
The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get tailored market studies; and more.
Most organizations are awash today in data and IT systems, yet they're still struggling mightily to use these invaluable assets to meet the rising demand for new digital solutions and customer experiences that drive innovation and growth. What's lacking are potent and effective ways to rapidly combine together on-premises IT and the numerous commercial clouds that the average organization has in place today into effective new business solutions. New research shows that delivering on multicloud e...
Isomorphic Software is the global leader in high-end, web-based business applications. We develop, market, and support the SmartClient & Smart GWT HTML5/Ajax platform, combining the productivity and performance of traditional desktop software with the simplicity and reach of the open web. With staff in 10 timezones, Isomorphic provides a global network of services related to our technology, with offerings ranging from turnkey application development to SLA-backed enterprise support. Leadin...
Take advantage of autoscaling, and high availability for Kubernetes with no worry about infrastructure. Be the Rockstar and avoid all the hurdles of deploying Kubernetes. So Why not take Heat and automate the setup of your Kubernetes cluster? Why not give project owners a Heat Stack to deploy Kubernetes whenever they want to? Hoping to share how anyone can use Heat to deploy Kubernetes on OpenStack and customize to their liking. This is a tried and true method that I've used on my OpenSta...
DevOps is a world surrounded by information, starting from a single commit and ending in roll out to production. In this talk, I'll introduce you to the world of Taboola DevOps data collection, to better understand what goes on under the hood. The system we've developed in-house helps us collect and analyse the entire DevOps process from the very first commit all the way to production. It provides us a full clear view with a drill-down toolset that helps keep us away from the dark side. ...
We at Capgemini have developed a cloud-native PaaS Solution called "Apollo". Apollo is built on top of following open source components. - Apache Mesos for cluster management, scheduling & resource isolation - Marathon or Kubernetes for Container orchestration - Docker for application container runtime, - Consul for service discovery via DNS - Weave for networking of Docker Containers - Traefik for application container load balancing
After years of investments and acquisitions, CloudBlue was created with the goal of building the world's only hyperscale digital platform with an increasingly infinite ecosystem and proven go-to-market services. The result? An unmatched platform that helps customers streamline cloud operations, save time and money, and revolutionize their businesses overnight. Today, the platform operates in more than 45 countries and powers more than 200 of the world's largest cloud marketplaces, managing mo...
With digital video content creation going viral and assuming the bulk of Internet traffic, how can the deluge of video content be analyzed effectively to derive insights and ROI? After all, video is not only huge in size, but it is complex given various visual, audio and temporal elements. Video summarization (a mechanism for generating a short video summary via key frame analysis or video skimming) has become a popular research topic industry-wide and across academia. Video thumbnail generation...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
As you know, enterprise IT conversation over the past year have often centered upon the open-source Kubernetes container orchestration system. In fact, Kubernetes has emerged as the key technology -- and even primary platform -- of cloud migrations for a wide variety of organizations. Kubernetes is critical to forward-looking enterprises that continue to push their IT infrastructures toward maximum functionality, scalability, and flexibility.
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio a...
Containerized software is riding a wave of growth, according to latest RightScale survey. At Sematext we see this growth trend via our Docker monitoring adoption and via Sematext Docker Agent popularity on Docker Hub, where it crossed 1M+ pulls line. This rapid rise of containers now makes Docker the top DevOps tool among those included in RightScale survey. Overall Docker adoption surged to 35 percent, while Kubernetes adoption doubled, going from 7% in 2016 to 14% percent.
Technology has changed tremendously in the last 20 years. From onion architectures to APIs to microservices to cloud and containers, the technology artifacts shipped by teams has changed. And that's not all - roles have changed too. Functional silos have been replaced by cross-functional teams, the skill sets people need to have has been redefined and the tools and approaches for how software is developed and delivered has transformed. When we move from highly defined rigid roles and systems to ...
Even if your IT and support staff are well versed in agility and cloud technologies, it can be an uphill battle to establish a DevOps style culture - one where continuous improvement of both products and service delivery is expected and respected and all departments work together throughout a client or service engagement. As a service-oriented provider of cloud and data center technology, Green House Data sought to create more of a culture of innovation and continuous improvement, from our helpd...