SYS-CON MEDIA Authors: Yeshim Deniz, Elizabeth White, Pat Romanski, Liz McMillan, Courtney Abud

Related Topics: @DXWorldExpo, Open Source Cloud, @CloudExpo

@DXWorldExpo: Article

Cloudant Merges BigCouch into CouchDB

DBaaS pioneer contributes database scalability and fault-tolerance framework to Apache CouchDB

"There are a lot of reasons people love CouchDB," said Adam Kocoloski, co-founder and CTO at Cloudant, as he announced last week that Cloudant had delivered on its promise to integrate core capabilities of its distributed database service to the open source Apache CouchDB project, "like its elegant programming model, data durability, flexible indexing, and, most of all, its unique way of replicating and synching data across data centers or devices."

CouchDB serves as the foundation of Cloudant's technology stack in the form of BigCouch, an open source variant of CouchDB that the company developed to support large-scale, globally distributed applications. After four years of operating BigCouch in production, Cloudant has merged the BigCouch code into the CouchDB codebase, making it possible to manage and replicate data with CouchDB at much larger scale.


Cloudant's Sam Bisbee featured in this recent Big Data Power Panel at 3rd Big Data Expo

Kocoloski continued:

"We're merging the horizontal scaling and fault-tolerance framework we built for BigCouch into CouchDB so people can more easily scale all that CouchDB goodness across multiple servers and keep it running nonstop. It's our way of saying thanks and helping to grow the community of CouchDB developers and users."

The open source BigCouch database project was developed in 2008 by the Cloudant co-founders, who had previously been using CouchDB for managing and distributing the petabytes of data generated every second by CERN's Large Hadron Collider. They developed a horizontal clustering and fault-tolerance framework for BigCouch that was inspired by the Amazon Dynamo research paper.

For the code merger, Cloudant engineers imported sections of BigCouch code into the Apache CouchDB repositories, adapting the database to run in a clustered environment and to better replicate databases across clusters and between data centers. Going forward, Cloudant will cease development of BigCouch, in order to participate in the CouchDB community and keep CouchDB and Cloudant clustering functionality in sync. Cloudant engineers will continue to make cluster-scaling and fault-tolerance enhancements within the CouchDB project and will reuse that code in Cloudant's database service.

"The code merger of BigCouch and Apache CouchDB is good for the open source community and developers that require a scalable Web-aware database," said Travell Perkins, CTO at Fidelity Investments. "As a classically trained computer scientist, I'm interested in the inner workings of my database solutions as much as the practical utility they provide dynamic data and use cases. I've tried a lot of NoSQL solutions over the years with varying degrees of success. After working with the distributed clustering capabilities being built into CouchDB, I think we are approaching the ideal JSON-centric database for enterprise workloads at scale."

"We're continuing work within the Apache project to integrate the clustering technology of BigCouch, but now we've set the stage and are welcoming more project committers to get involved," said Jan Lehnardt, Project Management Committee chair of the Apache CouchDB project. "Cloudant's work fine-tuning BigCouch database replication at large scale now gives Apache CouchDB a complete strategy for replicating data across distributed systems, whether nodes are Erlang clusters in the same data center or on the other side of the world. Developers have more options for moving data closer to their users and a simpler strategy for synchronizing that data throughout a larger system."

The key accomplishment of the merged code, according to Cloudant, is the BigCouch clustering capability. Among other improvements to Apache CouchDB, Cloudant has contributed a new compactor process that creates smaller and better-organized post-compaction databases. CouchDB users can now experience significant improvements in compaction and replication speed, as well as boosts in high-concurrency access performance. Additional improvements include: better index update speeds, updated aggregate reduce functions, smooth hot-code updates, improved logging, and streamlined libraries. Cloudant engineers also refactored internal code, removing complicated sections and boosting overall performance.

A preview of the merged software is available now, and a general release of CouchDB with the merged BigCouch functionality is targeted to be available following the Apache community release process.

More Stories By Jeremy Geelan

Jeremy Geelan is Chairman & CEO of the 21st Century Internet Group, Inc. and an Executive Academy Member of the International Academy of Digital Arts & Sciences. Formerly he was President & COO at Cloud Expo, Inc. and Conference Chair of the worldwide Cloud Expo series. He appears regularly at conferences and trade shows, speaking to technology audiences across six continents. You can follow him on twitter: @jg21.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Latest Stories
GCP Marketplace is based on a multi-cloud and hybrid-first philosophy, focused on giving Google Cloud partners and enterprise customers flexibility without lock-in. It also helps customers innovate by easily adopting new technologies from ISV partners, such as commercial Kubernetes applications, and allows companies to oversee the full lifecycle of a solution, from discovery through management.
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, discussed why containers should be paired with new architectural practices such as microservices rathe...
Using serverless computing has a number of obvious benefits over traditional application infrastructure - you pay only for what you use, scale up or down immediately to match supply with demand, and avoid operating any server infrastructure at all. However, implementing maintainable and scalable applications using serverless computing services like AWS Lambda poses a number of challenges. The absence of long-lived, user-managed servers means that states cannot be maintained by the service. Lo...
In 2014, Amazon announced a new form of compute called Lambda. We didn't know it at the time, but this represented a fundamental shift in what we expect from cloud computing. Now, all of the major cloud computing vendors want to take part in this disruptive technology. In his session at 20th Cloud Expo, John Jelinek IV, a web developer at Linux Academy, will discuss why major players like AWS, Microsoft Azure, IBM Bluemix, and Google Cloud Platform are all trying to sidestep VMs and containers...
Here to help unpack insights into the new era of using containers to gain ease with multi-cloud deployments are our panelists: Matt Baldwin, Founder and CEO at StackPointCloud, based in Seattle; Nic Jackson, Developer Advocate at HashiCorp, based in San Francisco, and Reynold Harbin, Director of Product Marketing at DigitalOcean, based in New York. The discussion is moderated by Dana Gardner, principal analyst at Interarbor Solutions.
Using serverless computing has a number of obvious benefits over traditional application infrastructure - you pay only for what you use, scale up or down immediately to match supply with demand, and avoid operating any server infrastructure at all. However, implementing maintainable and scalable applications using serverless computing services like AWS Lambda poses a number of challenges. The absence of long-lived, user-managed servers means that states cannot be maintained by the service. Lo...
With the rise of Docker, Kubernetes, and other container technologies, the growth of microservices has skyrocketed among dev teams looking to innovate on a faster release cycle. This has enabled teams to finally realize their DevOps goals to ship and iterate quickly in a continuous delivery model. Why containers are growing in popularity is no surprise — they’re extremely easy to spin up or down, but come with an unforeseen issue. However, without the right foresight, DevOps and IT teams may lo...
Isomorphic Software is the global leader in high-end, web-based business applications. We develop, market, and support the SmartClient & Smart GWT HTML5/Ajax platform, combining the productivity and performance of traditional desktop software with the simplicity and reach of the open web. With staff in 10 timezones, Isomorphic provides a global network of services related to our technology, with offerings ranging from turnkey application development to SLA-backed enterprise support. Leadin...
Take advantage of autoscaling, and high availability for Kubernetes with no worry about infrastructure. Be the Rockstar and avoid all the hurdles of deploying Kubernetes. So Why not take Heat and automate the setup of your Kubernetes cluster? Why not give project owners a Heat Stack to deploy Kubernetes whenever they want to? Hoping to share how anyone can use Heat to deploy Kubernetes on OpenStack and customize to their liking. This is a tried and true method that I've used on my OpenSta...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Kubernetes is a new and revolutionary open-sourced system for managing containers across multiple hosts in a cluster. Ansible is a simple IT automation tool for just about any requirement for reproducible environments. In his session at @DevOpsSummit at 18th Cloud Expo, Patrick Galbraith, a principal engineer at HPE, will discuss how to build a fully functional Kubernetes cluster on a number of virtual machines or bare-metal hosts. Also included will be a brief demonstration of running a Galer...
10ZiG Technology is a leading provider of endpoints for a Virtual Desktop Infrastructure environment. Our fast and reliable hardware is VMware, Citrix and Microsoft ready and designed to handle all ranges of usage - from task-based to sophisticated CAD/CAM users. 10ZiG prides itself in being one of the only companies whose sole focus is in Thin Clients and Zero Clients for VDI. This focus allows us to provide a truly unique level of personal service and customization that is a rare find in th...
Emil Sayegh is an early pioneer of cloud computing and is recognized as one of the industry's true veterans. A cloud visionary, he is credited with launching and leading the cloud computing and hosting businesses for HP, Rackspace, and Codero. Emil built the Rackspace cloud business while serving as the company's GM of the Cloud Computing Division. Earlier at Rackspace he served as VP of the Product Group and launched the company's private cloud and hosted exchange services. He later moved o...
92% of enterprises are using the public cloud today. As a result, simply being in the cloud is no longer enough to remain competitive. The benefit of reduced costs has normalized while the market forces are demanding more innovation at faster release cycles. Enter Cloud Native! Cloud Native enables a microservices driven architecture. The shift from monolithic to microservices yields a lot of benefits - but if not done right - can quickly outweigh the benefits. The effort required in monitoring,...
As you know, enterprise IT conversation over the past year have often centered upon the open-source Kubernetes container orchestration system. In fact, Kubernetes has emerged as the key technology -- and even primary platform -- of cloud migrations for a wide variety of organizations. Kubernetes is critical to forward-looking enterprises that continue to push their IT infrastructures toward maximum functionality, scalability, and flexibility. As they do so, IT professionals are also embr...