SYS-CON MEDIA Authors: Zakia Bouachraoui, Liz McMillan, Yeshim Deniz, Elizabeth White, William Schmarzo

Related Topics: @DXWorldExpo, Open Source Cloud, @CloudExpo

@DXWorldExpo: Article

Cloudant Merges BigCouch into CouchDB

DBaaS pioneer contributes database scalability and fault-tolerance framework to Apache CouchDB

"There are a lot of reasons people love CouchDB," said Adam Kocoloski, co-founder and CTO at Cloudant, as he announced last week that Cloudant had delivered on its promise to integrate core capabilities of its distributed database service to the open source Apache CouchDB project, "like its elegant programming model, data durability, flexible indexing, and, most of all, its unique way of replicating and synching data across data centers or devices."

CouchDB serves as the foundation of Cloudant's technology stack in the form of BigCouch, an open source variant of CouchDB that the company developed to support large-scale, globally distributed applications. After four years of operating BigCouch in production, Cloudant has merged the BigCouch code into the CouchDB codebase, making it possible to manage and replicate data with CouchDB at much larger scale.


Cloudant's Sam Bisbee featured in this recent Big Data Power Panel at 3rd Big Data Expo

Kocoloski continued:

"We're merging the horizontal scaling and fault-tolerance framework we built for BigCouch into CouchDB so people can more easily scale all that CouchDB goodness across multiple servers and keep it running nonstop. It's our way of saying thanks and helping to grow the community of CouchDB developers and users."

The open source BigCouch database project was developed in 2008 by the Cloudant co-founders, who had previously been using CouchDB for managing and distributing the petabytes of data generated every second by CERN's Large Hadron Collider. They developed a horizontal clustering and fault-tolerance framework for BigCouch that was inspired by the Amazon Dynamo research paper.

For the code merger, Cloudant engineers imported sections of BigCouch code into the Apache CouchDB repositories, adapting the database to run in a clustered environment and to better replicate databases across clusters and between data centers. Going forward, Cloudant will cease development of BigCouch, in order to participate in the CouchDB community and keep CouchDB and Cloudant clustering functionality in sync. Cloudant engineers will continue to make cluster-scaling and fault-tolerance enhancements within the CouchDB project and will reuse that code in Cloudant's database service.

"The code merger of BigCouch and Apache CouchDB is good for the open source community and developers that require a scalable Web-aware database," said Travell Perkins, CTO at Fidelity Investments. "As a classically trained computer scientist, I'm interested in the inner workings of my database solutions as much as the practical utility they provide dynamic data and use cases. I've tried a lot of NoSQL solutions over the years with varying degrees of success. After working with the distributed clustering capabilities being built into CouchDB, I think we are approaching the ideal JSON-centric database for enterprise workloads at scale."

"We're continuing work within the Apache project to integrate the clustering technology of BigCouch, but now we've set the stage and are welcoming more project committers to get involved," said Jan Lehnardt, Project Management Committee chair of the Apache CouchDB project. "Cloudant's work fine-tuning BigCouch database replication at large scale now gives Apache CouchDB a complete strategy for replicating data across distributed systems, whether nodes are Erlang clusters in the same data center or on the other side of the world. Developers have more options for moving data closer to their users and a simpler strategy for synchronizing that data throughout a larger system."

The key accomplishment of the merged code, according to Cloudant, is the BigCouch clustering capability. Among other improvements to Apache CouchDB, Cloudant has contributed a new compactor process that creates smaller and better-organized post-compaction databases. CouchDB users can now experience significant improvements in compaction and replication speed, as well as boosts in high-concurrency access performance. Additional improvements include: better index update speeds, updated aggregate reduce functions, smooth hot-code updates, improved logging, and streamlined libraries. Cloudant engineers also refactored internal code, removing complicated sections and boosting overall performance.

A preview of the merged software is available now, and a general release of CouchDB with the merged BigCouch functionality is targeted to be available following the Apache community release process.

More Stories By Jeremy Geelan

Jeremy Geelan is Chairman & CEO of the 21st Century Internet Group, Inc. and an Executive Academy Member of the International Academy of Digital Arts & Sciences. Formerly he was President & COO at Cloud Expo, Inc. and Conference Chair of the worldwide Cloud Expo series. He appears regularly at conferences and trade shows, speaking to technology audiences across six continents. You can follow him on twitter: @jg21.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Latest Stories
Organize your corporate travel faster, at lower cost. Hotailors is a next-gen AI-powered travel platform. What is Hotailors? Hotailors is a platform for organising business travels that grants access to the best real-time offers from 2.000.000+ hotels and 700+ airlines in the whole world. Thanks to our solution you can plan, book & expense business trips in less than 5 minutes. Accordingly to your travel policy, budget limits and cashless for your employees. With our reporting, int...
Nicolas Fierro is CEO of MIMIR Blockchain Solutions. He is a programmer, technologist, and operations dev who has worked with Ethereum and blockchain since 2014. His knowledge in blockchain dates to when he performed dev ops services to the Ethereum Foundation as one the privileged few developers to work with the original core team in Switzerland.
Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes. We are offering early bird savings...
DXWorldEXPO LLC announced today that the upcoming DXWorldEXPO | DevOpsSUMMIT | CloudEXPO New York will feature 10 companies from Poland to participate at the "Poland Digital Transformation Pavilion" on November 12-13, 2018. Polish Digital Transformation companies which will exhibit at CloudEXPO | DevOpsSUMMIT | DXWorldEXPO include All in Mobile, dhosting, Cryptomage, Perfect Gym, Polcom, Apius Technologies, Aplisens, ELZAB SA, TELDAT, and Rebug.io.
This session describes how Professional Services organisations can deliver within Technology-as-a-Service (IaaS) constructs, in private and public enterprise cloud scenarios. See how professional services can be packaged and funded by IaaS cash flows, based upon consumption of technology services. Learn how significant, IT infrastructure transformations can be funded through OPEX spending models with multi-year As-a-Services based contracts. Understand how the automation of repeatable services c...
Enterprises are universally struggling to understand where the new tools and methodologies of DevOps fit into their organizations, and are universally making the same mistakes. These mistakes are not unavoidable, and in fact, avoiding them gifts an organization with sustained competitive advantage, just like it did for Japanese Manufacturing Post WWII.
Whenever a new technology hits the high points of hype, everyone starts talking about it like it will solve all their business problems. Blockchain is one of those technologies. According to Gartner's latest report on the hype cycle of emerging technologies, blockchain has just passed the peak of their hype cycle curve. If you read the news articles about it, one would think it has taken over the technology world. No disruptive technology is without its challenges and potential impediments t...
In his session at 21st Cloud Expo, Michael Burley, a Senior Business Development Executive in IT Services at NetApp, described how NetApp designed a three-year program of work to migrate 25PB of a major telco's enterprise data to a new STaaS platform, and then secured a long-term contract to manage and operate the platform. This significant program blended the best of NetApp’s solutions and services capabilities to enable this telco’s successful adoption of private cloud storage and launching o...
The digital transformation is real! To adapt, IT professionals need to transform their own skillset to become more multi-dimensional by gaining both depth and breadth of a wide variety of knowledge and competencies. Historically, while IT has been built on a foundation of specialty (or "I" shaped) silos, the DevOps principle of "shifting left" is opening up opportunities for developers, operational staff, security and others to grow their skills portfolio, advance their careers and become "T"-sh...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Despite being the market leader, we recognized the need to transform and reinvent our business at Dynatrace, before someone else disrupted the market. Over the course of three years, we changed everything - our technology, our culture and our brand image. In this session we'll discuss how we navigated through our own innovator's dilemma, and share takeaways from our experience that you can apply to your own organization.
Having been in the web hosting industry since 2002, dhosting has gained a great deal of experience while working on a wide range of projects. This experience has enabled the company to develop our amazing new product, which they are now excited to present! Among dHosting's greatest achievements, they can include the development of their own hosting panel, the building of their fully redundant server system, and the creation of dhHosting's unique product, Dynamic Edge.
Cloud Storage 2.0 has brought many innovations, including the availability of cloud storage services that are less expensive and much faster than previous generations of cloud storage. Cloud Storage 2.0 has also delivered new and faster methods for migrating your premises storage environment to the cloud and the concept of multi-cloud. This session will provide technical details on Cloud Storage 2.0 and the methods used to efficiently migrate from premises-to-cloud storage. This session will als...
DXWorldEXPO LLC announced today that Nutanix has been named "Platinum Sponsor" of CloudEXPO | DevOpsSUMMIT | DXWorldEXPO New York, which will take place November 12-13, 2018 in New York City. Nutanix makes infrastructure invisible, elevating IT to focus on the applications and services that power their business. The Nutanix Enterprise Cloud Platform blends web-scale engineering and consumer-grade design to natively converge server, storage, virtualization and networking into a resilient, softwar...
The Transparent Cloud-computing Consortium (T-Cloud) is a neutral organization for researching new computing models and business opportunities in IoT era. In his session, Ikuo Nakagawa, Co-Founder and Board Member at Transparent Cloud Computing Consortium, will introduce the big change toward the "connected-economy" in the digital age. He'll introduce and describe some leading-edge business cases from his original points of view, and discuss models & strategies in the connected-economy. Nowad...