SYS-CON MEDIA Authors: Pat Romanski, Liz McMillan, Zakia Bouachraoui, Elizabeth White, Yeshim Deniz

Article

MongoDB Write Concern: 3 Must-Know Caveats

In this post, we discuss 3 gotchas when using MongoDB write concern.

'Write concern' in MongoDB describes the level of write acknowledgment you can expect from it. It's a rather important setting to remember in your write operations and its behavior is useful to understand, especially in distributed MongoDB deployments (i.e. replica sets and sharded clusters). In this post, we discuss 3 gotchas when using MongoDB write concern.

MongoDB Write Concern

MongoDB's documentation defines write concern as "the level of acknowledgment requested from MongoDB for write operations to a standalone mongod or to replica sets or to sharded clusters."

Simply put, a write concern is an indication of 'durability' passed along with write operations to MongoDB. To clarify, let us look at the syntax:

{ w: <value>, j: <boolean>, wtimeout: <number> }
Where*,
 w can be an integer | "majority" | , it represents the number of members that must acknowledge the write. Default value is 1.
 j Requests that a write be acknowledged after it is written to the on-disk journal as opposed to just the system memory. Unspecified by default.
wtimeout specifies timeout for the applying the write concern. Unspecified by default.

* You can find the detailed syntax in the Write Concern Specification documentation.

* Learn more about the different "tags" you can use for common write concern values in our Understanding Durability & Write Safety in MongoDB blog.

Example:

db.inventory.insert(
    { sku: "abcdxyz", qty : 100, category: "Clothing" },
    { writeConcern: { w: 2, j: true, wtimeout: 5000 } }
)

The above insert's write concern can be read as follows:  acknowledge this write when 'at least 2 members of the replica set have written it to their journals within 5000 msecs or return an error'. A write concern value for option was majority, meaning "requests acknowledgment that write operations have propagated to the majority of voting nodes, including the primary."

The importance of write concern is apparent. Increasing values of w increases the latency of writes while also decreasing their probability of getting lost. Choosing the correct values for write concern depends on the latency and durability requirements of writes being performed.

With that as the background on what a write concern is, let's move on to the three caveats to remember when using write concern.

CAVEAT 1: Setting write concern on replica sets without a wtimeout can cause writes to block indefinitely

The majority definition (applicable MongoDB 3.0 onwards) above states that acknowledgment is requested from a majority of the "voting nodes". Note that "If you do not specify the wtimeout option and the level of write concern is unachievable, the write operation will block indefinitely. "

This can have unexpected consequences, for example, consider a 2+1 replica set (i.e. a primary, a secondary and an arbiter). If your sole read replica goes down, then all writes with a write concern w option of "majority" will block indefinitely.  The same will happen if the w option is set to 2. Another extreme example is in the case of a 3+2 replica set (primary, 2 secondaries and 2 arbiters, not a recommended configuration). All "majority" writes will block even if a single data node is unavailable as the majority number, in this case, is 3.

The simplest way to alleviate this issue is to always specify a wtimeout value so the query can timeout if the write concern can't be enforced. However, in case of such timeout errors, MongoDB doesn't undo already successful writes made to some of the members before the timeout occurred.

There is also currently no setting to ensure a write reaches the majority of nodes that are currently reachable, so be careful about setting the value of write concern w based on the topology, desired durability, and availability.

CAVEAT 2: You might lose data even with w: majority

It seems intuitive that once a write has been acknowledged by the majority of voting members, its durability is guaranteed. However, that isn't the case! Remember that when the j option is unspecified, a write is acknowledged right after it has been written to memory.

So, such a write can be lost if a freak power outage takes out the majority of the nodes to which the write had propagated (and before syncPeriodSecs i.e. before it could be flushed to disk).

In order to ensure the durability of writes, it's best not to turn off journaling on your database and set the j option to true. In fact, starting MongoDB 3.6, the --nojournal flag has been deprecated for replica set members using the WiredTiger storage engine.

With a w value of "majority" and the j option unspecified on a replica set, the exact durability behavior depends on the value of the replica set configuration writeConcernMajorityJournalDefault. When set to true (and when journaling is enabled), it acknowledges writes after they have been written to the journals of a majority of voting members.

Aside: Even with journaling turned on, your writes might still get lost on the MMAPv1 storage engine if an outage occurs within commitIntervalMs duration. The WiredTiger storage engine, on the other hand, forces a sync of journal files when it receives a write with j option set to true. And, even with j set to false, an acknowledged "majority" write to a latest WiredTiger based deployment can be lost only when majority of the data nodes crash simultaneously.

CAVEAT 3: w: 0 while setting j: true doesn't improve write performance

This is easy enough to reason once you think about it, but equally easy to forget. Setting w option to 0 is usually done to write to the database in a "fire-and-forget" fashion - when you have a fair amount of confidence on the database infrastructure and care more about latency than the durability of every write. However, if you set the j option to true, your w option will effectively be overridden as the database will ensure that the write is written to the on-disk journal before returning.

If you're using write concerns to guarantee the success of your write operations, make sure that you remember these three crucial caveats! We're here to help, so feel free to connect with any questions through Twitter or by email.

MongoDB Write Concern: 3 Must-Know Caveats

More Stories By Vaibhaw Pandey

Vaibhaw Pandey is a Software Developer with interests in Distributed Systems, Databases and Web-scale technologies.

Latest Stories
DXWorldEXPO LLC announced today that Nutanix has been named "Platinum Sponsor" of CloudEXPO | DevOpsSUMMIT | DXWorldEXPO New York, which will take place November 12-13, 2018 in New York City. Nutanix makes infrastructure invisible, elevating IT to focus on the applications and services that power their business. The Nutanix Enterprise Cloud Platform blends web-scale engineering and consumer-grade design to natively converge server, storage, virtualization and networking into a resilient, softwar...
Digital transformation is about embracing digital technologies into a company's culture to better connect with its customers, automate processes, create better tools, enter new markets, etc. Such a transformation requires continuous orchestration across teams and an environment based on open collaboration and daily experiments. In his session at 21st Cloud Expo, Alex Casalboni, Technical (Cloud) Evangelist at Cloud Academy, explored and discussed the most urgent unsolved challenges to achieve fu...
Wasabi is the hot cloud storage company delivering low-cost, fast, and reliable cloud storage. Wasabi is 80% cheaper and 6x faster than Amazon S3, with 100% data immutability protection and no data egress fees. Created by Carbonite co-founders and cloud storage pioneers David Friend and Jeff Flowers, Wasabi is on a mission to commoditize the storage industry. Wasabi is a privately held company based in Boston, MA. Follow and connect with Wasabi on Twitter, Facebook, Instagram and the Wasabi blog...
Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app secu...
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO Silicon Valley 2019 will cover all of these tools, with the most comprehensive program and with 222 rockstar speakers throughout our industry presenting 22 Keynotes and General Sessions, 250 Breakout Sessions along 10 Tracks, as well as our signature Power Panels. Our Expo Floor will bring together the leading global 200 companies throughout the world of Cloud Computing, DevOps, IoT, Smart Cities, FinTech, Digital Transformation, and all they entail. As ...
Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures. Offering private, hybrid, and public cloud solutions, Atmosera works closely with customers to engineer, deploy, and operate cloud architectures with advanced services that deliver strategic business outcomes. Atmosera's expertise simplifies the process of cloud transformation and our 20+ years of experience managing complex IT environments provides our customers with the confidence and trust tha...
Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes. We are offering early bird savings...
Only Adobe gives everyone - from emerging artists to global brands - everything they need to design and deliver exceptional digital experiences. Adobe Systems Incorporated develops, markets, and supports computer software products and technologies. The Company's products allow users to express and use information across all print and electronic media. The Company's Digital Media segment provides tools and solutions that enable individuals, small and medium businesses and enterprises to cre...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO Silicon Valley 2019 will cover all of these tools, with the most comprehensive program and with 222 rockstar speakers throughout our industry presenting 22 Keynotes and General Sessions, 250 Breakout Sessions along 10 Tracks, as well as our signature Power Panels. Our Expo Floor will bring together the leading global 200 companies throughout the world of Cloud Computing, DevOps, IoT, Smart Cities, FinTech, Digital Transformation, and all they entail.
Daniel Jones is CTO of EngineerBetter, helping enterprises deliver value faster. Previously he was an IT consultant, indie video games developer, head of web development in the finance sector, and an award-winning martial artist. Continuous Delivery makes it possible to exploit findings of cognitive psychology and neuroscience to increase the productivity and happiness of our teams.
Yottabyte is a software-defined data center (SDDC) company headquartered in Bloomfield Township, Oakland County, Michigan. The mission of Yottabyte is to enable any organization to easily and economically realize the benefits of the cloud in their own datacenter. The Yottabyte Cloud Software natively virtualizes compute, storage, and network resources into a resilient, automated software-defined cloud infrastructure solution. Both Enterprise IT customers and Service Providers benefit from Yottab...
Every organization is facing their own Digital Transformation as they attempt to stay ahead of the competition, or worse, just keep up. Each new opportunity, whether embracing machine learning, IoT, or a cloud migration, seems to bring new development, deployment, and management models. The results are more diverse and federated computing models than any time in our history. Unfortunately, there is no single deployment model that solves all of your business goals. You need to maintain the fl...
Eric Taylor, a former hacker, reveals what he's learned about cybersecurity. Taylor's life as a hacker began when he was just 12 years old and playing video games at home. Russian hackers are notorious for their hacking skills, but one American says he hacked a Russian cyber gang at just 15 years old. The government eventually caught up with Taylor and he pleaded guilty to posting the personal information on the internet, among other charges. Eric Taylor, who went by the nickname Cosmo...