SYS-CON MEDIA Authors: Liz McMillan, Elizabeth White, Pat Romanski, Jason Bloomberg, Dana Gardner

Related Topics: Containers Expo Blog

Containers Expo Blog: Opinion

Not All Clusters Are the Same

Chief Architect Shares His Experience with Building Scalable Storage and Database Solutions

Aaron Passey, CTO of Clustrix, is sharing his expertise as an architect behind both the Isilon and Clustrix solutions. Isilon is now a public storage company; Clustrix launched this summer and is already solving database scalability problems in worldwide customer environments today. Check out the Clustrix blog for more posts from Aaron: http://www.clustrix.com/resources/blog/.

Industry Opinion from Aaron Passey, Clustrix CTO
There are a wide variety of techniques out there for clustering storage appliances. The question is: what problem are you really trying to solve? If you look at Isilon's clustered storage appliances (where I was chief architect), you'll see that the clustering is done at the block level. The block addresses are generalized into a generic (node, drive, block_num) tuple and the on-disk data structures simply use that generalized address everywhere a block address would normally be used. The communication on the back end of an Isilon cluster is block reads and writes, transaction messages, and lock messages (plus some other miscellaneous bits). Each read or write operation is controlled by the initiator, and the smallest granularity of locking is at the block level. Cache lives both at the disk and at the initiator. If you were to put it into an architecture category, you'd call it an Infiniband SAN (Storage Area Network). This is perfect for a file system. This architecture lends itself to zero-copy, extremely high-performance file access for streaming files, very low CPU utilization on the nodes holding the disks (which allows the addition of the accelerator nodes for high speed FibreChannel and 10 GbE), infinite scalability, and extremely low latency for operations on cached data.

However, it doesn't support high read/write concurrency on a single file. Imagine if you ran an OLTP database with a high write load using an architecture like that. With the locking done at the block level, you can never expect to get high concurrency for items smaller than a block. Every node that wants to write to a block would have to get an exclusive lock on that block, which invalidates other nodes' caches. If you had an active table with massive read/write load sitting on top of a cluster like this, performance would tank, dominated by lock contention. Then why do some databases take this approach to scale? How can you possibly make a shared-backend cluster resembling a SAN and expect it to scale with a database workload like some have done? How can you make an expandable storage engine plug-in and expect the entire database to scale? What works extremely well for a file system does not work at all for a database. We need a new approach.

Clustrix has a new approach. Rather than shipping the data blocks on the back end, we ship the queries. That may sound like an innocuous statement, but really it has a far-reaching impact on the architecture. I am pleased to say that our approach solves the clustered database problem, resulting in a database system that can handle high concurrency at any scale. Check out www.clustrix.com if you don't believe me.

More Stories By Georgiana Comsa

Georgiana Comsa is the founder of Silicon Valley PR, a PR agency with a unique focus on the data infrastructure markets. Georgiana's decision to found Silicon Valley PR was based on her own experience as a corporate PR professional working with other PR agencies; she noticed that there was a need for a specialized, rather than a general tech PR firm, with media, analyst, and vendor relationships that would benefit its clients. With Silicon Valley PR, companies get to leverage the power of traditional and digital media relations to generate highly targeted press coverage, contributing to tangible business wins, which help them launch and grow their businesses.

Latest Stories
Early Bird Registration Discount Expires on August 31, 2018 Conference Registration Link ▸ HERE. Pick from all 200 sessions in all 10 tracks, plus 22 Keynotes & General Sessions! Lunch is served two days. EXPIRES AUGUST 31, 2018. Ticket prices: ($1,295-Aug 31) ($1,495-Oct 31) ($1,995-Nov 12) ($2,500-Walk-in)
DevOpsSUMMIT at CloudEXPO will expand the DevOps community, enable a wide sharing of knowledge, and educate delegates and technology providers alike. Recent research has shown that DevOps dramatically reduces development time, the amount of enterprise IT professionals put out fires, and support time generally. Time spent on infrastructure development is significantly increased, and DevOps practitioners report more software releases and higher quality. Sponsors of DevOpsSUMMIT at CloudEXPO will b...
Yottabyte is a software-defined data center (SDDC) company headquartered in Bloomfield Township, Oakland County, Michigan. The mission of Yottabyte is to enable any organization to easily and economically realize the benefits of the cloud in their own datacenter. The Yottabyte Cloud Software natively virtualizes compute, storage, and network resources into a resilient, automated software-defined cloud infrastructure solution. Both Enterprise IT customers and Service Providers benefit from Yottab...
FinTech Is Now Part of the CloudEXPO New York Program. Financial enterprises in New York City, London, Singapore, and other world financial capitals are embracing a new generation of smart, automated FinTech that eliminates many cumbersome, slow, and expensive intermediate processes from their businesses. Accordingly, attendees at the upcoming 22nd CloudEXPO | DXWorldEXPO November 12-13, 2018 in New York City will find fresh new content in two new tracks called: FinTechEXPO New York Blockchain E...
Only Adobe gives everyone - from emerging artists to global brands - everything they need to design and deliver exceptional digital experiences. Adobe Systems Incorporated develops, markets, and supports computer software products and technologies. The Company's products allow users to express and use information across all print and electronic media. The Company's Digital Media segment provides tools and solutions that enable individuals, small and medium businesses and enterprises to cre...
In addition to 22 Keynotes and General Sessions, pick from 40 technical sessions in two tracks: (1) DevOpsSUMMIT (2) Cloud-Native & Serverless. EXPIRES AUGUST 31, 2018. Ticket prices: ($295-Aug 31) ($395-Oct 31) ($495-Nov 12) ($595-Walk-in) Does NOT include lunch. DevOps Institue Certification DevOps Institute Two-Day DevOps Certification Program EXPIRES AUGUST 31, 2018. Ticket prices: ($995-Aug 31) ($1,095-Oct 31) ($1,195-Nov 12) ($1,395-Walk-in)
Despite being the market leader, we recognized the need to transform and reinvent our business at Dynatrace, before someone else disrupted the market. Over the course of three years, we changed everything - our technology, our culture and our brand image. In this session we'll discuss how we navigated through our own innovator's dilemma, and share takeaways from our experience that you can apply to your own organization.
Today we introduced our New York & Silicon Valley combo sponsorship and exhibit opportunities with unmatched pre and post-show promotion. At CloudEXPO | DevOpsSUMMIT | DXWorldEXPO NY & CA, Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune...
Cloud adoption is a core component of digital transformation. Scaling the IT environment, making it resilient, and reducing costs are what organizations want. Hear from the author of the best selling Packtbook "Architecting Cloud Computing Solutions" as he presents and explains critical Cloud solution design considerations and technology decisions required to choose and deploy the right Cloud service and deployment models, that are aligned to your business and technology service requirements. T...
The current environment of Continuous Disruption requires companies to transform how they work and how they engineer their products. Transformations are notoriously hard to execute, yet many companies have succeeded. What can we learn from them? Can we produce a blueprint for a transformation? This presentation will cover several distinct approaches that companies take to achieve transformation. Each approach utilizes different levers and comes with its own advantages, tradeoffs, costs, risks, a...
Blockchain has numerous revolutionary ambitions, but technology hasn’t evolved enough to make them practical. What is the best use of blockchain technology today? How are asset owners and managers looking at blockchain to transform ownership structures? How will blockchain technology allow global investors to access new markets? What kinds of companies will take advantage of blockchain technology as a more efficient way to raise capital?
Since their inception, Total Uptime has been delivering on their promise of unprecedented uptime to major brands as well as Fortune 500 corporations in the United States and around the world. Their Cloud network spans 17 countries and delivers the unique ability to seamlessly route Internet traffic from one site to the next when our customers need us to. It is built on the best SSAE 16 SOC 2 Type 2 and CSAE 3416 Datacenters and boasts an impressive line-up of Network and Transit Provider Redunda...
​Blockchain, also known as Distributed Ledger Technology (DLT), can be used for more than just crypto currencies. This presentation discusses how blockchain and the Internet of Things (IOT) can be used for several supply chain management and logistics use cases. Real world examples that utilize the open source project Hyperledger will be discussed.
This session describes how Professional Services organisations can deliver within Technology-as-a-Service (IaaS) constructs, in private and public enterprise cloud scenarios. See how professional services can be packaged and funded by IaaS cash flows, based upon consumption of technology services. Learn how significant, IT infrastructure transformations can be funded through OPEX spending models with multi-year As-a-Services based contracts. Understand how the automation of repeatable services c...
Gym Solutions is a software as a service (SaaS) solution purpose-built to service the fitness industry, with over 1000 clients servicing over 2 million gym members across 40 countries making Perfect Gym Solutions one of the largest and fastest growing SaaS applications in the fitness industry. Their platform is a comprehensive package of modern modular products for the management of fitness clubs, sports facilities and gyms an end- to end solution, revolutionising the way that gyms are managed. ...