SYS-CON MEDIA Authors: Yeshim Deniz, Elizabeth White, Pat Romanski, Liz McMillan, William Schmarzo

Related Topics: Containers Expo Blog

Containers Expo Blog: Opinion

Not All Clusters Are the Same

Chief Architect Shares His Experience with Building Scalable Storage and Database Solutions

Aaron Passey, CTO of Clustrix, is sharing his expertise as an architect behind both the Isilon and Clustrix solutions. Isilon is now a public storage company; Clustrix launched this summer and is already solving database scalability problems in worldwide customer environments today. Check out the Clustrix blog for more posts from Aaron: http://www.clustrix.com/resources/blog/.

Industry Opinion from Aaron Passey, Clustrix CTO
There are a wide variety of techniques out there for clustering storage appliances. The question is: what problem are you really trying to solve? If you look at Isilon's clustered storage appliances (where I was chief architect), you'll see that the clustering is done at the block level. The block addresses are generalized into a generic (node, drive, block_num) tuple and the on-disk data structures simply use that generalized address everywhere a block address would normally be used. The communication on the back end of an Isilon cluster is block reads and writes, transaction messages, and lock messages (plus some other miscellaneous bits). Each read or write operation is controlled by the initiator, and the smallest granularity of locking is at the block level. Cache lives both at the disk and at the initiator. If you were to put it into an architecture category, you'd call it an Infiniband SAN (Storage Area Network). This is perfect for a file system. This architecture lends itself to zero-copy, extremely high-performance file access for streaming files, very low CPU utilization on the nodes holding the disks (which allows the addition of the accelerator nodes for high speed FibreChannel and 10 GbE), infinite scalability, and extremely low latency for operations on cached data.

However, it doesn't support high read/write concurrency on a single file. Imagine if you ran an OLTP database with a high write load using an architecture like that. With the locking done at the block level, you can never expect to get high concurrency for items smaller than a block. Every node that wants to write to a block would have to get an exclusive lock on that block, which invalidates other nodes' caches. If you had an active table with massive read/write load sitting on top of a cluster like this, performance would tank, dominated by lock contention. Then why do some databases take this approach to scale? How can you possibly make a shared-backend cluster resembling a SAN and expect it to scale with a database workload like some have done? How can you make an expandable storage engine plug-in and expect the entire database to scale? What works extremely well for a file system does not work at all for a database. We need a new approach.

Clustrix has a new approach. Rather than shipping the data blocks on the back end, we ship the queries. That may sound like an innocuous statement, but really it has a far-reaching impact on the architecture. I am pleased to say that our approach solves the clustered database problem, resulting in a database system that can handle high concurrency at any scale. Check out www.clustrix.com if you don't believe me.

More Stories By Georgiana Comsa

Georgiana Comsa is the founder of Silicon Valley PR, a PR agency with a unique focus on the data infrastructure markets. Georgiana's decision to found Silicon Valley PR was based on her own experience as a corporate PR professional working with other PR agencies; she noticed that there was a need for a specialized, rather than a general tech PR firm, with media, analyst, and vendor relationships that would benefit its clients. With Silicon Valley PR, companies get to leverage the power of traditional and digital media relations to generate highly targeted press coverage, contributing to tangible business wins, which help them launch and grow their businesses.

Latest Stories
Eric Taylor, a former hacker, reveals what he's learned about cybersecurity. Taylor's life as a hacker began when he was just 12 years old and playing video games at home. Russian hackers are notorious for their hacking skills, but one American says he hacked a Russian cyber gang at just 15 years old. The government eventually caught up with Taylor and he pleaded guilty to posting the personal information on the internet, among other charges. Eric Taylor, who went by the nickname Cosmo...
Most modern computer languages embed a lot of metadata in their application. We show how this goldmine of data from a runtime environment like production or staging can be used to increase profits. Adi conceptualized the Crosscode platform after spending over 25 years working for large enterprise companies like HP, Cisco, IBM, UHG and personally experiencing the challenges that prevent companies from quickly making changes to their technology, due to the complexity of their enterprise. An accomp...
The benefits of automated cloud deployments for speed, reliability and security are undeniable. The cornerstone of this approach, immutable deployment, promotes the idea of continuously rolling safe, stable images instead of trying to keep up with managing a fixed pool of virtual or physical machines. In this talk, we'll explore the immutable infrastructure pattern and how to use continuous deployment and continuous integration (CI/CD) process to build and manage server images for any platfo...
DevOpsSUMMIT at CloudEXPO, to be held June 25-26, 2019 at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Am...
Nicolas Fierro is CEO of MIMIR Blockchain Solutions. He is a programmer, technologist, and operations dev who has worked with Ethereum and blockchain since 2014. His knowledge in blockchain dates to when he performed dev ops services to the Ethereum Foundation as one the privileged few developers to work with the original core team in Switzerland.
It cannot be overseen or regulated by any one administrator, like a government or bank. Currently, there is no government regulation on them which also means there is no government safeguards over them. Although many are looking at Bitcoin to put money into, it would be wise to proceed with caution. Regular central banks are watching it and deciding whether or not to make them illegal (Criminalize them) and therefore make them worthless and eliminate them as competition. ICOs (Initial Coin Offer...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
Sanjeev Sharma Joins November 11-13, 2018 @DevOpsSummit at @CloudEXPO New York Faculty. Sanjeev Sharma is an internationally known DevOps and Cloud Transformation thought leader, technology executive, and author. Sanjeev's industry experience includes tenures as CTO, Technical Sales leader, and Cloud Architect leader. As an IBM Distinguished Engineer, Sanjeev is recognized at the highest levels of IBM's core of technical leaders.
We are in a digital age however when one looks for their dream home, the mortgage process can take as long as 60 days to complete. Not what we expect in a time where processes are known to take place swiftly and seamlessly. Mortgages businesses are facing the heat and are in immediate need of upgrading their operating model to reduce costs, decrease the processing time and enhance the customer experience. Therefore, providers are exploring multiple ways of tapping emerging technologies to solve ...
If a machine can invent, does this mean the end of the patent system as we know it? The patent system, both in the US and Europe, allows companies to protect their inventions and helps foster innovation. However, Artificial Intelligence (AI) could be set to disrupt the patent system as we know it. This talk will examine how AI may change the patent landscape in the years to come. Furthermore, ways in which companies can best protect their AI related inventions will be examined from both a US and...
This session describes how Professional Services organisations can deliver within Technology-as-a-Service (IaaS) constructs, in private and public enterprise cloud scenarios. See how professional services can be packaged and funded by IaaS cash flows, based upon consumption of technology services. Learn how significant, IT infrastructure transformations can be funded through OPEX spending models with multi-year As-a-Services based contracts. Understand how the automation of repeatable services c...
"Calligo is a cloud service provider with data privacy at the heart of what we do. We are a typical Infrastructure as a Service cloud provider but it's been designed around data privacy," explained Julian Box, CEO and co-founder of Calligo, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
DXWorldEXPO LLC announced today that the upcoming DXWorldEXPO | DevOpsSUMMIT | CloudEXPO New York will feature 10 companies from Poland to participate at the "Poland Digital Transformation Pavilion" on November 12-13, 2018. Polish Digital Transformation companies which will exhibit at CloudEXPO | DevOpsSUMMIT | DXWorldEXPO include All in Mobile, dhosting, Cryptomage, Perfect Gym, Polcom, Apius Technologies, Aplisens, ELZAB SA, TELDAT, and Rebug.io.
Enterprises are universally struggling to understand where the new tools and methodologies of DevOps fit into their organizations, and are universally making the same mistakes. These mistakes are not unavoidable, and in fact, avoiding them gifts an organization with sustained competitive advantage, just like it did for Japanese Manufacturing Post WWII.