SYS-CON MEDIA Authors: Zakia Bouachraoui, Elizabeth White, Pat Romanski, Liz McMillan, Yeshim Deniz

Blog Feed Post

My Top 10 Blogs of 2014

Bill Schmarzo

2014 was a good year for big data, and for some of my big data blogs.  This is my annual “My Top 10 Blogs” list.  Choosing 10 forces me to exclude some interesting topics, such as my Big Data MBA series from my teaching stint at the University of San Francisco.  Fortunately, my clients shared many real-world challenges so I had plenty of fuel for blogs.  Here’s to 2015 being an even bigger year for Big Data!

#10 (Tied) – Dear John Letter

A good friend of mine who took over a company that provides a statistics software package inspired this blog.  I have much admiration for my friend “John”, so I thought it only fair to share with him what challenges I thought he’d face when moving the software package forward, within a big data world.  Hopefully this is useful advice for any company looking to transition or build software for this brave new big data world.

Figure 1:  The Move to Actionable Analytics

Figure 1:  The Move to Actionable Analytics

#10 (Tied) – Waiting For Godot: Developing Competitive Differentiation

This blog really wasn’t about big data, but instead it challenged organizations to really understand where and how data, analytics and technology could be leveraged to provide business differentiation versus just business parity.  I really wanted the big data conversation to focus on business differentiation instead of what version of Hadoop the organization used.  Ugh!

Figure 2:  Examples of Software Products that Provide Business Differentiation

Figure 2:  Examples of Software Products that Provide Business Differentiation

#9 – Data Science – The More Actionable Data, the Better

Janet Yellen is the “Moneyball” queen of the Federal Reserve.  Janet Yellen caught my attention as an example of a leader who understands that one needs multiple perspectives (metrics) in order to thoroughly understand the state of the business.  Janet Yellen uses a dashboard of metrics to understand the job market and take decisive action.  Although more data is better than less data, it’s perhaps even more important to employ a range of metrics and scores that help you understand the situation and make better decisions.

Figure 3: Janet Yellen's Jobs Dashboard

Figure 3: Janet Yellen’s Jobs Dashboard

#8 – In the World of Big Data, Small Data Rules!!

This blog gave me a chance to revisit my childhood game of Strat-o-matic baseball.  This blog addresses the power of integrating “small” data – data captured via consumer comments, work orders, physician or mechanic notes, etc. – with the “big” data from your operational systems.  There are many opportunities for organizations to ask that one extra question – to capture the “small data” – that, when combined with the organization’s big data, can yield more accurate and actionable insights.  Organizations need a thoughtful process for ensuring the capture and central storage of this small data.

Figure 4: Boston Red Sox search for variables that are better predictors of performance

Figure 4: Boston Red Sox search for variables that are better predictors of performance

#7 – Save the Ugly Baby: the Pixar Success Story

This blog was the direct result of watching our data science team struggle with a particular client’s data.  The data science team was trying to explain to the client that the analytic process was fraught with failure, but that failures are progress because they teach you what doesn’t work.  I happened to be reading an article about innovation at Pixar, and the connection between the two processes became clear – be patient, keep testing, keep fine-tuning and eventually success will happen.

Figure 5:  Saving The Ugly Baby Process

Figure 5:  Saving The Ugly Baby Process

#6 – Best Practices for Analytics Profiles

This blog came very close to being #1 because it captures the analytic process that we use in every one of our big data and data science engagements.  Our data science team is always looking to build analytic profiles at the individual entity level, whether that’s customers, products, employees, students, slot machines, wind turbines, etc.  We like to designate these entities as the organization’s “strategic nouns,” those entities where superior knowledge and insights can help optimize key business processes and uncover new monetization opportunities.  And this approach works every time.  I love it!

Figure 6:  The Power of Analytic Profiles and Strategic "Nouns"

Figure 6:  The Power of Analytic Profiles and Strategic “Nouns”

#5 – Don’t Think Better; Think Different

This blog reflects that biggest problem that I see with organizations with respect to moving into the big data world: organizations are trying to apply old paradigms to new technologies and capabilities.  It just doesn’t work.  For example, I don’t know how many times I’ve seen organizations try to incrementally extend their BI and data warehouse capabilities by trying to integrate Hadoop and HDFS capabilities, and totally swing and miss.  Then they blame it on the technology and not their approach.  Ugh!

By the way, be sure to read the comment about “Don’t Think Business Functions; Think Business Initiatives” at the bottom of the blog.  Interesting observation shared with my by one of my Big Data MBA students.

Figure 7:  Evolution from Descriptive to Predictive to Prescriptive Analytics

Figure 7:  Evolution from Descriptive to Predictive to Prescriptive Analytics

#4 – Hadoop Data Modeling Lessons – by Vin Diesel

This blog took me nearly 12 months to write, and even now I’m not certain that I got it right. The Vin Diesel comment is a reference to his role as Xander Cage in the movie “XXX” where he admonishes the Prague police for not using their full firepower (heat seeking missile) in a standoff with a sniper. The way that a data scientist designs their data models is very different from how a data warehouse designer designs a data model.  In order to take advantage of the raw processing power of MPP and Hadoop architectures, the data scientists want long flat tables that eliminate joins.  And that’s what we’ve got – massive flat files against which the data scientists are running their analytics.  They’re sort of ugly, but very effective from an analytics perspective.

Figure 8:  Data Science - Flatten the Star!

Figure 8:  Data Science – Flatten the Star!

#3 – How I’ve Learned To Stop Worrying And Love The Data Lake

This is one of my favorite blogs and a topic that I discuss frequently. I especially like how the blog addresses the advantages of a data lake to the data warehousing and BI teams.  I’m a big fan of the data lake (even if I’m not a big fan of the name) and see more and more of our clients adopting the data lake not only to support their analytics environments, but more and more to free up expensive resources off of their data warehousing environments.  The data lake is really a winner, and 2015 will be the year of the data lake (and I’ll be speaking about the Big Data MBA and the Data Lake at the February Strata conference…hint, hint)!

Figure 9:  The Power of the Data Lake

Figure 9:  The Power of the Data Lake

#2 – Dynamic Duo of Analytic Power: Business Intelligence Analyst PLUS Data Scientist

This is one of my more recent blogs, and it has gotten lots of favorable feedback.  This blog took me a couple of months to get right.  I really struggled to understand the key differences between how the traditional Business Intelligence process works (which I knew well from my Business Objects days), and how the data science process works.  But after several engagements where I got a chance to work closely with our data science team, I think I finally nailed it!

Figure 10:  The Data Science Engagement Process

Figure 10:  The Data Science Engagement Process

#1 – “Big Ideas” Big Data Business Model Maturity Index

This one is clearly my favorite.  We turned the Big Data Business Model Maturity Index into a very cool, visually engaging animation.  The animation turns the Big Data Business Model Maturity Index into a living, and hopefully thought-provoking story with engaging visuals. It’s a very creative presentation, and it actually makes me look smart.  Heck, even my kids were impressed!!


Big Data Business Model Maturity Index

Big Data Business Model Maturity Index

So that’s it, my “Top 10 + 1” blogs.  2014 was a great year and I hope to have even more to share in 2015! Thank you for all your feedback and comments!

My Top 10 Blogs of 2014
Bill Schmarzo

Read the original blog entry...

More Stories By William Schmarzo

Bill Schmarzo, author of “Big Data: Understanding How Data Powers Big Business” and “Big Data MBA: Driving Business Strategies with Data Science”, is responsible for setting strategy and defining the Big Data service offerings for Hitachi Vantara as CTO, IoT and Analytics.

Previously, as a CTO within Dell EMC’s 2,000+ person consulting organization, he works with organizations to identify where and how to start their big data journeys. He’s written white papers, is an avid blogger and is a frequent speaker on the use of Big Data and data science to power an organization’s key business initiatives. He is a University of San Francisco School of Management (SOM) Executive Fellow where he teaches the “Big Data MBA” course. Bill also just completed a research paper on “Determining The Economic Value of Data”. Onalytica recently ranked Bill as #4 Big Data Influencer worldwide.

Bill has over three decades of experience in data warehousing, BI and analytics. Bill authored the Vision Workshop methodology that links an organization’s strategic business initiatives with their supporting data and analytic requirements. Bill serves on the City of San Jose’s Technology Innovation Board, and on the faculties of The Data Warehouse Institute and Strata.

Previously, Bill was vice president of Analytics at Yahoo where he was responsible for the development of Yahoo’s Advertiser and Website analytics products, including the delivery of “actionable insights” through a holistic user experience. Before that, Bill oversaw the Analytic Applications business unit at Business Objects, including the development, marketing and sales of their industry-defining analytic applications.

Bill holds a Masters Business Administration from University of Iowa and a Bachelor of Science degree in Mathematics, Computer Science and Business Administration from Coe College.

Latest Stories
In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at, will discuss how to use Kubernetes to setup a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace....
Kubernetes as a Container Platform is becoming a de facto for every enterprise. In my interactions with enterprises adopting container platform, I come across common questions: - How does application security work on this platform? What all do I need to secure? - How do I implement security in pipelines? - What about vulnerabilities discovered at a later point in time? - What are newer technologies like Istio Service Mesh bring to table?In this session, I will be addressing these commonly asked ...
Today most companies are adopting or evaluating container technology - Docker in particular - to speed up application deployment, drive down cost, ease management and make application delivery more flexible overall. As with most new architectures, this dream takes significant work to become a reality. Even when you do get your application componentized enough and packaged properly, there are still challenges for DevOps teams to making the shift to continuous delivery and achieving that reducti...
Signs of a shift in the usage of public clouds are everywhere Previously, as organizations outgrew old IT methods, the natural answer was to try the public cloud approach; however, the public platform alone is not a complete solutionThe move to hybrid, custom, and multi-cloud will become more and more prevalent At the heart of this technology trend exists a custom solution to meet the needs and concerns of these organizations, including compliance, security, and cost issues Blending Ser...
The KCSP program is a pre-qualified tier of vetted service providers that offer Kubernetes support, consulting, professional services and training for organizations embarking on their Kubernetes journey. The KCSP program ensures that enterprises get the support they're looking for to roll out new applications more quickly and more efficiently than before, while feeling secure that there's a trusted and vetted partner that's available to support their production and operational needs.
Skeuomorphism usually means retaining existing design cues in something new that doesn’t actually need them. However, the concept of skeuomorphism can be thought of as relating more broadly to applying existing patterns to new technologies that, in fact, cry out for new approaches. In his session at DevOps Summit, Gordon Haff, Senior Cloud Strategy Marketing and Evangelism Manager at Red Hat, discussed why containers should be paired with new architectural practices such as microservices rathe...
Between the mockups and specs produced by analysts, and resulting applications built by developers, there exists a gulf where projects fail, costs spiral, and applications disappoint. Methodologies like Agile attempt to address this with intensified communication, with partial success but many limitations. In his session at @DevOpsSummit at 19th Cloud Expo, Charles Kendrick, CTO at Isomorphic Software, presented a revolutionary model enabled by new technologies. Learn how business and develop...
When a company wants to develop an application, it must worry about many aspects: selecting the infrastructure, building the technical stack, defining the storage strategy, configuring networks, setting up monitoring and logging, and on top of that, the company needs to worry about high availability, flexibility, scalability, data processing, machine learning, etc. Going to the cloud infrastructure can help you solving these problems to a level, but what if we have a better way to do things. ...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It's clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Th...
With the rise of Docker, Kubernetes, and other container technologies, the growth of microservices has skyrocketed among dev teams looking to innovate on a faster release cycle. This has enabled teams to finally realize their DevOps goals to ship and iterate quickly in a continuous delivery model. Why containers are growing in popularity is no surprise — they’re extremely easy to spin up or down, but come with an unforeseen issue. However, without the right foresight, DevOps and IT teams may lo...
xMatters helps enterprises prevent, manage and resolve IT incidents. xMatters industry-leading Service Availability platform prevents IT issues from becoming big business problems. Large enterprises, small workgroups, and innovative DevOps teams rely on its proactive issue resolution service to maintain operational visibility and control in today's highly-fragmented IT environment. xMatters provides toolchain integrations to hundreds of IT management, security and DevOps tools. xMatters is the ...
DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE). Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...
When you're operating multiple services in production, building out forensics tools such as monitoring and observability becomes essential. Unfortunately, it is a real challenge balancing priorities between building new features and tools to help pinpoint root causes. Linkerd provides many of the tools you need to tame the chaos of operating microservices in a cloud native world. Because Linkerd is a transparent proxy that runs alongside your application, there are no code changes required. I...
"There is a huge interest in Kubernetes. People are now starting to use Kubernetes and implement it," stated Sebastian Scheele, co-founder of Loodse, in this interview at DevOps at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...