The Snowflake IPO was a big deal, and not merely because of the company’s enormous valuation.
In 2013 Cloudera co-founder Mike Olson confidently (and accurately) declared “a stunning and irreversible trend in enterprise infrastructure.” That trend? “No dominant platform-level software infrastructure has emerged in the last 10 years in closed-source, proprietary form.” Snowflake, a cloud-based enterprise data platform, may spell the end of that run.
Sure, we had Splunk, but Spunk squeaked through the hypothesis police before open source had found its feet, as Lightspeed partner Gaurav Gupta told me. MySQL, Apache Hadoop, MongoDB, Apache Spark... all of them (at least initially) open source.
But now... Snowflake. Is Snowflake a snowflake? Or is the era of open source infrastructure coming to a close?
In part the answer to that question depends on just how fiercely you’re prepared to defend the underlying assumption. After all, it’s simply not the case that all “dominant platform-level software infrastructure” is open source. This isn’t really to dispute Olson’s central thesis, because it’s absolutely true that the bulk of enterprise infrastructure has trended toward open source over the past 10 to 20 years.
As Gordon Haff puts it, “You can certainly construct a narrative for the infrastructure being heavily driven by open source: Most NoSQL, Hadoop, Kafka, Spark, Ceph, Jupyter, etc. But a lot in the space isn’t as well: lots of cloud services, Tableau, Splunk, etc.” And Snowflake, of course.
Though you’d never guess it from the energetic proselytizing of yesteryear, developers have never been overly religious about open source. The reason for that “stunning” trend is simply that open source made it easier for developers to get their jobs done thanks to high-quality, easily accessible open source data infrastructure. There are, of course, other benefits, such as the communities that often accompany open source projects, coupled with a desire to have more granular control of one’s software stack. But ultimately open source has won because it enables developers to “get ---- done.”
Which is why, for example, you’ll find developers happy to use open source software like Apache Airflow to load data into their proprietary Snowflake data platform. It’s not cognitive dissonance. It’s pragmatism.
Speaking of such pragmatism, Tom Barber suggests that the shift to managed cloud services somewhat negates “people’s interest in open source... because with SaaS you’re not paying for licenses but for a service, which changes the thinking somewhat.” After all, he continues, “Open source meant you didn’t pay for licenses but you still had to pay someone internal or external to install it, tune it, run it…. Most people can apt/yum install MySQL but tuning it requires in-depth knowledge.”
Or let’s express that another way, as Redmonk analyst James Governor does: “Cloud is a better distribution and packaging mechanism than open source ever was…. Convenience is the killer app. Managed services win.” Or, as Olson himself suggested to me,