(Liu-zishan/Shutterstock)
“DevOps” is really easy to say. However the easy phrase masks an uneasy pressure between growth and operations that may result in poor communication and finally poor administration of more and more advanced tech stacks. Addressing that DevOps pressure and communication hole additionally occurs to be problem that the founders of an observability startup known as Honeycomb purpose to unravel.
Honeycomb’s story begins circa 2012, when two engineers, Christine Yen and Charity Majors, discovered themselves working collectively at Parse, which developed a back-end for cellular apps. Yen was a developer whereas Majors was extra inclined in the direction of the infrastructure. A friendship blossomed between them, regardless of variations in jobs and pure abilities.
“She was an ops and infra engineer who I’m certain popped out of the womb carrying a pager. I used to be a product engineer who needed to construct cool experiences for our clients,” Yen mentioned. “I used to be petrified of manufacturing and, to be frank, broke a bunch of Charity’s infrastructure.”
Even again in 2012, gaps in observability have been apparent. Getting data out of logs was difficult, and it solely bought tougher because the software program stack advanced from monolithic functions operating on-prem in the direction of microservices operating within the cloud–not to mention immediately’s serverless and stateless apps operating in K8S clusters.
Parse was acquired by Fb in 2013, which uncovered Yen and Majors to a novel logging and metrics software known as Scuba. Constructed upon an in-memory time-series database, Scuba immediately allowed the engineers to visualise much more of the “nitty gritty” knowledge, resembling app IDs, SDK variations, and app variations, that have been beforehand off limits. That opened Yen and Majors’ eyes to a brand new world of observability element and efficiency troubleshooting.
“The whole lot was so quick you might churn by means of tons and tons and tons of knowledge, spit out a chart, after which say ‘Drill into this one app and this one SDK model,’” Yen mentioned. “As a result of we have been a platform, we wouldn’t know if there was a Russian relationship app that will launch, that will use a brand new endpoint in an sudden manner and take down our Mongo cluster. Nevertheless it occurred on a regular basis, and so we had to determine learn how to get forward of that.”
Emboldened by the Scuba expertise, Yen and Majors left Fb and based Honeycomb in 2016. The pair had an thought for a brand new type of observability software that would categorical operations ideas in a builders’ language, and (hopefully) bridge the hole between Dev and Ops, thereby bringing again some semblance of simplicity–or no less than understandability.
“What was these fairly comprehensible worlds that may very well be understood at a excessive degree the place all of us had Rails apps on 5 beefy EC2 situations, they usually have been your pets, proper? You give them names–cute issues–and also you get very pressured when one goes down,” mentioned Yen, who’s Honeycomb’s CEO. “And now it’s the pets versus cattle. Now we have now situations we’re biking by means of within the cloud. We’re shifting from these 5 app servers and monoliths to microservices on Kubernetes pods which might be continuously getting recycled.”
Honeycomb goals to make advanced observability knowledge readily consumable by builders (Picture courtesy Honeycomb)
At a technical degree, Yen and Majors, Honeycomb’s CTO, made a number of selections in 2016 that set it on its path. It determined there was no good cause to separate up the Holy Trinity of observability knowledge–logs, metrics, and traces–as some observability firms do. By storing logs, metrics, and traces collectively, it might be simpler to correlate particulars present in them, which might speed up the remediation of issues.
The second large architectural resolution the corporate made was it chosen an column-oriented database wherein to retailer all this observability knowledge. This made Honeycomb extra like Snowflake than Datadog, Yen mentioned.
“We didn’t invent column shops, and we’ll be the primary to let you know that there’s nothing, actually, all that particular about column shops,” she instructed BigDATAwire in an interview at re:Invent 2024 final week. “However we have been the primary individuals to essentially begin to query, ought to we perhaps not have logs over right here and metrics over right here endlessly? Ought to we perhaps attempt to discover methods [to keep them together], particularly in the event that they’re speaking about the identical factor that occurred in an utility?”
As an alternative of spending effort and time attempting to affix these knowledge sorts collectively to allow them to be analyzed, Honeycomb’s design naturally shops them collectively, utilizing the OpenTelemetry (OTEL) format. That not solely eliminates the necessity for knowledge integration, nevertheless it helps with the following functionality that Honeycomb is thought for: Offering context to the advanced metadata it collects, which in flip helps flip the technical infrastructure jargon that operations of us converse into the enterprise language that builders converse.
“It meant that beforehand Charity could be like ‘Christine, what did you do to the write throughput on my Cassandra occasion?’ And I’d be like, let me attempt to determine it out. I’m actually scared,” Yen mentioned. “It turned these conversations into, ‘Hey, Christine, I noticed elevated write throughput on my Cassandra situations. So I appeared, and plainly that elevated site visitors is pushed by this app on this endpoint. What’s occurring?’
“And I’d be like, Oh, that’s a regression,” she continued. “Now let me go look, as a result of now it’s in my world. Now I can go attempt to reproduce it in a check. Now I can go take a look at the enterprise logic. Now I perceive why it issues, as a result of I do know which apps you’re speaking about, and that’s the best way that it modified how we work collectively as an engineering staff.”
That method appears to be resonating, because the San Francisco-based firm has already signed 800 paying clients, together with firms like Vanguard, Fender Guitars, and Jack Henry & Associates. The corporate, which has raised about $147 million by means of a Collection D spherical, additionally final week picked up a 2024 BigDATAwire Readers’ Selection Award within the class of Greatest Large Information Product: DataOps and Observability.
Honeycomb isn’t the one observability firm that’s shaking up the established order today. Corporations which have moved their IT operations to public clouds are straining beneath mountains of logs, metrics, and traces, in addition to the huge prices of storing and processing all that knowledge, they usually’re open to new concepts. The incumbent observability distributors nonetheless have momentum, however a brand new guard is rising on this house, together with startups like Honeycomb.
Honeycomb is actively pursuing clients who’re bored with the enterprise practices of a few of its large observability software opponents and are on the lookout for one thing new to deal with the large storage and processing challenges of immediately, Yen mentioned.
“Try to be questioning what you’re doing along with your instruments to start with, and perhaps if what you’re doing along with your instruments is similar factor you’ve been doing for the final 20 years with telemetry, with APM, logging, and monitoring instruments, maybe the best way we method constructing software program is price a revisit,” she mentioned. “I believe perhaps that’s why your readers like us, as a result of we’re right here to query.”
Associated Gadgets:
BigDATAwire Reveals Winners of 2024 Readers’ and Editors’ Selection Awards
Honeycomb Faucets Observability Buzz to Elevate $50M Collection D
VCs Open Up the Checkbook for Observability Startups