Ganesh Datta
HostCTO & Co-founder of Cortex
Rob Zuber
CTO at CircleCI
March 26, 2026
In This Episode
In this episode of Braintrust, Cortex co-founder and CTO Ganesh Datta sits down with Rob Zuber, CTO at CircleCI, who has spent over a decade at the center of how engineering teams build and ship software. Rob shares his thinking on two challenges that are becoming harder to ignore as AI accelerates output: the quiet erosion of software quality, and the pressure to move fast without a clear sense of direction.
They discuss what made the best QA engineers so effective and why that mindset largely disappeared, how LLMs could help bring it back, and why engineering leaders need to think about metrics very differently depending on whether their teams are scaling a mature system or exploring uncharted territory.
You’ll learn
As testing shifted to automation, engineering lost the adversarial instinct that made great quality engineers effective. They would smell something off in a system and dig in—behavior that no scripted test can replicate.
LLMs have absorbed an enormous amount of public code, incident reports, and known failure patterns. With the right framing, such as asking what will break rather than what to build, they could surface edge cases no one thought to script.
LLMs are rewarded for producing things, so they produce things. Give them a different goal and they pursue that instead.
The real question is whether teams are testing the right assumptions. AI tools have dramatically lowered the cost of experimentation, which should mean more ideas tested more quickly, not just more of the same delivered faster.
For exploratory teams, the metrics that matter are adoption, user feedback, and whether assumptions are being validated. PR throughput is the wrong thing to optimize for in that context.
Quotes
"An LLM has a massive bag of tools because it reads every line of code that's ever been public. With a little bit of guidance in the right direction, you should be able to say: I built this thing. What's going to go wrong?"
Rob Zuber
CTO at CircleCI
"Knowing that a number has moved is not interesting. It's certainly not the end of a conversation. It's the very beginning of a conversation."
Rob Zuber
CTO at CircleCI
"If you're obsessed with metrics that are like delivery metrics, you're gonna be deeply disappointed."
Rob Zuber
CTO at CircleCI
Timestamps
(2:57)
The lost art of QA and what made the best quality engineers so effective.
(06:17)
How LLMs could rebuild an adversarial, curiosity-driven approach to testing.
(11:37)
Why framing the question correctly is what unlocks LLM potential for quality work.
(18:11)
Spec-first development and UAT as a model for LLM-assisted quality testing.
(21:34)
Moving fast matters, but direction and assumption-testing matter more.
(26:17)
Running in dual mode: rapid prototyping for discovery vs. scaled, resilient systems.
(32:37)
Why DORA and delivery metrics are only the start of a conversation, not the end.
(34:52)
Why teams adopting AI tools need freedom from delivery metric pressure to actually learn.
Other episodes
The platform engineering playbook for velocity, quality, and AI readiness at SIXT
In this episode of Braintrust, Cortex co-founder and CTO Ganesh Datta sits down with Boyan Dimitrov, CEO of SIXT, one of the world's largest mobility providers operating in over 100 countries. Boyan shares the story behind SIXT's engineering transformation, from shipping software once or twice a month to running nearly 10,000 deployments a month, and explains how extreme standardization became the engine driving both velocity and quality at the same time.
They discuss the pull-and-push model SIXT uses to drive platform adoption without mandating it from the top, how Boyan built a business case for platform investment by starting with specific problems rather than a platform-first vision, and how years of foundational standardization work is now paying significant dividends as SIXT accelerates its AI strategy.
March 12, 2026
Boyan Dimitrov
CTO of SIXT
How Thrive Market's SVP of Engineering thinks about reliability culture
In this episode of Braintrust, Cortex co-founder and CTO Ganesh Datta sits down with Randy Shoup, SVP of Engineering at Thrive Market. Randy shares lessons from his leadership roles across multiple companies and explains how measurement and transparency can help teams build stronger engineering cultures.
Randy and Ganesh chat about how fear can block progress, why recovery speed matters more than trying to prevent every failure, and how teams improve through steady, incremental gains. They also discuss a few practical ways to build trust around metrics so organizations can use visibility for learning instead of punishment.
February 26, 2026
Randy Shoup
SVP of Engineering at Thrive Market
