Engineering teams rely on certain metrics to assess their ability to deliver quality products, on time. This is a useful exercise, but execution has been lacking—with metric collation often handled via spreadsheet, or stand-alone tool. Neither approach is ideal for two reasons:

1) How—or more specifically where—metrics are collected silos them away from business context.

2) Because metrics are untethered from root cause, developers who otherwise want to improve productivity have begun to treat them as distractions, or worse—demerits. 

This isn’t an indictment of measurement. It’s an indictment of the means.

Last week we published a blog positing that the only way to bolster engineering metrics with the context needed to be useful—and the tools needed to be actionable—is to move them into the system of record. Today, Cortex is excited to take the first step in this journey with our latest product, Eng Intelligence. Here, we’ll share what this means for Cortex customers, and hopefully soon, engineering teams everywhere.

Introducing Cortex Eng Intelligence

Eng Intelligence is a new module within the Cortex platform that was built to reverse how teams think about metrics related to efficiency and productivity. By positioning this analysis right alongside context from Catalogs and Scorecards, metrics once viewed in isolation—like deployment velocity and PR count—become indicators for deeper investigation, allowing leaders to uncover bottlenecks, and unblock their team.

How it works

Eng Intelligence works alongside any Cortex platform package, but is further enhanced by the full suite of capabilities including Initiatives, which enable users to time-bound projects according to specific goals and deadlines. There are three key steps to using Eng Intelligence effectively:

Aggregate: Eng Intelligence aggregates data from already-connected version control, ticketing, deployment, and on-call solutions to calculate critical metrics according to your organization’s priorities. This data is then presented by team, group, or individual, and can be filtered by time to help users spot trends between key metrics like PR size and velocity. By default this data is accessible to team owners and managers, but can be configured by role based on the organization’s requirements. 

Contextualize: When users want to better understand what’s behind each trend spotted, they can dive into team and group detail pages to view teammates, the health of software they own, and any ongoing initiatives already in place. Oftentimes this analysis will expose oversights in operational standards otherwise enforced by Cortex. For example, if a Production Readiness scorecard lacks a rule to add a ReadMe file, users may see deployment velocity fall over time, as new developers struggle to understand the software they inherit.

Act: The above example can be remedied by adding or updating a Product Readiness Scorecard with a ReadMe requirement, which tells Cortex to continuously scan new and existing software for alignment. Another example might be noticing that MTTR is down across all teams. In this scenario, users can create a new rule as part of an existing Operational Efficiency Scorecard, or even carve off a time-boxed initiative to ensure all new and existing services pass SLO's defined in Datadog.

Use Cases

Current users of Eng Intelligence have already uncovered a number of use cases for this solution. A key theme throughout the below examples relates to the importance of centralizing all engineering context. Productivity is an output of systemic inputs, so productivity metrics need system-wide context to make those metrics meaningful. 

Trend Analysis

In situations where leaders identify undesirable trends, just being able to see more context has helped teams prevent over-rotating on targets. Time to review may be up, but what if incidents are down? Viewing data in context helps leaders make nuanced decisions that keep their teams from spiraling around a single metric. 

DORA/SPACE Alignment

Whatever your ideal metric framework, Eng Intelligence enables you to put that data to work for your team, rather than against them. Compare things like PR time to review, or average open to close time within or across teams, and trace performance back through team composition, projects, and software health to understand root cause, and prescribe next steps.

Alert Centralization

Reduce cognitive load for developers by centralizing tasks for both software and productivity improvement. Devs receive alerts for both use cases, prioritized and contextualized against one another, rather than worrying about balancing multiple tools to understand where to focus next. Report on productivity progress within or right alongside reporting on other standards alignment like production readiness or operational efficiency.

Operational Excellence

Create achievable goals for your team to move towards improvement at their own pace, given other workloads. Identify which parts of these programs should take priority over others (perhaps PR close time feels most urgent) with Initiatives that are automatically ordered by importance for each developer in their personalized Developer Homepage.

Resource Allocation

One team might be hitting every target while another team struggles to keep their head above water. Comparing metrics across teams improves resource allocation, and can even help make the case for new headcount. In fact, It was reviewing our own Eng Intelligence instance that led our Director of Engineering to recognize that she was the bottleneck on deployments during a recent holiday. She now has approval to hire an additional manager to help.

Team Accolades

One use case for Eng Intelligence amongst our pool of early adopters has been the ability to acknowledge outstanding effort, speedy onboarding, or traits of high collaboration, such as a high number of PR reviews. With detailed data available, praise can be context-specific and accompany every step of the work – not just the final product.

Making productivity a shared language

We built Eng Intelligence to unify how developers and executives think about the true power of engineering metrics—as intelligence that can be used to unblock engineers from doing work that drives the business forward. With the right language, informed by the right context, we think how developer productivity is measured can inspire feelings of drive rather than fear. 

Over time, managers and developers can see why certain metrics may be underperforming, and actually work to implement programs to help, all in the same platform. In fact, we’ve seen companies increase key metrics like deploy frequency by as much as three times per team just by showing developers what they needed to do and how they could contribute to that metric. 

Results like these cut through the debate: Counting lines of code and doling out PIPs or pizza parties doesn’t work nearly as well as building awareness, alignment, and clarity. Ideally, that’s what measuring developer productivity actually does. 

Productivity not punishment

While we won’t pretend the discussions and debate about developer productivity will stop here, we hope, whether you try Eng Intelligence or not, you reconsider the role these metrics should play in the work your team undertakes on a daily basis.

Measures of efficiency will always be unique to the context of each company. By reversing how these numbers are leveraged, you can refocus on the activities that enable you to improve them on a more consistent, and reliable basis. 

If you’d like to know more about Eng Intelligence, or get a demo from our team of experts, contact us today!