Cortex Catalogs: More than Microservices

Cortex is an Internal Developer Portal (IDP) that reduces friction in developing software that stays aligned to standards of excellence. While all IDPs offer some cataloging functionality, the ability of that catalog to match existing business logic can mean the difference between a true system of record, and yet-another-[siloed]-tool.

In this blog we’ll look at Cortex’s early origins as a “microservice catalog,” how the platform evolved to support increasingly diverse architectures, and why in 2024 fully customizable catalogs to track infrastructure, APIs, pipelines, products, teams, tools—anything you want—are a non-negotiable for ensuring success.

Microservice origins

“Uber is the classic case of microservices gone wrong.” Prompt any member of the Cortex team with this quote and they’ll finish it before you can. It’s our origin story, and comes from our CEO Anish’s account of what led him to found Cortex. Though it’s nearly identical to the backstory of the other two co-founders, Ganesh Datta and Nikhil Unni, who faced similar scalability challenges as their companies expanded microservice development. 

That was 2019, when microservice architectures were still a relative rarity. While “service catalogs” weren’t new, they did mean something a little different to the IT teams using them to capture data about support services on offer. When Cortex emerged with a way to track ownership, origins, and standards in service-oriented architectures the term “service catalog” took on a new meaning, and stuck.

By 2020, Cortex had evolved from a cataloging and scoring solution to a full-spectrum IDP serving all parts of the software development lifecycle. And by 2021, it was obvious that it wasn’t just businesses with microservices that would benefit. Those with complex mono repos, multi repos, or hybrid development environments all share a need to not only improve visibility over increasingly complex architectures, but drive consistency at scale. These organizations may have a need for “service” catalogs, but also needed a way to represent the resources, infrastructure, teams and other elements that relate to those services—all of which were already defined in existing business logic.

Cortex software catalogs: a comprehensive system of record

While Cortex’s most popular use case was microservice management, the platform was intentionally designed to be extensible, allowing it to support any type of entity as well as their unique relationships and hierarchies. Catalogs that don’t match your business’s data model can be extremely hard to integrate into your team’s daily workflows, forcing teams to change their vocabulary or split it between multiple systems creates friction, which is counter to the goal of any well-designed portal. 

Because of this, software catalogs that can account for all software entities (not just microservices) have become an increasingly important requirement for anyone considering an Internal Developer Portal—especially as development environments continue to evolve in parallel. 

Cortex enables users to define their own catalog and entity types, to match existing business logic, or as a means to reconciling out dated information architectures.

Populating or "hydrating" catalogs is easy, as Cortex connects to 50+ popular developer tools, and can accomodate custom data from internal sources just as easily. It can also ingest a full suite of API services and endpoints from ubiquitous sources such as GitHub or Kubernetes, greatly reducing up-front setup. And new software can be scaffolded, registered, deployed, and cataloged directly from Cortex, streamlining the entire development lifecycle.

Customizing your catalog to reflect your software data model

Your organization may refer to the above catalog types with different nomenclature, or may want to elevate a sub-type of Infrastructure, for example. However you define and relate the elements of your ecosystem can be replicated in how you structure your Cortex catalogs. This is facilitated in three ways:

1. Create any catalog: No need to fit entities into the concept of “services,” “resources,” “domains,” or “teams” — create any catalog category and tag it to any type
2. Define any entity: Bring in any entity type as above, with their own unique characteristics
3. Prioritize any view: Select your top-line entities for display in the “all entities” and “all catalogs” menu to enable easier navigation for end-users

Some use cases for creating custom entity types and other deep customization include:

• Creating a catalog of API endpoints
• Generating a searchable inventory of application components
• Indexing dependency libraries and creating a navigable hierarchy between them
• Organizing multi-cloud k8s clusters and deployed container services
• Managing application packages and artifacts across multiple dependant teams
• Adapting service catalogs to feature flags, to keep them in sync with feature state 

Importantly, the rote parts of defining this information architecture come out of the box. Users can define the catalog and data types that best fit their existing business logic, without having to re-engineer obvious relationships like how alerts and services relate to one another. 

How Cortex customers are using custom catalogs

Cortex customers create an average of five custom catalogs beyond the default types provided at onboarding (service, resource, team, domain). 

The below illustrates how real customers have applied catalog customziation to ensure their IDP information architecture matches core business logic already deeply engrained across all parts of the business.‍

eCommerce platform: Created a catalog type for supported storefronts 

Real Estate platform: Created catalog types for each major product

Travel service: Created a catalog type to track all known entities running in a particular datacenter

Investment firm: Created a Divisions catalog type which includes multiple teams

Medical service: Created catalog type for Azure spring applications

Transportation service: Created a helm chart entity type for observability

Why you need a software catalog

Organize and track all of yout software components, across teams and over time

It’s not hard to track software in a spreadsheet, if you have <50 components, and the free time of a full time eng manager or TPM to do it. But after that, you’ll start to lose track of ownership, changes to dependencies, packages being used, excessive open tickets, connection to vulnerability scanners, etc. This ultimately compounds the time it takes to do anything in development, incident response, and deployment. If devs can’t find the right answer to questions they have about software components, they’ll burn hours piecing the puzzle togetehr themselves. A software catalog that connects to your identity provider and all the tools in your ecosystem makes it easy to track all services as they (and your team) evolves. 

Accelerate developer onboarding and time to productivity

Just gaining initial context about software they’re inheriting, tools their team uses, best practices they’re meant to follow, or what software components and templates are available to them is often the #1 cited challenge for developers in new roles—whether new to the company or just moving into a different group. Comprehensive software catalogs that mirror your business logic can drastically reduce onboarding time 

Map all the technologies and tactics your company uses

Make it easier for your teams to see what technologies are available to them, and measure which allow your teams to perform the best. From there, you can try to standardize the technologies your company uses, which will improve efficiency. Plus, if you need to adopt new technologies, your catalog lets you do that much faster.

Streamline security vulnerability identification and fixing

Addressing security vulnerabilities is of the utmost importance. Yet, it takes many companies around two months (at least) to find and fix critical vulnerabilities in their applications. Software catalogs ensure you can assign each service an owner. That way, every service has someone keeping security in check.

For more information on how Cortex can help you create clarity an alignment across teams and tools, hop into the product, or schedule a custom demo today!