Cortex Query Language (CQL) is the powerhouse behind many features in the Cortex developer portal. CQL is an abstraction layer between your catalog and all the third party tools you have – it’s available when you need it, and hidden when you don’t. When you first begin using Cortex, we use CQL behind the scenes to run your Scorecards, laying the foundation for more advanced use cases later on. Once you’re comfortable with using Cortex, you’ll find that CQL empowers you to gain insight into your services and track their maturity in a whole new way.
Structured metadata
Your first experience with Cortex will be importing services and resource into the respective Catalogs — for most of our users, this is the first time they’ve ever cataloged every one of their components (outside of a spreadsheet!). This phase is crucial: a catalog provides a structured metadata store for all of your components, replacing loose confluence pages with a centralized system with standardized data models. After you’ve built out your Catalog, you’ll begin working with Scorecards to survey essential information about your services.
After building a Scorecard or two, you’ll roll them out to your engineers and gather feedback. At this stage, you’ll realize that some services may have specific requirements, and simple rules don’t quite capture that. For example, your most critical services probably require a higher level of code coverage than less critical services do. As you productionalize your Scorecards, you’ll begin to discover these edge cases. This is where CQL shines.
CQL gives you access to all of the metadata about a service, including metadata you’ve defined yourself, as well as all the data from your third party tools – from “does the service have SLOs defined” to “verify in DataDog that those SLOs are passing and have an acceptable error budget.” Within Scorecards and the Query Builder, you can access every single data point about every single service in your Catalog. Because CQL is a power-user feature, you don’t have to dive into it every time and can stick to our out of the box queries and rules. Instead, you can choose to dive into CQL only when you need to accurately capture a complex situation.
The structured information stored in the Service Catalog, and exposed via CQL allows you to access valuable insights about your services that would be impossible to collect otherwise. For example, when the Log4j vulnerability was detected, many Cortex users found that their Catalogs already contained all of the information about deployments, ownership, and repositories that they needed. Because CQL can query and combine all of this disparate information, customers of Cortex could quickly generate a list of all services that were affected by the Log4j vulnerability, along with all of the owners for the relevant services, and jump right into mitigation – instead of spending weeks searching for services and planning for the effort
“Cortex was a critical piece of our Log4j mitigation effort. The ability to easily identify owners and correlate Snyk reporting really allowed us to focus on getting patched instead of managing spreadsheets and lists.”
— Shawn Burke, Distinguished Engineer at SoFi
Accurate representation
Once you’re comfortable with Cortex, you can use CQL to tailor your Scorecards and define more complex rules. CQL allows you to capture edge cases, set different thresholds, and combine data sources. This flexibility allows you to define rules as expressions, so your Scorecards accurately reflect your services.
Let’s say, for example, that you want to make sure every Javascript or Typescript project has a yarn.lock or a package-lock.json file. Without CQL, you’d have to set up two different rules in your Scorecard. But, because there are no cases in which a service would have both files, any service measured by the Scorecard would always fail at least one rule. Developers then grow frustrated and gamification falls apart.
However, with the ability to define rules as CQL expressions you can easily write a single rule that captures this requirement: ‘needs a yarn.lock OR package-lock.json file.’ As long as the service passes the logical expression, it passes the rule and the service earns the points it deserves.
At an even more advanced level, CQL allows you to dynamically generate JIRA queries, look up files in your repository and parse JSON, YAML, and XML files, and work with data from third party integrations. This accuracy and flexibility is crucial for gamification, keeping developers engaged with standards and best practices.
Powerful queries
While you’ll engage with the Catalog and Scorecards regularly, there will be times you want to explore your service catalog and all of the data it contains. CQL lets you treat your catalog as a true data store, capturing not only the information you’ve provided to Cortex, but all the data that exists within the tools you’ve integrated with Cortex.
CQL empowers you to combine data from the Catalog with live integration data, so Cortex acts as a single point of access for all of your service information. Cortex’s Catalog is not a static data source, but a living database.
Let’s say, for example, every service has a CI pipeline file that’s named after the service, and you want to find all of the services that are using an older version of the file format. Without CQL, you’d write a script that loops over every repository, loads the file, and asserts this constraint (without ever notifying the service owner about it!). Instead, using CQL, you could combine service metadata with Git data to run a query like jq(git.fileContents(tag + “-ci.yaml”), “.version”) = “1.1”.
The true power of CQL is in its flexibility. Users don’t need to worry about the details — the abstraction of CQL simply allows access to any information you need.
The Cortex philosophy
A developer portal should be a flexible tool — every organization has its own needs, and portals should likewise be adaptable to meet this variety of needs. Cortex believes that a dev portal shouldn’t restrict you to what its API exposes or limit the ability of your Scorecards to track information to a pre-canned set of data.
CQL was designed with this same philosophy. Rather than restrict developers, it enables them to explore and understand their services on a deeper level with less effort, minimizing the burden on developers and engineering leaders while exponentially increasing the value and ROI of a Service Catalog.
To see how CQL can empower your teams to unlock valuable insights about your services, book a demo of Cortex today.
