You might have heard of Spotify Backstage (backstage.io), an open-source option for building internal developer portals. But what is Spotify Backstage? This article will help answer your questions about Backstage and explain important considerations to think about if you’re deciding whether to adopt in your team.
In the last two decades we saw DevOps emerge as a way to push back on the complexity and entropy creeping into the software development ecosystem. This thoughtful and considered approach to development processes has been a positive development, and helped rationalize some of the chaos involved in building software. Before DevOps, each development team, and sometimes each developer, had to navigate a tech stack that resembled a jungle of dependencies, tooling and spotty documentation.
While DevOps was a necessary first step, it didn’t solve the explosion of developer tools on its own. Getting a handle on the huge amount of infrastructure tooling - software components, open-source plugins, SaaS, ml models, Kubernetes etc. - required more than just sound processes. This led to the creation of internal developer portals and internal developer platforms, or IDPs. These tools centralized best practice and built golden paths to help developers focus on building software without being overwhelmed by a complex working environment.
The most widely used developer platform was created by the impressive engineering team at Sweden’s Spotify, called Backstage. This began as an internal project called “System Z” at Spotify to help engineers discover and understand all the company’s software, services, docs, APIs, and tools. This project provided a UX layer to help engineers quickly answer questions about software ownership, deployment, and dependencies.
What is Spotify Backstage?
Spotify Backstage is an open platform for building developer portals to create, manage, and explore software from a single UX layer. Backstage was originally developed at Spotify as an internal tool and then was open-sourced in 2020. Its functionality includes a software catalog, software templates, docs like code, and a variety of plugins that can extend the platform.
The success of Backstage points to a broader demand for developer portals and service catalogs. This is visible in several contributing trends:
- Growing complexity of engineering organizations. Today’s engineering teams are tasked with new projects and delivering detailed use cases at the fastest ever rate. As more high-growth “unicorn” tech companies emerge, there’s an increasing need for tools that help manage this complexity.
- Autonomous teams. The popular “Spotify Model” involves having small “squads” of engineers, around 7-10 people, that own projects end-to-end. Developer portals and service catalogs help teams maintain their independence while enforcing standardization and ease of discovery.
- Rise of developer tools. There’s been an explosion in developer tools in recent years, because every company is now a software company, and because increasing developer productivity can make a huge impact on a business’s bottom line. Developer portals and service catalogs help organize the growing number of developer tools that teams use internally.
Backstage Features — Out of the Box
Backstage includes several features for organizing software and related documentation:
Backstage Software Catalog
Currently in alpha, the Backstage Software Catalog is a unified source of metadata and ownership information about all the software that your team works on, like microservices, APIs, data pipelines, libraries, and more. The catalog is based on metadata YAML files. You can track third-party software, too, so you have one place to view all the external tools you use to manage your code.
Backstage Software Templates
Currently in beta, Backstage Software Templates let you define a code skeleton with variables that can be filled in. Then, you can push your template to GitHub or GitLab, where it can be used by the team to standardize the creation of new services (or even something like an onboarding guide).
Backstage Search
Also in alpha, Backstage Search lets you choose from modular frontend and backend components to build the right search tool for your organization. In addition to searching your software catalog, you can add support for other sources like your plugins, wiki, or even Stack Overflow. All these results will show up directly in Backstage to reduce context-switching.
Backstage TechDocs
Backstage TechDocs allows engineers to write technical documentation as Markdown files that live together with the code. The idea is to remove all the friction to creating docs and make them really easy to find, so docs actually get used. A Backstage plugin will automatically create and render a docs site + metadata, using MkDocs.
Backstage Plugins
As an open-source project, Backstage has a growing number of plugins that can help you customize the platform to your needs. Plugins are React components that are added to each service’s page. It’s possible to build your own plugin — we built a plugin that lets you add Cortex Scorecards to Backstage.
What engineering problems does Spotify Backstage solve?
Without a centralized platform or portal like Backstage, developer teams are left to fend for themselves. This causes them to become stuck into silos, harming their developer experience and ultimately reducing productivity.
Using an IDP like Backstage helps with the following:
Visibility and Discovery: Backstage provides a centralized platform where all services and software components are registered. This makes it easier for developers to discover existing resources and understand how they fit into the broader system architecture, enhancing visibility across teams and projects.
Documentation and Collaboration: By integrating documentation directly with the development workflow, Spotify Backstage ensures that relevant information is easily accessible to all team members. It also facilitates collaboration by providing tools for sharing knowledge, feedback, and best practices within the developer community.
Standardization: Backstage provides templates and scaffolding for creating new services and components. This helps ensure consistency in development practices and reduces the complexity of managing multiple technologies.
Infrastructure Management: The platform offers a unified view of an organization's infrastructure, allowing developers to manage and monitor their services in one place. This simplifies infrastructure management tasks and helps developers quickly identify and resolve issues.
Customization: Recognizing that no two development environments are exactly alike, Spotify Backstage is designed to be highly customizable, enabling teams to tailor the platform to fit their specific needs and workflows. This flexibility ensures that developers can optimize their use of the platform for maximum productivity.
Challenges of Working with Backstage
While Backstage provides many advantages, there are challenges to be aware of when working with this tool, primarily because it involves a significant engineering investment. As Backstage is an open-source tool, the deployment, configuration, and maintenance falls on the end-user. There is documentation, but direct support may be limited and is not guaranteed.
Learning Curve: like all complex platforms, Backstage comes with a steep learning curve. This can slow initial adoption as developers spend significant time understanding the architecture, features, and best practices for effective use.
Not quite React: Backstage's design patterns are increasingly deviating from standard React, which is why we're seeing long and longer timelines to launch. You'll now need full time engineers well-versed in their increasingly domain-specific design to do even the simplest things like integrate with a directory.
Adoption struggles: The VP of Engineering at Spotify recently noted that global adoption of Backstage is 99% at Spotify... and less than 10% in any other organization. This may be due to a lack of impetus to log in, since Backstage lacks a prioritization and alerting mechanism.
Customization Complexity: While customization allows teams to tailor the platform to their specific needs, this leads to some internalized complexity. Getting it right requires deep understanding of the platform's internals and may necessitate additional coding, testing, and validation efforts.
Integration Challenges: Integrating the platform with existing tools and systems can present difficulties. This is especially true if APIs or connectors are limited, undocumented, or incompatible with current workflows, leading to potential disruptions or bottlenecks in development processes.
Maintenance Overhead: Maintaining such a wide-ranging platform, especially when highly customized or integrated with numerous other services, can significantly increase the overhead. Regular updates, bug fixes, and compatibility checks become necessary to ensure smooth operation.
Scalability Concerns: As teams and projects grow, the platform must be able to scale accordingly. Scalability concerns arise if the platform struggles with increased load, leading to performance degradation or the need for substantial architectural changes. For this reason platforms such as Backstage typically require a dedicated internal platform team.
Limiting scoring sources: If you purchase the additional plugin package for scoring, you'll still only be able to perform software checks manually via API, using "facts" or metadata in your catalog.
Upkeep and Documentation: Effective use and maintenance of the platform require up-to-date documentation, which can be a challenge to maintain, especially as customizations or integrations increase in complexity. Outdated documentation can hinder new user onboarding and complicate troubleshooting efforts.
Unfulfilling work: Many engineers don't want to just build integrations and plugins, which means the task is usually traded between teams, slowing down cycle times.
Unlike other Cloud Native Computing Foundation (CNCF) projects, Backstage does not come with pre-built Docker containers to deploy and will require an investment of development hours to create the Backstage app, apply customizations, and deploy to production. It’s important to note that Backstage is built using Node.js and Yarn, and depending on your tech stack, additional expertise and work to integrate these technologies may be needed.
Backstage is vendor-agnostic and can be hosted on a variety of cloud platforms, e.g. AWS vs. Google Cloud. Data can be ingested from any arbitrary source. These advantages, however, can be a double-edged sword. Backstage does not come with pre-built Docker containers to deploy, so teams will need to create a Backstage app, make customizations on it, build a Docker image from it, and then deploy to a Kubernetes environment. This will require the following technologies:
As explained here, your Kubernetes deployment will also need to be configured to access a database—Backstage supports PostgreSQL or Sqlite as the built-in database options.
Why do engineering teams choose Cortex instead of Backstage?
Backstage is a good option for teams that want to prioritize the effort of building and maintaining their own portal. But it's not the only option. Commercial developer portals have sprung up in response to the extreme overhead and increasingly niche frameworks of Backstage.
One such solution is Cortex—a cloud-hosted or on-premises alternative to Backstage. Over the last several years, Cortex has listened to the Backstage developer community, and has prioritized three things:
1) Completely custom catalogs fed by 50+ OOTB integrations: Unite all your engineering tools and data for always-up-to-date service or resource level information on owners, dependencies, operations, security, and more
2) Fully flexible scorecards with the ability to push targeted initiatives: Set standards based on any data in your catalog, and drive progress against time-sensitive initiatives like audits, migrations, and more. Get devs to actually use the portal you've created.
3) Developer self-service with scaffolding: Leverage custom templates for scaffolding new software without re-writing code. Set requirements like owner defined, and repo connected, and enable devs access to one-click actions like spinning up a resource or making an API call.
Compared to Backstage, the advantages of Cortex are a lower TCO, faster setup, higher adoption, and easier extensibility.
To get started with Cortex, check out our demo here.
