.avif)
In a single day, a single developer might work with half a dozen or more tools and integrations. Although each of these might independently streamline an aspect of development, a lot of time can be wasted switching between all these tools, introducing obstacles into the typical development workflow.
A developer homepage aims to address these issues by providing a unified, personalized view into a developer’s tooling. While a global view can offer high-level insights, a developer homepage presents insights that are specific and relevant to an individual user.
Unification
Unification is a core component of a developer homepage — its essential feature is unifying a bunch of distinct tools. Let’s say you’re working with Pagerduty, GitHub and Jira: without a homepage, you have to go to GitHub to review pull requests, log into Jira to see tasks assigned to you, and navigate to Pagerduty to examine on-call rotations and time-off requests. Even with just three tools, there’s a cost associated with using them distinctly, whether that looks like dozens of tabs open in a browser or extra time spend switching between services.
Implementing a developer homepage
Identity
Building out the concept of identity is the first step in implementing a developer homepage. This includes data about who you are, and how you fit into your teams. Who you are may vary across tools, but because name and email address are likely to be consistent, these details are used as the first set of filtering criteria. Identity enables you to map information across different tools to make sure data presented on the homepage is relevant.
Note that even this core information can be different across different tools. For example, you might have a middle initial included in your name on one platform, but not another, or an API might not return a complete response, or does not include an email address in its response. In that case, a developer homepage relies on OAuth instead, and prompts a user to authenticate the integration with the homepage. This method offers a more complete picture of relevant information.
To determine how you fit into your team, a developer homepage ingests team or group data from a third-party platform, and slots you into the appropriate teams. For example, Cortex will grab all GitHub teams that exist for an organization, and identify the teams you belong to.
Ownership
Identity informs the data that a developer homepage wants to fetch, or the data that you personally own. Rather than making a request for all pull requests in GitHub, you can query for all PRs that involve a specific user, which might include things you authored or assigned, or things that were assigned to you.
Once querying for individual ownership, the developer homepage should query for things related to teams you belong to. This way, it’s picking up things that you’re individually tagged on and things that your team is responsible for, giving you a more complete picture in your developer homepage.
Data fetching
With identity and ownership established, the developer homepage can begin fetching data, grabbing open PRs that aren’t archived and that involve a specific GitHub username.
During this stage, it’s recommended to combine queries in order to avoid rate limits. Instead of fetching on user-level queries, you can issue one massive query. By avoiding a rate limit, you minimize the cost for using GitHub servers and the number of trips that need to be taken. Note that GraphQL and REST API have completely distinct rate limit resources — make certain calls on each to take advantage of this.
A lot of the data fetching process revolves around identifying the right keywords. Cortex cycled through a lot of iterations as we noticed components that wouldn’t work and/or wouldn’t be best for particular use cases. This process took time to figure out how to leverage APIs to get the right information.
For example, a massive query that includes a bunch of users generated more results than GitHub API’s 1,000-result limit. To address this problem, we made a new query that was based on specific time intervals — since then, we haven’t encountered rate limits.
Large datasets
Large organizations in particular are likely to face challenges with extremely large datasets. In one case, Cortex learned that it was making 187,000 API calls a day to maintain updated data in the developer homepage.
We were able to address this by applying a classic computer science concept: rather than grabbing the full state of the world, only grab deltas. The change from full ingestion to querying for recent updates addressed this problem, but introduced another — if there’s a failure, this approach might miss a time window, so you have to refresh the query. A system has to be more tolerant in this scenario, especially in the event that the refresh fails.
In the long term, Cortex plans to implement something more event-driven, like webhooks, in order to further reduce API calls. Failure resiliency is still important in this scenario — to reduce outbound requests not generating information, that mechanism should do some pulling in addition to event-driven webhooks.
It’s also recommended that you set custom sync rates across different providers and information, and reduce or increase the frequency of ingestion appropriately. For example, you can fetch on-call rotations less frequently than GitHub PRs to maximize the benefit to developers.
Action items
With all of this data from all of these tools in a single place, a developer homepage can combine information and generate insights. In this way, a developer homepage provides developers with actionable data.
For example, a developer homepage might inform you if a given GitHub repo doesn’t have a code owners file, or doesn’t have two reviews required to merge branches. Through a Datadog integration, the homepage can show you whether SLOs are defined for specific services, and through Pagerduty, you can confirm that every service has an on-call rotation established.
Because the concepts of identity and ownership are established for a given user, a developer portal creates business logic that gives developers action items, one of the most powerful aspects of a developer homepage.
The future of the developer homepage
Moving forward, Cortex plans to combine the concept of polling with event-based data fetching to get the best of both worlds: reducing API requests and achieving greater consistency over time. We’re also looking to integrate with more providers, so the developers homepage reflects your entire ecosystem. Eventually, the developer homepage will incorporate all the providers in your tech stack in one place, from source control to monitoring tools.
We also incorporate ease-of-use features based on user feedback. Recently, we added badges to indicate when PRs are outdated, if they can be reviewed quickly, or if they’re deemed complex. This way, users get more information at a quick glance, and know what to set aside time for.
To see how Cortex can help you unify all your tools, book a free demo with us today!
