You might be surprised (maybe not) to learn that the IDP you choose directly impacts the success of your implementation. Different IDPs prioritize user experience in unique ways, leading to significant differences in their architectures, integration models, and capabilities.
Does an IDP’s integration model really matter that much?
Absolutely it does.
While we won’t delve into every possible model, for the sake of brevity, we will compare two primary integration models - direct integration and agent models - and the specific use cases each one enables. Here’s what you’ll learn:
Agent-based model: how this model requires data to be pushed into the IDP, necessitating constant data ingestion for real-time visibility, querying, and analysis
Direct integration model: how this model allows for real-time visibility, data access and interactions directly from the source without constant data ingestion
Let’s dive a little deeper.
Why the integration model matters
It’s true – IDP’s have different methodologies for their integration models. But which model is better? In other words – which integration model satisfies your current IDP needs while providing a framework to introduce future IDP use cases with ease?
Consider Cortex’s direct integration model. We’ve been designing, building, and implementing our IDP with some of the world’s best engineering teams since 2019. We’ve learned a lot. When evaluating other integration models we found many customer’s use cases could not be met with an agent model.
Cortex competitors, like Port, use an agent-based model that requires pushing all relevant data into Port for querying. An agent-based model requires constant data ingestion and unfortunately, limits IDP users ability to: leverage the platform for real-time visibility and querying, impacting ad-hoc analysis capabilities.
So, why does the integration model matter?
It’s all in the use case, baby. You want to use your IDP to its potential, right? After all, driving alignment to standards, mastering production readiness, and bringing visibility and context to your engineering teams is what brought you – at least partially – to Cortex in the first place.
You’re wondering which specific use cases and outcomes can be achieved with one model and not the other?
Tell me you’re using an agent model without telling me you’re using an agent model. I’ll go first.
Imagine being able to pull arbitrary data from all of your third-party tools (could be code quality, alerting, git, security and compliance tools, etc) for scoring.
It’s very possible… but agent-based models struggle with this because they rely on pre-ingested data. Any new or ad-hoc queries require significant engineering efforts to manually configure the agent to capture and push this data, often resulting in delays and incomplete information.
And your team needed an IDP in place yesterday, remember?
With an agent model you will also need constant data ingestion to have real-time visibility into simple queries like:
Which monitors are failing right now?
Who is currently on-call for this service?
Who owns this service?
Is there a Slack channel for this microservice?
Can I alert them via the IDP?
I need to investigate every GitHub repository with Jenkinsfiles and ensure each is using the proper build steps. Can I do that in the IDP?
Or take it a step further:
I need to see a list of every service failing their uptime monitors AND haven’t been deployed in the last 24 hours.
Unfortunately, these use cases are only possible with creating an agent for every integration and constant data ingestion with an agent-based model.
You might be wondering: if these gaps exist today with different integration models, what limitations will we face in the future due to these same models?
Don’t worry, friend. We’ll dive into the future state shortly.
The emerging technology of IDPs
IDPs are still considered an emerging technology. And yet they’ve captured the hearts, minds, and attention of engineering teams (and engineering leadership) everywhere. Not because they’ve solved the long-standing visibility dilemma, but because of what they can do with the right visibility and integration model.
Agent-based model provisioning
Setting up an agent-based model involves multiple complex steps, including assessing requirements, determining data flow, and synchronizing frequency, among others. This process requires constant data ingestion and robust infrastructure to handle high volumes of data transmission, making it a challenging and resource-intensive approach.
This agent-based model uses a push method. Push models can be more complex compared to other systems, sometimes requiring additional infrastructure such as message brokers or real-time communication protocols. If an organization is handling a large number of simultaneous push notifications, it may require robust infrastructure to handle high volumes of data transmission.
Additionally - agent-based tools force you to modify the system being monitored, breaking the cardinal rule of integrations.
Constant data ingestion isn’t always a feasible solution for businesses. So how do teams enable real time visibility without constant data ingestion?
Direct integration model provisioning
Direct integration models
Cortex's direct, live integrations enable real-time visibility into the current state (e.g., on-call, service health, active incidents) without constant data ingestion.
Direct integration models are straightforward and typically work like this:
API endpoint exposure
SaaS platform exposes API endpoints that allow external systems to request data or perform actions.
Authentication
SaaS platform receives a request from an integrated system and verifies its authenticity using credentials or tokens.
Request processing
The SaaS platform processes the incoming API request, which might include actions like retrieving data from its database, updating records, or even performing specific actions. (Hey /cortex Slack Bot. I want to see service information like owners, current on-call, links, and timeline events. Easy. Use /cortex service <service-tag>.)
Data provision
The integrated system processes the received data and syncs it with its own database or system. This process might include updating records, triggering actions, generating reports, and more.
Direct integration models are easier and faster to implement. Free from the requirements of complex infrastructure for real-time communication, engineering teams have real-time visibility into their data when needed.
This not only makes scaling easier and less expensive but also puts control into the hands of the experts running the IDP. It’s a more tailored approach to match your organization's specific needs.
The future state and IDPs
Earlier, we hinted about the future state of IDPs and if these integration models will impact the next wave of features we expect to see from IDPs.
Let’s explore how the integration models we’ve discussed will shape the emerging, really cool future state of IDPs.
Self-service and automation
You see Cortex talk about self-service and automation as a core pillar of our IDP and you’re curious – are self-service and automation capabilities limited with an agent model as well?
You're a techie. And you have techie dreams (don’t we all?).
You want to leverage self-service and automation to enable developers to build better from the start. You can imagine your team using self-service capabilities like automated scaffolding, sending notifications (e.g., Slack) and creating repositories and resources using pre-approved boilerplate templates upon request.
Cortex's direct integration model unlocks advanced use cases like self-service scaffolding, automated notifications, and dynamic scorecards that are difficult - or impossible - with agent-based models.
Imagine:
Developers can self-service automated scaffolding which serves two purposes: significant time savings while driving standardization.
DevOps built and approved CI/CD, notifications, configuration, and versioning.
SRE’s pre-approved and resources, receive notification upon creation, and have immediate visibility into the resources with clear ownership and scorecards.
Engineering managers can drive production readiness and align teams to clearly defined standards.
Working in any of these roles, you’re all too familiar with the painful parts of the job. With the painful parts removed, engineering teams can unlock the coveted ability to focus.
Focus? Of all things, focus?
Yes, new friend. Focus.
What would you give to have a system that effortlessly and efficiently removes the painful, time-consuming, tedious parts of your job (context switching, jumping between apps, monitoring health of resources and services, and more). Imagine investing that time in top-tier, high-value activities that result in innovation.
See? Focus.
In fact, we recently participated in the very first Total Economic Impact report for Internal Developer Portals. And the results were amazing. Check out the TEI for Cortex here.
Achieving success with your Internal Developer Portal
It’s all about results; they determine how you measure success.
Success – for an Internal Developer Portal – is defined by the ability to efficiently solve specific use cases when needed. Unknowns are just that - unknown. You shouldn’t have to bear the burden of configuration management at the agent-level of an IDP before you’ve even used it. By selecting a direct integration model, you’ll benefit from real-time visibility into the current state while unlocking advanced use cases without additional engineering effort.
With the right IDP, you will not only be set on the quickest path to attaining maximum ROI, but you will also have the foundation prepared to embrace and achieve standardization, production readiness, and self-service in your engineering workflows.
See it for yourself: book a live demo to see Cortex in action.
