Internal Developer Platforms have a lot of benefits for the organization like increasing development velocity and developer productivity. But, as with any other project, on the road to success, there are many challenges that you need to overcome. In this blog post, we’re going to talk about the top 3 challenges we identified in our discussion with platform engineers and vendors in the industry.
Challenge 1: Catch 0 birds with one stone
When starting your journey to building an IDP, you have already done research or you have something in your mind that you think would be a great platform. Other stakeholders may have a different perspective on what would be a great platform which can be challenging.
Many platforms suffer from a lack of focus that causes the team to deliver a platform which got nice capabilities but none of them is valuable for the end users. For example, you decide that your platform should get data from many platform tools and aggregate them into a single UI. Then you implement integration with your logging, metrics, CI/CD, source control repository, and 3 automated workflows. Implementing those integrations can be time-consuming and eventually, you understand that smart links to these systems would be enough to solve the challenge and starting with API docs would create more impact.
Many platform teams start to run and then, delivery becomes challenging because they try to deliver an all-in-one solution that looks AMAZING instead of focusing on the main challenges and solving them in the best way.
Just a note - a portal is important but something it’s not the first challenge to solve.
How to Overcome Challenge 1?
Here you should make sure you focus on what matters for your users & stakeholders. Here is a list of questions that you must answer before starting to implement anything:
What are the main challenges?
Which of the challenges we want to solve first?
Who are the main stakeholders?
What's good in the current solution?
What would be the most valuable solution for this challenge?
Is catalog the answer to it?
Can we validate that with a user?
How long is going to take to implement it?
Based on those answers, you can start to define what to implement.
Challenge 2: Build & Operational Resources
Building a platform requires resources. The team that’s going to own the platform needs to perform tasks in various fields in the platform like infrastructure, cloud, DevOps, and even frontend development. You can build a platform but if it does require a platform and you don’t have someone who knows frontend development, you would find yourself struggling to deliver the last piece in the puzzle.
The team is going to put in neverending efforts in improving the platform and the delivery in the organization. Some organizations allocate a group of 3-4 people from different teams for 2-3 quarters to create an initial product, it does deliver something but not always for positive ROI.
How to Overcome Challenge 2?
To manage your resources wisely and benefit from the ROI, you should plan and execute well.
The first thing you need is a team & resources. I would say that the most impactful team I saw had a product manager and 3-4 engineers that knew how to deliver and own the product. They know the organization pretty well. With a good team, you can plan, deliver and create an impact. Please remember, that delivering a good platform will increase the development velocity. It can be the most impactful internal facing team in the company.
Usually, if you have a clear goal you know what you should deliver and what the ownership of the team is going to be. Here are 3 different models:
1. Dev Portal Team - Usually responsible for the portal only and not the full platform. it means that platform capabilities are out of scope.
2. Platform Team - The team is usually responsible not only for the platform but also for other capabilities.
For example, workflow, catalog automation, observability, and other platform capabilities.
This team varies based on the required capabilities, complexity, etc.
3. Platform + Cloud Team - This kind of team is responsible for everything from creating cloud resources to the portal itself.
I would say that there is no one-size-fits-all, pick what fits for you.
If you create a team that is responsible for something that an existing team owns today. You may take responsibility for some activities of the existing team with the resources like team members.
The 2nd thing is the tech, you should pick the technology that will allow you to deliver fast and use your pros while mitigating your cons and risks. The thing that usually takes most of the time is building a portal. You can create your own portal or buy one, Let’s check the options we have:
1. Build - You need to build everything. What Spotify did when they created Backstage.
2. Backstage-based implementation - Opensource implementation model, you use the core Backstage and integrate tools and plugins into it.
3. Managed solutions - As with any managed solution, here you pay for the platform portal services.
The first thing is to understand how many resources you have and what's your goal because it's not an easy question to answer.
Let me show a few examples of teams.
Challenge 3: How to build & manage the software catalog?
The catalog is a DB with all of your software, there are plenty of objects such as services, APIs, systems, applications, and dependencies. All of those need to be managed. You can do a one-off but it will not be enough, there must be a way to make sure the catalog is up to date to unlock its value. The saddest story is a portal with outdated content. The developers will lose trust and not use it at all.
Maintaining the catalog is a long working effort that should have a smart solution to make sure that it’s dated and automated. Examples of the most common pitfalls:
A new service is created and not added to the portal
A new API was added and the old one was deprecated but only the old one was available in the portal
The dependencies aren’t relevant anymore
Each one of these will cause your effort to create a platform to turn into a graveyard for old information.
How to Overcome Challenge 3?
We want everything that we own to be up to date. So the first thing is that - don’t put into the platform things that you can’t update as needed. For example, if your platform holds API information and you don’t have an easy way to import the swagger (maybe automatically) - you may want to hold the feature until you have a solution because it’s going to be outdated in a month. Release
How can we manage our catalog information?
You might have a few things on your mind. Let's break down a few solution options:
1. Automated - We don't trust people, we need to find a way to automate it.
One solution can be to fetch information from live data like your deployment or integrate it to your CI/CD guaranteeing that every change will generate a swagger file and push it to the API docs.
2. Enforced - If it's not automated, enforce the update.
Services will not be created without the file, changes to an API file will require a commit to the docs too.
3. Make it standard - In some org, things do not exist if they're not in the catalog.
I personally don't believe in this method for a long time but it works in some companies.
When you build a service that is consumed by other teams in the organization, it may be your artifact besides the running service itself.
Comments