A how-to manual for engineer-led discovery in software platforms

348 points•amortize•3 days ago•73 comments•

73 comments

dabedee1 day ago
> Platform teams are engineering-led rather than product-led. There is almost never a product manager handing you a roadmap, no revenue line to follow, and no market to lose.

It's precisely because of this framing and mentality that platform teams don't actually serve people well and are usually highly dysfunctional towers of people inventing work.

The fix for having no market is to act like the teams you serve could leave. This whole article lists signals, and none of them is that. Being captive does not mean the users or internal teams don't have other options and don't notice. Being product-led means caring about your users. A platform teams should be product-led, not engineering-led in that very narrow meaning. Otherwise you invent work as this article so wonderfully exposes.

DanielHBabout 18 hours ago
Previously I worked at a very large company, my team was mostly isolated form the rest of the tech stack of that company.

Suddenly came a mandate to try to integrate our systems together, the first goal was to use their authorization system (which involved, I kid you not, setting up 3 separate EKS clusters that talked to each other and if the main one goes down, all go down) that the core Platform team of the company was setting up.

Worse even, the plan was to use us as "guinea" pigs for their new systems before rolling out to the rest of the company because our team was smaller and therefor wouldn't be as impacted by problems on their side.

I was vehemently opposed to this, all our other experiences with their stuff was just a huge amount of pain. I remember clearly a meeting we had where I said: "Why would I use your stuff that is not well documented and that I don't understand when I can use open source stuff that is documented and that I can understand". Their only argument was that said they would have internal support for us.

I came up with this concept that I tried to push on to them, that their stuff shouldn't be this monolithic tower of babel monster, but instead should be a buffet where I can pick and choose what to use. Our requirements were very different from their stuff and I didn't want to run into problems because of stuff that had 0 benefit to us.

I ended up leaving that job mostly because of this.

WorldMakerabout 11 hours ago
I've had similar fights over the years, on both sides of the discussion.

"If our internal design system is barely half as well documented than Bootstrap then teams will still use Bootstrap. We should take a page from Bootstrap's documentation and include as much detail as we can, on everything."

"As a platform team, your product is this API my app calls. My beta environment should be pointing to your Production, never your beta environment. If that means you need to support better multi-tenant from the same 'app', then you need better multi-tenant. Your outage impacts my testing, which impacts my velocity, which impacts my deadlines. Your backwards incompatible updates need to be planned and scheduled and rolled out like a Production update, every time."

Product mentality is still very useful in platform development. If you can't sell your platform on its documentation and its stability and you must sell your platform on mandate and top-down control, you probably aren't doing as much to help your engineering culture as you think you are.

mahboiabout 8 hours ago
This sucks. Our team strategy in these situations was to just wait and hope the mandate goes away. Or I'd personally de-prioritize that stuff. Meanwhile a couple of us supported our smaller team's platform.

Things never got too bad for us, but they easily could've, and it'd probably be like your scenario where I quit. Was lucky enough that a couple of big layoff waves didn't impact us but did drastically reduce the amount of bogus technical mandates coming in.

aleqs1 day ago
You're assuming product people don't 'invent' work, which is faaaar from true. The more technical the area, the more nonsense 'product' generates, that engineering then has to either redo, clean up or fight. The 'product' people on 'product-led' teams also generally own the internal comms and marketing (internal) side of things, which often has the effect of silencing engineering (and many other negative side effects). (Not always but often imo)

Engineers should understand their customers and their product (whether internal or external), should understand their metrics, and should be able to make intelligent, informed decisions. Sticking a non-technical 'product' (aka marketing) person into the mix is generally net-negative imo.

hilariouslyabout 14 hours ago
I think the problem is one of management and prioritization and decision making - when multiple engineering teams disagree, how do you decide? Management consistently hires people that solve management problems, and mediating disagreements or picking product direction are core management functions that have been completely left behind as managers become purely MBA number people.
majormajor1 day ago
I don't think they're making any claims about non-platform teams never having similar work-invention problems.

They're just saying a good platform team engineer cares about their users.

Which is fairly at-odds in spirit with the quoted idea of "there's no market to lose."

The original article does say "talk to your users" but it also de-emphasizes this by having it among an apparent laundry list of other signals: crashes, costs

Focus on cost without talking to your users - aka the people who care about the spend? You might spend a lot of time reducing a number that isn't very important right now. Focus on crashes but don't talk to your users? You might address some things with easy workarounds ("i hit retry") while ignoring much more painful toil. This is captured in the details for those sections, but not in the headlines.

There's also some interesting stuff in there in some of the bullets, like overloaded use-cases and partner-to-prototype, but again, that's just more specifics on how to talk to your users.

I think the article would be a lot more helpful to a lot more people if it was a "Guide to Talking to Your Users" and then framed each of those specifically as: user discovery, and how to talk about the given part.

dominotwabout 13 hours ago
product lead doesnt mean lead by 'product people'
oldnewthingabout 14 hours ago
Somewhat true. Platform teams have to be engineering & product led. I run a platform team and we work very closely with our customers to understand what problems they are facing and identify platform level solutions for those problems. But we are still hampered by the team not using the platform for their everyday work, reducing the empathy they have towards its papercuts. However, being purely product focused can lead you astray of your users. Our PMs invent capabilities that no one really asked for ignoring the rather large backlog of capabilities that they already need. That's also bad.

The problem with every platform team is that, impact is hard to nail down. Does X using your platform Y to generate revenue mean your platform Y has indirect impact same as X? Most companies don't think so and end up destaffing their platform teams. Until, the destaffing ends up in much more inefficiency because everyone is inventing their own crooked wheel, spending time on the same capabilities and detracting from product development.

dominotwabout 13 hours ago
> Platform teams have to be engineering & product led.

No they have to be product led.

> Our PMs invent capabilities

this is not a good reason for it to be not.

carlmrabout 21 hours ago
>It's precisely because of this framing and mentality that platform teams don't actually serve people well and are usually highly dysfunctional towers of people inventing work.

Spot on, I've worked in multiple small, medium and large companies. And these internal platforms almost always turn into ivory towers of useless churn.

Even if being forced to use it internally, in a lot of cases the projects ended up buying the same solution (or a working version of it) from an outside vendor when it came to fulfill actual customers needs on the outside.

If that wasn't possible we built what we needed ourselves.

If you have internal customers, I still believe you need internal incentive alignment. Otherwise what the platform builds and what is actually used and needed drift apart.

And you need at least one real customer project to dogfood the platform initially. Because most platforms start out way too generic with pure architecture astronauts [0] at the helm.

If you don't have a project-0 to test those assumptions against, you don't need to start that platform.

[0] https://www.joelonsoftware.com/2001/04/21/dont-let-architect...

WorldMakerabout 11 hours ago
> And you need at least one real customer project to dogfood the platform initially. Because most platforms start out way too generic with pure architecture astronauts [0] at the helm.

The Rule of Three in Refactoring applies to this scale, too. If you are building a "platform" for one customer project, that's a one-off and quite possibly YAGNI. If you are building a "platform" for two customer projects, that's likely coincidence and still possibly doesn't show company-wide need. If you can build for at least three customer projects, that is a pattern, that provides real guidance on exactly how generic things need to be and better ideas how to abstract it.

flowerlad1 day ago
> Being product-led means caring about your users. A platform teams should be product-led, not engineering-led in that very narrow meaning.

And where do product people get ideas for new features? If you're Microsoft you look at what successful competitors are doing. If you're anyone else you look at customer pain points, exactly as described in the article.

dirtbag__dad1 day ago
> The continuous struggle is to find ways to increase the value our platform provides to the users of the system.

IME as a staff+ platform engineer*, it is dead obvious what increases user value. What is truly challenging is building a story around why anything should be worked on at all, when platform sits the furthest from customers.

I previously worked with a seasoned PM from FAANG who had low technical chops but somehow placed themself in the final decision maker seat for platform work.

They relentlessly blocked work that I described repeatedly in details: data quality was hurting, latency was dog shit, etc. But “bad data model? What do our customers care about our data model and pipelines that are confusing to maintain?”

It wasn’t until I showed some basic charts about our core database being oversubscribed and at risk of a more serious incident. At that point, we finally had the same understanding of the problem at the highest level and I was granted (lol) approval.

This admittedly took me almost a year to figure out. Others just trusted my judgement. The takeaway for me was really good tho. Even technical people probably don’t know wtf is going on in your domain, and metrics gets everyone on the same page, because numbers and charts are easy to understand.

* Platform is used too broadly so hard to say what the author exactly means by this.

Gareth321about 20 hours ago
I'm the PM side of this debate with my TL. I've tried meeting him on every level of understanding. Graphs, story telling, Jira issues, business cases, ITIL, OLAs and SLAs, customer interviews, user interviews, you name it. He strongly resists having to justify the things he wants to work on. His justifications are usually some combination of "it's too complicated to explain," and "this code is a mess." Problem is, I need to account for their hours. I need to explain why we're spending money on that thing - to people who are even less likely understand what my TL is saying.

I say this not to diminish your frustration but to appreciate that you tried to meet him at his level, and provide some context for what the other side can feel like.

dirtbag__dadabout 3 hours ago
I can validate that is frustrating. An inability to articulate more than the code is bad and it’s complicated is surprising from a TL.

That is probably the case in most companies, and as I said it’s not that hard to identify what is impacted by that dog shit code.

Having a spike/PRD/TRD process, really anything written by a human long form, can help build alignment. This is because you have ample time to ask questions and dive deeper, and it’s not personal it’s just part of the process.

I’m not sure whether the charts I created were themselves the convincing piece of evidence. They just happened to be where I found common ground easier and then maybe the rest just clicked

sarchertechabout 12 hours ago
Presumably the TL has a manager. This whole I need to justify what I’m working on to my manager and my other manager is wild to me.
ambicapterabout 12 hours ago
> It wasn’t until I showed some basic charts about our core database being oversubscribed and at risk of a more serious incident.

What were the charts that convinced them? Just high load % on your databases?

dirtbag__dadabout 3 hours ago
More or less. I set up DBM on datadog and built a dashboard with common db perf metrics. You can set thresholds as dotted red lines across the y-axis of charts to articulate moments of danger.
makeitdoubleabout 15 hours ago
> metrics gets everyone on the same page, because numbers and charts are easy to understand.

Those are just not to convince a manager though, it gives numbers to set KPIs, or at least some benchmark to evaluate the improvements, and at the end of the day solid reasons to justify promoting the team.

From both sides of the fence, fixing problems that only the engineers on the ground can properly value leads to everyone being miserable in the long run.

fsloth1 day ago
"that work does not exist unless an engineer invents it."

This is so strange. To my mind the only purpose companies hire engineers is to support business. The staff engineer should not need a project manager to tell what is interesting for business aspects - even though the goals are likely mostly technical.

I do realize this does not hold up always. But to me if you can't provide some reasoning for your work in business metrics you are participating in an academic exercise.

OtherShrezzingabout 15 hours ago
>To my mind the only purpose companies hire engineers is to support business.

Lots of hires, especially in places big enough to have dedicated teams for platform work, are for empire building, rather than to support business needs. In these situations, companies hire engineers because a higher-level manager needs to increase their headcount, in order to pad out their CV for their next move.

[0] https://www.investopedia.com/terms/e/empirebuilding.asp

merrvkabout 11 hours ago
Absolutely spot on.The time main output is unnecessary busywork that is purely designed to pad out CV’s
devmor1 day ago
I think you're operating on a different definition of "work".

The OP is discussing tasks to do, you sound like you are discussing things that need to be done. The OP isn't saying there aren't things to be done, they are saying that no one is going to tell them what needs to be done, so they must discover and formalize it.

"Inventing Work" is a bit tongue-in-cheek.

nmehner2 days ago
"inventing work" = "requirements engineering"

"Inventing work" is a strange phrase to use imho.

mytydev1 day ago
I was thinking the same thing. The word that I would be more inclined to use is used multiple times throughout the article. Discovery.
juancn1 day ago
I use the "what's going to kill us next" philosophy. Figure out what that is and do something to avoid it.

Wash, rinse, repeat.

r3trohack3r1 day ago
“Deal with the alligator closest to the boat”

Read the full thread on Hacker News →

Related stories