I approach from my street and if the highway is empty, almost instant green. Otherwise I have to wait some time until the road clears. During peak hours when the road doesn’t clear, it just seems to go green after a fixed time.

With such a setup, the traffic lights controller can't actually see that there's only 4-5 cars approaching on the main road – either they're far away enough that they don't even register on the detector for the main road yet (in which case a vehicle waiting on the crossroad will naturally trigger the phase change without delay), or even if they're already near enough to the intersection to trigger the detector, the traffic lights controller can't actually know that it's "only" 4-5 vehicles and its strategy might hence be that if the main road has had more than x seconds of uninterrupted green time, a demand from the side street will be granted immediately, regardless of traffic conditions.

With a state-of-the-art system it'd be different – with more and better sensors the traffic lights controller could actually track individual vehicles (or platoons of vehicles) as they approach the intersection and try making the kinds of trade-off you mention – but somebody needs to pay for that kind of upgrade. (And I suspect we haven't necessarily reached the point where the price premium for that kind of advanced traffic lights controlled compared to classic traffic actuation is negligible enough for it to be the default at least for all new intersections and equipment renewals, either…)

Of course, it helps that infrastructure adjustments over the decades ensures that many people that don't need to be in cars, aren't in cars.

Seriously though, speaking as a South African living basically next to Sandton, there are plenty of traffic lights near to me which cause complete chaos when they are out. They will cause significant delay for the entire working day on certain routes, e.g. anywhere on Winnie Mandela (formerly William Nicol) Drive.

There is however one intersection (Stirling and Balmoral Ave) on the way to my kid's school on a non-major road which definitely works better when its traffic light is out, and this is at any time of day in my experience.

Some traffic lights definitely should rather be mini traffic circles, e.g. on Peter Place by the one office park and had the exact same frustration for the one on Grosvenor Road near to Bryanpark.

As the Johannesburg Road Agency (like all City of Johannesburg Municipal companies) is so incredibly incompetent, our traffic lights are incredibly unreliable, I shudder to think how much it sucks out of our economic output all the people sitting in traffic longer than is needed simply due to an out of order traffic light.

And the local police also don't help with regular roadblocks while at the same time being terrible at law enforcement (yes, you'd think roadblocks are a sign of law enforcement, in practice though they're more for soliciting bribes while screwing up traffic).

If the lights are at an intersection where a very busy road crosses with a less busy road then turning it into a four way stop just turns it into a traffic jam on the very busy road. I’ve seen this happen during rush hour in the United States multiple times.

When it’s not rush hour though maybe the situation is different? Not sure.

Ideally, yes, practically, no. In reality you also have to consider what comes downstream of a traffic light. It doesn't help at all if one traffic light has perfect throughput only to put everything into gridlock in the next node.

That means in most real traffic systems each intersection lane will have to have less then ideal throughput because you don't want to overwhelm the next section, as this could be even worse for the system overall.

If you have some engineering background you can also think about this as impedance matching. If you have a 4 lane road narrowing to 2 lanes you have a sudden impedance increase, leading to flow issues. If you put an intersection inbetween you now have a way to manage that impedance mismatch with signaling times, but you have to both look at the impedance before and after the intersection.

The second reason: trying to incentivize people onto trains, buses, bikes, foot, etc.

Cambridge, MA does this. The light are purposefully timed to not let you travel through multiple greens on the tertiary roads in particular, less so secondary, etc.

Cambridge also gives a substantial amount of time to pedestrians as part of every cycle- no "beg button", because the city does not consider someone on foot to be less deserving of crossing time - specifically to not have the car "be king". By making it relatively fast to get around on foot, they reduce the incentive to drive.

They also heavily structure the residential neighborhood streets with one-way directions designed to make it very difficult to use the streets people live on as cut-throughs when the main roads jam up because that leads to noise, pollution, and danger to people on foot and bike, particularly because the kind of person to use a neighborhood as a cut-through is likely also driving quickly and aggressively.

The Old County lights at least are staggered so they are almost always green at the same time. The Central Expressway ones, not so much - there've been times I've been stuck for 20 minutes trying to cross the 100 feet from Moffett to Castro in Mountain View, because the light timing never lined up with the train signals.

I don't think you need a vast conspiracy to point out that connecting traffic signals costs more (fancier signal equipment, running cable, maintaining the cables, etc) and the benefits may be subtle, hard to predict, and maybe even hard to measure.

Additionally, you have a lot of people that would prefer to slow traffic. Depending on local culture, investing in traffic control can be difficult (although upgrading traffic signal systems may be better able to accomodate priority for transit and bicycle detection)

In San Francisco (as well as other cities I've lived in) they put police in the middle of intersections during rush hour to guide traffic (i.e., stop cars before they drive into an intersection that they'll get stuck in the middle of). Seems woefully inefficient