The technical musings of Iain Cambridge.
177 comments
I’d remove “go” from the above, i.e. I think same applies to other stacks.
Even using GitHub domain links in code comments gets problematic long term. Ie when a migration happens and those links start pointing nowhere.
I have seen too many cases of requirements on internal projects that prohibit devs from working because oops the vpn is down now or oops gitlab is under load and the pipeline for your dependency won't finish for the next hour.
Even if you don't want to go as far as vendoring (which can get a bit out of hand with, say, NPM), a middle ground is using Artifactory or whatever as a proxy.
Then be more deliberate about when you update. AI may even help here where static analysis doesn't, because you might be able to use inference to see if a dep upgrade is even needed. Unless it has a severe vuln you probably don't need to track latest.
It's not only an issue of availability or depending on circumstances you can't control, but a matter of reproducibility hence dependability.
I want to be able to compile critical software when a zombie apocalypse hits, and I have a growing discomfort about trusting outside parties for their availability and benevolence.
Everything else is fine. Generics, errors, whatever...
Why can't you just search-and-replace? Presumably all of them refer to "GitHub.com" and not much else code will, so I'd think this was an exceptionally easy case.
Even easier for comments, since them being obsolete for a few hours during a migration doesn't exactly break anything.
It was horrible. A team of people spent weeks. Different projects depended on different versions of the same internal libraries, we had to create a branch for each depended-on commit and make a version of that commit with the new URLs. The expressed goal was to end up with a system that's "exactly the same" as the old, just on a new host. Changing which version of libraries projects depends on would introduce unnecessary risk.
And the result is a code base where bisects are broken and where it's impossible to build an old version of any of the code without a ton of work.
The "module name is network path" is a convenient convention but not at all some "limitation" of the tooling.
Even in the case where it’s an internal only Lu array, it can be complicated to refactor a library name.
In this terminology, the article basically amounts to saying "This namespace can fail! The solution is, use another!"
But that doesn't get you anywhere, because the new namespace can fail too. In fact it is almost certain that it will fail sooner than "github.com's DNS and hosting" as a namespace.
There is no solution where you tie your software release to a namespace that can't fail because there is no namespace that can't fail. It doesn't matter if your favorite language uses DNS or has a centrally blessed repository or if it distributes a blessed list of package names with the language itself or anything else. The namespace can fail.
Therefore, the only thing you can really do is be resilient against failures, and in a lot of ways, the only practical solution to resilience is just to assume that if, in the future, someone has problems getting a package due to a namespace failure, they will not helplessly disintegrate into a puddle of tears while your code is lost forever, but that the future person will instead solve this perfectly solvable problem.
> it will fail sooner than "github.com's DNS and hosting"
When it does fail you can fix it. When github fails all you can do is stare at their status page.
You can also just use "replace github.com/example/example => gitlab.com/example/example" in your go.mod file and everything will keep working. That seems like a very pre-mature optimization for something that doesn't really matter.
The idea is: the package maintainer pushes a final version of the old package that is a shim, and which has a deprecation notice. The shim imports the new package, and forwards all calls to the new package (and I suppose type aliases and variable aliases as well). The shim includes `//go:fix inline` annotations on all exported symbols, so that when users run `go fix` it rewrites their code to use the new package, by inlining their usages of the old package (which is now simply a shim that references the new package).
Not perfect. There is still a bit of a discoverability problem. Not all tooling warns on deprecated packages/functions (go toolchain doesn't, but gopls and staticcheck do). And users need to know to run `go fix`.
1: https://go.dev/blog/inliner#example-renaming-ioutilreadfile
// Deprecated: use example.com/mod/v2 instead.
module example.com/mod
As library user, it's your responsibility to keep your stuff updated. There may also be utilities in the various automatic dependency updaters that can migrate these things."replace example => github.com/example/example" at first, then change to "replace example => gitlab.com/example/example" or similar when there's a migration
(i don't use go, but i hope this is possible and i am confused if it's not)
You know, normal development task, small amount of story points..
One day, we’re all going back to vendoring dependencies.
Maybe at that point it won't be using git either, or maybe it will - who knows.
I use dotnet and I never liked seeing dlls and binary files in my diffs. I would argue if we are adding vendor code to our projects, we should demand the FULL source code instead of dlls. Maybe it is already possible with things like x unit. I have never given it much thought... But then that vendoree code has to come from somewhere as well, right? I mean there is something to be said about provenance or something here?
Sorry if this feels like a stream of consciousness because it is ↔
The answer to the question of why THAT is the case - is not so easy to answer. The main benefit I can see with using lock files instead of vendoring is that it saves a lot of storage and diff history from entering your repository. So clones are much faster, backups smaller etc.
I think Go used to work this way (automated vendoring) but it’s the only language I can think of that ever did in terms of standard tooling. It would be instructive to learn why that changed.
Deepcopy [1] is a Go package that is still heavily used despite its creator has disappeared 9 years ago. At least GitHub is a stable and trusted host as a distribution point and communication point for users.
Signed modules / including the signature hash would also solve a lot, e.g. it'd mean domain sales no longer silently inherit full permissions. It's sorta a shame that Go keeps doing such a good job at a minimum-viable wheel-rewrite, but then lets it linger for so long without catching up to the rest of the programming world.
How many mainstream languages have content addressed imports? I can’t think of any, so I assume I’m misunderstanding your meaning of the term because you seem to be suggesting that it is common and Go is the outlier for lacking it?
You already can use SHA in go.mod in exactly the same way you would use a version string.
I don’t know about using multiple proxies in go mod though.
If github is "taken down" you can't just resolve the whole domain differently, you need to only resolve the packages differently. If your "Golang domain" is taken down, it should be a lot easier to hotfix until something proper is implemented (the quickest and dirtiest would be a hostfile-entry).
It is being retired because very few people use it. It kind of sucks but this is how domains work, it’s not real estate.
Read the full thread on Hacker News →
Related stories
- Lobsters · 86 points · about 1 year ago
- Don't couple your Go code to GitHubiain.rocksLobsters · 48 points · 3 days ago
- Hacker News · 8 points · 6 days ago
- Hacker News · 14 points · 8 days ago
- DEV Community · 17 points · 10 days ago
- DEV Community · 14 points · 11 days ago