I always suspected a UK government would eventually reverse course on "coding in public".
So I've been backing up all government code. For six years.
Today I finally generated a report on what has been deleted. Its much worse than I realised.
I always suspected a UK government would eventually reverse course on "coding in public".
So I've been backing up all government code. For six years.
Today I finally generated a report on what has been deleted. Its much worse than I realised.
Before you WELL ACTUALLY me: Yes, a lot of this is garbage test repositories or things that don't matter.
But a lot is not. And you should take notice when (for example) the entire output of the Met Office Informatics Lab vanishes.
@jonty a graph of deletions over time would be interesting. For instance, is this a recent development or just normal slow churn of repos over time?
@ret I only just started logging the date the repos vanished today, so I'll have that for the future.
I know for a fact that a large amount (e.g. nhs) was in the last few months.
@jonty collating lists into per-department FOI requests inquiring why the deletions happened would be interesting.
There was a scare a few months ago about these new LLMs being good at finding security vulnerabilities in code.
Hopefully that means they were only temporarily pulled offline until these teams had a chance to run these models against the code before some malign entities could do so.
I’m not holding my breath on them coming back imminently, though.
@jonty uhmm WELL ACTUALLY this is a big fucking deal
@jonty In the unlikely event that you hadn't spotted this already, it looks like the Internet Archive might be helpful here, e.g. the Informatics Lab repos were still around in Feb 25: https://web.archive.org/web/20250203062854/github.com/informatics-lab
@m Already running code to populate approximate dates from a few sources, including that!
@jonty Exactly. The fact that some of the deleted repositories are trivial doesn’t make the disappearance of the significant ones any less concerning. When an entire body of work, like the Met Office Informatics Lab’s output, vanishes, that deserves scrutiny and preservation—not dismissal as “just old repos.”
@jonty HMRC & nhs don’t surprise me
@jonty
Maybe archiving code of public interest is a good idea. Here in Germany we also have a couple applications developed more or less openly (often the code is quite hard to find), maybe I should start archiving a bit of that.
Your post is bookmarked for me to read your findings tomorrow.
:neocat_heart:
@jonty thank you for service
@jonty how much has been archived on @swheritage ?
Maybe I've misunderstood something: I pseudo-randomly checked about a dozen or so of the listed "deleted" repositories in and they all seem to be available on the MS/GH web interface (I didn't try git clone) right now. I couldn't find any that were missing on MS/GH. In what sense are they "deleted"?
In any case, requesting archiving on https://archive.softwareheritage.org with a few clicks is wise.
@boud @jornfranke @swheritage You are probably looking at the mirror URL, not the origin. Look at the org name.
@jonty That’s an impressive six-year effort, and unfortunately the results sound like exactly why independent archiving matters. Public code should remain auditable and reusable, not quietly disappear when priorities change. A report showing what was deleted is a valuable piece of accountability. 👏
@jonty I worked at uktrade there are a lot more than 1 missing. Though in theory they were putting them back - I tink in practice they never put the code for any of the websites back and most of it is down after the issue they had about a year ago.
(Issues are normal, and under the previous CTO didn't result in taking everything down for ages)
@64kb I only know what's missing since I started archiving, don't suppose you can give me an example of something you think I should know of?
@jonty Privatise. Everything.
Then one morning wake up in a cyberpunk dystopia.
Everything sucks. Except for 0.1% of ultra-rich.
@jonty cc @kirschner, maybe of interest!
@jonty Bye bye transparency.
@jonty goddamn this is insane. Like, this is the type of thing the Trump administration did when they took over.
Except I don’t get what’s different now that could cause this. We haven’t had an extreme political party take control, and nothing publicly suggests they’d delete code like this. What’s going on?
@jonty Six years of quiet backups is what makes this useful rather than just alarming. Anyone can notice a repo is gone. Almost nobody has the before state to prove what was in it.
The 100% on informatics-lab is the number that stands out. That is not pruning, that is an organisation being removed from the public record.
@jonty Not surprised. Why would the government do anything that is genuinely for the public good? This is not what governments do or what they were created for.
@jonty We paid for that. Who controls it now? Has it been given away to private companies?
@jonty following the US, their mutant offspring, like a good little subservient…
@jonty it's almost certainly not a sinister change of heart, though, it's just various departments and agencies panicking about security and doing "something" about it. That's been quite a theme recently "we have to go closed source because we can't have bad actors exploiting our code"
...which is wrong, but it's not some big thing. It's just a phase of security theatre people are going through.
@kludgekml The intent doesn't make any difference: The policy before was that code stayed in the open and you worked in the open. That is no longer being adhered to.
Even when code had security issues, it got fixed in the open, or - at worst - temporarily pulled while it was being fixed.
I've been talking to government departments and doing this for years. Things have changed. Enormous departments are looking to go dark and *not come back out again*.
"I always suspected a UK government would eventually reverse course on "coding in public"."
I think intent _does_ make a difference if that's your headline, and I think some of the replies suggest that I'm not alone (one way or another).
I'm glad you were sceptical enough to make the mirror, and I agree that it's not good that the NHS and MoJ have abandoned coding in the open, but I think the meaning of that is less apocalyptic.
We can just wait and see which of us is right though. If GDS go dark I'll sadly concede!
@kludgekml ...the first line is "why I built this" - it's not a declaration that it's happened.
I mean, I don't know what to say here. It's like we're looking at different toots. I think you imply one thing, you say you don't. Fair enough.
I do have a procedural question that's hopefully simpler: under alphagov you list two repos (govuk-rota-generator and govuk_ab_analysis). But only one of those is vanished (govuk-rota-generator). The other is still public: https://github.com/alphagov/govuk_ab_analysis - why does that show up?
@jonty Great piece of work, thanks.
Edited: I see all the dates are in the git repo.
@jonty Thank you very much. That is really helpful, especially for @johas who coordinates the @fsfe 's work on "Public Money? #PublicCode"