I miss the summer.

#unix_surrealism #comic #cve

whelp that’s such a massive TLS failure I first thought that it’s some fake ai-hallucinated news.

Update everything.

EVERYTHING

Every docker container you have that does outbound networking holds a forgeable TLS chain so security is dead.

https://emudhra.we-have-a-problem.com/

Single-homing all of the banking, transaction processing, ALL OF THE BUSINESS in a single GCP region. lol.

eu-west2 is the new us-east1

I should also fix the mastodon images... But at least I’m not running an international bank!

If the only service that's not "major outage" is your status page, is it really partial?

I wonder if the traffic team is having fun out there.

Here's a little nifty trick: you can patch your runtime systemd config with things like

systemctl set-property --runtime k3s.service 'DeviceAllow=/dev/zfs rw'

(apparently k3s needs to see /dev/zfs now, go figure)

You know shit's good when you have to strace the damn thing to figure where it's failing at.

I'll just update this little unrelated bit of confi...KUBERNETES EXPLODES VIOLENTLY

@kajer @homeassistant as someone who inflicts the pwned passwords db check on users under my control at every opportunity (at least until I complete this project to get rid of passwords altogether), even I think this feature should be explicitly opt-in in Home Assistant. Calling out to external APIs is exactly what Home Assistant shouldn't be doing!

I do think the amount of vitriol that's ended up being spilled about this is overblown however. The initial thread came in hot (not unreasonably so, even if it's not how I'd've gone about it) but everyone has ended up egging each other on into a proper rage about it.

I'm seeing this increasingly commonly now (OSS maintainer makes a bad decision for a good reason, and the response is a dog pile before a discussion even happens). I don't know if it's just the general malaise about the state of the tech industry getting to everyone, but it's concerning nonetheless.

Раптово (моя) епоха nvim почала для мене завершуватись. Раніше вже бачив helix, але от зараз помітив що він прям дуже легкий і більшість фіч насправді можу замінити іншими TUI/CLI.

So how did we get in here?

I had firewall rules that were named ROUTE64_xxx and would allow bgp sessions to route64. Over time I added bgptunnel and lagrange, but I just extended the address sets of the rules to allow traffic in.

My ipv4 net is single-homed and I don't feel that great about it, so I wanted to improve that and relocate the primary route64 route to my PoP in Amsterdam. Given I ran out of connections in route64, the plan was to tear down the old one and to spin up the new ones. I had 3 uplinks after all, right? LOL.

This is exactly the amount of negligence that my OCD doesn't allow me at work. It's just that homelab doesn't qualify for "production" in my mind and I was like meh whatever, yolo this thing. Stopped the old r64 bgp sessions old router, deleted them and created new ones in the admin panel. Then told chatgpt to update the config and move the router setup.

pulumi up tripped on a totally unrelated change and then I hit the stupid mikrotik export bug, so I wasn't paying that much attention to the actual diff. Even if I would, there are 100s of random IP addresses and I'd never really eyeball route64 change taking lagrange down too. It did (same address set named route64 - chatgpt happily wiped "all route64 connections from this router").

So now route64 is down, bgptunnel is down for a week (I never checked if it was functioning! It's just not replying on GRE), and the change brings lagrange down too, taking the total number of upstreams to 0. The /24 is withdrawn from the global announcements and in half an hour everything stops working (I'm happily asleep by then, though).

The recovery was easy, at least. Emergency manual rules for lagrange are in and I'm regaining control of my subnet. Route64 remains hung in the air as their AMS router doesn't want to talk to me and I don't want to talk to the FRA one.

Yay, my first vibe-coded network outage!

In all fairness, it's on me. I did review the firewall rule changes and I thought, yeup, that totally 100% looks legit and makes perfect sense. Managing too many things on one physical router is quickly becoming a large source of pain lately.

`ssh_dispatch_run_fatal: Connection to 2a0b:abcd:ef01:ffff::254 port 22: Connection corrupted`

huh. Is something trying to escape the machine? Is it a rogue AI? Is this all how antropic and openai chatbots escape their confinement??

The iPhone Duo proportions felt oddly familiar so I had to go search the closet.

Yep. It's similar in shape and size to my favourite handheld device of all times. The only difference is that Palm slides down to reveal extra screen, not unfolds.

Tfw you want to brew an espresso but your pressure sensor companion app asks to create a free account

As soon as I notified the offending party about the complaint, they immediately followed up with the information requested. It's funny that they use stolen linkedin data to reach out.

slowly but surely my infra moves off @tailscale and onto my ASN with self-manged east-west links. Tailscale is magic when it works but also magic when it doesn't and troubleshooting magic is very hard.

Besides, east-west gateways are very easy to set up. It's just a trivial¹ envoy config that validates the SPIFFE certs which are very easy to get working with a TPM² keeping the machine identity secure. And of course there are no routing problems³ when you manage your infra.

1) lol
2) lmao
3) that's actually true, as long as you have your own ASN and the routing problems just become properly scaled NOC problems like "why the fuck this upstream drops my ipv4 egress"

Feels oddly satisfying to file those GDPR violations to DPC.