We got paged the other night, so now I know a lot more about ZFS internals than I ever thought I would.
https://groups.google.com/a/chromium.org/g/ct-policy/c/RcKh1_LxoB0/m/7qlqM04fBwAJ
We got paged the other night, so now I know a lot more about ZFS internals than I ever thought I would.
https://groups.google.com/a/chromium.org/g/ct-policy/c/RcKh1_LxoB0/m/7qlqM04fBwAJ
@filippo If you'd like someone to throw this in the direction of the OpenZFS mailing lists and/or issue tracker, I can do it. The comments in the systemd service unit¹ are pretty clear and easy to follow, and it definitely sounds like an OpenZFS issue although I don't know how they'd go about fixing it.
(I don't see any cgroup related changes in the development OpenZFS on a quick skim, so it's probably not fixed there.)
¹ https://config.tuscolo.sunlight.geomys.org/#%2fetc%2fsystemd%2fsystem%2fmemcg-reclaim%40.service
@cks Oh yeah please do!
I’m also not sure what a fix would look like exactly, but would love for someone to look at it.
@filippo To make sure I understand this: none of the cgroups involved here have (cgroup) memory limits, so this is purely an issue of the dentries+znodes+etc being charged (only) to a .service cgroup and not being touched by global reclaim?
The good news is this may already be fixed, in a 2025 change: https://github.com/openzfs/zfs/commit/a7a144e655850b4160943e4ba315eb9a5dc2b2fe
(I believe this is in 2.4 but not anything before it.)