So this seems to do it for new connections:

sudo sysctl -w net.mptcp.enabled=0

But still leaves active sockets using MPTCP. In my setup, this works:

sudo systemctl restart forgejo