This is fascinating. There were trans and intersex athletes competing at the Olympics in the early years, but everything changed during/after the 1936 Olympics in Nazi Germany, when the Nazis decided to go after Zdeněk Koubek, a trans man competing for Czechoslovakia
https://www.vox.com/culture/364032/trans-athletes-olympics-2024