IRBs are a vital mechanism for ensuring that research has respect for persons and avoids causing harm.
But they are woefully unprepared for dealing with CS research.
Folks at Stanford and Google ran an experiment where the basically A/B tested car congestion in cities by tweaking the routing weights in Google Maps. Without informing anyone, and without consent.
Now: their results are cool! They do suggest that this can be a good way to ease automobile traffic! It can save time and emissions!
But check out this part of the paper:
"The study protocol and data use were reviewed by the Stanford University Institutional Review Board (IRB). Based on the analysis involving exclusively non-identifiable data, the IRB determined that this work does not meet the definition of research involving human subjects as defined in 45 CFR 46.102 or 21 CFR 50.3, and therefore did not require formal IRB review."
What are we even doing here if changing the traffic patterns of entire cities does not have "human subjects"