The first rule of the papers-reading club is that you’re unfortunately already in the papers-reading club and I will send you links and summaries

Yet again hearing how some skilled work never truly mattered now that it's getting automated, while it absolutely did and was a point of professional pride for many.

It's a significant aspect of 'deskilling', which is a known consequence of automation. Pretending otherwise is needlessly unkind.

this seems more accurate given the frequency of errors right now

Rasmussen maps these as a space across two dimensions: abstraction (purpose to physical) and decomposition (whole-system to individual components). Each component can have a physical set of properties, but also a high level purpose it is connected to; so do aggregated portions of a system.

He then describes how workers do their work by skipping around abstraction layers, going up and down the hierarchies based on both purposes (high-level goals and functions) and reasons (bottom-up signals) but also across various aggregations (whole systems vs. components).

The cool thing though is that he then maps a problem-solving path as a line walking that conceptual path within a "conceptual reasoning space" of operators that is event-independent. Neelam Naikar (https://doi.org/10.1016/j.apergo.2016.06.001) re-renders diagrams Rasmussen had made in a few publications in the mid 80s:

This is the "abstraction-decomposition space". It maps a troubleshooting workflow from a system symptom ("no communication to tape or disk") via a diagonal walk to isolate a faulty component ("short-circuited transistor").

This is such an incredible visualization of so many complex elements!

Although there have been changes in cognition and systems theory, I wanted to bring up some stuff Rasmussen published in the 80s that was really elegant.
I'll use 3 diagrams he published, covering the ideas of abstraction hierarchies and how people operate systems when troubleshooting them.

The first one is the abstraction hierarchy from Coping With Complexity (1981). This shows how requirements and design tends to go top-down based on goals that must be met by the system. However, signals of problems tend to go bottom-up, from a local issue showing up as disrupted functionality.

The 2nd one (same paper) shows aggregation, where a nuclear power plant can be described as a whole conceptual system (top), as functional sub-units with material elements (middle), or as granular units that get closer to the material world (bottom).

This is a very good text on comparing ecology and organizational dynamics when it comes to harvesting signals; big fan of this one.
https://psychsafety.com/a-practical-guide-to-looking-at-things-properly/

Wrote up a bunch of stuff about some patterns in system design, about the tension, contrast, and possibility of composing approaches of analytical decomposition to increase control, and of complexity-aware stances for emergence, and some pitfalls of either stance:

https://ferd.ca/control-and-complexity-tension-in-systems-design.html

Quick little brainstorm-y blog post about traditional versus resilience engineering focuses (foci?): https://surfingcomplexity.blog/2026/08/02/traditional-versus-resilience-engineering-views/

Delegating pressure to the final individual, who is now tasked with continuously fixing the entire system’s misalignments through their personal choices.

RE: https://mastodon.social/@grimalkina/116919270665096152

I was a reviewer for this book so I can give you some spoilers about it: it rules, get a copy of it, start telling everyone about cumulative culture, unbuild the myths that keep your teams down, make shit be meaningful.

One of the ironies about AI agents in ops tasks is that it feels like there has never been as much interest in creating a forgiving environment with proper structural support than through promising to remove humans from it, finally forcing a less individualistic and blameful approach to design.

at this point why not just host my repos on the staging servers, github

Your AI ops tools are making your team worse at incidents; not because the tools are bad, but because they're good. The better the tools get, the harder it is for your team to handle the incidents that the tools can't.

The productivity gains are real. But they come with a side effect most companies aren't accounting for.

The operational work that AI is taking over was also how your engineers built deep system knowledge: the dependency chains, the failure modes, what "healthy" looks like. That knowledge was a side effect of doing the work. Nobody planned it. Nobody measured it. Now that AI handles the work, nobody is building the knowledge. You discover that the hard way.

This pattern predates AI. For example, auto-scaling eroded knowledge of scaling dynamics long before AI came along. But AI is automating a broader range of work, faster.

Incidents are, by definition, the situations the automation can't handle. As AI gets more capable, the left-overs get messier. This is true even if you aren't using AI for incident response. The knowledge erosion happened before the incident started.

@mononcqc pointed out a revealing asymmetry: coding assistants are marketed as augmenting the engineer; AI ops tools are marketed as replacing the work entirely. That tells you what the market thinks operational work is worth: not much. If your company agrees, it's going to underinvest in the human knowledge that effective incident response requires. https://ferd.ca/the-picture-they-paint-of-you.html

In aviation, autopilots gradually took over more routine flight tasks and pilot skills suffered. Lisanne Bainbridge studied what automation does to human skills and gave the dynamic a name in a seminal 1983 paper: "the ironies of automation." In aviation, this played out over decades. With AI in ops, it's playing out in months and weeks.

So, deploy AI ops with your eyes open: keep your engineers connected to their systems. Run exercises without AI tools. If your team can't function without them, you want to know that now, not during the next hard incident.

More in my blog: https://grtcrcl.com/4vtRRGe

And definitely something I'm writing about in my forthcoming book: https://im4ds.com/

Even after years, one of the weirdest parts of gardening to me is needing to harden the seedlings before transplanting them. Like “yes hold on a minute I gotta take the plants out so they can play outdoors for a while, but they gotta be in before streetlights turn on” is a real and necessary thing.

The ongoing stream of software engineering pieces that mention that the future is in writing spec but never bother to define what a specification is or at what abstraction levels it should be is appalling; arguably, tickets are a spec, the code is a spec, and work between both is connecting dots.

The underlying argument as far as I can tell is that we might now operate at a higher level of abstraction in spec definition, but without stating which level it is or how we can switch across abstraction layers for various degrees of specificity, this whole thing is meaningless.

“decisions with lasting social consequences are attached to a future moment when the technology is assumed to have revealed its true form, rather than addressed in the present in which it already operates. […] it defers responsibility by attaching accountability to a moment that never quite materializes”

Good opinion piece that lines up with a lot of complaints I’ve had about this whole thing: https://link.springer.com/article/10.1007/s00146-026-02970-6

I was reading discord’s incident review (https://discord.com/blog/behind-the-scenes-of-the-3-25-26-voice-outage) and it had this quote:

“To confirm this finding, we reached for Recon, an amazing tool for diagnosing issues in Elixir and Erlang production systems (the related Erlang in Anger is more-or-less required reading as all on-call engineers end up scouring its pages eventually).”

This is kind of wild. I wrote these over 10 years ago to help coworkers and to support oncall at a past employer when I was still trying to convince the org to keep investing in our team and its solutions.

Even though I eventually gave up and moved on, seeing the stuff I designed and published to support real world investigations and improvisation in the heat of the moment (in running systems that were about to collapse!) still see use, years after I last personally made use of them, is really neat.

Infinite love to my SRE coworkers, one of whom casually dropped this single line in a retro: "the industrial hourly deploy train and its consequences have been a disaster for society"

Self-blame isn't blameless. Saying "my bad, I'll be more careful next time" looks like accepting responsibility, but really it's just stopping everyone from learning.

Most writing about blameless culture is about managers not blaming engineers, and engineers not blaming each other, because blame shuts down learning. Your company has probably gotten reasonably good at that. What most still miss: engineers blame themselves, and the damage is the same.

"I should have caught that" is a thought-terminating cliché that sounds like insight, but isn't. The moment it lands, the exploration stops. "Sam made a mistake" becomes the story, and the systemic factors go unexamined: the slow dashboard, the missing canary phase, the runbook last updated 18 months ago, the schedule pressure.

Self-blame looks like accountability. Your best engineer absorbs the weight. Your VP walks out thinking the culture is healthy. It isn't.

This is the same "heroic" pattern you've trained your team to avoid during incidents, just moved later in time. The hero who fixes the outage alone is the same hero who, in the review, takes the hit so the team can move on. Individual heroics both during the incident and after. Neither builds organizational capability.

The fix: when you hear "my bad, I'll be more careful next time," someone needs to gently redirect. "What were you seeing at the time? What information did you have? What made the action seem right?" That's where the contributing factors live. That's where the learning lives.

More in the book I'm writing about incident management (im4ds.com), based on experience at Slack, Google, and consulting clients (greatcircle.com/im). Longer version: https://grtcrcl.com/4vJb2gb

When did you last hear "my bad, I'll be more careful next time" in a post-incident review — and what happened to the exploration next?

Writing my action items on a piece of paper that I will burn to ward off spirits