New blog post: "Knowledge refinement in the digital era"
How can computational knowledge move from research into education and general culture?
https://blog.khinsen.net/posts/2026/09/25/knowledge-refinement.html
A related thread by @jack :
"In the digital era, it has become widely accepted that software and electronic datasets are representations of scientific knowledge.... For now, however, this is mostly wishful thinking. While code and data certainly are scientific *information*, it is difficult to consider them to be *knowledge* as long as they don't participate in the knowledge refinement process."
https://blog.khinsen.net/posts/2026/09/25/knowledge-refinement.html
"Whereas nobody would seriously consider publishing a lab notebook as a journal article after just some light editing, many researchers seem to believe that computational notebooks, such as the popular Jupyter notebooks, could replace journal articles."
https://blog.khinsen.net/posts/2026/09/25/knowledge-refinement.html
"Like knowledge refinement, code refinement makes code more reliable, more accessible, and long-lived, but only from the "code as a tool" perspective. It tends to reduce rather than augment its value as a knowledge repository."
https://blog.khinsen.net/posts/2026/09/25/knowledge-refinement.html
@khinsen that point about notebooks resonates a lot with me. I've seen so many people (imho naively) argue that the academic publication should be dead and replaced by a Jupyter/quarto etc notebook. And it's so alien to me, it's as if those people never read papers to learn something precisely for that refinement step they provide. I don't think I thought of it using that term but it's very fitting!
We have discussed this before, but for me the history of the ideas are important, thus relation of previous levels of refinement to the new ones.
Secondly, there should be a group fo people that does that refinement. Eventually this is about democracy. My blog post about "Democracy as a welfare service" talks about research guiding democratic decision making.
This of course demands the refinement of knowledge and the financing of such endeavor.
Thus we need to understand this process from a sociological point of view...
Radical changes need to happen in the formation and refinement of knowledge, especially if citizen science is incorporated in this effort for change.
@arunisaac That would make software a bad map - which may well be true!
@khinsen Why can't learners see and interact with more concept maps and fewer paragraphs? The former are readily searchable/refinable/visualizable by computation without natural language processing tools (e.g., LLMs). Maps also support nonlinear assimilation which respects what learners already know rather than requiring a linear scan through territory learners may not yet be prepared for.
@bassethorn We have certainly not yet explored all the possibilities of digital knowledge representations, including visual ones. So what you describe may well be the way of the future!
The main obstacle I see to experimenting with such ideas is the (justified!) conservatism of scientific publishing. Long-time archival and new experimental representations are unfortunately conflicting requirements.