So, open thread: how much do you document on *your* projects, how much duplication is worth the continued upkeep as the ecosystem continues to evolve, how much effort do you put into that initial onboarding experience when it's not really under your control? Does it differ for contributors *to* your library vs. users *of* your library? I would love to make peace with *some* long-term criteria for where to draw this line.
@glyph for Amaranth and Glasgow I've standardized on pipx and pdm. this seems to work ok; I never truly know who falls off if they don't tell me but if a ChemE with zero prior programming experience can do world-class work in six months I'm not at least failing hard
@glyph i actually place more focus on "how miserable is it past onboarding" than specifically "how miserable is onboarding". yes, the latter narrows down the first step of the funnel. but (and perhaps this is the domain talking) i think people are quick to discount mildly painful first experience if the rest of it is smooth, over the inverse
@glyph probably my proudest moment in this regard is hearing isabel (the ChemE i mentioned) say that "if you follow the glasgow installation instructions to the letter they never fail, regardless of the OS and environment". it took a lot of work to get there
@whitequark ❤️ some credit goes to the packaging ecosystem of course but even now that is a huge achievement, it's still very easy to forget that one command or configuration setting that you needed to apply early in the process that every python dev _already has_ but a new user won't
@whitequark shoot I should probably actually make a list of all of those that remain, I wonder if I could concretely list them out
@glyph specifically the thing that makes it possible is me working with the python-libusb1 maintanier to ship a DLL, the wasmtime-py maintainers to ship DLLs, and to myself tp ship YoWASP. it's not primarily a python packaging thing. PEP517 helps for sure but that's more of a developer angle not end user angle
@whitequark I think that the pyproject.toml revolution has made it so that many different choices are valid and work fine and it really is amazing that we've finally gotten to this point. pdm definitely works well, so does poetry, so does (sigh) uv, or hatchling. I use a bunch of different tools, sometimes _different_ tools than the project itself recommends, and rarely have problems
@glyph this is true but it's also more complicated than that. until recently (we've just started using that PDM feature) lockfiles weren't portable, for example
@whitequark yeah. I imagine Glasgow has a lot of very specialized needs that ram straight into the edges of what is currently made easy by the ecosystem
@glyph nope, it's one of the most boring python libraries around. you can check it out, all it has is a literal handful of dependencies and some entry points. it does rely on multiple native code dependencies but that's all handled by just normal pip mechancis
Nov 7, 2026, 8:22 PM@whitequark @glyph Maybe it's got better recently as I spent a lot of the last year in C# land, but I've never successfully built/run/deployed a Poetry project using PDM
Nov 7, 2026, 8:23 PM@whitequark @glyph Ugh, the lockfile situation is really unfortunate, yeah. I've got a whole section in the pyreading README about how screwed up the lockfile situation is.
Nov 7, 2026, 8:28 PM@whitequark @glyph Oh, `marker` finally landed in pylock, huh? That's really helpful! The main blocker for me was always having locks for optional dependency groups...
Nov 7, 2026, 8:21 PM@glyph @whitequark Even more that pyproject.toml itself, having PEP 517/8 define what cross-tooling builds should look like has been an absolute godsend. It's nothing short of miraculous to me that `pdm build` can build pure Python, Python+Rust, Python+C++, and Python+Zig/WASM packages using the exact same invocation.
@xgranade @whitequark yeah the number of times that a project with native dependencies worked so well on the first try that I didn't _notice_ it had native deps until months or even years later is truly staggering. if you had a native dep in 2011, it wasn't that I'd notice; my entire week was ruined
Nov 7, 2026, 8:25 PM@glyph @whitequark I'm working with enough obsolete packages that I still always notice native deps, and have developed a healthy fear of any packages that use dynamic linkage instead of static linkage, but that's absolutely my weird vantage point in trying to wrangle a cursed codebase that I didn't initially create.
@glyph I have resolved that if I ever get back into programming on my own projects, I will make the documentation a centrepiece; I will strive to my full ability to make the documentation, specifically, a work of art.
Nov 7, 2026, 8:17 PM@glyph airflow has reams and reams of this sort of documentation and it pisses me off. _I_ can never find things that I know are written somewhere
@ash haha. so, excellent coverage, terrible information architecture?
@glyph When I started webdev 10ish years ago, the setup process was the biggest barrier for me. I am mostly in PHP (I know), so not fully analogous, but I have to assume some of the steps are similar when starting from scratch. I never really got too hung up on PHP's principles, but the fact I had to set up XAMPP and learn how to make it work with whatever was the right version of the language, and do a bunch of server management that I could not find explanations for which worked for me... -c
@glyph I’ve struggled with this for a long time, especially with the bonus level of “project-wide” documentation, where the project is spread across many repos, only 1-2 of which are relevant to most contributors. What’s common across repos, what is unique, what is generic to Python standard practices? Most of our packages are simple pure Python where it feels silly to write down “pip install -e ." A hundred times when should work for _every_ Python package, but it needs to be discoverable for the person coming for the first time. Taking a “user stories” approach to documentation has really helped, but I think we still have a lot of work to do.
@minrk except now everybody's doing `uv run` and `pip install` is old-fashioned and wrong! how do you update that across nine bazillion repos all at once?
(not me. I love being old fashioned and wrong, and I will be doing `pip install -e` until it stops working. but, you know, the kids these days.)
@glyph Provide lowest common denominator instructions aimed at the target audience. This could be as simple as "Install the following dependencies...", or if you can be confident a specific package manager exists (like pip) then use that.
Level these up with 0-N per-OS/package manager instructions ("Copy this apt command line to install everything..."), as many as you can comfortably support. Which may be 0.