My essay on rogue AI has been republished by the Bulletin of the Atomic Scientists — “Rogue AI didn’t breach Hugging Face, human decisions did.” Grateful for the invitation! https://thebulletin.org/2026/09/rogue-ai-didnt-breach-hugging-face-human-decisions-did/
Forgive me, I have blogged again: https://scottrichmond.me/on-programming-pedagogy/
In which I argue that rather than less coding, we ought to teach more, in learning situations heretofore untroubled by code.
hello there!
For #linux #audiogame players, this is going to be really good news indeed.
Remember the tolk speech dispatcher proxy I made, the one that used the same way of interfacing with speech dispatcher as nvda2speechd, aka another server binary which hosts the websocket endpoint, and based on what was sent from the windows dll to the websocket, it would do the connections to speech dispatcher? that worked, for a while, and probably people who play games relying on tolk using audiogame manager are still using it, so it works to this day.
However, this solution is janky, I always thought it to be that way, but I didn't know of any alternative at the time, so I accepted this to just be the way things are for now.
Every once in a while, I keep revisiting the idea of trying to make this better, so I went to the wine HQ channel on irc.
After a long stream of messages exchanged with @r in that channel, I learned not only that it's possible to do so, in rust of all things, which I didn't think possible at all, but also that wine has two ways of going about this, winelib and unixlib. The details aren't very important, all that's important is that we can, actually, create a windows dll which interfaces with linux components through wine, even if yes, the connection point is kinda painful and arcane because of the way wine makes you specify things.
That discussion resulted not only in me knowing this stuff, but also in them making a better version of the tolk spd proxy, which doesn't use any third party server binary at all, so you don't have to run any external processes. The code is here:
https://github.com/ArcaneNibble/tolk2spd
There are details on how to build this, or run it from the precompiled artifacts present on the releases page in the readme of the repository, although building it yourself is quite involved.
If this ends up included in audiogame manager, considering that it uses a wine prefix per game, we may not have to include the environment variable whatsoever, because audiogame manager controls the prefix, so the file can be moved in the right place inside there.
The original discussion included this as well, but I was initially asking if something like the mechanisms presented above exist because I was wondering, would it be possible to bridge Microsoft's MSAA or UIA to at-spi? the outcome of the experiment with tolk2spd demonstrates this can be done, although the current way in which things have to be specified would make it really hard, as it already made things relatively cryptic for this project, but this is considered accepted complexity since wine has no other way of going about it. What do people think? would bridged accessibility API's help us?
A few days ago, I read a blog post by @ploum about why we apparently need more of RMS and the free software foundation, because of increasing corporate control, surveillance and corporations poisoning the commons
https://ploum.net/2023-06-19-more-rms.html
I realize this post has been published almost three years ago, so the author could probably have different opinions about the situation now especially because of recent developments, however there are many people who hold the opinion that all we need is more free software and the rest of the issues go away, even today, so I thought, let's talk about it.
Before we do however, an important note, that I agree with most of the article, about how companies are using free software to further fascism worldwide, we work for free while those companies get a lot of value from our code, etc etc, that's not the stuff I have issue with. The actual issue is the opinion that we need more of the free software foundation as it currently exists today, as well as opposing the people who want to abolish the FSF and replace it with something else, because if software is free the whole commune will just fix itself eventually.
First off, I find it problematic that RMS is mentioned in that article a lot, despite this being about free software in general and the free software philosophy in particular. Granted, the author does say that they don't encourage that particular person explicitly or that he represents the whole free software movement, but they refer to him in place of his philosophy, which does a detriment to both their article and the free software movement.
However, let's not forget who RMS is. He's a sexist, an advocate for pedophilia and necrophilia, and the author did acknowledge some of this, but only vaguely saying that RMS is controversial, although they didn't say anything specific besides we might not like what he says and how he says it, which is missing the forest for the trees. My question is, how can we, as a community, consider this person fundamental to free software today, someone without whom free software, or better said, the free software foundation, can not continue to exist?
yes, he created the philosophy, he did a lot of awesome advocacy and other work in the beginning, nobody says everything he did is null. However, that's one thing, and another thing entirely is the author saying that when criticizing him, we only care about feels good communication. Condensing all those issues into feels good communication is well, not a very inspired choice of words at best, at worst a show of privilege because of actively dismissing all the concerns and lived experiences of people who say this about RMS, but I'm not one to assume the worst especially taking into consideration the rest of the article, so I'll go with the first word interpretation.
But alright, let's say that doesn't exactly matter now, being a bad choice of words and all, because the most important thing is how the free software philosophy is all we need, according to the post. Alright then, let's talk about this point
When the fascist takeover of the ruby gems package manager by DHH and his pals happened, then a few members of that community had to try their best to do some damage control, alone for the most part, what did the FSF do? when the fascist company Anduril, through an undisclosed employee planted in the steering committee of nix OS, tried a hostile takeover of that project based on free software, then bragged on the platform formerly known as twitter about how they will keep doing fascism with free software, no matter what other people think, to a thread from DHH from what I remember, what did the FSF say or do? To my knowledge, nothing related for one, and for another, just stuff they always say, about how Apple and Microsoft products are proprietary and we need freedom respecting options, which is true, but not any kind of answer to what's happening in the free software community they claim to support.
Let's put it slightly differently, then. How many women do you think are in free software, contributing to code, writing docs, making awesome tools? Well, clearly not a lot, considering how many spaces which are practically men only I see. Why? well, you can't look further than the free software foundation, which harbors a very much sexist individual with no change in sight, and where a lot of the members there presumably agree with him.
How many members of the LGBTQ+ community are doing free software? Let's just say that a lot of people who think that open code is the only thing that matters also think those people shouldn't exist, see hyprland, omarchy, xlibre, etc for that one. The most ironic thing though, it's members of the LGBTQ+ and disabled communities who built a lot of the stuff these people depend on, sometimes outside of the FSF or any usual free software channels specifically because of the way these people run their spaces, but they conveniently choose to forget that fact. And in this case, the problem is not even stuff like a company uses free software to poison the commons as was put forward in the article, those projects are in the commons if we say the commons is free software, but those fascist people themselves poison the commons by being in the commons.
Do we need to talk about accessibility and disabled people in free software? hmm yes, I suppose we do, since we're here. How many disabled people work with the free software foundation, do you think, how many of those people have left this space entirely because of antagonism from maintainers because they dared to ask for certain improvements so they could access apps at all?
Outside of a few awesome communities I'm gonna talk about below, what do you think is the response to one of us opening an issue about accessibility to a random foss project? In most cases, it's something akin to patches welcome, although said in different ways. And I mean sure, I get that maintainers are already stressed out by a bunch of feature requests, have a life outside of foss and so on, and that's probably some kind of answer which would work if it was a relatively trivial feature, didn't add much to the application and therefore could be as well done by someone else, even if I wouldn't necessarily use that phrase myself and I don't think it's a nice way of talking to people to begin with.
However, accessibility is a core part of the operating system, the desktop, the app, and it's not something that's optional for us, we literally can't use it at all otherwise.
Did the FSF ever say how important accessibility and inclusion is? did they mention that intersectionality exists, and we don't live in a vacuum? a good guess would say no, and if they did, they only implied it in one article or another, instead of consistently saying it across most of their messaging.
Why isn't free software put on the same pedestal as accessibility, inclusion, being against discrimination, and other similar humanities? The answer I came up with has been said by a lot of other people too, but I'm gonna say it anyway. It's because a lot of those people are privileged, white, cis and kinda rich men, who believe that it's all gonna be fixed by one more foss project, one more version of the gpl or whatever, the same way they believe that if they vote one more time, everything's gonna be solved. If we don't address these issues with our most vocal organization of the free software space, we'll always be seen as those toxic people who don't care to the people who have their own fights, people who we could have as allies, but whom we instead push away because of our behaviors and position on critical issues, that is, no position whatsoever.
at this point, I feel like I should introduce a plug to @tante and their essay, who is free software for?
https://tante.cc/2025/03/03/who-is-free-software-for/
This is a very awesome essay about the issue. We're not the borg, people shouldn't be able to benefit from free software only if they can code on it, not everybody is capable of coding, and if they are, many projects are complex and don't provide enough documentation for newcomers, so the person willing to contribute a patch doesn't know where to begin in most cases.
So you might then be saying, can the FSF really put forward a point about everything that happens in the foss community? well no, probably not, but that's not the point. The point is to answer to big stuff like this explicitly, but also, to clarify their position regarding the pressing issues which have always existed in society, but which are now finally heard a bit more than before, enough so that they can't say they don't know about them. I believe Drew Devault was right indeed in that essay, and generally I don't agree with Drew Devault on many things, but even so. I think it's time to hold the FSF accountable, not only for refusing to let RMS go, but also for ignoring the festering toxicity and discrimination that happened in this space for a very long time indeed.
Alright, fwew, I got that part out of my system. I talked about the bad until now, as you can probably see, so from now on I'm gonna talk about the good, the awesome people who do awesome things outside of the FSF, and the communities which help us the most.
First, as a surprise to probably no one, is the gnome community. Not only that there has been a huge lot of work over the last few years towards making gtk do the right thing in most cases, so that developers wouldn't have to do many explicit things for making something accessible(thanks to all the people who made that possible btw!), but also when I went in the matrix rooms of a bunch of gtk apps, sometimes even apps in gnome core or circle, and mentioned an accessibility issue, instead of saying patches welcome in a derogatory way, they said so stuff's broken? let's see how we can fix it. Afterwards, begun a collaboration and a dialog in order to fix it, and by the time the next few releases rolled around, most of the accessibility issues got fixed. Of course, as many people would point out, rightly so, it wasn't always that way, some gtk maintainers did answer in that way to accessibility related requests in the past, gnome3 was terrible for example, etc. Yes, I also realize that the sovereign tech fund grant helped with allowing the knowledgeable people to work on this full time, but this could as well been heavily promoted and sponsored, even if partially, by the FSF themselves, they have enough money to sponsor critical projects like this I assume, because I can't imagine they get such a low amount of donations that this isn't possible whatsoever.
Second on this list is kde. I mean, for the people not aware, they got from an almost unusable desktop in 5.x, to the level where a friend of mine exclusively uses KDE for everything, because they like it better than gnome, and now they can actually use either.
And for the first time in a long time, we have an actual choice, gnome is almost entirely accessible, kde's progress is impressive, and in the tiling window manager space we have a very scriptable sway, but also the niri compositor has screen reader accessibility built into it for many functions.
The free desktop organization. Banning the hyprland developer because of his toxic community, as well as making derogatory comments and stuff like that, if nothing else, is more than many other organizations would do in that context, including the FSF.
Ironically enough, the free software foundation europe, if nothing else but because they renounced RMS because they recognized what dangers that person brings to the image of free software as a movement. Furthermore, their site is more accessible and well-structured than the original FSF site, but that might be just me. They should of course talk much more about the issues mentioned above, however the first step has been made.
So, to address the previous question, who reacted to the fascist activity in the foss space? well, some people in the gnome community I talked to, people on the fediverse, people who develop matrix tooling, etc, so in other words, people from a lot of places and backgrounds in the foss space, except the FSF.
So, I think we need an organization which values accessibility, diversity, inclusion and combats toxicity in its spaces a lot more than the FSF. We need a place where developers can learn more about making their apps accessible and their space less toxic and more safe, we need people vocally talking about this stuff in the free software spaces which matter, so that people would be able to know this stuff exists and that it's important, just as important as the code being free or that nobody can redistribute it under any other license.
I know we can't get all this in a legal document, law stuff will never be able to help with this anyway, but if the organizations we trust with promoting our vision, our code and our community are as gatekeeping and toxic as the FSF, then no wonder we don't have a lot of people organizing in other fields talking to us. So, we can't make it part of the license, but it's very important that we make it part of the spirit of our movements and ways of organizing going forward, because you know the saying, nobody's free until everybody's free.
And one more thing, when all's said and done, I don't care at all whether a drone used by ICE, or the army, or another such group, to shoot and kill people with impunity is using free, open software with reproducible builds, no, I don't care whatsoever. What I care about is that people from our own communities are being targeted by fascists who want to kill them for existing, and I care about not helping fascism with our free software and not having it in our communities, because it doesn't matter what free software powers a machine of war, as long as we have no opinion on it.
I published a paper from a first-person perspective about how features on #Mastodon leave enormous loopholes for harassing behavior (OA, 🔗 ⬇️ )
I came over here in the Nov22 wave & never left, but I no longer invite people to join without giving (loud) fair warning about drawbacks. I can't in good faith recommend Mdon to anyone who's more likely to be on the receiving end of harassing behavior
We have proof of concept, but the stakes are *urgent* to improve!
https://openjournals.uwaterloo.ca/index.php/JoCI/article/view/6644
The First National Website Dedicated To Documenting ICE License Plates Is Here, from #LATaco
https://lataco.com/ice-license-plate-tracker
@BetaCuck4Lyfe @firebreathingduck well, if it's gonna take a revolution, so be it, there have been many more before it, there will probably be others after. Sometimes, speaking truth to power means just that, screw it all because the goddamn fascists are burning the planet and us with it, and eventually the option of not making noise vs trying to make any future at all, not only a better future, for humanity have to be wayed, and one of them would be found wanting. One guess as to which one that's gonna be, :p
Also, we don't have to wait for the revolution TM, we can start doing mutual aid, community projects and organising starting now, because this indefinite waiting, vote blue no matter who and everything can be fixed by voting rhetoric is also harmful.
Politics aren't gonna save us, compromising with the fascists isn't gonna save us either, so there aren't many other ways of going about this. There are a couple things we can do to technically not require a revolution right now, but the list is shrinking by the day.
The friend.com ads may be ragebait marketing but their motivations are sincere - whoever did this understands..
I'm running another Ofcom Q&A for people who run small, low-risk sites in the UK, who want to know more about compliance with the Online Safety Act https://buttondown.com/indie-and-community-web-compliance-/archive/7-ofcom-virtual-qa-event-23-october/ - send your questions in by 15 October
I got interesting news from the fedora accessibility room today! this concerns blind and visually impaired #linux users exclusively, however:
For a bit of time, specifically pipewire >=1.4, one can start pipewire as root. That includes the regular daemon and the alsa layer, because jack emulation is a library loaded inside programs. Anyway, more recently than that, pipewire-pulse got the ability to be launched via root, as a system service:
https://gitlab.freedesktop.org/pipewire/pipewire/-/commit/dea6fa7f4c3053050d2819afa80265e5ffb39730
I'm not sure if ubuntu can use any of this yet, I suppose not, but most other distros which aren't debian based should be able to do so, for example arch, gentoo, probably nix and fedora starting with 42. This means that your system can start talking much, much sooner without the use of scripts, as long as you enable the system services instead of the user ones, or well, apparently the user units don't conflict, weird as that might sound. Either way, the same security is achieved, because who cares if root apps can listen to your microphone, the battle is lost if those apps are already root anyway!
In particular, this means that one can start espeakup with the system and it'll speak as soon as possible. Not in the initramfs, not at the enter decryption key prompt, but that's still huge progress in case your system crashes and so on.
What do y'all think, does this change anything, or it's still the same for you because you use scripts anyway, or because your systems rarely crash in such a way where that'd be required?
does anyone remember foss alternatives to linktree? I remember hearing of some, but I forgot details, and I'd rather ask the fedi about this than search online and possibly find malicious tools, or stuff that wants to sell me something, you know how it goes.
I'm not sure how many people on here run archlinux, but recently, a contribution I made to speech dispatcher, namely the pipewire module I talked about a while ago on here, got in the arch package.
Before wooping with joy and other similar feelings because something I made, with the help of the maintainer of course, got into a mainline distribution people actually use, not just in my computer, it'd be really nice if I could know whether it made any tangible difference.
So then, here are afew of the things I hoped to come true with the introduction of a pipewire native audio module:
Now, I'm absolutely no statistician, and I don't know how to even begin to measure any of this. So, here is me asking help from the wider fediverse.
For the people who know how any of this part works, can you take afew measurements of the performance of speechd, both with and without the pipewire support enabled? I'm primarily interested in the situations I mentioned before, especially the latency, does well under load and consumes less resources aspect.
For anyone who sees this, can I please have a boost? :p it would be incredibly helpful for me to know if this improved anything, even if one bit, and if it was worth going on and on about latency improvements regarding pipewire. I tried to do local tests with sound first, the results were positive, and so was my experience with using speech dispatcher with this module enabled. It could be because of my older computer, but I definitely feel a difference, although not a huge one. If there are other bottlenecks, I'm not sure they're in audio anymore, unless my code is flawed, which could definitely be and a deeper review on that by someone who knows audio stuff better than me should probably be done at some point, but I'm trying to get an inkling into the benefits this had, if any.
Reupping last week's newsletter: do we need a union for hopeful technologists? https://buttondown.com/justenoughinternet/archive/a-union-for-hopeful-technologists/ This is from a UK perspective, where policy and influence is dominated - if not overwhelmed - by US tech thinking
I, for one, am fed-up of centralising technofascists and am thinking of new routes to organise. Wrote this, which is mostly questions, in the hope of finding some answers and a whole load of collaborators https://buttondown.com/justenoughinternet/archive/a-union-for-hopeful-technologists/
Take it from somebody with chronic tinnitus from going dancing in my 20s, omg get ear plugs.
We have a serious democracy problem in #romania indeed, from a cleptocratic government no matter who's in power, to disability discrimination, to all sorts of human rights issues perhaps, those who live here surely know of what I speak. However, with all the defects of the system, we never expected, after the revolution, to ever return to the bolshevik way of thinking, to what is clearly fascism for all to see, invoked through pretenses of religion and faith, like the nazis do.
So, no matter if you live here or in other countries, please, mobilise and vote! I know the other candidate isn't perfect, but I'm not endorsing if the game isn't perfect, I won't play, type rhetoric, I categorically will not gamble and play games with the future of my nation, neither should you, this is a serious matter demanding our collective attention.
I said, on multiple occasions, that I hate the political class, I hate my nation because of how weak and passive it is in the face of what politicians do to it, but that's one thing, and quite another is being led by a person with strong ties to russia, an antivaxer and antiscientist, a person who defends the actions of nazis and played a role, however insignificant, in russia's illegal annexation of Crimea.
For whoever voted because of hate of the other, whoever or whatever that other may be, well, they're probably either nazi or nazi sympathisers themselves, same as this dude who made his campaign on the nazi, enshitified platform called Tictock.
To those who were misguided in voting for him, those who didn't know he's a putin sympathiser and who didn't read into some of his earlier commentary before voting for him, sunday, the 8th of december, is the time to rectify that.
For people who are disappointed with the system and think that the other candidate is a part of it, remember, we're voting for a president who can be pressured into doing the right thing, we vote so that we can play a better game another time. And think of it from this angle, if she wins, we will manage to beat americans at democracy, that would be a start, don't you think?
If this dude wins, there are no more games to play, because he will install himself as the fascist dictator of romania for a very long time, and remember the romanian revolution from 89? yeah, our collective spirit is more broken than then, we have been disilusioned with a lot of the political class and have gotten used to backstabbing people ourselves and behaving like them, in the hope that some crums will fall our way, and he knows that very well and will put that to good use, to ensure that no revolutions will ignite for a very long time, if ever.
We over here, who oppose the system, and human rights activists especially, are a primary target for his campaign and fascism in general. Remember, fascism is a government by the few, for the few, and if you, someone who voted for him, think you're one of the few, you're most likely wrong, and personally, I'd rather die than be a part of the circle of the few.
We often have discussions here about monopolies, unfair practices by employers against their workers, human rights issues and such, which aren't even a concern here in Romania, because currently our economy is bad enough that the average citizen doesn't even think about this stuff, or at least that's my impression when explaining these issues to people. However, if this dubious fascist sprung from the hat of foreign influencers and secret so called information services, people voting based on hate of the other and specially rigged social media algorythms wins, we'll never get to the point where we can care about that stuff whatsoever, because we will probably be at best a province of russia after some time in his mandate.
So, for anyone who sees this, please boost it, I'd be very grateful for that, since after all, my life, as well as the lives of other activists and intellectuals, are on the line here.
Let's show the Ticktock bros that we can raise awareness organically, through the new internet, one without algorythms, without algorythmic manipulation, predatory survailance and dark patterns, one where community and community building is valued more than anything else.
For people who actually live here and are directly affected by the results of this election because of that, I feel your pain regarding the system, I consider myself a radical opposer of the current pollitical class, however from one disappointed person to another, now's not the time!
we have to break the wheel, play a new game, on our terms, not those dictated to us by social media overlords, foreign influences and unscrupulous conspiracy theory pushers. So, whatever opinions you may have for the other candidate, please make the right choice for the continued democracy of our country, because there's no turning back from this.
To everyone who boosted or replied to this post, anticipatory thanks! to those who ended up even considering voting differently, or voting at all for that matter, because of what I said, I'm happy to have changed perceptions, even if only a bit.
It's not too late to save democracy, but we must act now, because it's our last chance!
Only together we can expunge fascism!
Together, as one, we have to show them that we, don't, give, an inch! our country is more important than anything else now, and fascism is definitely not what the people who fought for our freedom during the revolution would consider freedom.
So, for them and their memory if not for yourselves, keep fighting with your vote, don't let their sacrifice go to waist!
a short while ago, I got followed by an account called @matthew@caneandable.social.
This account has no posts, no bio, no fields in the bio even if there's no text, very few following people, but a lot of followers. This leads me to think that it's a bot. Can anyone confirm? I don't know a lot about the cainandable.social community, but this is possibly a malicious account, so I feel obliged to warn people about this, especially in light of the all too recent spam waves on the fediverse caused by open signup instances being flooded by bots and trols.
So yeah, if any admin of that community hears this, or perhaps if a member encounters this message, I recommend you try to investigate and perhaps forward it to the admins.
This, to me, looks like a bot on a following spree, and if I'm right, what will most likely follow is either our posts being used to train ai or some stupid stuff like that, or maybe we'll get spam with mentions of our accounts in there and lots of hashtags, like in previous spam waves.
I'm not sure who remembers, but back in the day, I started going in the matrix rooms of various projects I found #accessibility issues with, and then I begun helping devs fix those issues, for those who remember, I even called it IHelpedMakeItAccessible, with a hashtag for whatever reason, perhaps for discovery.
True, we rarely need something as concerted as that nowadays, especially because of the wonderful work the #gtk people did to implicitly make apps with some known defects accessible, basically by computing accessible control names for various conditions in which they might be missing from the actual controls, but those are just heuristics anyway. However, what about the qt apps? what about gtk apps which could still do with some real accessibility work? what about a category of apps we don't have accessible on linux whatsoever in gui form, such as ebook readers?
So then, I'm thinking about furthering this more than I can on my own, because we really do need more apps accessible on #linux, especially with microsoft making windows less and less desirable to stay on as time goes by with them continuing to be unchallenged by most regulators.
With that in mind, what do you think would be the best way of achieving this? I thought of making a matrix room or similar, devoted to free software accessibility, but crucially, for as many disabilities as we can make more or less compatible to each other when it comes to accessibility needs. This is very similar to what @pixelate did with his fossibility project, however this is more than a hub for devs to find accessibility testers, more than a potential wiki. In this space, we basically unite the disparate communities which formed around the linux desktop, and give ourselves a space to share all the cool lifehacks to make our tools work the way we want them to. Even better, we can use this to our advantage, and present a much more united position when it comes to the accessibility related issues, to freedesktop people, as well as toolkit and application developers. The same way small tech companies who could barely stop fighting among themselves for scraps bought each other to form monopolies, who were then able to present united arguments to their regulators to shut down opposition, we can do the same, but for the right cause this time, so that we can present saner arguments to people who genuinely want to make this complicated stuff better for everyone.
More important however, we could gather there to also help code the accessibility fixes ourselves, because we come there from all sorts of different backgrounds, some working with gtk, some with qt, others with web tech, and there's need for that and probably much more where we're going.
So, what do you think? or should we do something else? I feel like we should try our best to speed linux along even faster than it'd go organically, to get to the point where accessibility goes as fast as the rest of the freedesktop technology space. Also, the more concerted our voices are, the more we could reach other linux distros than what we're used to, for example endever OS, manjaro and garuda linux come to mind now that calamares is accessible, but also it'd be nice to reach out to valve, convincing them to enable accessibility in their steam client since it's mostly just a flag they had enabled once during the beta when they switched to a new version of the chromium embedding, and then we could perhaps get a screenreader properly working on the steamdeck, which would be amazing. What do you think?
this is, like a lot of my posts where I'm not replying to threads, concerning #blind users of #linux. With that out of the way, let's get into it
So, I was technically able to do this for some time now, around three days or so, but now I got enough courage to actually do it. This involves a lot of courage indeed, because this is my primary and only system, and the experiment in question could have cost me speech on the computer, which is a pretty big deal
So yeah, what did I do that's so dangerous? I stopped the pulseaudio server, entirely, and by that I mean the pulse pipewire compatibility layer, pipewire-pulse. Foolishly as it might sound from the outside, I tryed this to see if I could still get speech. Not so foolishly after all, and also in a show of spectacularly defying expectations, because I'm writing this now, I do, very much so, in fact, as an unintended consequence, my system feels snappier too, incredible, right?
As many of you are pretty technical, since using linux as a VI person kinda pushes you in that direction, it should not come as any surprise to you that speech dispatcher uses pulseaudio, aka pipewire-pulse, to play sound, because usually when that crashes, you get no speech and such. So then, how is this possible? no, I'm not using alsa, or any of the other audio backends in there. The rest of this huge post will be devoted to that question, as well as explaining some background around why this matters and how things were before. Note: this would probably only make a lot of positive change to a particular group of people, those who value the amount of latency of your audio systems, either because you're musicians, want to be, are working with specialised audio software, are using a complicated hardware setup with lots of nodes which have to be balanced in the right way, or just because you can, etc etc, those kinds of people. For most of you, this may be a novelty, may get some limited use out of it due to decreased cpu load and more fast feeling interfaces in your desktop environments, but for you, this is mostly a nice read and some hipped enthusiasm I suppose.
It all started when I was talking to afew linux musicians, if I recall correctly, also someone who might have been involved in a DAW's development. I was talking about accessibility, and finally arrived to someone telling me that speech dispatcher requests a very low latency, but then xruns quite a number of times, making pipewire try to compensate a lot. I could be misremembering, but they also said that it throws off latency measuring software, perhaps those inside their DAW as well, because the way speech dispatcher does things is making pw increase the graph latency.
So then, what followed were afew preliminary discussions in the gnome accessibility matrix room, about how audio backends work in spd. There, I found out that plugins mostly use a push model, meaning they push samples to the audio server when those become available, in variable sizes as well, after which the server can do the proper arrangements for some kind of sane playback of those. Incidentally or not, this is how a lot of apps work with regular pulseaudio, usually using, directly or indirectly, a library called libpulse_simple, which allows one to basically treat the audio device like some kind of file and have things done that way, where the library could also sometimes introduce buffering of its own before sending to pulse, etc. Of note here is that pulse still has callbacks, but a lot of apps don't use that way of doing things.
Back to the problem though, this was fine for pulse, more or less anyway, because pulse didn't symbolise the media graph in any way, there was simply no such concept there, you had only apps being connected to devices via streams, so there wasn't a way with which to get apps to syncronise their rates to something which can more or less be sent directly to the soundcard. So, when apps invariably started to diverge the rate at which they pushed samples to pulse, as far as I understand, pulse took the latency of the slowest stream and add it to everyone else to attempt to syncronise, because, after all, pulse would still have to mix and send everyone's frames to the soundcard, and because there either was no poling model or no one wanted to implement it, that was the best choice to make in such environments.
Enter low-latency software, #jack and #pipewire. Here, latency minimising is the most important thing, so samples have to be sent to the soundcard as soon as possible. This means that the strategy I outlined above wouldn't work here, which gets us neetly in the concept of an audio graph, which is basically all the sound sources that can play, or capture, in your systems, as well as exactly where sound is played to and captured from. Because of the low-latency factor however, this graph has to be poled all at once, and return samples similarly fast, in what the graph driver calls a cycle. The amount for which apps can buffer audio before they're called again, aka a graph cycle duration, is user-adjustable, in jack via the buffer size, in pipewire via the quantum setting. But then, what happens to apps which don't manage to answer as fast as they get called by the server? Simple, even simpler than the answer of pulse, alsa, etc to the problem, and their various heuristics to try to make sound smooth and insert silence in the right places. The answer is, absolutely nothing at all, if an app didn't finish returning its alotted buffer of samples, not one more or less than that, the app would be considered xrunning, either underrunning or overrunning based on the size of the samples buffer they managed to fill, and their audio, cutting abruptly with perhaps afew bits of uninitialised memory in the mix, is sent to the soundcard at a fixed time, with the others. This is why you might be hearing spd crackle weirdly in vms, that's why you hear sometimes other normal programs crackle for no good reason whatsoever. And you know, this is additive, because the crackling spreads through the entire graph, those samples play with distortion on the same soundcard with everything else, and everyone else's samples get kinda corrupted by that too. But obviously, if it can get worse, it will, unfortunately, for those who didn't just down arrow past this post. There are afew mechanisms of reducing the perceived crackling issues from apps which xrun a lot, for example apps with very low sample rates, like 16 khz, yes, phone call quality in 2024(speaking of speech dispatcher), can get resampled internally by the server, which may improve latency at the cost of a degraded quality you're gonna get anyways with such a sample rate, but also te cpu has to work more and the whole graph may again be delayed a bit, or if an app xruns a lot, it can either be disconnected forcefully by pipewire, or alternatively the graph cycle time is raised at runtime, by the user or a session manager acting on behalf of the user, to attempt to compensate, though it'll never go like regular pulse, but enough to throw off latency measuring and audio calibration software.
So, back to speech dispatcher. After hearing this stuff, as well as piecing together the above explanation from various sources, I talked with the main speech dispatcher maintainer in the gnome a11y room, and came to the conclusion that 1, the xrunning thing is either a pipewire issue or a bug in spd audio plugins which should be fixed, but more importantly B, that I must try to make a pipewire audio backend for spd, because pw is a very low-latency sound server, but also because it's the newest one and so on.
After about two weeks of churn and fighting memory corruption issues, because C is really that unsafe and I do appreciate rust more, and also I hate autotools with passion, now my pr is basically on the happy path, in a state where I could write a message with it as it is now. Even on my ancient system, I can feel the snappyness, this really does make a difference, all be it a small one, so can't wait till this will get to people.
If you will get a package update for speech dispatcher, and if you're on arch you will end up doing so sooner or later, make sure you check the changes, release notes, however your package repositories call that. If you get something saying that pipewire support was added, I would appreciate it if as many of you as possible would test it out, especially when low-latency audio stuff is required, see if the crackling you misteriously experienced from time to time with the pulse implementation goes away with this one. If there are issues, feel free to either open them against speech dispatcher, mention them here or in any other matrix rooms where both me and you are, dm me on matrix or here, etc. For the many adventurers around here, I recommend you test it early, by manually compiling the pull request, I think it's the last one, the one marked as draft, set audio output method to pipewire in speechd.conf, replace your system default with the one you just built by running make install if you feel even more adventurous , and have fun!
I tested this with orca, as well as other speech dispatcher using applications, for example kodi and retroarch, everything works well in my experience. If you're the debugging sort of person, and upon running your newly built speechd with PIPEWIRE_DEBUG=3, you get some client missed 1 wakeups errors, the pipewire devs tell me that's because of kernel settings, especially scheduler related ones, so if y'all want those to go away, you need to install a kernel configured for low-latency audio, for example licorix or however that is spelled, but there are others as well. I would suggest you ignore those and go about your day, especially since you don't see this unless you amp up the debugging of pipewire a lot, and even then it might still just be buggy drivers with my very old hardware.
In closing, I'd like to thank everyone in the gnome accessibility room, but in particular the spd maintainer, he helped me a lot when trying to debug issues related to what I understood from how spd works with its audio backends, what the fine print of the implicit contracts are, etc. Also, C is incredibly hard, especially at that scale, and I could say with confidence that this is the biggest piece of C code I ever wrote, and would definitely not want to repeat the experience for a while, none of the issues I encountered during these roughly two weeks of development and trubbleshooting would have happened in rust, or even go, or yeah, you get the idea, and I would definitely have written the thing in rust if I knew enough autotools to hack it together, but even so I knew that would have been much harder to merge, so I didn't even think of it. To that end though, lots of thanks to the main pipewire developer, he helped me when gdb and other software got me nowhere near solving those segfaults, or trubbleshooting barely intelligible speech with lots of sound corruption and other artefacts due to reading invalid memory, etc.
All in all, this has been a valuable experience for me, it has also been a wonderful time trying to contribute to one of the pillers of accessibility in the linux desktop, to what's usually considered a core component. To this day, I still have to internalise the fact that I did it in the end, that it's actually happening for real, but as far as I'm concerned, we have realtime speech on linux now, something I don't think nvda with wasapi is even close to approaching but that's my opinion I dk how to back up with any benchmarks, but if you know ways, I'm open to your ideas, or better, actual benchmark results between the pulse and pipewire backend would be even nicer, but I got no idea how to even begin with that.
Either way, I hope everyone, here or otherwise, has an awesome time testing this stuff out, because through all the pain of rangling C, or my skill issues, into shape, I certainly had a lot of fun and thrills developing it, and now I'm passing it on to you, and may those bugs be squished flat!
today, I heard about a #ai, aka #llm, tool for linux, so I went ahead and tryed it, even though I'm not an AI fan as people who follow me probably know by now.
the first impressions here are, well, very positive. Not only because I could run a tiny AI model on such a horrible hardware like mine, but also because the application is, for one, not an electron app, but a native ui app with gtk, and for another, overwhelmingly accessible as far as I'm concerned, especially since this is a first find, without me having to talk to the developers or anything.
For the curious, the app is called alpaca. It uses ollama under the hood, similar to open web ui or llm studio, with the exception that it's not written in electron or any bullshit like that.
The app guides you through choosing a model, you got a model manager button and so on, it's relatively easy. It's also possible to create your own models in a way, which I guess are just the model file or preexisting model you downloaded, combined with some embeddings and custom settings, or custom responses, I'm not sure yet. Either way, it's very interesting all things considered. O yeah, if you care about that stuff, you're also able to save chat history, import and export chats and probably many more things I don't know about, just been using the app for afew minutes at most.
For people who like this kind of thing, especially if they are using linux, this is an app I definitely recommend. It's best to install it via flatpak, if nothing else, because that's what I did and I can confirm it works well.