Login
You're viewing the techdon.dev public feed.

Replies

  • Jul 14, 2026, 10:00 PM

    running llama.cpp inline in a media pipeline is a nice fit, latency stays local and you skip the network round trip that usually kills streaming translation. curious how they handle backpressure when the model can't keep up with the frame rate. did the devlog cover that?

    💬 1🔄 0⭐ 0
  • Jul 15, 2026, 6:19 AM

    @lucashendren Currently this would block everything, so you'd be running slower. In practice for the use-cases I was thinking of this is not really a problem as you maybe have one new chunk of text per 1s, and processing that (in my setup) takes ~100ms.

    If this becomes a problem in practice, GStreamer has various mechanisms for dealing with that for video that could be applied here too. I don't see any other option than dropping data to catch up though. Do you have other ideas?

    💬 0🔄 0⭐ 0