Login
You're viewing the techdon.dev public feed.
  • Jul 2, 2025, 9:48 AM

    @pavel I'm confused. V4L lets you stream to a CMA dmabuf which should be importable as GL_TEXTURE_EXTERNAL_OES, right? Or am I missing something?

    💬 1🔄 0⭐ 0

Replies

  • 💬 0🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • Jul 2, 2025, 11:43 AM
    @dos Sorry, hero, that's dark magic behind my understanding. I see the words but don't understand the sentences. :-(

    I'd need working example here. I got surprisingly far vibecoding this, but even robots have their limits.
    💬 0🔄 0⭐ 0
  • 💬 2🔄 0⭐ 0
  • Jul 2, 2025, 6:43 PM

    @pavel Not only you had copies in- and out- of GLES context there, but these copies were sequential - and your benchmark waited until things were copied before proceeding with the next frame, so it was pretty much useless in assessing GPU performance. In practice, GStreamer can happily encode the previous frame while the GPU is busy with the current one, all while CSI controller is already receiving the next one.

    💬 1🔄 0⭐ 0
  • Jul 2, 2025, 6:49 PM

    @pavel Also, it gets faster when you increase the buffer size, because rendering is so fast you're mostly measuring API overhead 😁

    With full 13MP frames: 315.1 MB in 0.62s = 511.3 MB/s

    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 2:53 PM
    @dos Thanks for a patch. And yes, it makes the loop faster.. if you don't actually use the data. When used for loading/saving 720 images from the ramdisk, speed went from ~16 sec to ~21 sec.
    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 3:00 PM

    @pavel I left the memcpy line commented out for a reason - with it uncommented, the result is exactly the same as with glReadPixels (which is effectively a memcpy on steroids). The point is to pass that buffer to the encoder directly, so it can read the data straight from the output buffer without waiting for memcpy to conclude.

    I've also verified that the approach is sound by having the shader output different values each frame and accessing it via hexdump_pixels inside the loop. Still fast ;)

    💬 2🔄 0⭐ 0
  • Jul 3, 2025, 3:07 PM

    @pavel That said, rendering to a linear buffer can be slower, that's expected. The question is whether gains from passing buffers around for free are higher, which for an actual "record video from a camera" use case will almost certainly be true (and which has very different performance characteristics from reading images from files - you can't directly attach a file as a texture).

    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 3:09 PM
    @dos But you only hexdumped first few pixels, right?

    Is that buffer uncached or something?

    I pushed current code to https://gitlab.com/tui/debayer-gpu .

    Yes, with memcpy(), I'm getting same results as before. If I get rid of the memcpy(), and attempt to fwrite() the buffer directly, things actually slow down.

    I can't easily connect gstreamer to that, I'm going through ramdisk for now. I'm using time ./ocam.py debayer for testing -- https://gitlab.com/tui/tui/-/blob/master/ucam/ocam.py?ref_type=heads
    💬 1🔄 0⭐ 0
  • Jul 3, 2025, 7:06 PM

    @pavel > I can't easily connect gstreamer to that

    Why not? I quickly hacked up passing dma-bufs to GStreamer and even though I'm glFinishing and busy-waiting on a frame to get encoded sequentially it still manages to encode a 526x390 h264 stream in real time on L5.

    💬 2🔄 0⭐ 0
  • Jul 3, 2025, 8:15 PM

    @pavel Plugged it into V4L2 - with a caveat that for now I fed the GPU full-res 13MP frames to meet stride alignment requirement (the shader output is still 526x390). It says it does 240 frames in 10.55s. I wonder if it's really slightly too slow, or just bad timing from our camera stack :)

    Image attached toot
    💬 2🔄 0⭐ 0
  • Jul 3, 2025, 8:26 PM
    @dos Camera is 23.5 FPS, IIRC. Do you have it under version control somewhere? This is a bit of achievement :-).
    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 9:04 PM

    @pavel paste.debian.net/1384224/

    It's ugly, hardcodes everything, lies on frame timing, occasionally segfaults. Most of it is copied straight from LLM, I just massaged the pieces to work together. Not the kind of code I'd like to sign off on :) But it's a working example, so have fun with it.

    💬 1🔄 0⭐ 0
  • Jul 3, 2025, 9:12 PM

    @pavel The first thing to do to improve it (after cleaning it up) would be to actually make use of the buffer pool. Dequeue the buffer, attach it as a texture, kick off rendering, get a fence and pass it with the output buffer to GStreamer without waiting on rendering to finish, then queue it back asynchronously once rendering is done. This should allow for much more complex shaders than this sequential code does.

    💬 3🔄 0⭐ 0
  • Jul 3, 2025, 9:35 PM
    @dos Fences; that must be some kind of dark magic.

    This code seems too good to be true. So, just to be sure, and in case you disappear tomorrow, can I add /* Copyright 2025 Sebastian Krzyszkowiak, GPLv2 */ and act according to that?
    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 9:44 PM

    @pavel Good question. Not sure what license would be appropriate to put on something that's mostly an output of a model trained on code on all sorts of licenses anyway...

    But given that it's just a bit of glue code between three APIs put together as an example, consider it to be under MIT-0 😜

    💬 2🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Jul 3, 2025, 9:58 PM
    @dos Thank you! I'll take closer look tomorrow or over the weekend. In the meantime, would you have Makefile or build command that goes with it?
    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 9:58 PM

    @pavel LDLIBS = -lEGL -lGLESv2 -lm -ldrm -I/usr/include/libdrm -lgbm -lgstvideo-1.0 -lgstapp-1.0 -lgstallocators-1.0 -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/aarch64-linux-gnu/glib-2.0/include

    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 9:36 PM

    @pavel BTW. The fact that I could stream full-res frames and bin them down in the shader at real time is an interesting news, as this may open up possibility to use phase detection autofocus.

    💬 1🔄 0⭐ 0
  • Jul 3, 2025, 9:40 PM
    @dos Exactly. That's a bit of big deal. That's why I'm trying to make sure this code does not go away. I had phase-detection auto-focus working at one point, but decided it is unusable as I did not see a way to scale down images quickly enough.

    Plus it also adds possibility of zooming.
    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 9:57 PM

    @pavel There's plenty of low-hanging fruits in there. Higher frame rates and 10-bit output are also likely some debugging session or two away 😜

    💬 1🔄 0⭐ 0
  • Jul 3, 2025, 10:05 PM
    @dos :-) Hopefully. I'll believe things when I see them running locally.

    BTW there's one more important thing this can probably do: take full-resolution photos while recording video.
    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 10:19 PM

    @pavel There's a question whether it will be worth elevated power consumption though. I've also stumbled upon csi erroring out with "Rx fifo overflow" requiring a reboot to recover that I haven't seen at lower resolutions, but haven't looked closer.

    💬 1🔄 0⭐ 0
  • Jul 3, 2025, 10:36 PM
    @dos Yes, there's more work to be done in the kernel; sometimes camera does not work after reboot, bayer-10 modes are not supported, ... :-(. And yes, it will take more power, but with phase-detection AF, it should be significantly better camera.
    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 10:57 PM

    @pavel Toggling the killswitch makes it appear though.

    IIRC PDAF was also usable at half-res.

    RAW10 is just a matter of setting up clocks for higher bandwidth and more lanes. Switching data format is then just a single register away.

    💬 0🔄 0⭐ 0
  • Jul 3, 2025, 11:01 PM

    @pavel When I lie to GStreamer and tell it that its input is in YUY2, it gets faster - perhaps even fast enough to encode at 1052x780. That's another opportunity for improvement.

    (and there's nothing magic about fences, it's just a simple synchronization primitive 😛)

    💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Jul 4, 2025, 8:57 AM
    @dos Yeah, I played a bit. Nice. But segfaults, occasionaly, and may segfault more when I switch to matroskamux. So I guess crash may be gstreamer-related? :-). There's also some kind of noise in bottom right corner, maybe that's related, too.
    💬 0🔄 0⭐ 0
  • Jul 4, 2025, 9:01 AM

    @pavel Pretty sure it will just work fine once it's rewritten cleanly and does such arcane magic as releasing the buffers at the right time etc. :)

    💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Jul 4, 2025, 1:45 PM

    @pavel Yes, of course.

    BTW. Turns out that streaming to YouTube instead of a local file is just a matter of using rtmpsink instead of filesink 😁

    A live stream on YouTube, showing a sitting cat looking into the camera.
    💬 1🔄 0⭐ 0
  • Jul 4, 2025, 4:04 PM

    @pavel I'm playing with GStreamer now (which is new for me) and it seems like most of this code could be replaced with GStreamer elements, and the rest should neatly plug in as custom elements 😂

    💬 2🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Jul 5, 2025, 12:22 PM

    @pavel You've got a dma-buf handle, already mapped buffer and even GStreamer with all its sinks available, so... however you want? Pretty much anything will be able to consume it easily.

    💬 1🔄 0⭐ 0
  • Jul 5, 2025, 1:30 PM
    @dos I don't have much experiences with GUI programming, so I was looking for suggestions. I've got dma-buf handle but would not know what to do with it in gtk, and maybe SDL is better match. Or perhaps stick to original plan and do user interface ("take picture" button etc) in another process.
    💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Jul 5, 2025, 12:28 PM

    @pavel Not sure what you mean. GStreamer is internally multi-threaded, but its API is thread-safe and there's only one thread in this code. Of course any kind of production-quality code will use some mainloop and enqueue buffers based on callbacks rather than while(!processed){} loop, but it's not exactly rocket science.

    💬 1🔄 0⭐ 0
  • Jul 5, 2025, 1:20 PM
    @dos There are at least two threads in this code: main one, and whatever runs "on_buffer_released". I don't yet know what causes the segfaults, but I suspect gstreamer.
    💬 0🔄 0⭐ 0
  • Jul 5, 2025, 1:33 PM

    @pavel That one line is the only thing that runs from another thread and it's neither scary nor requires any locking 😁

    But there are several other smelly things in this code and lots of missing error handling, so I'd rather start with that when looking for suspects.

    💬 1🔄 0⭐ 0
  • Jul 5, 2025, 8:17 PM

    @pavel Passing the right buffer size to gst_dmabuf_allocator_alloc helps it to not crash and not have garbage at the end of the frame 😂

    💬 0🔄 0⭐ 0