I worked at a fairly big tech co years before the AI boom. People did large scale refractoring across huge code bases back then. With refactoring tools. And properly written robots.

Applying changes to code at scale, opening PRs automatically, basic interaction with human reviewers, making sure tests pass, getting things merged when ready. All that already existed before LLMs. And it was actually reliable and not capable of hallucinating terrible things.

It's like we've forgotten how to automate things without LLMs and openclaw now...

I was browsing @acegikmo's media tab on bluesky to find something and my god why is everything she posts so PRETTY????

Every single PoC and visualization she posts could be a figure in a high quality textbook or video.

Even the random photos of her desk and cat look professional!!!

I want to be like that ;;

https://bsky.app/profile/freya.bsky.social

Apparently I was so annoyed I didn't realize I typoed Swift as Switch... ^^;;

The slop machine even hard coded the exact same camera matrix from my example dumps, down to the fp64 rounding error (400 is actually a result of a calculation but the slop machine isn't smart enough to guess what it is).

(Also the math here is wrong, the slop version's entire take on resolutions and aspect ratios is nonsense)

Apparently someone already wrote a whole production Swift wrapper for the AppleCVA face tracking framework I discovered. I'm pretty sure it was vibe coded.

This sucks. I'm doing this both to help people and to teach myself Swift and macOS programming. What's the point now?

I spent a whole day reverse engineering Apple binaries to work out the API and required input data. I didn't post the API header or docs but I did post logs with a lot of the required info. Enough for a slop machine to figure it out I guess.

This is really demoralizing. I was planning to stream this weekend to work on all this. Now I don't really feel like it any more.

I guess you can't even post reverse engineering logs now without risking people slopping their way ahead of you.

RE: https://vt.social/@lina/116562210481202674

It's kind of crazy how *simple* the private FaceKit API is. It's just a pure C API that uses NSDictionary for arguments and configuration. Three functions (init, get semantics, and process).

You give it a CVImageBuffer and face rects and it gives you all the tracking data.

There's even a Lite version of the API that I think can do face detection internally (otherwise you need to do it yourself by using something like Vision.framework).

I haven't tried it yet because ARKit doesn't use it, and I don't know for sure if it's equivalent or more limited.

But yeah... Apple should just open this up. There's no reason to tie the awesome face tracking model to the ARKit behemoth and iDevices only! The raw FaceKit API works with any video input, you could feed it a recording or static photos if you wanted.

🚀 IT WORKS!!!! 🚀

ARKit iPhone face tracking, without an iPhone. Just a webcam on any Mac.

All the blend shapes. All the features. No magic camera required. It just works.

Now I just need to plug this into VBridger/VTS and clean things up a bit ✨

Logs: https://gist.github.com/hoshinolina/ba7548a50563146fa1caf3d2bd8ea3be

BTW, it seems to be using around 65% CPU time (of one M2 core), or 80% if I tell it to force CPU-only processing.

My code can probably be optimized, but yeah, it's not a very heavy model or anything. Doesn't even need a GPU/NPU really I think.

This is the exact same code and machine learning model that ARKit uses under the hood on iPhones, with the same quality.

Apple doesn't publicly support it on macOS through ARKit... but they've been shipping it secretly as a private API.

🚀 IT WORKS!!!! 🚀

ARKit iPhone face tracking, without an iPhone. Just a webcam on any Mac.

All the blend shapes. All the features. No magic camera required. It just works.

Now I just need to plug this into VBridger/VTS and clean things up a bit ✨

Logs: https://gist.github.com/hoshinolina/ba7548a50563146fa1caf3d2bd8ea3be

So close... it's accepting my face tracking request, but then it crashes in a thread in a Swift release function.

It's all "dynamically typed" CFDictionary stuff, so I'm probably doing something horribly wrong with the types or memory management... (I'm a Swift newbie^^;;)

OK, so it actually wants me to locate the faces *first*, then I give it the face bounding boxes and it does the face tracking.

The Vision framework can do face detection, so this shouldn't be difficult.

FKTracker: (AppleCVA) [com.apple.AppleCVA:FaceKit] Creating FaceKit with options: { "add_debug" = 1; "failure_fov_modifier" = "0.5"; "network_failure_threshold_multiplier" = 1; "num_tracked_faces" = 1; "rgb_only" = 1; "robust_tongue" = 0; "use_recognition" = 0;}

FKTracker: (AppleCVA) [com.apple.AppleCVA:FaceKit] CVAFaceTracking created (0x7feaaee60).

Reddit: "Lol this person is insufferable and clearly has no idea what they're talking about and doesn't understand the technology"

🙄

Even the mod called me out without doing the experiment themselves. Guess I'll stick to other communities.

I'm honestly quite disappointed with /r/vtubertech. I expected better than this from the VTuber community.

I get that this is an *extremely* widespread myth, but people are getting irrationally angry at the idea that maybe Apple just has a really good face tracking model and it's not the camera.

Literally all they have to do is just... cover up the depth camera on an iPhone and see how the tracking still works practically the same, with just the front selfie cam.

But oh well, I guess I'll just write and release "VBridger for macOS except it's using the webcam" instead 😇

I told Reddit that ARKit face tracking is just really good webcam tracking, and gave them a simple test that proves it.

They're all downvoting me without doing the test.

Look what I just found... the ARKit face tracking code is hiding in macOS. I'm going to make this work.

Typical ML argument: "If I can read something legally, why can't I train an LLM on it?"

Humans are capable of reading things and later writing a similar thing that is still a copyright violation. If I go and write a book that follows the plot line of Star Wars, that's still a copyright violation, even if no text is literally the same. If I play the melody to a song on my piano and release it without the appropriate mechanical cover license, that's also a copyright violation.

The reason this does not happen often is that, as humans, we are aware that that's plagiarism and there are rules. Sometimes it happens by accident, and people still get sued and lose.

LLMs have no such awareness and routinely output things which are blatant copyright violations when appropriately prompted. That means the model weights encode that work, and therefore, are themselves a derivative work.

Your brain encodes a massive amount of copyrighted information. You are not a walking copyright violation because humans aren't data, can't be copied and distributed en masse, have human rights, etc. This is why "mind reading machines" are a classic dystopian plot point (monetizing your thoughts etc).

An LLM is not a human, does not have human rights, nor human privileges. It is data, and if it encodes copyrighted information, that's a derivative work. If you aren't following the license of the training data, that's a copyright violation.

tfw just got wall-jumpscared by @lina (i mentioned that i was going to look at a backup delay issue and then forgot to actually look at it, or post about such until much later)

root@mastodon ~# rclonewho are you see discord
root@mastodon ~#

root@mastodon ~# autorestic -c /etc/autorestic.yml exec -b hetzner stats
Using config:   /etc/autorestic.yml
Using lock: /etc/.autorestic.lock.yml


    Executing on "hetzner"

root@mastodon ~# who are you see discord

BTW, there is almost certainly a model size and architecture threshold here. If I train an image upscaler on copyrighted data, for example, it's quite improbable that it will be able to generate copyrighted data from an unrelated input (for typical upscaler designs and sizes). Most likely that is safe.

Is there a size threshold for LLMs where, for typical training data distributions, the LLM is highly unlikely to "memorize" any copyrightable information? Is that size threshold a usable LLM? These are open questions (that nobody seems to be interested in researching).

I'm willing to bet if such a size threshold exists, it's much smaller than the minimum LLM size useful for "general purpose" prompting though.

Yes, this means that anyone downloading "open" models potentially puts themselves in as much legal risk as downloading a movie does.

Just *using* a cloud based LLM may or may not be safe depending on how a bunch of legal unknowns go, if the output happens to not qualify as a copyright violation. Of course, since you have no idea whether it will, you're basically playing copyright Russian roulette.