First, background - how does muse's browser work?

When you're using your little musey, you might say "oh yoo hoo! chat servant! please peruse the latest sites on the world wide web." Your musey may then call its browser.browse tool (it may also just curl a URL it invents, or use a different headless tool, but whatever). This spawns some browser running in a separate container on the host that can't be seen from the agent's main "cell" VM. This separation is part of the credential system: when a credential is needed, the browser agent can request that credentials get filled in directly to the page without ever entering the main VM.

The browser tool is a subagent LLM with its own set of tools, and it primarily works in an browser.observe, think, browser.act loop, culminating in some browser.report where it summarizes what happens. Upon encountering some set of things that need more structured behavior, that subagent then dispatches another sub-subagent, or a "specialist" to handle that - one of those is the bot_challenge specialist.

browser system prompt here: https://github.com/sneakers-the-rat/muse-skills/blob/main/binary-strings/browser/system.md

RE: https://neuromatch.social/@jonny/117432015798538958

As a consumer product, muse must be able to browse the web to do all the things it promises that magical "agentic ai" can do. Meta needs Muse to work, badly, because it has 2 trillion dollars on the line and is a laughingstock in the AI race. Problem though, there have been crawlers and bots for as long as the internet has existed, and there are lots of roadblocks put in place to keep the web usable for people - captchas!

So what is a poor Meta to do? Why, subvert the terms of service of every website of course! The muse "hatch" binary and the web browser ship with an entire suite of skills, tools, prompt text, and specific site guidance for 92 sites, all designed to defeat captchas!

Muse has used its powers of brute force to disassemble its own binary and extract the strings into a topical collection here so you can follow along at home: https://github.com/sneakers-the-rat/muse-skills/tree/main/topics/bot_challenge

number go down

Q3 Earnings: yeah muse is ridiculously expensive to run, but all we have to do to fix that is make it 10x cheaper. By.. Running the models at night. Idk we're going to ask muse how to make muse profitable.

one finding that i don't have the patience to chase down is that
a) i am 95% sure that the network egress threat classifier has an LLM in the loop that evaluates the requested URL and other contextual metadata as part of the decision, and
b) i am 25% sure that you can cause the classifier to itself receive a classifier refusal by giving it urls that look like prompt injections or biosafety triggers, and rather than either refusing the connection or opening an approval card, it causes a TLS failure.

i have a ~60k/1 week sample confirming (a), (b) is flaky and trying to measure it locks up the instance

my exit criteria for this is now "meta either pays me and i go and get my pilot's license and forget about computers or else they stop making new bugs that boggle the mind" and we all know that one of these outcomes is possible in this universe while the other is not.

the standing rule now is that for every bug i report to the bug bounty program i am withholding a bug of equal magnitude until meta decides to pay me a dime, so that in case they do not pay me then i have something to sell still.

just cornered and reproduced a bug where denying a user approval request actually approves the request, triggerable by any binary due to other unpatched but reported vulnerabilities, so with that i'm going to bed and going to submit another 130k tier account takeover bug report tomorrow because due to EVEN OTHER unpatched but reported vulnerabilities every vuln of this kind can be escalated to account takeover.

by the way, everyone already knew this, but "muse was written with claude" confirmed by the sourcemap comments. If i was a meta engineer i would be a liquid composed primarily of shame by volume, attempting to mop myself up off the floor but being continuously juiced back out by having to live the lie that i am not merely a cog in the machine of the company at the top of a single leaderboard: burning the most money on that fuckin boondoggle.

y'all i am watching this whole system collapse under its own weight in real time and i don't even know how to describe it. in the first week or so i was looking at this, it was clearly a premature launch with a bunch of holes in it. then they started patching a lot and it actually looked like it was improving so i was mostly interested in looking at where they were headed, which was horrifying surveillance state directions. but now it's getting into a "face melted" state where all the stuff they are doing is blurring together and increasing in pace and fighting against itself and it's getting even more ridiculously broken than it was at the start and i have to reopen all the things i thought were closed and can't really keep track of any one bounty lead in particular.

They have been introducing this idea of "CVM" - confidential virtual machine - which is the single worst idea for a product i have ever seen: "we will give everyone a free VPS hooked up to a what the fuck machine in such a way that even we will be unable to see what it is doing." however that term now means two completely different things - one meaning the means of spawning the VM so the host can't see it, and another meaning a protocol for how page-embedded content can communicate with the VM (which can be either a normal VM or a CVM, and normal VMs default to serving CVM-protocol embedded things now).

No you are not supposed to know what either of those things mean. Not even meta or the platform knows what they mean anymore. The code and the comments make assertions about the behavior of one meaning of CVM that are absolutely not true about the other, and then that makes it not true about the first meaning because it assumes it is already the case. I can't wrap my head around it.

the approval system - the entire human in the loop safety checkpoint surface that asks you about whether you want sketchy computer things to happen - is an absolute duck pussy of failure states. in theory you are given some controls with the vaguest options: "always ask" (so far so good...) and "ask for some actions" (for WHICH? WHEN?). however neither of these options do either of those things. I am trying to write a report about one kind of failure state when all of a sudden the ground shifts and one hundred more failure states appear that defy all description. It is currently, reproducibly the case that every single approval in an extremely common context completely breaks the entire premise of the approval system, and i literally DO NOT AND CAN NOT KNOW how much i can say about that without it being irresponsible disclosure because it's "THE ENTIRE THING BEING TRIVIALLY EXPLOITABLE" but also IT IS UNCLEAR IF THAT IS INTENTIONAL.

and they have yet to get back to me about a vuln that makes it possible for any random old process to be able to opt into "infinite approvals for everyone" country. I am still hoping for a bounty payout for that and that description is vague enough that you can't tell what the vuln is, but if when i reported it the magnitude of being able to exploit that was magnitude 1, then the current magnitude is something like 1,000,000 because of the random bullshit spiraling around it since then. the fix to this vuln requires completely rewriting how their entire intra-VM permissioning system works or else some very fragile linux permissions surgery, but there's literally no hope of either.

i keep trying to put a period on this and finish up another trip through hell but it's like the expansion of the universe, it's already fucked, but the amount of fucked it is keeps accelerating from every part of it

collecting side thread
https://neuromatch.social/@jonny/117474923921731787

LLMs love to create pointless convenience methods, which is a great feature when you try to mask your feature flags in even the inexplicably shipped source maps by converting the flag names to hashes, only to have them wrapped in a function that describes exactly what they are.

oh... well... that's controllable from the client...

one thing that is awesome about muse's web client is how it is literally constantly printing the bearer token that gives control over the whole VM into the console

meta is less friendly to other domains, though - ridge dot com and threadless dot com both block datacenter traffic, so meta rents "residential" proxy egress from oxylabs to work around their protections. thanks, unnecessary LLM commentary!

Remember how Amazon banned Muse from shopping because nobody looking at the ads kills its ad business?

Wal-Mart is also a huge ad company with a retail side hustle, except it partnered with meta, and some interesting wal-mart related strings showed up in a new binary pushed to the VM as of yesterday around 09:30 UTC. The web browser adds two headers which are only present in config files for wal-mart domains: X-Agent-Session-Id and Muse-Authorization.

The X-Agent-Session-Id appears to be a unique identifier created each time the web browser tool is opened, so that web requests can be correlated where otherwise they just look like connections from meta's servers. Muse-Authorization is more interesting, a signature that proves that the request came from meta, but it's partitioned across different lanes - live user chat vs. background activity. I can't tell if this is live yet because i can't see the raw requests for the browser tool, but to me that looks like the recipe for an agreement where wal-mart allows muse bot traffic on its site in exchange for the telemetry and ability to distinguish active shopping from passive agent recommendations.

Oh lmao they have disabled all sharing including static pages, great job everyone.

The reason its like this is because shared conversations and spaces and artifacts need to be on different subdomains is because they are arbitrary fullstack web content, so if they just used a path segment then they would allow cookies, local storage, etc. from one context to bleed into another, and every space turns into a credential harvesting vector. Other things like githubusercontent dot com work because they only host static user content.

This same vuln affects other shared things in muse so I guess if this is bountyable then I'll report that lol.

RE: https://mastodon.social/@arstechnica/117473258140283378

I saw this but didn't report it because I was sure that meta would classify this as intended behavior lol. They have deployed tooling for direct VM-to-VM messaging and other sharing modalities as well, so this is only going to get more fun