NoLemurs

note that you probably need to trust the remote household with your data

Probably not actually. Borgbackup (and any sensible encrypted backup system really), does client side encryption. So the data should be encrypted before it's even sent down the wire. The remote household shouldn't be able to access the data without the encryption key/password.

I mean, they will have to trust the remote household not to destroy the data, but that's pretty much an unavoidable limitation.

You’d better give us money and not try to regulate us.

Actually, right now, the western AI companies desperately want regulation.

An unregulated environment is one where they're forced to compete on price against the Chinese companies and startups running open-weight models. All signs point to them losing that battle.

OpenAI/Anthropic's wish list right now is a bunch of regulation to prevent the use of foreign models, and to require expensive to implement "safety" requirements that make it hard for startups to compete.

You absolutely shouldn't be angry!

The existence of powerful open-weight models is the single thing that's going to prevent us from descending into the full corporate dystopia everyone's afraid of.

It's going to hit the stock market hard once everyone figures out what's going on though.

[forget]
keep-daily = 10
keep-weekly = 4

Daily backups are cheap because of de-duplication. I keep a few weekly backups in case it takes me a bit to notice something went wrong. I don't usually back up logs.

I could probably drop the weekly backups without much risk, but I've got the space.

The hilarious part is, yes, I don't care how fast my launcher is.

But... also, Steam is by far the slowest, least responsive program on my computer.

"Faster than Steam" feels like a joke.

I don't use a lot of non-open source applications (and not a lot of desktop applications generally). Have they gotten so bad that the slowness of Steam doesn't feel bad to people anymore?

the models only need to be right some of the time and they still constitute a threat.

Right!?!

And with how cheap and fast they are to run, they don't have to be right that often as long as a separate run can verify/validate claims. In a lot of ways what the LLMs do should probably be thought of as just a very sophisticated sort of brute force approach. You're effectively just searching the space of combinations of known vulnerabilities. It doesn't require particular cleverness, just an encyclopedic knowledge, a reasonable way to prune the search space, and the ability to put in the compute time.

I must be missing something, because you're getting a bunch of upvotes. I don't disagree with anything you said, but it feels like a total non-sequitur to what I was saying.

Yes - proper air-gapping is a good idea. But asking the big AI labs to air gap is kind of pointless. Liability already gives them legal incentives to avoid incidents. If that didn't stop them, a letter signed by a bunch of academics sure won't. More importantly, open-weight models capable of doing all the things that we're concerned about are already in the hands of bad actors around the world.

This letter is asking us to lock the front door of the house when the back wall is missing.

The ones in Russia definitely are. They don't have to be local models - you run them in the cloud like anyone else.

The idea that the frontier labs have LLMs with special capabilities that make them dramatically better than the top open-weight models is largely marketing.

The reality is that most systems are poorly secured, and LLMs make exploiting vulnerabilities fast, cheap, and easy. What would have taken a trained security researcher weeks before can be done by a much less skilled attacker with an LLM in a few hours.

This is stupid. Open-weight models that can largely do what the OpenAI/Anthropic models did are already out in the wild.

Do these people not understand that you can't put the cat back in the bag?

It doesn't even have to be only for known devices. Once you've compromised one device, you can just phone home to have a full powered LLM do your work.

The only real use cases for this would be on closed networks with no internet access, or networks where the traffic is closely monitored. Like, yeah, if I'm trying to hack the CIA, maybe I consider something like this. For the vast majority of devices and networks, this is inefficient and unnecessary.

I mean, it is kind of cool. But vanishingly few organizations have tight enough security that this is worth doing.

I think people are right that the solution here isn't to try a different project.

Instead of being confrontational about it, though, I'd recommend a different aproach.

evelyn: .lol 😜

you: I know! I know. The branding is pretty silly. But this really is the right tool for the task. Unesco and Wikimedia use it. And we can do our own branding for $50/month. 🤷

A lot can be fixed by just acknowledging people. They're trying to evaluate something they don't understand, and having a reaction. It's not even unreasonable. The right thing to do is just acknowledge and move on.

Usually that's all it takes. Don't let yourself be embarrassed when you're being reasonable. If they won't move on, that's the point where I start thinking about job applications somewhere else.

Doesn't mean they're wrong!

I'll just upvote the both of you. We all need to be more aware of our echo chambers.