This is a most excellent place for technology news and articles.
I don't know that it's wise to trust what anthropic says about their own product. AI boosters tend to have an "all news is good news" approach to hype generation.
Anthropic have recently been pushing out a number of headline grabbing negative/caution/warning stories. Like claiming that AI models blackmail people when threatened with shutdown. I'm skeptical.
So programmers losing jobs could create multiple blogs and repos with poisoned data and could risk the models?
lol nice BSD brag thrown in there
found that with just 250 carefully-crafted poison pills, they could compromise the output of any size LLM
That is a very key point.
if you know what you are doing? Yes, you can destroy a model. In large part because so many people are using unlabeled training data.
As a bit of context/baby's first model training:
Just the former is very susceptible to this kind of attack because... you are effectively labeling the training data without the trainers knowing. And it can be very rapidly defeated, once people know about it, by... just labeling that specific topic. So if your Is Hotdog? app is flagging a bunch of dicks? You can go in and flag maybe 10 dicks and 10 hot dogs and ten bratwurst and you'll be good to go.
All of which gets back to: The "good" LLMs? Those are the ones companies are paying for to use for very specific use cases and training data is very heavily labeled as part of that.
For the cheap "build up word of mouth" LLMs? They don't give a fuck and they are invariably going to be poisoned by misinformation. Just like humanity is. Hey, what can't jet fuel melt again?
So you're saying that the ChatGPT's and Stable Diffusions of the world, which operate on maximizing profit by scraping vast oceans of data that would be impossibly expensive to manually label even if they were willing to pay to do the barest minimum of checks, are the most vulnerable to this kind of attack while the actually useful specialized LLMs like those used by doctors to check MRI scans for tumors are the least?
Please stop, I can only get so erect!
On that note, if you're an artist, make sure you take Nightshade or Glaze for a spin. Don't need access to the LLM if they're wantonly snarfing up poison.
if that's true, why hasn't it worked so far then?
I used to think it wasn't viable to poison llms, but are you saying there's a chance? [a meme comes to mind]
You and me. We just need 248 more volunteers and we can save the world!
Isn't this applicable to all human societies as well though?
Well, I'm still glad offline LLM's exist. The models we download and store are way less popular then the mainstream, perpetually online ones.
Once I beef up my hardware (which will take a while seeing how crazy RAM prices are), I will basically forgo the need to ever use an online LLM ever again, because even now on my old hardware, I can handle 7 to 16B parameter models (quantized, of course).
So what websites should be targeted?
