this post was submitted on 19 Oct 2026
418 points (99.3% liked)
Technology
88927 readers
2935 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It's still a risk. You can train an LLM to respond maliciously to a very narrow range of requests that would not be easy to spot in general testing. If you know how the LLM is used, you can target that very specifically. It doesn't have to be as blatant as "always deny the Tiananmen Square massacre ever happened".
Given how thoughtlessly people use AI, and how opaque some of the systems are, I think this is a legitimate concern.
And yes, every government outside America should ban the use of ChatGPT and Claude. Obviously.
But you don't need to rely on testing. Deepseek is open source so they could just go through the code
Well, open weight is not open source. There's a lot of black box still left in those models.
It's still far more open to investigation and analysis than the likes of ChatGPT, though. Lots of work has been done to allow some degree of investigation into how the "brains" of an LLM work and in the case of Deepseek you can actually apply those techniques to the model you know you're using.
there is no comparison between software and services, though. self-hosting deepseek might be, subjectively, the wrong choice for your lab, but chatgpt is wrong for everyone, every human and every animal and plant too
this is coming from a k2 7b user btw. it's good enough for me!
edit: i replied to the wrong fucking comment and I'm too angry to fix it
Thanks for at least not being so angry you didn't bother adding that edit, I was very "wut?" There. :)
I'm not aware of any realistic way to vet published model weights like that.
It works for some open-source software, because you can download the source and compile it yourself. But most of the "open" models don't publish their entire training set, or enough details on the training for it to be truly reproducible. DeepSeek doesn't publish their data sets, and they probably couldn't, legally, since they "collaborate with third-party data providers to obtain proprietary datasets". And even if they did, it would be prohibitively expensive to train it on your own, unlike simply compiling open-source code.
The actual code for training LLMs is relatively trivial. The secret sauce is the training data and fine-tuning.
You have agents classify what each portion of the network responds to and how. One compares it to filling out a very large crossword puzzle: https://already-clean.goodfire.com/
That's the theory anways, not sure if it will stand the test of time...
Sounds interesting, thanks for the link. It's a little over my head tbh but I'll definitely take a closer look.