What just happened? Another incident has taken place that illustrates the need to be careful what you tell AI. A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to "shoot up" the Sheriff's office. After a human reviewer examined the statements, they were reported to police.
According to the arrest report, Carli Michelle Heller, of Bonita Springs, Florida, wrote on September 26 that she would attack the Sheriff's office. She later said that she uses Anthropic's chatbot like a "diary."
Claude's safety systems flagged the entry and it was escalated to a human reviewer. After deciding it was a credible threat, the reviewer reported it to law enforcement.
The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.
Deputies identified Heller and visited her home. She was detained without incident before an LCSO intelligence detective took over the investigation.
Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.
Anthropic isn't going to be taking any chances when it comes to anything it deems a potential threat. Last month, it was reported that OpenAI and Sam Altman are being sued by British Columbia over claims that the company could have prevented a mass shooting in the Canadian province.
The shooter, eighteen-year-old former pupil Jesse Van Rootselaar, had previously been flagged by OpenAI's safety team for her conversations about gun violence, but OpenAI never alerted police because the conversations did not meet the threshold for legal referral.
In June, Florida also sued OpenAI and Altman, alleging that ChatGPT had contributed to real-world harms, including the 2025 Florida State University shooting.
The latest incident is another reminder to think before you enter something into a chatbot that could get you into trouble. It's certainly not a private diary whose contents are for your eyes only.
Reports last month revealed that human contractors reviewing Microsoft Copilot's image editor can see users' prompts, uploaded photos and AI-generated edits. Documents show that some of those assignments contain sexual, disturbing or potentially illegal material, though the reviewers are not there to flag the content – only to assess whether the output is accurate.
I'm building a fully private assistant with local Home Assistant, local model, and a local Wikipedia database for lookups. Will probably make a post/release of it soon once I finish replacing all my Google Nest displays.
I'm coding a Kiwix mcp (I assume you're using kiwix as your wikipedia source?) that might be of interest to you. I'll post about it on !localLlama when done.
I don't like any of the extant Kiwix MCP tools that I found because they leave formulation, ranking, search and interpretation up to the LLM.
I'm looking at a different way to get exact spans with re-ranking via TinyBERT right in front of the llm so that it doesn't have to do 3 jobs and hallucinate itself into stupid, which is what small LLMs tend to do when given multi-hop tools.
I'm testing it with Ling Tiny 3, miniCPM5-2b, Granite 4-3b and Qwen 3.5-2b and 4b, with the goal of making sure that it actually generalizes. It's been a massive pain in the ass and I've basically proven RAG from first principles...but when I'm done, you won't need to vectorize the source into any kind of database nor "just trust me bro" the llm is searching Kiwix properly.
I'd like to subscribe to your newsletter.
I want to do this, but only got as far as setting up a minimal Home Assistant. Only put a few hours into it, though, then got sidetracked by trying to turn an esp32 into a speaker/microphone puck.
Well, if you'd rather a ready made solution or two
https://www.projectnomad.us/
https://futureproofhomes.net/
https://clawbox.com/
Like the poster above you, I'm rolling my own. More work...but a lot cheaper
PS: you know about this one, right?
https://shop.m5stack.com/products/m5stack-core2-esp32-iot-development-kit?variant=35960244109476
Up until the 80s, there was still some debate over whether wiretaps should be legal. It's sad that we have come to accept them as normal.
It was less worrisome back then. The cost to have someone listen to and transcribe your conversations was quite high. They'd really have to want to go after someone to do it.
Now, turning voice conversations into searchable text is nearly free.
Not only that - wiretaps have Judicial Oversight in that they require a Court Order (well, in most of the World, not sure if that's the case in present day America) which a Judge supposedly only issues if there's reason to suspect the person whose lines are tapped of committing or preparing to commit a crime, whilst Antrophic's surveillance of people's conversations with their chatbot has no Judicial Oversight at all and has no minimum legal threshold that must be reached before it is allowed to happen.
I think this is one of the greatest dangers AI presents. Sure it makes some of us stupider, sure it drinks our water and eats our GPUs and raises power prices... but once upon a time, it took actual human effort to run a dragnet, even with judicial approval.
Now, you plug in an LLM to the giant pile of corporate surveillance data and ask it to find troublemakers. Anonymity is dead and its death made retroactive and since there's no government oversight, the government can just lease access to the tools they used to need approval to use.
Where does this apply?
To the English-speaking world and probably beyond.
People will fight you tooth and nail to prevent something they view as immediately threatening to them. On the other hand, they'll pay you to implement something they view as immediately useful to them. In either case, they don't care at all about any non-immediate repercussions for anything.