Technology

You can just sandbox AI and block any network access via firewall. Trivial

This is completely irrelevant - by default local models do not have access to the internet, because you have to provide/activate the relevant tools. They can't access the web by themselves regardless of where you are, the environment has to be set up for it. Even if you provide a web search tool, they can't do anything outside of providing search parameters - what happens with them is defined by the tool, not the LLM.

These things work by providing the LLM with a system prompt that explain them what they have to do to get access to specific tools. They get provided the specific syntax they have to use it, and the environment they are running inside is then parsing the correctly formatted plain text the LLM outputs.

Specific example for the system prompt in Newelle, a all-in-one suite on Linux to run models, to provide tools in general and the web search tool in detail:

# Tools  

## Overview  
You have access to tools that extend your capabilities. Use them when they are relevant to the user's request.  

## Invocation Format  
When using a tool, output **only** a single valid JSON object:  

\`\`\`json  
{  
  "tool": "tool_name",  
  "arguments": {  
    "arg_name": "arg_value"  
  }  
}  
\`\`\`  

## Rules  
1. Output only the JSON object — no explanations, markdown, or extra text before or after.  
2. Ensure valid JSON: no comments, trailing commas, or extra text.  
3. Use only the tools listed below and only their defined arguments.  
4. **After invoking a tool, stop generating immediately.** Wait for the result before continuing.  
5. Some tools are shown in **compact form** (only name and description, no `parameters`), marked "(compact: ...)". Calling a compact tool directly with guessed or missing arguments is an error and will be rejected.  
## Available Tools  

\`\`\`  
{TOOLS}  
\`\`\`  

The Search tool in detail:

{  
  "name": "search",  
  "description": "Perform a search query on the internet, you can specify the number of results to return and if you want to only return the links and titles.",  
  "parameters": {  
    "type": "object",  
    "properties": {  
      "query": {  
        "type": "string"  
      },  
      "tool_uuid": {  
        "type": "string"  
      },  
      "only_links": {  
        "type": "boolean"  
      },  
      "max_results": {  
        "type": "integer"  
      }  
    },  
    "required": [  
      "query"  
    ]  
  }  
}  

Edit: I can't for the love of god find out how to escape the 3 backticks for the JSON-Object correctly without either breaking the code blocks or the backslashes showing up. In the original prompt, the backslashes do not exist.