I'm unfamiliar with AI chatbots that you pay for. What is a token?
Technology
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
In very simple terms, a token is more or less a word. You pay per input and output tokens (your prompts and the answers) as they correlate the most closely to the energy expended by the LLM to process your request.
No wonder. Since deepseek has open license, they have to compete with 3rd party providers, and in case of smallest models with local generation.
Damn, fire sale already?
Thank you daddy Xi
Still gonna self host it instead (maybe)
FYI the flash model is ~158 GB
How are they running it? Doesn't the model have to fit in (V)RAM? Does Nvidia have such huge memories in the H cards?
Lots of GPUs together.
For self hosting it essentially needs to fit in VRAM + RAM but it'll take a lot of CPU for the part in RAM
Deepseek probably uses those big fancy H cards and not one but several together to increase VRAM.
The destiled models?
How fast do you burn through tokens that $4 for a million of them was a lot of money?
If you use it for Q&A, that's a lot of tokens. If you use it to write software somewhat autonomously, it's easy to go through a million tokens every few hours. Do that every day and you'll be paying over $100 a month at that rate.
They have to cut the price because its behind the frontier models. No one would buy it otherwise
The ones paying attention and on a budget would still use them. "The best" of anything is usually not cost effective.
Even before reducing the prices, they were already 2 to 3 times cheaper than equivalent alternatives from Anthropic's ($3in, $15out) and OpenAI's ($1.75in, $14out) at $1.74in and $3.48out. Now they're around 10x cheaper.
Edit: Deepseek V4 Flash is the leading model on OpenRouter

so much for no one buying it
Also, the big models are in the infant stages of turning the monetization screws. This could be a tactic to knock their owners' legs out from under them. If they can't turn up their prices as much or as fast as they had hoped due to the competitive pressure, venture capital may begin to divest sooner. At their current debt levels, that would literally end OpenAI and I believe also Anthropic.
If a temporary "permanent" price cut wipes out some competitors, that will make for smoother sailing in the future for them to raise prices more than they otherwise would have been able to, quickly recouping the cost of the price reduction and then all profit after that.
If it knocks out small competitiors then the big players will buy them up for the compute alone.
I'm not aware of any noteworthy small competitors in the space. Who do you mean?