this post was submitted on 23 Aug 2026
416 points (89.2% liked)

Political Memes

12439 readers
2462 users here now

Welcome to politcal memes!

These are our rules:

1) Be civilJokes are okay, but don’t intentionally harass or disturb any member of our community. Sexism, racism and bigotry are not allowed. Good faith argumentation only. No posts discouraging people to vote or shaming people for voting.

2) No misinformationDon’t post any intentional misinformation. When asked by mods, provide sources for any claims you make.

3) Posts should be memesRandom pictures do not qualify as memes. Relevance to politics is required.

4) No bots, spam or self-promotionFollow instance rules, ask for your bot to be allowed on this community.

5) No AI generated content.Content posted must not be created by AI with the intent to mimic the style of existing images

founded 3 years ago
MODERATORS
 
you are viewing a single comment's thread
view the rest of the comments
[–] kuberoot@discuss.tchncs.de 1 points 2 days ago (1 children)

Again, this is an expression of your ignorance, not reality.

OLMo 2 was trained on Wikipedia and other fully public forums. Its training sources and data is fully accessible and open.

Decided to check out the first example quickly. It's hard to dig through the information, but following the chain of sources:

In the first stage, which covers over 90% of the total pretraining budget, we use the OLMo-Mix-1124, a collection of approximately 3.9 trillion tokens sourced from DCLM, Dolma, Starcoder, and Proof Pile II.

As part of DCLM, we provide a standardized corpus of 240T tokens extracted from Common Crawl

So, it's using data from Common Crawl. What data, exactly? That'd be harder to dig up. DCLM has a repository, but they don't make an effort to point out how, or if, they're filtering the data.

What I can quickly find is information from Common Crawl itself, which is the ultimate source of data. On that I can immediately see only two things:

  1. They have an opt-out list, and from what I understand, they're scraping websites by default, providing mechanisms for you to block them or opt-out... If you're even aware they exist.
  2. According to their stats page, they have 313423 pages scraped from github.com. I doubt github has that many pages of non-user-generated content, so the question is... What data is going in there, who wrote it, and did anybody agree to that? What about other websites from those top domains, such as blogspot.com, wordpress.org, readthedocs.io? I doubt they got permission to use the 17175161 pages of content from blogspot they've scraped.

So yeah, maybe there's a "good" model out there I'd actually accept, but I've seen "open" models being released, and I can't possibly check all of them, just checking the websites for one of the sources for one of the models took 20 minutes here, if I wanted to verify this properly I'd have to setup the tooling to query the terabytes of data for this source, and all others, and even there I'm not sure if I'd find answers.

[–] Fawkes@lemmy.zip 1 points 1 day ago (1 children)

Okay, so you actually take issue with internet scraping itself? Let me get things straight, do you have a problem with LLMs themselves, or the data used to train them? It seems like an obvious "Yeah, the data is the issue," to me but I want to clarify because I HAVE seen the other stance before.

And if it is just the data, does that mean your REAL issue is the act of internet scraping itself? Because then you're also going to have to condemn every search engine and indexing algorithm in the world and go back to pre-indexing days when we just typed in the URL as the only method.

Or is it the monetisation that you don't like? Because then the models I provided should be exempt because they're 100% free.

[–] kuberoot@discuss.tchncs.de 1 points 9 hours ago

It's not as simple as one singular aspect, and I don't know if I might've been fine with it if it was actually open and free from the start. But where I draw the line on ethics is with the usage of data without permission to create models intended to reproduce the data - which means laundering the data, reproducing the patterns from it to create competition/replacements for the people who made the creative work it's trained on.

The monetization is a more complicated aspect, because it is also fact that at this point the big GenAI companies are dominating the market and consuming an unreasonable amount of resources to grow, and with that any use of LLMs is supporting that market. Using the services of those companies obviously supports them, publicly using LLMs supports the legitimacy of the market, and if the current trend continues, private use of selfhosted LLMs is making yourself dependent on them, and it sure seems like the tech market is doing its best to make the necessary hardware unavailable to people, meaning you're setting yourself up to be dependent on the companies in the future.

But, that's the fun thing - this is a short explanation of my opinion, because this is a complex topic, and I'm not writing this out every time, and people wouldn't read it. My specific opinion is that governments need to catch up on copyright law, and hopefully GenAI is a bubble that bursts soon. If anti-GenAI sentiments become more widespread, maybe it'll happen. If not, then I'll have to eventually give up on my morals, but for now I can only hope it doesn't come to that.