When you post on reddit, instagram etc. you're giving the company your valuable data. (And it often is valuable, you put your thoughts into it and all that). Consider if you take 15 minutes to create value content, you'd get paid $5 or sth if you were doing it commercially. Now all that value is donated to big tech companies. (that's the only reason why they have value at all.)
So what's the problem with that? The problem is that reddit and others can instantaneously put your years-long effort behind a paywall. I'm not joking. Look at these:
when i want to see content on reddit (for research) or instagram (for local community) they don't allow me to see the data that users have contributed. Reddit claims that it blocks VPNs for "security reasons", meanwhile i'm using VPNs in browser's incognito mode to not be tracked. Instagram wants you to sign in sothat they amass more users and can take leverage of the network effect even more.
I can understand restricting who can post to limit bot activity, but reading must always be possible. After all, it's our data. That's exactly what happens when you give your data to big companies to "communicate with other people": your data becomes a source of income for them, at our expense.
You can download archives of Reddit comment and submissions from Academic Torrents.
I'm not aware of any similar public archives of Fediverse content.
Interesting, if I'm not misreading that index you have 2007 to 2015, then jan 2024 onwards monthly (at 30-50gb a file) but not the 9 years in the middle
hmm i'm not entirely sure either but it seems like fediverse content should be easily archivable.
probably this is another engineering challenge to think about once that i have checks notes a few months of free time ...
Doesn’t every lemmy instance federate all the content anyway?
Not holistically, as an archive. When new content is posted on an instance it gets pushed out to any other instances that are subscribed to it. That's the only time the content is sent out. So if you start up a new instance but don't subscribe to anything nothing will be sent to you, and you won't get any historical content sent to you from beforehand. The Fediverse isn't a "flood fill" mesh.
Gotcha. But if someone ran a lemmy archive instance continually it would work.
Sure, if they subscribed to literally everything and had started doing so right from the beginning.