I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

you are viewing a single comment's thread
view the rest of the comments

[–] rimu@piefed.social 76 points 23 hours ago (2 children)

Lemmy does this to itself, too - when you create a post it federates out and then every Lemmy instance that receives it makes a GET request to retrieve the post from the original server. Even though it just received the exact same data. Some other fedi software is even worse - Akkoma does a GET of the id of every activity it receives, not just for posts but for Likes too.

It's insane.

Anyway, proper caching at the reverse proxy / Cloudflare level solves it. The same kind of caching setup would do the same for sites serving HTML, like NPR too.

[–] Ghoelian@piefed.social 6 points 16 hours ago (2 children)

That's weird. Is that how it's supposed to work, or did they just implement it in a bad way?

[–] rimu@piefed.social 2 points 6 hours ago

I'm not sure but my guess it's an attempt to deal with the problem where sometimes federation activities arrive in the wrong order (especially after an instance has been offline for a while) - by making a GET to the origin server you can be sure you've got the latest version of the content. There are probably better ways to achieve that, though.

PieFed prioritises efficiency over correctness so it just accepts whatever is sent to it. In 3 years no one has complained about seeing stale content although I'm sure it happens occasionally.

[–] The_Decryptor@aussie.zone 11 points 14 hours ago

It's an access control method, called "authorized fetch". Rather than broadcasting the data you just announce the source of it, and then any client that wants to show it needs to negotiate with the originating server to request the data (And then blocklists and such come into play)

Say you've got 3 users, A, B, and C. A has blocked C, B follows A, and C follows B. Now A makes a post, B sees it in their feed and reboosts it, what does C see?

If B's server sends the full response along, then C can see the contents of A's post even though they've been blocked by A. But if all it sends is the URL of the post, C's server needs to make a request for it (providing authentication), and then A's server can reject it, so C sees nothing.

[–] SorteKanin@feddit.dk 1 points 16 hours ago

How else can you be sure the content is valid though? Lemmy posts are sent as Announces from the community, not from the actor posting the post. The community could just fake a post if you just trust it.