You made this on your phone on the bus ride to and from work.
I cleaned the cat box yesterday and considered that an accomplishment.
Fuck.
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
No low-effort posts. This is subjective and will largely be determined by the community member reports.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
You made this on your phone on the bus ride to and from work.
I cleaned the cat box yesterday and considered that an accomplishment.
Fuck.
Congrats on the cat box cleaning!
Hey fellow scener, cool project!
Just a few thoughts/questions:
BTRFS and ZFS support real deduplication via copy on write, and would eliminate all current disadvantages of symlink and hardlink deduplication. It just works.
yeah that's a good point, I'll add an option to take advantage of this if you know you're running on a filesystem where that works as intended.
Why have it be one huge python source file?
oh don't worry, it's all separate files during development -- there's a build-stage which bundles everything up into a single file for distribution. But thanks for the concern :D
What do you use to bundle into one file?
copyparty-sfx.py is a custom packer (see this reply) created by make-sfx.sh, and copyparty.pyz is a standard zipapp, created by make-pyz.sh. The zipapp has more disadvantages than the sfx.py, so that's the default/recommended build.
Ah, so you have compiled it into one file? Didn't know that was possible for python, what tool do you use for this?
sooo this is one of the things that started with someone saying "wouldn't it be funny if..."
if you open copyparty-sfx.py in a text editor, you'll see how -- but please make sure to use an editor which is able to handle about 600 KiB of comments which contain invalid utf8 / binary garbage 😁
I ended up rolling my own packer since I wanted optimal encoding efficiency, and everything I could find would do stuff like base85 or ucs2 tricks, but it turns out python is perfectly happy with binary garbage in comments if you declare that the file is latin-1 so it realizes all hope is lost :D
the only drawback of the sfx.py is that it needs to extract to $TEMP before running, so that's the slight advantage of the zipapp (the .pyz alternative), but that suffers from some performance reduction in return, and is more hermetic (doesn't let you swap out the bundled dependencies with fresh versions as easily if necessary)
Elderly raspberry pi B [✓]
Large portable drive gathering dust [✓]
Guess I'm setting up a locally hosted file server in the near future.
Clearly a labour of love 👍
Maybe support for some music streaming apps (subsonic?) would be cool?
That's a neat idea -- I've heard that a lot of stuff uses the subsonic API under the hood, so I'll see what it would take to become compatible with that. At first glance it looks like I'd have to mine and index way more information about audio files, but could still be doable :>
I have a question, and I want to emphasise thar this is not criticism but a request for dive into technicalities.
In the video you mentioned copyparty has an one-way sync tool. Is there a good reason why it's not two-way, or is this just something you weren't motivated to do?
No worries, good question :>
The problem with bidirectional filesync is that it's an absolutely massive can of worms, very easy to mess up, and the consequences of messing up are usually the worst kind (loss of data). There's an insane amount of edgecases to keep in mind, and you need to get every edgecase right every single time, otherwise you might wipe someone's vacation photos, or suddenly downgrade someone's keepass database to an older version... And stuff like syncing multiple devices to the same server makes it balloon further.
I've started becoming more confident in copyparty's filesystem-index database, but it's still just a hint/guideline, with the filesystem being the only source of truth -- it's still not something I'd trust with tracking sync-state against one or more clients.
The bigger guys who offer bidirectional sync (nextcloud, syncthing, etc.) have spent years perfecting their logic, so I'd like to leave this in their capable hands.
Sometimes I feel so new to setting up my own digital ecosystem because I look at a thing and think "that's so cool" but struggle to imagine it at home. So could someone help me understand.
This would be a replacement for something like Google Drive or Proton Drive? The actions I would use this for would be:
So I would spin this up on my NAS or my main PC and replace those services and accomplish those actions using this software?
Are there other services or actions I'm missing? Am I misunderstanding the premise entirely?
I think Copyparty would be great for that purpose. The only thing you're missing is a way to expose it to the internet, such as a public IP or some tunnel
Yep! Depending on what your home connection looks like, you have a few options:
if you are lucky enough to have your own private IP-address and are able to open ports, then you're almost done already -- you can put copyparty on some port (or keep the default 3923), and then anyone could connect to it by going to https://your.ip.address:3923/
(with this approach, you will want to create your own HTTPS certificate so the traffic is properly encrypted -- the best option here is to get a domain and get a certificate for the domain)
however, if you are behind CGNAT, meaning your internet provider has given you a shared IP-address, then people cannot connect directly to your home-PC. One way around that issue is by setting up a machine somewhere on the internet which bridges the gap back home to your PC. Cloudflare offers this as service, and this is explained in the copyparty readme -- see the "at home" section for one way to do that.
if you are against using Cloudflare for idealistic reasons (they are becoming quite powerful since they run a whole lot of the internet), then you can set up a cheap VPS which serves the same purpose. That's my setup, and how you are accessing the copyparty demo server right now -- I have the cheapest VPS you can get from Hetzner. The VPS is running nginx, and it forwards the traffic to my homeserver through an SSH tunnel. I haven't documented this approach in the copyparty readme, but I have a feeling a lot of other people have :>
I would probably remove python 2 support, it was end of life when the project was started.
As long as it's not causing any issues or drawbacks for modern python versions (and it isn't), I don't see any reason to do that -- on the contrary, I know people are running copyparty on retro equipment, so I'd very much prefer to keep it for as long as possible :>
Can we have Gopher support?
I was thinking of that!! But then I realized that even Firefox removed gopher support by now, so the joke was dead on arrival :P
Firefox isn’t that great for FTP either. Gopher still exists :-) and I’d love to use it even more.
I bumped into copyparty the other month when looking for a software that could let me transfer files to a friend with the ability to pause/resume. Didn't bother with it, tried another software instead. Never really got it to work so I gave up on it.
Bumped into the YouTube video today, decided to give copyparty a shot, damn sir you've written a fine piece of software. It's so easy to get up and configure. The UI is a bit janky, but charming at the same time. Thanks for all the hard work!
"It's janky. It's charming. It works. It's Copyparty." ®