Peergos release v0.1.3

Published: Sun 15 September 2019

We've been hard at work, so hard in fact that we're only now writing another blog post. We've sped everything up, enabled native builds, added garbage collection, added S3 storage support and fixed several bugs.

Everything in this post is contained in our latest release, v0.1.3, which you can try out on our server

The most obvious change is that we've sped most things up by ~2.5X, notably uploads and downloads. Our download speeds are now comparable to Dropbox, despite the extra overhead of decryption!

Faster downloads
2.5X faster downloads

The next big feature is transaction-aware garbage collection. This is critical to ensure our disks don't unnecessarily fill up.

If you change 1 byte of a file bigger than 5 MiB, then we reencrypt that 5 MiB section of file. The new cipher text for this section has nothing in common with the old cipher text (otherwise it wouldn't be very good encryption!!). This means that now 5 MiB of blocks are unused and wasting space. Eventually, with enough modifications you'd fill up the disk with all this garbage.

Too much garbage
Too much garbage

The operation that cleans up these unreferenced blocks of data is called garbage collection. The difficult part is making sure we don't garbage collect data that someone is in the middle of writing (before they commit it). At the moment, this is achieved with a process wide lock. Once IPFS implements native transactions we can remove this global lock and get much better throughput.

The next addition was statically compiled native binaries. Until now, to run Peergos yourself you had to independently install Java, which is ~100 MiB. This is an extra barrier that makes it harder to use, and which some people aren't willing to do. The native builds:

  1. are smaller in total size (32 MiB compared to a 13 MiB jar plus a 100 MiB JVM);
  2. use much less RAM than a full JVM;
  3. start much faster than a JVM (4 ms compared to 660 ms).

The last feature in particular enables us to have an interactive command line interface which doesn't require running a daemon. We'll be writing more about this soon! The only negative of native image builds is we don't yet have reproducible builds for it.

The final feature in this release is support for using an S3 compatible service for data storage instead of the local hard disk.

As well as all these new features we've fixed a host of bugs in both the server and the client. So head on over to peergos.net and check it out!

RECENT POSTS

Back to Top