Fast seeking and encrypted history

Published: Thu 01 August 2019

Another week, another set of cool new Peergos features. We've been hard at work and wanted to tell you about our latest new features: super fast seeking within a file, encrypted history (fix the back button), and storage usage and quota display.

Imagine you were watching a long movie in Peergos using our built in streaming video player, and wanted to skip ahead to where you left off last time, around two hours in. Previously this involved downloading the encrypted metadata for every 5MiB chunk of the file (a few network requests), decrypting the location of the next chunk, and repeating this until you reach the desired offset. The links between the chunks are encrypted so that the outside world (or the server) can't deduce the size of your files. This was SLOOOOOW. To seek a gigabyte or so in took about 5 minutes, or more depending on your internet connection latency! No one wants to wait around that long. so we came up with a new scheme for deterministic, but still private subsequent chunk locations. Before this change each chunk location was 32 random bytes. In the new scheme we calculate the location of a subsequent chunk as the following, where the secret is 32 random bytes that are stored in the encrypted metadata of the first chunk of the file:

next location = sha256 hash(current location + secret)
How we now calculate the locations of later parts of a file

This means that to seek ahead we don't need to do any IO at all. We just do a bunch of hashing locally. This is ridiculously fast and doesn't leak to the world the connections between your chunks. We can now seek many gigabytes into a file in about 5s, which is mostly the time to retrive the eventual chunk of the file.

http://localhost:8000/#{%22nonce%22:%22hl6U6X/ytjRGT+pi5wmJFq...,%22ciphertext%22:%22oK9JAGotzDOELvIon...}
A URL with the encrypted path in the fragment.

The second feature this week is encrypted history. Until now, if you press the browser back button you get logged out of Peergos. Super annoying! Now, every time you change directory we store the path, padded and encrypted, in the url fragment. If we didn't encrypt it then there'd be a log of every directory you visited in your browser history. The other requirement is that we catch changes to the url (e.g. caused by pressing the back button) and try to decrypt and interpet the resulting path. This makes navigation much more intuitive.

We also add to the history if you open a file in Peergos, and if you log in through one of these links then we will restore you to the correct directory, and open any file you had open. This allows you to store "deep bookmarks" to things in Peergos which don't reveal anything to someone who get's access you your bookmarks!

Space usage display
Space usage display

The third feature is a display of your current space usage and allowance. This lets you see at a glance how close to your limit you are, and you can request more from the admin for your storage node. That could be a friend who's running a Peergos server for you and gives you more storage for free. Or later down the line we will add the ability for a hosting provider to charge money for storage. After you've requested more storage an admin can approve it next time they log in.

Request more space
Request more space

Want to be part of the future? Create an account on https://peergos.net or self-host your own private personal datastore.

Happy Peergossing!

-The Peergos team

RECENT POSTS

Back to Top