Discussion:
P2P core/daemon in Base?
(too old to reply)
Le Baron d’Merde
2018-02-22 02:51:58 UTC
Permalink
Dear Fellows,

Nothing new and probably will not be implemented, specially by me who can't write code, but I am sharing this idea anyway.

The point would be to have a P2P core/daemon in Base with some API to be used by clients, including pkg, freebsd-update, and portsnap becoming P2P based/capable, and would also allowing any user that desire to become a FreeBSD mirror[1]. What would reduce the load on the FreeBSD infrastructure.

Other than those, idealistic, that built-in P2P core to be also capable of handling
"modern" needs, like a net/syncthing equivalent client, and the usual torrent clients (CLI,
GTK, Qt etc.).

Some time ago, while digging in the Syncthing resources I found some people willing
to use that to keep files synced between thousands of servers, and so I imagine this
kind of feature would (very?) be appealing at enterprise level.

This piece of software could of course be integrated with IPFW, Capsicum or wherever be necessary, possible or desired.


1 - The FreeBSD mirror capability would be a separated feature allowing for those who
desire to partially mirror the FreeBSD packages, etc. For instance in a way that user
to mirror from just the latest packages to everything.
--
Best Regards.
LBdM.
Dave Cottlehuber
2018-02-22 13:25:56 UTC
Permalink
Post by Le Baron d’Merde
Dear Fellows,
Nothing new and probably will not be implemented, specially by me who
can't write code, but I am sharing this idea anyway.
The point would be to have a P2P core/daemon in Base with some API to be
used by clients, including pkg, freebsd-update, and portsnap becoming
P2P based/capable, and would also allowing any user that desire to
become a FreeBSD mirror[1]. What would reduce the load on the FreeBSD
infrastructure.
At present if nobody's downloading 11.1R.txz then there's no load to the mirrors. Most P2P implementations require some form of seed node that keeps track of active peers & blocks, so it's not quite a free lunch.

I've an incomplete erlang-based version of https://tools.ietf.org/html/rfc7574
called PPSP (Peer-to-Peer Streaming Protocol), and I always felt a C or
rust-based implementation of it would be a great addition to FreeBSD whether ports
or base.

PPSP is a multi-peer streaming mix of:

- merkle tree across arbitrary blocks of data
- lightweight 'torrent-ish' UDP protocol for requesting and sharing blocks
- optional live-streaming (i.e. stream starts before full data is available)
- optional cryptographic signatures

For example, I'd love to be able to "zfs send" via a multipeer protocol and have it Do The Right Thing in transferring this to multiple "zfs recv" peers, or have our custom pkg repo pull stuff off the adjacent servers rather than stumble across
the Pacific Ocean for the data. [Yes I'm aware pkg can be cached already].

If somebody is interested in working on this please get in touch, I would put some
funds in and can offer some context on the spec as I was involved in it.

Also, if it's something the FreeBSD Foundation might consider jointly supporting,
I would help with the paperwork & submission.

A+
Dave

Dave Cottlehuber
+43 67 67 22 44 78
Managing Director
Skunkwerks, GmbH
http://skunkwerks.at/
ATU70126204
Firmenbuch 410811i

Loading...