Freenet development over Freenet

This is a brief guide for distributed development between anonymous and non-anonymous participants over freenet. I try to mirror all required content here.

Intro and short background

This guide shows how to use infocalypse to contribute to freenet repositories anonymously. Also it shows how to get these changes back into the git repos on github.

For hosting in freenet it uses the very efficient infocalypse extension of Mercurial. Infocalypse supports incremental updates, additional redundancy, reinserting other peoples repositories without being able to tamper with them, minimizing the number of requests needed to get a repo and a whole lot of other nice features.

Since infocalypse works for Mercurial and most freenet development is done in git (and on github), we use hg-git to transform the repository from git to Mecurial and the other way round (using the dulwich library which implements the git protocol in Python). That bridge is transparent, so no history gets lost when pushing the data to freenet and back.

Anonymous coders can then use infocalypse to retrieve the repositories from freenet, hack on them and upload their new versions directly into freenet. Afterwards non-anonymous hackers can retrieve these changes and push them back into github.

Preparation

Base requirements (mirrors in brackets):

These will be out of date, soon. But they should suffice for in-freenet development for a long time.

Get and install infocalypse

This will use the commandline on GNU/Linux. If you want to provide instructions for GUI users, feel free to fork this repo.

Done. You just added freenet capabilities to your DVCS and updated the repo over freenet.

Contributing to an existing project

To contribute to a project anonymously, you need someone who uploads the repository into freenet (lets call him core devel) and someone who can get your changes back to the core devs, for example by pushing them onto github.

So we can break up contributing into 3 tasks:

  1. Getting and forking repos from freenet and
  2. Mirroring freenet repos from github into freenet,
  3. Transferring the changes back onto github.

As an example we will do that with the pyFreenet repo (since inserting and downloading the fred-repo takes some time).

Getting and forking repos from freenet

To get a repo, we first initialize an empty repo and then use infocalypse to pull the changes from the given REQUEST_KEY:

(And wait a bit. For me this call on a test repo took 3 and a half minutes)

--aggressive tells infocalypse to search harder for the most current USK revision.

Then we hack on the repo (if you are more comfortable with git than with Mercurial, you can use hg-git to push it to a git repo and pull back the changes once you are done).

To share our work, we push the code back into freenet:

This gives us ANOTHER_REQUEST_KEY.

Now share the request key.
Never share the insert key, except if you want to pass on maintenance of your in-freenet repo.

If you want to get a notice when a repo gets updated, just bookmark it like any other freesite and fred will keep you up to date.

Mirroring freenet repos from github into freenet

Shown with the example of pyFreenet.

Requirements

To mirror a repo from github to freenet, you need hg-git and dulwich.

Enabling hg-git

For installing and activating hg-git, let’s just ask the hg-git website:

First, make sure that you have a working C compiler on your system. On Windows, one option is the free Microsoft Visual C++. (Be sure to install the 2008 version for compatibility reasons.) On Debian-style Linux, run apt-get install python-dev.

Secondly, run easy_install hg-git. If you don't have easy_install available, you can get it as part of Python's setuptools package: apt-get install python-setuptools.

Lastly, make sure the following is in your ~/.hgrc:

[extensions]
hgext.bookmarks =
hggit =

Getting the repo

Now you can simply clone the repository

And upload it to freenet

hg fn-create gives you a REQUEST_KEY at the end. For my pyFreenet upload it gave me this:
USK@-sEkUkv7SClGNhjpvFF9TtmGHnoiwvThkepMUTy7Fh4,Rciq6x0804hb~rNzxmINfkuSCgAEvHZjJNHQcUo0nuQ,AQACAAE/pyfreenet.R1/0

Share the request key to enable others to get the repo.

Never share the insert key, except if you want to pass on maintenance of your in-freenet repo.

Transferring the changes back onto github.

To get the changes back onto github, we simply do the reverse path of getting the repo from github: Pull from freenet, then push to github. But this time we use an authenticated ssh connection for interacting with github.

So essentially, you need to contact a core dev and ask him to do the following:

Hacking on the main freenet repo

If you want to hack directly on fred, you can get it with the following key: Fred now has a subrepo which BREAKS YOUR ANONYMITY if you clone because it gets Mercurial to access Github. If you really, really want to do this before we resolved this, look at the sourcecode of this site. If you want to help fix this, please contact ArneBab via IRC #freenet @ irc.freenode.net

It is uploaded once a week. Nag ArneBab on Sone if something breaks :)

Common tasks

(some repetition to come)

Sharing a Mercurial code repo over freenet

Getting and forking a repo

Sharing a Github repo over freenet

Included here, because github is currently used as non-anonymous development platform.

Get hg-git and dulwich.

To get the changes back onto github, do the reverse:

Getting info on a repo

hg fn-info

Making sure that a repo stays available

Content in freenet can fall out if it is not requested. These are tipps to make sure that repos you consider important stay available.

Your own repo

Reinsert it. There are different levels of reinsertion. 1 and 4 can only be done by the creator, all the others can be done by anyone who downloaded the repositories. The levels:

level: 1 - re-inserts the top key(s) 2 - re-inserts the top keys(s), graphs(s) and the most recent update. 3 - re-inserts the top keys(s), graphs(s) and all keys required to bootstrap the repo. This is the default level. 4 - adds redundancy for big (>7Mb) updates. 5 - re-inserts existing redundant big updates.

Now reinsert, since this is our own repo, we use all levels:

Get it in a new repo from time to time to ensure that it is still available:

Unbreak a repo

If a repository becomes really unretrievable, you can simply recreate it:

Just use hg fn-create --uri INSERT_URI again (with the key from hg fn-info).

Sponsoring the repo of someone else

Reinsert all parts for which you do not need the insert key:

Getting additional information on infocalypse

hg help infocalypse

Contributing to this guide

If you wish to contribute, just clone and fork this repo.

Request key: USK@ugGhE7lfI-QyFYL6yuZt7uyhlrVfvIRo0xkQxd2qV~c,L1bCwDm5Hh5qgqml7SBQcF85vL8IPvzN4WihjWIKH8I,AQACAAE/infocalypse-dvcs-over-freenet.R1/10

The not-just-downloaded files (and dulwich) are in static/

Currently the site gets parsed by pymarkdown_minisite, which is not yet included in this repo (⇒TODO: Add a small parser). Since it is not yet included, you can either edit the index.txt (preferred) or directly work on the static/index.html to add something to the site.

Then just upload the static/ folder via freesitemgr (from pyFreenet) or jSite - and share the code via hg fn-create --uri INSERT_URI.

Infocalypse: code sharing over Freenet

-- 2014-09-24 10:26:26 --


Created with pyMarkdown Minisite
using the layout from the pybrary.