Notes on releasing Freenet

Inhaltsverzeichnis

Also see ./scripts/README.md

Using the release VM image

Boot the system

qemu-system-x86_64 -smp 2 -m 2048 -enable-kvm -hda debian_wheezy_amd64_standard.qcow2 -nographic -net nic -net user,hostfwd=tcp::10022-:22

Test releasing

Personalize Setup

Firstoff create a new SSH key and change the authorized keys. You won’t want to have someone logging into your release VM with the default key from this image. 

rm -r ~/.ssh
ssh-keygen
cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys

Create a new GPG key: 

gpg --gen-key # use size 4096 and otherwise default values

Create your own fake insert key and use it: 

GPGEMAIL="myself@local"
GPGUSERNAME="myself"

# create a new insert key and never store it in unencrypted form
sudo swapoff -a # disable all swap
# cut removes the trailing /
echo NEWKEY=$(fcpgenkey | tail -n 1) | cut -d / -f 1 > /run/user/1000/.freenet.insertKeys
# encrypt the key
gpg -er "${GPGEMAIL}" /run/user/1000/.freenet.insertKeys
cp /run/user/1000/.freenet.insertKeys.gpg ~/.freenet.insertKeys.gpg
# remove the unencrypted version
rm /run/user/1000/.freenet.insertKeys
sudo swapon -a # re-enable swap

# setup git
GPGLONGID="$(gpg --list-secret-keys --with-colons "${GPGEMAIL}" | head -n 1 | cut -d : -f 5)"
git config --global user.name "${GPGUSERNAME}"
git config --global user.email "${GPGEMAIL}"
# also setup the gpg key to avoid getting cryptic error messages
git config --global user.signingkey "${GPGLONGID}"

Prepare Plugins

With the example of Sharesite 

cd ~/
git clone --recursive https://github.com/freenet/plugin-sharesite  plugin-Sharesite
cd plugin-Sharesite
git checkout master
git checkout .
# (do changes, i.e. pull a branch - this can be done outside the release VM!)
# create a signed tag
# first get the message from the NEWS file
TEMPFILE="$(mktemp /tmp/msg.temp.XXXXXXXXXX)"

TAG="$(grep -m1 Sharesite NEWS | sed 's/.*Sharesite //' | sed s/://)"
grep -m2 -B100 Sharesite NEWS | grep -m1 -A100 | head -n -1 > "$TEMPFILE"
git tag -s -F "$TEMPFILE" "$TAG"
# share the tag
git push --tags
# now release the plugin (must be done from the releae VM because it need access to the website - this mightchange with the new site)
cd ~/scripts
./release-plugin Sharesite # runs ant, expects to create a file dist/Sharesite.jar
# once the update finishes you can get the keys via wget http://127.0.0.1:8888/uploads/listKeys.txt
# copy the plugin jar for release
cp ~/plugin-Sharesite/dist/Sharesite.jar ~/FreenetReleased/Sharesite.jar
cd ~/fred
# now adjust fred to point to the released plugin

# edit src/freenet/pluginmanager/OfficialPlugins.java and adjust
# .loadedFrom in the addPlugin call to point to the new CHK. Do NOT
# change minimumVersion, because it blocks the node from starting (if
# I second guess the code correctly). Only use recommendedVersion if
# there is a minumumVersion. In the case of Sharesite, the version is
# realVersion in src/plugin/Plugin.java To add a default plugin, edit
# the default freenet.ini somewhere (TODO: Find the defaultfreenet.ini
# )

# and commit the change
echo "    Update Sharesite to $TAG

    Built from the tag $TAG.
" > "$TEMPFILE"
git commit -F "$TEMPFILE" --gpg-sign# =<email>

Release Fred

Ensure that fred is at master (with the changes you want to add already in master). 

cd ~/fred
git checkout master
git checkout .
cd -

Now release a build. 

BUILD_NUMBER="1476"
cd ~/scripts
./release-build "${BUILD_NUMBER}"
# follow the process
# If something goes wrong, just add 
# set -x 
# as the first non-comment line of the script which fails (this outputs every command it runs).

Finally test the release: Testing a test release.

The initial setup you’re building on is described in Fakes for testing.

Do real releases

In addition to the Test releasing, get the insert key and the jarsigner certificate, encrypted to your new gpg key.

Get access to osprey.vm.bytemark.co.uk and https://github.com/freenet/fred (via github oauth token).

Login to your image: 

ssh -XY -p 10022 user@localhost

Get your long GPG ID: 

gpg --list-secret-keys --with-colons | head -n 1 | cut -d : -f 5

Now Adapting this image for real Freenet releases.

to do for switching to real releases

  • [X] re-clone fred and add ArneBab remote: git clone git@github.com:freenet/fred; cd fred; git checkout master; git checkout .; git remote add ArneBab git@github.com:ArneBab/fred-staging-1; git pull --no-ff --gpg-sign ArneBab stable-1476
  • [X] ensure that I’m on master git branch.
  • [X] re-add fred/lib and check ant.
  • [X] activate asset uploading in scripts/release-build (uncomment)
  • [X] adjust the server in ~/.freenetrc (from localhost to [myusername]@...osprey...)
  • [X] setup a website deploy environment on osprey (copied the ~/.freenetrc from ~operhiem1/.freenetrc, adjusted it to point to my home dir and cloned the script, fred, wininstaller and website repositories)
  • [X] replace the testing insert key file with the real insert key file
  • [-] run ./release-build 1476 – I got failures at releasing the javadoc. Deployed the website to testing.freenetproject.org by adjusting ./remote-deploy-website to add –testing to ./deploy-website
  • [X] cd ~/scripts/; ./remote-deploy-website; ./insert-update

My release process

To do a release, I first create a branch <i.e. stable-1476) in my personal fred repo (github.com/ArneBab/fred-staging-1) with the changes (including an update to the NEWS.md) and file a pull-request.

Then I ask for review of that pull-request.

To update translations I use cd ~/scripts/; ./transifex-pull (needs a section [https://www.transifex.com] with hostname, password, token and username).

Afterwards I cd into ~/fred and git pull --no-ff ArneBab <branch>; git commit -m "merge branch <branch>" --gpg-sign. --no-ff forces the creation of a merge commit so I can abort the merge commit and create a new one as signed merge.

That done, I can cd ~/scripts/; ./release-build 1xxx and follow the steps.

Preparation

debian stable qemu image: https://wiki.debian.org/QEMU#Setting_up_a_stable_system 

# get wheezy (7.0), since there isn’t jet a premade image for jessie (8.0, stable since 2016-09-17)
if test ! -f debian_wheezy_amd64_standard.qcow2; then
    wget https://people.debian.org/~aurel32/qemu/amd64/debian_wheezy_amd64_standard.qcow2
    wget https://people.debian.org/~aurel32/qemu/amd64/README.txt
    gpg --keyserver pgp.mit.edu --recv-key C376A8DAF1BCDB73
    gpg --check-sigs C376A8DAF1BCDB73
    gpg --verify README.txt
    grep debian_wheezy_amd64_standard.qcow2 README.txt
    md5sum debian_wheezy_amd64_standard.qcow2 # should fit the README TODO: find something stronger than md5
fi
# qemu with curses for text-based usage
qemu-system-x86_64 -m 2048 -enable-kvm -hda debian_wheezy_amd64_standard.qcow2 -curses
# root pw: root, user: user, user-pw: user
# install what we need. Start with GNU screen and git.

setup sudo for user 

# as root
apt-get install sudo
visudo

with visudo add a line which says: 

user    ALL=(ALL:ALL) NOPASSWD: /usr/bin/apt-get
user    ALL=(ALL:ALL) NOPASSWD: /sbin/shutdown -h now
user    ALL=(ALL:ALL) NOPASSWD: /sbin/swapoff -a
user    ALL=(ALL:ALL) NOPASSWD: /sbin/swapon -a

Building fred

Prepare the Debian stable virtual machine

Install the requirements for building fred: 

# first priority: avoid systemd (I consider systemd as inherently flawed and a security risk)
# from https://www.debian.org/releases/jessie/amd64/release-notes/ch-information.en.html#systemd-upgrade-default-init-system
echo 'Package: systemd-sysv
Pin: release o=Debian
Pin-Priority: -1' > /etc/apt/preferences.d/local-pin-init
# update the package listing
apt-get update
# install GNU screen and GNU Emacs to get a better interface
apt-get install screen emacs
# pv is useful for any larger shell interaction (progress bar)
apt-get install pv
# add another randomness source
apt-get install haveged
# do a full upgrade: I need jessie for gradle. This will take some time.
# edit /etc/apt/sources.list and replace wheezy with jessie
sed -i s,wheezy,jessie,g /etc/apt/sources.list
apt-get update
apt-get upgrade # start with a minimal upgrade
apt-get dist-upgrade # update everything
apt-get autoremove
# install packages needed to build Freenet (search: apt-cache search <name>)
apt-get install openjdk-7-jdk openjdk-7-dbg ant ant-optional ant-doc git libjna-java libjna-posix-java python python3 gradle libobjenesis-java python-setuptools zip wine
# install wine32 for the old AHK-based windows installer releases
dpkg --add-architecture i386 && apt-get update && apt-get install wine32
shutdown -h now

boot again with nographic, since curses does not work with jessie. Add -smp 2 since we’ll want to run a Freenet node while doing other stuff, so this should help. 

# -net nic -net user,hostfwd=tcp::10022-:22 # for ssh access
# -net nic
qemu-system-x86_64 -smp 2 -m 2048 -enable-kvm -hda debian_wheezy_amd64_standard.qcow2 -nographic -net nic -net user,hostfwd=tcp::10022-:22 
# now you can login to your VM with ssh -XY -p 10022 user@localhost

Get the requirements

su - user # ensure we’re no longer root
for i in scripts fred pyFreenet; do 
    # --recursive gets submodules
    git clone --recursive https://github.com/freenet/$i
done
git clone -b master --recursive https://github.com/freenet/fred fred-upstream
cd fred-upstream
# track all branches
for remote in `git branch -r | grep -v /HEAD`; do git checkout --track $remote ; done
git fetch --all
git checkout master
git checkout .
git submodule update --init --recursive
cd -
# use junit4 and hamcrest from debian:
cp /usr/share/java/junit4-4.11.jar fred/lib/junit4.jar
cp /usr/share/java/hamcrest-core-1.3.jar fred/lib/hamcrest-core.jar
# alternatively get junit and hamcrest as described here: https://github.com/junit-team/junit4/wiki/Download-and-Install
# wget -O ~/fred/lib/junit4.jar http://search.maven.org/remotecontent?filepath=junit/junit/4.12/junit-4.12.jar
# wget -O ~/fred/lib/hamcrest-core.jar http://search.maven.org/remotecontent?filepath=org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar

Setup pyFreenet 

echo 'PATH="$PATH:$HOME/.local/bin"' >> ~/.bashrc
source ~/.bashrc
cd pyFreenet
git remote add ArneBab https://github.com/ArneBab/lib-pyFreenet-staging
# for legacy python2: fcp
git checkout master
git checkout .
python setup.py install --user
# for python3: fcp3 and freenet3
git pull ArneBab py3
git checkout py3
git checkout .
python3 setup.py install --user
cd -

Install the requirements for releasing fred: 

cd fred
wget https://www.bouncycastle.org/download/bcprov-jdk15on-154.jar -O lib/bcprov-jdk15on-154.jar 

# requirements for next
wget https://maven.java.net/content/repositories/releases/net/java/dev/jna/jna/4.2.2/jna-4.2.2.jar -O lib/jna-4.2.2.jar
wget http://central.maven.org/maven2/org/mockito/mockito-core/1.9.5/mockito-core-1.9.5.jar -O lib/mockito-core-1.9.5.jar
wget https://maven.java.net/content/repositories/releases/net/java/dev/jna/jna/4.2.2/jna-4.2.2.jar.asc -O lib/jna-4.2.2.jar.asc
wget http://central.maven.org/maven2/org/mockito/mockito-core/1.9.5/mockito-core-1.9.5.jar.asc -O lib/mockito-core-1.9.5.jar.asc
gpg --keyserver pgp.mit.edu --recv-key 4DB7BC57DFDBCEA4
gpg --verify lib/jna-4.2.2.jar.asc
gpg --keyserver pgp.mit.edu --recv-key A1B4460D8BA7B9AF
gpg --verify lib/mockito-core-1.9.5.jar.asc

# prepare some remotes
git remote add xor https://github.com/xor-freenet/fred-staging
git remote add ArneBab https://github.com/ArneBab/fred-staging-1

cd -

Install Freenet nodes

mkdir ~/FreenetNode
cd ~/FreenetNode
wget --ca-certificate=~/freenetproject.pem 'https://freenetproject.org/assets/jnlp/freenet_installer.jar' -O new_installer_offline.jar
java -jar new_installer_offline.jar -console
# follow instructions: just click enter and then 1

# lie about completing the wizard to get Freenet running with default values
./run.sh stop
sed -i 's/^End$/fproxy.hasCompletedWizard=true/' freenet.ini
echo 'End' >> freenet.ini
./run.sh start
# freenet will start automatically on your next login
cd -

Try to release next with gradle

cd fred
# try gradle
./gradlew tasks # local version, thanks to nextgens it works now!
cd -

This isn’t ready yet and has quite a few experimental parts. Therefore, after ensuring that it builds and the tests work, we’ll turn to releasing from maaster.

Release master with ant

cd fred
# get contrib
git submodule update --init --recursive
echo lib.contrib.get = true >> override.properties
ant
# if one test fails
ant -Dtest.haltonfailure=false
# and check which tests fail
# to just check quickly whether the code compiles:
ant -Dtest.skip=true
cd -

If one test fails, run all tests and collect all the failures: 

for i in $(find test -iname *.java); do 
  ant -Dtest.class=$(echo $i | sed -e s,test/,, -e s,/,\\.,g -e s,.java,,) || echo $i >> failing-tests.txt
done

Testing release

Fake key

# generate a request and an insert key
INSERT_KEY="$(fcpgenkey | tail -n 1)"
fcpupload -w --spawn ${INSERT_KEY}jar-1475 dist/freenet.jar
# adjust the update key in a Freenet node to USK@/.../testing-jar/1475

Original update key: USK@O~UmMwTeDcyDIW-NsobFBoEicdQcogw7yrLO2H-sJ5Y,JVU4L7m9mNppkd21UNOCzRHKuiTucd6Ldw8vylBOe5o,AQACAAE/jar/1475

Testing update key: USK@TXZFS-xDhTkR~pOi8du7ANdIyE0FPU7cNzvrQA5ZAJ0,WAQaWq3pbnCCLVvOnNR-6ftlrMtuLsYEPgdVXohViUg,AQACAAE/testing-jar/1475

maybe also https://maven.java.net/content/repositories/releases/net/java/dev/jna/jna-platform/4.2.2/jna-platform-4.2.2.jar

I need a GnuPG key on the box so an existing maintainer can send me the update key in a safe way. 

gpg --gen-key # use size 4096 and otherwise default values

Simplest uploading 














verify a fred build





Run prepared Freenet node







cd ~/FreenetNode
./run.sh start
cd -







Install dependencies for verification:







cp scripts/freenetrc-sample ~/.freenetrc
sed -i 's,insertKeys="~/.freenet.insertKeys.gpg",insertKeys="$freenetRoot/.freenet.insertKeys.gpg",' ~/.freenetrc
scripts/set-freenetrc-base
# export all variables in ~/.freenetrc
sed -i 's/^\(\w*=.*\)/export \1/' ~/.freenetrc
# get dependencies
mkdir -p FreenetReleased/dependencies
wget --ca-certificate=~/freenetproject.pem https://downloads.freenetproject.org/alpha/freenet-ext.jar -O FreenetReleased/freenet-ext.jar

wget -O ~/FreenetReleased/dependencies/bcprov-jdk15on-154.jar https://www.bouncycastle.org/download/bcprov-jdk15on-154.jar 
wget -O ~/FreenetReleased/dependencies/bcprov-jdk15on-154.jar.asc http://repo2.maven.org/maven2/org/bouncycastle/bcprov-jdk15on/1.54/bcprov-jdk15on-1.54.jar.asc
gpg --verify ~/FreenetReleased/dependencies/bcprov-jdk15on-154.jar.asc

wget --ca-certificate=~/freenetproject.pem https://freenetproject.org/assets/keyring.gpg -O freenetkeys.gpg
gpg --import freenetkeys.gpg
cd ../scripts
./verify-build
# if this fails, check whether the versions are the same:
unzip -p /tmp/tmp.*/inserted-freenet.jar META-INF/MANIFEST.MF
# vs.
cat /tmp/tmp.*/unpacked-built/META-INF/MANIFEST.MF












create release






Requirements for the script setup-release-environment





create the group freenet-buildwin:







# as root:
addgroup freenet-buildwin
adduser user freenet-buildwin







Get launch4j and IzPack:







cd ~/fred/lib/
wget -O launch4j.tgz https://downloads.sourceforge.net/project/launch4j/launch4j-3/3.9/launch4j-3.9-linux.tgz
sha1sum launch4j.tgz # sourceforge says this should be 9d4c377af0149389da9ad3c3f1394fc5a655f540
sha256sum launch4j.tgz # archlinux says this should be e6e9a83927585d16efcb82f41d4ae480f14eccc19ced611a59f31fffb5ca549b see https://aur.archlinux.org/packages/launch4j/
tar xf launch4j.tgz
rm -rf launch4j
mv launch4*/ launch4j
wget  https://oss.sonatype.org/content/repositories/releases/org/codehaus/izpack/izpack-standalone-compiler/4.3.5/izpack-standalone-compiler-4.3.5.jar
wget https://oss.sonatype.org/content/repositories/releases/org/codehaus/izpack/izpack-standalone-compiler/4.3.5/izpack-standalone-compiler-4.3.5.jar.asc
# check the signature
gpg --keyserver pgp.mit.edu --recv-key 3B58205B9D7013A9 # long ID
gpg --verify izpack-standalone-compiler-4.3.5.jar.asc
mv izpack-standalone-compiler-4.3.5.jar standalone-compiler.jar







Get plugins and libraries:







cd ~/FreenetReleased
if test -e ~/fred/dist/freenet.jar; then 
    cp ~/fred/dist/freenet.jar ./
else
    wget --ca-certificate=~/freenetproject.pem -O freenet.jar https://downloads.freenetproject.org/alpha/freenet-build01475.jar
fi
# get a verified bcprov:
gpg --keyserver pgp.mit.edu --recv-key B341DDB020FCB6AB
wget -O dependencies/bcprov-jdk15on-154.jar https://bouncycastle.org/download/bcprov-jdk15on-154.jar
wget -O dependencies/bcprov-jdk15on-154.jar.asc http://repo2.maven.org/maven2/org/bouncycastle/bcprov-jdk15on/1.54/bcprov-jdk15on-1.54.jar.asc
gpg --verify dependencies/bcprov-jdk15on-154.jar.asc
# TODO: find a way to verify is-unicode.exe
wget -O dependencies/isetup-5.5.9-unicode.exe http://www.jrsoftware.org/download.php/is-unicode.exe 

# you can get the plugins from the web:
# wget --ca-certificate=~/freenetproject.pem -O JSTUN.jar https://downloads.freenetproject.org/alpha/plugins/JSTUN/JSTUN-v5.jar
# wget --ca-certificate=~/freenetproject.pem -O KeyUtils.jar https://downloads.freenetproject.org/alpha/plugins/KeyUtils/KeyUtils-v5026.jar
# wget --ca-certificate=~/freenetproject.pem -O Library.jar https://downloads.freenetproject.org/alpha/plugins/Library/Library-v35.jar
# wget --ca-certificate=~/freenetproject.pem -O seednodes.fref https://downloads.freenetproject.org/alpha/opennet/seednodes.fref
# wget --ca-certificate=~/freenetproject.pem -O ThawIndexBrowser.jar https://downloads.freenetproject.org/alpha/plugins/ThawIndexBrowser/ThawIndexBrowser-v5.jar
# wget --ca-certificate=~/freenetproject.pem -O UPnP.jar https://downloads.freenetproject.org/alpha/plugins/UPnP/
# or just copy them from your running node, which is actually safer:
cp ~/FreenetNode/plugins/*jar ~/FreenetReleased/
cp ~/FreenetNode/seednodes.fref ~/FreenetReleased/
cp ~/fred/lib/*jar ~/FreenetReleased/dependencies/
cd -







Get legacy windows requirements:







# see ~/wininstaller/build.cmd and ~/scripts/release-wininstaller
# use via 
#   wine cmd /c build.cmd
# add jsign for signing the installers
cd ~/
git clone https://github.com/ebourg/jsign
cd jsign
git checkout de5661f376c33cdd23f4a95dd9d7c549f6fa3661 # the new maven task still does not work
git checkout .
mvn package
# FIXME: Need to update scripts/sign-exe to call 1.4 instead of 1.3
cp jsign/target/jsign-1.4-SNAPSHOT.jar ~/FreenetReleased/dependencies/jsign-1.3-SNAPSHOT.jar
cd -
# FIXME: and because that does not work:
wget -O ~/FreenetReleased/dependencies/jsign-1.3-SNAPSHOT.jar https://github.com/ebourg/jsign/releases/download/1.3/jsign-1.3.jar
gpg --keyserver pgp.mit.edu --recv-key F513C419E4B9D0AC
# this is http, but only provides the data to check with the just specified key
wget -O ~/FreenetReleased/dependencies/jsign-1.3-SNAPSHOT.jar.asc http://central.maven.org/maven2/net/jsign/jsign/1.3/jsign-1.3.jar.asc
gpg --verify ~/FreenetReleased/dependencies/jsign-1.3-SNAPSHOT.jar.asc
# add the dependencies
cd ~/wininstaller
for i in freenet.jar freenet-ext.jar seednodes.fref; do
  cp ~/FreenetReleased/$i ~/wininstaller/res/install_node/
done
cp ~/FreenetReleased/dependencies/bcprov-jdk15on-154.jar ~/wininstaller/res/install_node/
for i in JSTUN.jar KeyUtils.jar ThawIndexBrowser.jar UPnP.jar Library.jar; do 
  cp ~/FreenetReleased/$i ~/wininstaller/res/install_node/plugins/
done
# autohotkey
mkdir ~/ahk
cd ~/ahk
# generic url: wget https://www.autohotkey.com/download/ahk-install.exe
# actually used url:
wget https://www.autohotkey.com/download/1.1/AutoHotkey_1.1.24.04_setup.exe
wine32 AutoHotkey_1.1.24.04_setup.exe # install to the default location, folder selection did not work for me
cp  ~/.wine/drive_c/Program\ Files/AutoHotkey/Compiler/* ~/wininstaller/res/tool_ahk/
# run the installer
wine cmd /c build.cmd
# get innosetup for the newer installer, sadly from untrusted SSL certs
wget --no-check-certificate https://www.jrsoftware.org/download.php/is-unicode.exe?site=2
mv is-unicode.exe\?site\=2 innosetup-5.5.9-unicode.exe
wine32 innosetup-5.5.9-unicode.exe







(innosetup is later run with the separate user freenet-buildwin to
avoid corrupting the mostly clean environment. This could be done
here, too)










Fakes for testing






Create a fake GPG key and fake an encrypted insert key at ~/.freenet.insertKeys.gpg:







# generate some randomness
for i in {1..100}; do find / 2>/dev/null >/dev/null ; done &
# create a gpg key without interaction
cat >foo <<EOF
     %echo Generating a basic OpenPGP key
     Key-Type: RSA
     Key-Length: 4096
     Name-Real: myself
     Name-Comment: someone
     Name-Email: myself@local
     Expire-Date: 0
     Passphrase: user
     # Do a commit here, so that we can later print "done" :-)
     %commit
     %echo done
EOF
gpg --batch --gen-key foo
# configure gpg to use long form keys, because the short format is susceptible to fake key attacks
echo >> ~/.gnupg/gpg.conf
echo keyid-format long >> ~/.gnupg/gpg.conf
# create a new insert key and never store it in unencrypted form
sudo swapoff -a # disable all swap
# cut removes the trailing /
echo NEWKEY=$(fcpgenkey | tail -n 1) | cut -d / -f 1 > /run/user/1000/.freenet.insertKeys
# encrypt the key
gpg -er myself@local /run/user/1000/.freenet.insertKeys
cp /run/user/1000/.freenet.insertKeys.gpg ~/.freenet.insertKeys.gpg
# remove the unencrypted version
rm /run/user/1000/.freenet.insertKeys
sudo swapon -a # re-enable swap







Create a (fake) jarsigner certificate:







keytool -genkeypair -keyalg RSA -sigalg SHA256withRSA -keysize 4096 -dname "cn=Arne Babenhauserheide, o=The Freenet Project Inc, c=US" -alias "freenet" -storepass "SomePassphrase" -validity 365 -keystore ~/.keystore
sed -i 's/jarsignerAlias="alias"/jarsignerAlias="freenet"/' ~/.freenetrc
sed -i 's/jarsignerStorePassword="password"/jarsignerStorePassword="SomePassphrase"/' ~/.freenetrc
sed -i 's/jarsignerCodeSigningKeyPassword="password"/jarsignerCodeSigningKeyPassword="SomePassphrase"/' ~/.freenetrc
mkdir -p ~/.gnupg/freenet/code-signing
keytool -importkeystore -srckeystore ~/.keystore -srcalias "freenet" -destkeystore ~/.gnupg/freenet/code-signing/code-signing.jks -deststoretype jks -destalias freenet -destkeypass "SomePassphrase"







Release to a FAKE server and FAKE origin repository:







sed -i 's,targetHost="osprey.vm.bytemark.co.uk",targetHost="localhost",' ~/.freenetrc

# as root:
for i in /var/www /var/www/emu /var/www/emu/l10n /var/www/freenet-website-testing-not-subbed/money-not-subbed-in /var/www/freenet-website /var/www/freenet-website-testing /var/www/downloads/alpha /var/www/downloads/alpha/plugins; do 
    mkdir -p "${i}"
    chown -R user "${i}"
done

# as user:
# passwordless ssh access to localhost
ssh-keygen
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

# fake upstream with fake origin
cd ~/
git clone --bare ~/fred-upstream ~/fred-origin
cd ~/fred-upstream
git push --all ~/fred-origin # pull from origin does not work
cd ~/fred
git remote set-url origin ../fred-origin
git fetch --all
git checkout -b master origin/master
cd ~/







Set your git username and email. MUST use the same email as the gpg key!







git config --global user.name "myself"
git config --global user.email "myself@local"
# also setup the gpg key to avoid getting cryptic error messages
git config --global user.signingkey $(gpg --list-secret-keys --with-colons | head -n 1 | cut -d : -f 5)












Do a release







cd ~/scripts
./setup-release-environment
cd -







Now see scripts/release-build.





This is an example run to release 1476 to the testing update key I setup earlier:







# run with ./release-build 1476
cd ~/scripts
./update-bookmarks
./update-version 1476
# tag-build merges into next. I cannot do that, yet.
./tag-build 1476 # remembers the info: you can repeat this. Beware that this will upload to github!
# get back to master
cd ~/fred
git checkout -b master origin/master
git checkout .
cd ~/scripts
./release-fred --dry-run build # remove dry run to actually run this
cd













Requirements which need to be kept private






FPI code signing certificate







keytool -importkeystore -srckeystore code-signing.p12  -srcstoretype pkcs12 -srcalias "freenet project inc's comodo ca limited id" -destkeystore ~/.gnupg/freenet/code-signing/code-signing.jks -deststoretype jks -destalias freenet -destkeypass "password"












GitHub OAuth token with `publicrepo` access set in `~/.freenetrc` under `gitHubOAuthToken`







Seednodes





Put the seednodes directory onto the virtual machine: Boot with SSH forwarding and scp it there.







# start the virtual machine with ssh forwarding
qemu-system-x86_64 -smp 2 -m 2048 -enable-kvm -hda debian_wheezy_amd64_standard.qcow2 -nographic -net nic -net user,hostfwd=tcp::10022-:22 
# switch to another terminal and copy the seedrefs
scp -P 10022  path/to/seedrefs.tar.xz.bin user@localhost:

















Preparing the qemu image for upload into Freenet







# create a sparse tarball
tar cJSvf freenet-release-image.tar.xz debian_wheezy_amd64_standard.qcow2












Testing a test release






Go to the node.updater config in advanced mode:





http://127.0.0.1:8888/config/node.updater?fproxyAdvancedMode=2





Enter the inverted key:







fcpinvertkey $(gpg -d ~/.freenet.insertKeys.gpg | sed s/^NEWKEY=//) | sed s,^S,U, | sed s,$,/jar/1475, ; echo












Adapting this image for real Freenet releases






Alternate section title: How to undo the faking.





You need:




    

  • Upload (SSH) access to osprey

    • 

  • Push access to the fred repo on github: https://github.com/freenet/fred

    • 

  • The real insert key for freenet updates

    • 

  • The real jarsigner FPI code signing certificate

    • 

  • Your own GnuPG key, stored only on the virtual machine on an airgapped computer and signed by other freenet devs (you can use that to receive the update key from the previous maintainer).

    • 





Then adjust ~/freenetrc:







# set the real values
PATHTO_REALINSERTKEY="path/to/realinsertkey.gpg"
PATHTO_REALJARSIGNERCERT="~/realjarsignercert"
REALJARSIGNERPASSWORD="realjarsignerpassword"
GPGLONGID="mygpgid"
GPGEMAIL="mygpgemail@local"
GPGUSERNAME="mygpg name"
GITHUBOAUTHTOKEN="sometoken"

# undo the faking
cp "${PATHTO_REALINSERTKEY}" ~/.freenet.insertKeys.gpg
sed -i 's/jarsignerStorePassword="SomePassphrase"/jarsignerStorePassword="'"${REALJARSIGNERPASSWORD}"'"/' ~/.freenetrc
sed -i 's,targetHost="localhost",targetHost="osprey.vm.bytemark.co.uk",' ~/.freenetrc
cd ~/fred
git remote set-url origin ssh://git@github.com/freenet/fred
cd ~/
git config --global user.name "${GPGUSERNAME}"
git config --global user.email "${GPGEMAIL}"
# also setup the gpg key to avoid getting cryptic error messages
git config --global user.signingkey "${GPGLONGID}"







If you want to keep the release machine truly airgapped, you might want to avoid pushing directly do github and instead push to a local bare git repo and then synchronize over an SD card (do not use a USB stick: they are attack vectors). The same goes for rsyncing to osprey directly: You might want to largely keep the fake setup and only change the upload key, but retrieving the binary blob from the local Freenet and transferring that to an online system via an SD card.











Autor: Arne Babenhauserheide


Created: 2017-11-03 Fr 11:19


Emacs 25.2.1 (Org mode 8.2.10)


Validate