UP | HOME

Using GNU Guix for software development

When I started working as a software developer at Disy, I also switched my main distribution at home to GNU Guix.

Guix enables me to run updates without fearing that losing power during the update could kill my system (I had that happen with Gentoo), it provides a stable base-system with per-user installed packages as well as environments that can install packages into the local shell much like language-specific package managers like npm. Also it aligns well with my ideals of using purely free software.

But I mostly do proprietary software development at work, and advanced features of Guix make some of that problematic. This page lists the workarounds I’m using.


PDF (drucken)

IntelliJ (Ultimate)

We’re using IntelliJ for Java development, because it is the best tool out there for a 1.5 Million lines Java codebase. But it requires running software which is not installed via the package manager.

I run it via the file ~/.local/bin/idea.sh which sets a custom LD_LIBRARY_PATH and the system-JDK, and then runs the idea.sh script which comes with IntelliJ.

Part of the library path is required for our starters, because of native code that has to find the libraries and libstdc++.

I find the right libstdc++ via ls /gnu/store/*gcc*/lib/libstdc++*.

#!/bin/bash

# show commands
# set -x

# # add cache-sync trap to ensure the caches are synchronized however this script ends. The script must also be at the end of the script.
function sync-caches () {
#     sudo umount /home/arne/.IntelliJIdea2019.3/system/caches
#     rsync -rauv /run/user/1000/intellij/caches/ /home/arne/.IntelliJIdea2019.3/system/caches
#     # sudo umount /home/arne/.IntelliJIdea2019.3/system/index
#     # rsync -rauv /run/user/1000/intellij/index/ /home/arne/.IntelliJIdea2019.3/system/index
      return
}
# trap sync-caches EXIT QUIT ABRT TERM KILL
# Hack: trap cannot work after exec, so we’re forking a second process which syncs when the current process ends.
{ while kill -0 $$; do sleep 5; done; sync-caches; } &

cd ~/Disy/
# ld library path:
# /gnu/store/i6l1579g80387rda658jy9cfqq82643d-sqlite-3.28.0/lib/ for libsqlite
# /gnu/store/gm253g15q0wp85pwdj34jcbdsh1yxmmn-gcc-10.1.0-lib/lib/ for libstdc++
# /gnu/store/rkq6ipys8hf5hw66jkzzw4nfr6ncq96a-fontconfig-2.13.1/lib/ for fontconfig
# /gnu/store/rykm237xkmq7rl1p0nwass01p090p88x-zlib-1.2.11/lib/ for libz

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/home/arne/.guix-profile/lib:/gnu/store/i6l1579g80387rda658jy9cfqq82643d-sqlite-3.28.0/lib/:/gnu/store/gm253g15q0wp85pwdj34jcbdsh1yxmmn-gcc-10.1.0-lib/lib/:. IDEA_JDK="$(dirname $(dirname $(realpath $(which java))))" exec -a "$0" bash -x idea-IU-193.6911.18/bin/idea.sh
# LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/gnu/store/7pb3iqfxq2rvyr7ym6j892s90sg7w79a-openjdk-14.0-jdk/lib/:/gnu/store/rykm237xkmq7rl1p0nwass01p090p88x-zlib-1.2.11/lib/:/gnu/store/i6l1579g80387rda658jy9cfqq82643d-sqlite-3.28.0/lib/:/gnu/store/xqjpihbfdc62d2q0sn8i0y0g1xpzbr1s-gcc-9.3.0-lib/lib/:/gnu/store/rkq6ipys8hf5hw66jkzzw4nfr6ncq96a-fontconfig-2.13.1/lib/:. exec -a "$0" bash -x idea-IU-193.6911.18/bin/idea.sh

This is paired with a sudoers entry to avoid the need for a password on sync in /etc/config.scm:

(define %sudoers-specification
  (plain-file "sudoers" "\
root ALL=(ALL) ALL
%wheel ALL=(ALL) ALL
%wheel ALL=(ALL) NOPASSWD: /run/current-system/profile/bin/mount --bind /run/user/1000/intellij/caches /home/arne/.IntelliJIdea2019.3/system/caches
%wheel ALL=(ALL) NOPASSWD: /run/current-system/profile/bin/umount /home/arne/.IntelliJIdea2019.3/system/caches
%wheel ALL=(ALL) NOPASSWD: /home/arne/.guix-profile/bin/mount --bind /run/user/1000/intellij/caches /home/arne/.IntelliJIdea2019.3/system/caches
%wheel ALL=(ALL) NOPASSWD: /home/arne/.guix-profile/bin/umount /home/arne/.IntelliJIdea2019.3/system/caches
"))

For info on the trap, see http://redsymbol.net/articles/bash-exit-traps/

The fork-hack to run code after exec is from that other guy on stackoverflow.

Maven

I run a custom version of Maven via a simple script:

#!/bin/sh

NOTE='\033[1;33m'
NONE='\033[0m'

echo "[${NOTE}NOTE${NONE}] Running Maven with:"
java -version

~/Disy/opt/apache-maven-3.6.1/bin/mvn "$@"

npm

npm is a bit of a beast: If I use what we create via maven, it dies because it does not find its libraries. Therefore I install it manually with the system-installed npm and then run npm-cli.js directly:

npm install npm@6.13.4 ; node_modules/npm/bin/npm-cli.js install; node_modules/npm/bin/npm-cli.js start -- --locale=de

To simplify this, I moved it into a shell-script at ~/.local/bin/npm:

#!/usr/bin/env bash
# if ! echo $PWD | grep -q Disy; then
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/home/arne/.guix-profile/lib:/gnu/store/gm253g15q0wp85pwdj34jcbdsh1yxmmn-gcc-10.1.0-lib/lib/:.
exec -a "$0" guix environment --ad-hoc gcc-toolchain@10.2 -- /home/arne/.guix-profile/bin/npm "$@"

Firefox

Update 2020-10: There are currently no updates to the flatpak package. I get the warning “Info: org.gnome.Platform is end-of-life, with reason: The GNOME 3.34 runtime is no longer supported as of 14th August 2020. Please ask your application developer to migrate to a supported platform.”

I actually tried to package Firefox for Guix, but packaging a few hundred Rust packages is quite some task. I stopped at more than 200 packaged ones, because the version numbers retrieved from guix import crate NAME did not match the version numbers needed but rather always retrieved the most recent version.

Though it ultimately failed, this is the script I used to automatically import many Rust packages:

for i in $(for i in $(grep -o ',[^ ()][^ ()]*' more/packages/gnuzilla.scm | sed s.,.. | xargs); do
               grep -q "name \"$i\"" more/packages/gnuzilla.scm || echo $i;
           done | sort -u | grep rust | sed s/^rust-//  | xargs); do
    guix import crate $i >> more/packages/gnuzilla.scm || \
        (sleep 30 && guix import crate $i >> more/packages/gnuzilla.scm);
done

The actual solution to get Firefox was to install flatpak and get Firefox from there:

guix install flatpak
flatpak remote-add --user --from org.mozilla.FirefoxRepo https://firefox-flatpak.mojefedora.cz/org.mozilla.FirefoxRepo.flatpakrepo
flatpak install --user org.mozilla.FirefoxRepo org.mozilla.FirefoxDevEdition

Now I run Firefox via a small script at ~/.local/bin/firefox:

#!/run/current-system/profile/bin/bash
flatpak run org.mozilla.FirefoxDevEdition "$@"

Zoom

For Zoom I use the same method as for Firefox:

flatpak install flathub us.zoom.Zoom
#!/run/current-system/profile/bin/bash
flatpak run us.zoom.Zoom

Custom Emacs profile

All my planning files and my setup to develop Javascript are in a specialized Emacs setup that I run with disymacs:

#!/usr/bin/env bash
HOME="${HOME}/Disy" emacs "$@"

There is a full custom setup in ~/Disy/.emacs.d so I can more easily synchronize the config between office and home office.

Connect to VPN

I use openconnect and a custom script to connect to our VPN.

guix install openconnect
#!/run/current-system/profile/bin/bash
if [ "$EUID" -ne 0 ]; then 
  echo This script needs root priviledges. 1>&2
  echo Executing sudo --login $(realpath "$0") in 3 seconds 1>&2
  for i in {1..3}; do
      echo -n .
      sleep 1
  done
  echo " " now executing sudo --login $(realpath "$0")
  exec sudo --login $(realpath "$0")
fi
# Connect via VPN to disy and ensure that the nameservers are correct
function resolv-disy-undo {
    echo "undoing VPN connection"
    cp /etc/resolv.conf.head-default /etc/resolv.conf.head
    # ensure that the manual DNS servers are available
    # let dhcp do the rest # TODO: this only worked in Gentoo. Find out how it works in Guix
    cp /etc/resolv.conf /tmp/resolv.conf && cat /etc/resolv.conf.head /tmp/resolv.conf > /etc/resolv.conf
    # /lib/dhcpcd/dhcpcd-run-hooks
}

# ensure that the change is undone when this script ends, see http://redsymbol.net/articles/bash-exit-traps/
trap resolv-disy-undo EXIT HUP INT QUIT ABRT TERM KILL

if test ! -e /etc/resolv.conf.head-default; then
    cp /etc/resolv.conf.head /etc/resolv.conf.head-default || exit 1
fi

# /etc/resolv.conf.head-disy contains our internal name servers, 
# one of them duplicated, because dhcpcd only uses the first three nameservers.

cp /etc/resolv.conf.head-disy /etc/resolv.conf.head
# /lib/dhcpcd/dhcpcd-run-hooks # update /etc/resolv.conf

echo
echo When requested, type your PASSWORD directly followed by the AUTHENTICATOR-TOKEN 1>&2
echo

(sleep 30 && sshfs -o allow_other,defer_permissions USER@NODE:/path/to/disk /path/to/disk) &

openconnect --protocol=gp gp.disy.net --csd-wrapper="$HOME/.guix-profile/libexec/openconnect/hipreport.sh" -u USER

resolv-disy-undo
reset

Run tomcat with Java 8

I run my system with OpenJDK 12, and we develop with Java 11, but our secondary Tomcat setup needs Java 8. So I create a local environment with Java 8 (via icedtea 3.7).

# download and unpack tomcat and enter the folder, then call
guix environment --ad-hoc icedtea@3.7.0
bin/startup.sh

Versiontracking via magit and monky

To make git usable I use magit. It gives me a good balance of efficiency and control.

For Mercurial, my preferred version tracking tool, I now use monky to have similar integration into Emacs as with magit.

Full update of installed packages

guix pull && guix update -k can be slow if substitutes cannot be found, so everything has to be built locally. I still don’t know why that happens, but I now have a way to avoid it: I just re-install all installed packages.

#!/usr/bin/env bash
# re-install all guix packages
guix pull && guix package -I | cut -f 1,3 | sed 's/\t/:/' | xargs guix install

I saved this as ~/.local/bin/update-guix-full.

Alfaview

For lectures with so many people that jitsi and nextcloud meet their limits. European alternative to Zoom.

First get the debian image and unpack it. Then unpack the data tarball.

#!/usr/bin/env bash
cd ~/Downloads/alfaview/data/opt/alfaview || exit 1
exec -a "$0" guix environment --ad-hoc gcc-toolchain@10.2 -- bash -c "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/home/arne/.guix-profile/lib:/gnu/store/gm253g15q0wp85pwdj34jcbdsh1yxmmn-gcc-10.1.0-lib/lib/:."' exec -a "$0" ./alfaview'

Adapt the paths, then createthis file as ~/.local/bin/alfaview and make it executable with chmod +x ~/.local/bin/alfaview.

Unlock cores

In my Guix config I throttle my cores to reduce power consumption. But for work I often need full performance in short bursts — for example when IntelliJ re-indexes sources or when I rebuild after changes. For this, I have a small core-unlocker script saved at ~/.local/bin/boost-cpu:

#!/usr/bin/env bash
# boost CPU for MIN minutes
if [[ x"$1" == x"" ]]; then
    echo "Error: missing argument."
    echo "usage: $0 <minutes>"
    exit 1
fi
if ! test $1 -eq $1 2>/dev/null; then
    echo "Error: argument '$1' is not a number."
    echo "usage: $0 <minutes>"
    exit 1
fi
if [[ x"$1" == x"--help" ]]; then
    echo "$0 <MIN>: boost CPU speed for MIN minutes."
    exit 0
fi

SEC="$((($1 * 60)))"

for i in $(seq 1 $SEC); do sudo cpupower frequency-set -u 6000000; sleep 1; done

Date: 2020-02-19 Mi 00:00

Author: ArneBab

Validate