Migrating to Multiarch

DEVS: If you make significant changes (this includes ordering!) to this guide, you MUST (!) perform a clean migration in a chroot/container/VM to make absolutely certain your changes don’t break things. You MUST NOT (!) accept assertations by ANYONE that “it should be right”.

Follow this guide very closely and carefully. If things go wrong, ask for help in #exherbo-multiarch or #exherbo.

  1. sync all repositories and install sys-apps/paludis[=scm][-python]. During migration, [python] will cause the build to fail but can be reenabled after you are finished.

    # cave sync
    * /etc/paludis/options.conf *
    sys-apps/paludis -python
    * /etc/paludis/package_unmask.conf *
    sys-apps/paludis scm

    As cross has now been merged, you will need a small workaround for paludis to build fine for amd64.

    * /etc/paludis/bashrc *

    You should drop that line once paludis is built.

  2. switch to the standard profile

    Switch away from ${location}/profiles/amd64/multilib in arbor.conf. Make sure you’re using a standard profile, e.g. profiles = ${location}/profiles/amd64

  3. create the /usr/host symlink to point to the active arch

    # ln -s x86_64-pc-linux-gnu /usr/host
  4. setup path and flags in /etc/paludis/bashrc and environment variables in /etc/env.d/00basic and /etc/profile to handle the new path configuration

    the following example is for building packages for i686-pc-linux-gnu and x86_64-pc-linux-gnu on an amd64 system.

    * /etc/paludis/bashrc *
    x86_64_pc_linux_gnu_CFLAGS="-pipe -O2 -march=native"
    i686_pc_linux_gnu_CFLAGS="-pipe -O2 -march=native"
    x86_64_pc_linux_gnu_CXXFLAGS="-pipe -O2 -march=native"
    i686_pc_linux_gnu_CXXFLAGS="-pipe -O2 -march=native"
    * /etc/env.d/00basic *
    * /etc/profile *

    Make sure to remove the unprefixed *FLAGS variables from /etc/paludis/bashrc.

    To make those changes take effect you need to regenerate and source the env files:

    # eclectic env update
    # source /etc/profile
  5. configure options to select appropriate targets for paludis in /etc/paludis/options.conf


    I will not be held responsible for your failure to do so. You WILL render the system irrecoverable if you do not select the CHOST in targets suboptions.


    * /etc/paludis/options.conf *
    */* targets: x86_64-pc-linux-gnu i686-pc-linux-gnu
  6. configure split_debug_location, tool_prefix for your installed repository and another installed repository for the cross compilation host

    A paludis environment can at present handle at most one cross compilation host. If you need another you can put it in a separate environment and use cave --environment for it.

    tool_prefix is the prefix for the toolchain components which is being used to generate the binaries for the target platform.

    split_debug_location is the location where the split debug information is stored for the packages that are installed.

    cross_compile_host is the target host on which the binaries will run.

    * /etc/paludis/repositories/installed.conf *
    split_debug_location = /usr/x86_64-pc-linux-gnu/lib/debug
    tool_prefix = x86_64-pc-linux-gnu-
    * /etc/paludis/repositories/i686-pc-linux-gnu.conf *
    format = exndbam
    location = ${root}/var/db/paludis/repositories/cross-installed/i686-pc-linux-gnu
    name = i686-pc-linux-gnu
    split_debug_location = /usr/i686-pc-linux-gnu/lib/debug
    tool_prefix = i686-pc-linux-gnu-
    cross_compile_host = i686-pc-linux-gnu
  7. install sys-devel/legacy-gcc and set it as your gcc provider

    # cave resolve --preserve-world sys-devel/legacy-gcc
    # eclectic gcc set legacy-gcc-4.9
  8. build binutils

    # cave resolve --preserve-world binutils
  9. build gcc[-threads]

    * /etc/paludis/options.conf *
    sys-devel/gcc -threads
    # cave resolve sys-devel/gcc
  10. build linux-headers, glibc

    linux-headers needs to be built for all cross compilation targets.

    # cave resolve --preserve-world sys-kernel/linux-headers
    # cave resolve --preserve-world --make cross-compile sys-kernel/linux-headers

    glibc needs to be built for all cross compilation targets.

    # cave resolve --preserve-world sys-libs/glibc
    # cave resolve --preserve-world --make cross-compile --no-dependencies-from sys-libs/glibc sys-libs/glibc

    Dependencies handling isn’t perfect. glibc has a run-time dependency upon timezone-data, but for cross compilation hosts that can’t be built before gcc. Therefore we have to use --no-dependencies-from during the migration.

  11. build gcc dependencies

    # cave resolve --preserve-world gmp --uninstalls-may-break '*/*'
    # cave resolve --preserve-world mpfr
    # cave resolve --preserve-world mpc
    # cave resolve --preserve-world zlib --no-dependencies-from zlib --uninstalls-may-break '*/*'

    Without --no-dependencies-from cave will complain about lots of unsatisfied multibuild_c:* dependencies which are not relevant for this migration. Similarly, --uninstalls-may-break is needed to let the software depending on zlib be broken temporarily.

  12. build language runtimes (take 1 of 2)

    # cave resolve libatomic libgcc libstdc++ --no-follow-installed-dependencies
    # cave resolve --make cross-compile --no-follow-installed-dependencies libatomic libgcc libstdc++
  13. build gcc[threads]

    Enable reporting the correct threading model from the driver to ensure that the runtimes are built appropriately.

    * /etc/paludis/options.conf *
    sys-devel/gcc threads
    # cave resolve --preserve-world gcc --no-follow-installed-dependencies
  14. select gcc as your gcc provider

    # eclectic gcc set 4.9
  15. build language runtimes (take 2 of 2)

    # cave resolve libatomic libgcc libstdc++ --no-follow-installed-dependencies
    # cave resolve --make cross-compile --no-follow-installed-dependencies libatomic libgcc libstdc++
  16. build sed, grep, ncurses, readline, gawk

    If you’re wondering, yes, gawk is built twice. You’d better follow this or you’ll get screwed

    # cave resolve --preserve-world sys-apps/gawk --no-follow-installed-dependencies
    # cave resolve --preserve-world sys-libs/ncurses --uninstalls-may-break '*/*'
    # cave resolve --preserve-world sys-libs/readline --uninstalls-may-break '*/*'
    # cave resolve --preserve-world sys-apps/gawk --no-follow-installed-dependencies
    # cave resolve --preserve-world sys-apps/sed sys-apps/grep --no-follow-installed-dependencies
  17. build pkg-config, libtool, gtk-doc and eclectic

    # cave resolve --preserve-world dev-util/pkg-config sys-devel/libtool dev-doc/gtk-doc app-admin/eclectic --no-dependencies-from dev-util/pkg-config --no-dependencies-from sys-apps/util-linux --no-dependencies-from dev-libs/libxml2 --no-dependencies-from sys-libs/cracklib

    libtoolize may fail to find /bin/grep if you don’t rebuild libtool. Similar issues can occur with the gtk-doc stuff.

  18. rebuild paludis and its deps

    If you don’t, cave will just abort when resolving any set or throwing any exception.

    Yes, this is a PITA, but we have to make sure all the dependencies are installed in the right order or everything will be broken.

    You might have to build more dependencies before building paludis itself depending on your options.

    # cave resolve --preserve-world sys-apps/file --no-follow-installed-dependencies
    # cave resolve --preserve-world app-arch/xz --uninstalls-may-break '*/*' --no-follow-installed-dependencies
    # cave resolve --preserve-world dev-libs/openssl --uninstalls-may-break '*/*'
    # cave resolve --preserve-world dev-libs/expat --uninstalls-may-break '*/*'
    # cave resolve --preserve-world dev-lang/python:2.7 app-admin/eclectic-python:2 --no-follow-installed-dependencies  --uninstalls-may-break '*/*'
    # cave resolve --preserve-world dev-libs/libxml2 --uninstalls-may-break '*/*' --no-follow-installed-dependencies
    # cave resolve --preserve-world app-arch/libarchive --no-follow-installed-dependencies
    # cave resolve --preserve-world app-arch/bzip2 --uninstalls-may-break '*/*'
    # cave resolve --preserve-world dev-libs/pcre
    # cave resolve --preserve-world sys-apps/paludis --no-follow-installed-dependencies

    Once that done, bash being really dumb can tell you it does’t find /usr/bin/cave anymore. source /etc/profile will solve this issue.

  19. purge sys-devel/legacy-gcc

    # cave uninstall sys-devel/legacy-gcc
  20. Nuke the old locale archive

    # rm /usr/lib/locale/locale-archive
  21. rebuild all installed packages in their installed slots, except for skeleton-filesystem-layout (see dependency loops that need to be solved manually)

    # cave resolve --everything installed-slots --without skeleton-filesystem-layout
  22. check that there are no owned files in the following directories (if those “directories” are actually symlinks, all is well; ignore them):

    # for dir in /lib64 /lib /bin /sbin /usr/lib /usr/lib64 /usr/bin /usr/sbin /usr/libexec /usr/include ; do cave owner ${dir} ; done
  23. remove those directories and create the following symlinks (don’t forget to backup your kernel modules!)

    # rm -rf /{bin,sbin,lib} /usr/{bin,sbin,lib,libexec,include}
    # ln -s usr/host/bin /bin
    # ln -s usr/host/sbin /sbin
    # ln -s usr/host/lib /lib
    # ln -s host/bin /usr/bin
    # ln -s host/sbin /usr/sbin
    # ln -s host/lib /usr/lib
    # ln -s host/libexec /usr/libexec
    # ln -s host/include /usr/include
    # ln -s bin /usr/host/sbin
  24. install skeleton-filesystem-layout

    # cave resolve --preserve-world skeleton-filesystem-layout

Please report bugs encountered, workarounds applied and update this document for them as appropriate.

Required Compatibility Path Hacks

Paludis requires the following symbolic links:

  1. /bin/sh -> /usr/${CHOST}/bin/bash
  2. /usr/bin/env -> /usr/${CHOST}/bin/env

Known Hurdles for Migration

What follows is an accumulated list of known subtle sticky points encountered when migrating a system (please add more to this list as discovered):

  1. dev-lang/perl needs build_options: -recommended_tests on the initial rebuild, after that tests are fine

  2. After rebuilding perl it is necessary to rebuild all dependents to use the updated perl path:

    # cave resolve --preserve-world perl --reinstall-dependents-of perl
  3. Some of the base system libraries and tools have a complicated rebuild path when using systemd. The correct ordering should be (with [] indicating a necessary option setting):

    # cave resolve --preserve-world sys-apps/coreutils
    # cave resolve --preserve-world app-text/xmlto
    # cave resolve --preserve-world sys-apps/dbus[-systemd]
    # cave resolve --preserve-world sys-apps/util-linux[-udev][-systemd]
    # cave resolve --preserve-world dev-libs/glib
    # cave resolve --preserve-world sys-apps/systemd
    # cave resolve --preserve-world sys-apps/dbus[systemd]
    # cave resolve --preserve-world sys-apps/util-linux[udev][systemd]
    # cave resolve --preserve-world sys-fs/e2fsprogs
  4. For gobject-introspection to build and work, since it uses ldd, you will need to add a symlink

    # ln -s lib /usr/x86_64-pc-linux-gnu/lib64
  5. To be able to bootstrap icedtea7 or sbcl, you’ll need to add this symlink

    # ln -s usr/x86_64-pc-linux-gnu/lib /lib64

Current Breakage

Copyright 2013 Saleem Abdulrasool