ppc64 boot wrapper linking fails with LLD 21.1.7

[Gelbpunkt]

Hi there, I'm building Linux 6.17.11 for ppc64 (big-endian) and seeing the following error when the boot wrapper is being linked:

+ powerpc64-unknown-linux-musl-ld -m elf32ppc -T arch/powerpc/boot/zImage.lds -Ttext 0x4000000 -z noexecstack -o arch/powerpc/boot/zImage.pseries -Map wrapper.map arch/powerpc/boot/pseries-head.o arch/powerpc/boot/of.o arch/powerpc/boot/epapr.o ./zImage.18795.o arch/powerpc/boot/wrapper.a
powerpc64-unknown-linux-musl-ld: error: section: .dynamic is not contiguous with other relro sections

the same happens not just for pseries, but also e.g. for pmac:

+ powerpc64-unknown-linux-musl-ld -m elf32ppc -T arch/powerpc/boot/zImage.lds -Ttext 0x400000 -z noexecstack -o arch/powerpc/boot/zImage.pmac -Map wrapper.map arch/powerpc/boot/of.o arch/powerpc/boot/epapr.o ./zImage.18796.o arch/powerpc/boot/wrapper.a
powerpc64-unknown-linux-musl-ld: error: section: .dynamic is not contiguous with other relro sections

Just to clarify: powerpc64-unknown-linux-musl-ld is LLD, my entire toolchain is only LLVM, no GNU binutils or GCC are installed.

I also saw a couple of other errors related to some disallowed relocations against the opal symbol when building with CONFIG_PPC_POWERNV enabled, but disabled it to work around it for now. I can post logs for that as well if it helps.

I was able to build 6.17.8 with LLVM 21.1.5 a few weeks ago just fine, so there's very likely a change in between one of the two that broke.

[Gelbpunkt]

The following diff makes the zImage.pseries link fine, I assume it also applies to the others:

--- a/arch/powerpc/boot/wrapper
+++ b/arch/powerpc/boot/wrapper
@@ -261,9 +261,11 @@ pseries)
     link_address='0x4000000'
     if [ "$format" != "elf32ppc" ]; then
        link_address=
-       pie=-pie
     fi
     make_space=n
+    pie=-pie
+    notext='-z notext'
+    rodynamic=$(if ${CROSS}ld -V 2>&1 | grep -q LLD ; then echo "-z rodynamic"; fi)
     ;;
 pmac|chrp)
     platformo="$object/of.o $object/epapr.o"

but I have no idea whether this is correct :)

[nathanchance]

Thanks for the report! I am sure I came across this when trying to fix #1601 but just never filed a separate issue for it.

I am surprised this was not seen with the earlier combination since I am pretty sure this would have been an issue earlier. Is there an in-tree target that reproduces this or do you have a .config that does? I can try to reproduce this locally then bisect based on that testing.

This may also be resolved with linker script changes but that might be harder to verify.

[Gelbpunkt]

Thanks for the report! I am sure I came across this when trying to fix #1601 but just never filed a separate issue for it.

Looking at that issue, it may be hard to reproduce this in a setup that isn't exactly mine, since ld is ld.lld, etc. I've tried to reproduce this by copying the configuration from the system that hits this issue onto my Fedora laptop and building it with make ARCH=powerpc LLVM=1, but no luck, I cannot reproduce it.

...unless I apply the following patch to force it to use LLVM binutils and LLD:

git diff

diff --git a/arch/powerpc/boot/wrapper b/arch/powerpc/boot/wrapper
index 1efd1206fcab..a4906e0c263a 100755
--- a/arch/powerpc/boot/wrapper
+++ b/arch/powerpc/boot/wrapper
@@ -51,9 +51,6 @@ format=
 notext=
 rodynamic=

-# cross-compilation prefix
-CROSS=
-
 # mkimage wrapper script
 MKIMAGE=$srctree/scripts/mkuboot.sh

@@ -128,7 +125,6 @@ while [ "$#" -gt 0 ]; do
     -C)
        shift
        [ "$#" -gt 0 ] || usage
-       CROSS="$1"
        ;;
     -D)
        shift
@@ -193,7 +189,7 @@ if [ -z "$kernel" ]; then
     kernel=vmlinux
 fi

-LC_ALL=C elfformat="`${CROSS}objdump -p "$kernel" | grep 'file format' | awk '{print $4}'`"
+LC_ALL=C elfformat="`llvm-objdump -p "$kernel" | grep 'file format' | awk '{print $4}'`"
 case "$elfformat" in
     elf64-powerpcle)   format=elf64lppc        ;;
     elf64-powerpc)     format=elf32ppc ;;
@@ -221,7 +217,7 @@ ld_version()

 ld_is_lld()
 {
-       ${CROSS}ld -V 2>&1 | grep -q LLD
+       ld.lld -V 2>&1 | grep -q LLD
 }

 # Do not include PT_INTERP segment when linking pie. Non-pie linking
@@ -230,7 +226,7 @@ nodl="--no-dynamic-linker"

 # suppress some warnings in recent ld versions
 nowarn="-z noexecstack"
-if "${CROSS}ld" -v --no-warn-rwx-segments >/dev/null 2>&1; then
+if "ld.lld" -v --no-warn-rwx-segments >/dev/null 2>&1; then
        nowarn="$nowarn --no-warn-rwx-segments"
 fi

@@ -369,7 +365,7 @@ epapr)
     link_address='0x20000000'
     pie=-pie
     notext='-z notext'
-    rodynamic=$(if ${CROSS}ld -V 2>&1 | grep -q LLD ; then echo "-z rodynamic"; fi)
+    rodynamic=$(if ld.lld -V 2>&1 | grep -q LLD ; then echo "-z rodynamic"; fi)
     ;;
 mvme5100)
     platformo="$object/fixed-head.o $object/mvme5100.o"
@@ -385,7 +381,7 @@ esac
 vmz="$tmpdir/`basename \"$kernel\"`.$ext"

 # Calculate the vmlinux.strip size
-${CROSS}objcopy $objflags "$kernel" "$vmz.$$"
+llvm-objcopy $objflags "$kernel" "$vmz.$$"
 strip_size=$(${CONFIG_SHELL} "${srctree}/scripts/file-size.sh" "$vmz.$$")

 if [ -z "$cacheit" -o ! -f "$vmz$compression" -o "$vmz$compression" -ot "$kernel" ]; then
@@ -438,14 +434,14 @@ fi

 # Extract kernel version information, some platforms want to include
 # it in the image header
-version=`${CROSS}strings "$kernel" | grep '^Linux version [-0-9.]' | \
+version=`llvm-strings "$kernel" | grep '^Linux version [-0-9.]' | \
     head -n1 | cut -d' ' -f3`
 if [ -n "$version" ]; then
     uboot_version="-n Linux-$version"
 fi

 # physical offset of kernel image
-membase=`${CROSS}objdump -p "$kernel" | grep -m 1 LOAD | awk '{print $7}'`
+membase=`llvm-objdump -p "$kernel" | grep -m 1 LOAD | awk '{print $7}'`

 case "$platform" in
 uboot)
@@ -460,7 +456,7 @@ uboot)
 esac

 addsec() {
-    ${CROSS}objcopy $4 $1 \
+    llvm-objcopy $4 $1 \
        --add-section=$3="$2" \
        --set-section-flags=$3=contents,alloc,load,readonly,data
 }
@@ -490,18 +486,18 @@ if [ "$platform" != "miboot" ]; then
         text_start="-Ttext $link_address"
     fi
 #link everything
-    ${CROSS}ld -m $format -T $lds $text_start $pie $nodl $nowarn $rodynamic $notext -o "$ofile" $map \
+    ld.lld -m $format -T $lds $text_start $pie $nodl $nowarn $rodynamic $notext -o "$ofile" $map \
        $platformo $tmp $object/wrapper.a
     rm $tmp
 fi

 # Some platforms need the zImage's entry point and base address
-base=0x`${CROSS}nm "$ofile" | grep ' _start$' | cut -d' ' -f1`
-entry=`${CROSS}objdump -f "$ofile" | grep '^start address ' | cut -d' ' -f3`
+base=0x`llvm-nm "$ofile" | grep ' _start$' | cut -d' ' -f1`
+entry=`llvm-objdump -f "$ofile" | grep '^start address ' | cut -d' ' -f3`

 if [ -n "$binary" ]; then
     mv "$ofile" "$ofile".elf
-    ${CROSS}objcopy -O binary "$ofile".elf "$ofile"
+    llvm-objcopy -O binary "$ofile".elf "$ofile"
 fi

 # post-processing needed for some platforms
@@ -510,7 +506,7 @@ pseries|chrp)
     $objbin/addnote "$ofile"
     ;;
 coff)
-    ${CROSS}objcopy -O aixcoff-rs6000 --set-start "$entry" "$ofile"
+    llvm-objcopy -O aixcoff-rs6000 --set-start "$entry" "$ofile"
     $objbin/hack-coff "$ofile"
     ;;
 cuboot*)
@@ -538,18 +534,18 @@ ps3)
     # copied to offset 0x100.  At runtime the bootwrapper program copies
     # the data at __system_reset_kernel back to addr 0x100.

-    system_reset_overlay=0x`${CROSS}nm "$ofile" \
+    system_reset_overlay=0x`llvm-nm "$ofile" \
         | grep ' __system_reset_overlay$'       \
         | cut -d' ' -f1`
     system_reset_overlay=`printf "%d" $system_reset_overlay`
-    system_reset_kernel=0x`${CROSS}nm "$ofile" \
+    system_reset_kernel=0x`llvm-nm "$ofile" \
         | grep ' __system_reset_kernel$'       \
         | cut -d' ' -f1`
     system_reset_kernel=`printf "%d" $system_reset_kernel`
     overlay_dest="256"
     overlay_size="512"

-    ${CROSS}objcopy -O binary "$ofile" "$ofile.bin"
+    llvm-objcopy -O binary "$ofile" "$ofile.bin"

     run_cmd dd if="$ofile.bin" of="$ofile.bin" conv=notrunc   \
         skip=$overlay_dest seek=$system_reset_kernel          \
@@ -567,7 +563,7 @@ ps3)
     # reached, then enter the system reset vector of the partially decompressed
     # image.  No warning is issued.
     rm -f "$odir"/{otheros,otheros-too-big}.bld
-    size=$(${CROSS}nm --no-sort --radix=d "$ofile" | grep -E ' _end$' | cut -d' ' -f1)
+    size=$(llvm-nm --no-sort --radix=d "$ofile" | grep -E ' _end$' | cut -d' ' -f1)
     bld="otheros.bld"
     if [ $size -gt $((0x1000000)) ]; then
         bld="otheros-too-big.bld"

With that applied, I can succesfully reproduce the problem:

  GEN     Makefile
  CALL    ../scripts/checksyscalls.sh
  WRAP    arch/powerpc/boot/zImage.pseries
ld.lld: error: section: .dynamic is not contiguous with other relro sections

I'll attach the defconfig.txt that I used (GitHub requres the txt extension for uploads). I'm able to reproduce on Fedora by running the following after applying the diff above:

# move defconfig.txt to arch/powerpc/configs/saved_defconfig
make ARCH=powerpc LLVM=1 saved_defconfig
make ARCH=powerpc LLVM=1 -j$(nproc)

If you also re-enable CONFIG_PPC_POWERNV, you should be able to reproduce the other error as well:

ld.lld: error: relocation R_PPC_ADDR16_HA cannot be used against symbol 'opal'; recompile with -fPIC
>>> defined in arch/powerpc/boot/wrapper.a(opal.o)
>>> referenced by opal-calls.o:(.text+0x16) in archive arch/powerpc/boot/wrapper.a

ld.lld: error: relocation R_PPC_ADDR16_LO cannot be used against symbol 'opal'; recompile with -fPIC
>>> defined in arch/powerpc/boot/wrapper.a(opal.o)
>>> referenced by opal-calls.o:(.text+0x1a) in archive arch/powerpc/boot/wrapper.a

ld.lld: error: relocation R_PPC_ADDR16_HA cannot be used against local symbol; recompile with -fPIC
>>> defined in arch/powerpc/boot/wrapper.a(opal-calls.o)
>>> referenced by opal-calls.o:(.text+0x3e) in archive arch/powerpc/boot/wrapper.a

ld.lld: error: relocation R_PPC_ADDR16_LO cannot be used against local symbol; recompile with -fPIC
>>> defined in arch/powerpc/boot/wrapper.a(opal-calls.o)
>>> referenced by opal-calls.o:(.text+0x42) in archive arch/powerpc/boot/wrapper.a

ld.lld: error: relocation R_PPC_ADDR16_HA cannot be used against symbol 'opal'; recompile with -fPIC
>>> defined in arch/powerpc/boot/wrapper.a(opal.o)
>>> referenced by opal-calls.o:(.text+0x5a) in archive arch/powerpc/boot/wrapper.a

ld.lld: error: relocation R_PPC_ADDR16_LO cannot be used against symbol 'opal'; recompile with -fPIC
>>> defined in arch/powerpc/boot/wrapper.a(opal.o)
>>> referenced by opal-calls.o:(.text+0x5e) in archive arch/powerpc/boot/wrapper.a