Gen btf.v7.draft #10518
Draft
Gen btf.v7.draft #10518
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Rename the member of `struct object` holding the path to BTF data if provided via --btf arg. `btf_path` is less ambiguous. Acked-by: Eduard Zingerman <eddyz87@gmail.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Increase the lifetime of parsed BTF in resolve_btfids by factoring load_btf() routine out of symbols_resolve() and storing the base_btf and btf pointers in the struct object. Acked-by: Eduard Zingerman <eddyz87@gmail.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Instead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids. Acked-by: Eduard Zingerman <eddyz87@gmail.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
resolve_btfids builds without compiler warnings currently, so let's enforce this for future changes with '-Wall -Werror' flags [1]. [1] https://lore.kernel.org/bpf/1957a60b-6c45-42a7-b525-a6e335a735ff@linux.dev/ Tested-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
This patch implements kconfig re-sync when the pahole version changes between builds, similar to how it happens for compiler version change via CC_VERSION_TEXT. Define PAHOLE_VERSION in the top-level Makefile and export it for config builds. Set CONFIG_PAHOLE_VERSION default to the exported variable. Kconfig records the PAHOLE_VERSION value in include/config/auto.conf.cmd [1]. The Makefile includes auto.conf.cmd, so if PAHOLE_VERSION changes between builds, make detects a dependency change and triggers syncconfig to update the kconfig [2]. For external module builds, add a warning message in the prepare target, similar to the existing compiler version mismatch warning. Note that if pahole is not installed or available, PAHOLE_VERSION is set to 0 by pahole-version.sh, so the (un)installation of pahole is treated as a version change. See previous discussions for context [3]. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/kconfig/preprocess.c?h=v6.18#n91 [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.18#n815 [3] https://lore.kernel.org/bpf/8f946abf-dd88-4fac-8bb4-84fcd8d81cf0@oracle.com/ Reviewed-by: Nicolas Schier <nsc@kernel.org> Tested-by: Nicolas Schier <nsc@kernel.org> Tested-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Subsequent patches in the series change vmlinux linking scripts to unconditionally pass --btf_encode_detached to pahole, which was introduced in v1.22 [1][2]. This change allows to remove PAHOLE_HAS_SPLIT_BTF Kconfig option and other checks of older pahole versions. [1] https://github.com/acmel/dwarves/releases/tag/v1.22 [2] https://lore.kernel.org/bpf/cbafbf4e-9073-4383-8ee6-1353f9e5869c@oracle.com/ Acked-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Nicolas Schier <nsc@kernel.org> Tested-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
A selftest targeting resolve_btfids functionality relies on a resolved .BTF_ids section to be available in the TRUNNER_BINARY. The underlying BTF data is taken from a special BPF program (btf_data.c), and so resolve_btfids is executed as a part of a TRUNNER_BINARY build recipe on the final binary. Subsequent patches in this series allow resolve_btfids to modify BTF before resolving the symbols, which means that the test needs access to that modified BTF [1]. Currently the test simply reads in btf_data.bpf.o on the assumption that BTF hasn't changed. Implement resolve_btfids call only for particular test objects (just resolve_btfids.test.o for now). The test objects are linked into the TRUNNER_BINARY, and so .BTF_ids section will be available there. This will make it trivial for the resolve_btfids test to access BTF modified by resolve_btfids. [1] https://lore.kernel.org/bpf/CAErzpmvsgSDe-QcWH8SFFErL6y3p3zrqNri5-UHJ9iK2ChyiBw@mail.gmail.com/ Acked-by: Eduard Zingerman <eddyz87@gmail.com> Tested-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Currently resolve_btfids updates .BTF_ids section of an ELF file
in-place, based on the contents of provided BTF, usually within the
same input file, and optionally a BTF base.
Change resolve_btfids behavior to enable BTF transformations as part
of its main operation. To achieve this, in-place ELF write in
resolve_btfids is replaced with generation of the following binaries:
* ${1}.BTF with .BTF section data
* ${1}.BTF_ids with .BTF_ids section data if it existed in ${1}
* ${1}.BTF.base with .BTF.base section data for out-of-tree modules
The execution of resolve_btfids and consumption of its output is
orchestrated by scripts/gen-btf.sh introduced in this patch.
The motivation for emitting binary data is that it allows simplifying
resolve_btfids implementation by delegating ELF update to the $OBJCOPY
tool [1], which is already widely used across the codebase.
There are two distinct paths for BTF generation and resolve_btfids
application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to
ensure correct linking. The patch doesn't change this approach, only
the implementation is a little different.
Before this patch it worked as follows:
* pahole consumed .tmp_vmlinux1 [2] and added .BTF section with
llvm-objcopy [3] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1
into a .tmp_vmlinux1.bpf.o object [2], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [4],
updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a *detached* file with
raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
(potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For kernel modules, creating a special .bpf.o file is not necessary,
and so embedding of sections data produced by resolve_btfids is
straightforward with objcopy.
With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows deleting elf_update() call and satellite
code (like compressed_section_fix [5]).
Endianness handling of .BTF_ids data is also changed. Previously the
"flags" part of the section was bswapped in sets_patch() [6], and then
Elf_Type was modified before elf_update() to signal to libelf that
bswap may be necessary. With this patch we explicitly bswap entire
data buffer on load and on dump.
[1] https://lore.kernel.org/bpf/131b4190-9c49-4f79-a99d-c00fac97fa44@linux.dev/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n110
[3] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c?h=v1.31#n1803
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n284
[5] https://lore.kernel.org/bpf/20200819092342.259004-1-jolsa@kernel.org/
[6] https://lore.kernel.org/bpf/cover.1707223196.git.vmalik@redhat.com/
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Tested-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
kernel-patches-daemon-bpf
bot
force-pushed
the
bpf-next_base
branch
3 times, most recently
from
5007636 to
0ac68c9
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.