Update struct btf_header to add a new "kind_layout" section containing a description of how to parse the BTF kinds known about at BTF encoding time. This provides the opportunity for tools that might not know all of these kinds - as is the case when older tools run on more newly-generated BTF - to still parse the BTF provided, even if it cannot all be used. Also add support to bpftool to dump metadata about BTF; its size, header information and kind layout section. The ideas here were discussed at [1], with further discussion at [2]. Patches for pahole will enable the kind layout addition during BTF generation are at [3], but even absent these the addition of the kind_layout feature in the final patch in this series should not break anything since such unknown features are simply ignored during pahole BTF generation. Changes since v4: [4]: - removed CRC generation since it is not needed to handle modules built at different time than kernel; distilled base BTF supports this now - fixed up bpftool display of empty kind names, comment/documentation indentation (Quentin, patches 8, 9) Changes since v3 [5]: - fixed mismerge issues with kbuild changes for BTF generation (patches 9, 14) - fixed a few small issues in libbpf with kind layout representation (patches 2, 4) Changes since v2 [6]: - drop "optional" kind flag (Andrii, patch 1) - allocate "struct btf_header" for struct btf to ensure we can always access new fields (Andrii, patch 2) - use an internal BTF kind array in btf.c to simplify kind encoding (Andrii, patch 2) - drop use of kind layout information for in-kernel parsing, since the kernel needs to be strict in what it accepts (Andrii, patch 6) - added CRC verification for BTF objects and for matching with base object (Alexei, patches 7,8) - fixed bpftool json output (Quentin, patch 10) - added standalone module BTF support, tests (patches 13-17) Changes since RFC - Terminology change from meta -> kind_layout (Alexei and Andrii) - Simplify representation, removing meta header and just having kind layout section (Alexei) - Fixed bpftool to have JSON support, support prefix match, documented changes (Quentin) - Separated metadata opts into add_kind_layout and add_crc - Added additional positive/negative tests to cover basic unknown kind, one with an info_sz object following it and one with N elem_sz elements following it. - Updated pahole-flags to use help output rather than version to see if features are present [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@xxxxxxxxxxxxxx/ [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@xxxxxxxxxx/ [3] https://lore.kernel.org/dwarves/20250528095349.788793-1-alan.maguire@xxxxxxxxxx/ [4] https://lore.kernel.org/bpf/20231112124834.388735-1-alan.maguire@xxxxxxxxxx/ [5] https://lore.kernel.org/bpf/20231110110304.63910-1-alan.maguire@xxxxxxxxxx/ [6] https://lore.kernel.org/bpf/20230616171728.530116-1-alan.maguire@xxxxxxxxxx/ Alan Maguire (9): btf: add kind layout encoding to UAPI libbpf: Support kind layout section handling in BTF libbpf: use kind layout to compute an unknown kind size libbpf: Add kind layout encoding support libbpf: BTF validation can use kind layout for unknown kinds btf: support kernel parsing of BTF with kind layout selftests/bpf: test kind encoding/decoding bpftool: add BTF dump "format meta" to dump header/metadata kbuild, bpf: Specify "kind_layout" optional feature include/uapi/linux/btf.h | 11 + kernel/bpf/btf.c | 96 ++++-- scripts/Makefile.btf | 2 +- tools/bpf/bpftool/bash-completion/bpftool | 2 +- tools/bpf/bpftool/btf.c | 90 ++++- tools/include/uapi/linux/btf.h | 11 + tools/lib/bpf/btf.c | 316 ++++++++++++++---- tools/lib/bpf/btf.h | 20 ++ tools/lib/bpf/libbpf.map | 1 + .../selftests/bpf/prog_tests/btf_kind.c | 176 ++++++++++ 10 files changed, 627 insertions(+), 98 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_kind.c -- 2.39.3
