This patchset implements a new type of map, instruction set, and uses
it to build support for indirect branches in BPF (on x86). (The same
map will be later used to provide support for indirect calls and static
keys.) See [1], [2] for more context.
This patch set is a follow-up on the initial RFC [3], now converted to
normal version to trigger CI. Note that GCC and non-x86 archs are not
supposed to work.
Short table of contents:
* Patches 1-6 implement the new map of type
BPF_MAP_TYPE_INSN_SET and corresponding selftests. This map can
be used to track the "original -> xlated -> jitted mapping" for
a given program. Patches 5,6 add support for "blinded" variant.
* Patches 7,8,9 implement the support for indirect jumps
* Patches 10,11 add support for LLVM-compiled programs containing
indirect jumps.
A special LLVM should be used for that, see [4] for the details and
some related discussions. Due to this fact, selftests for indirect
jumps which directly use `goto *rX` are commented out (such that
CI can run).
There is a list of TBDs (mostly, more selftests + some limitations
like maximal map size), however, all the selftests which compile
to contain an indirect jump work with this patchset.
See individual patches for more details on implementation details.
Changes since RFC:
* I've tried to address all the comments provided by Alexei and
Eduard in RFC. Will try to list the most important of them below.
* One big change: move from older LLVM version [5] to newer [4].
Now LLVM generates jump tables as symbols in the new special
section ".jumptables". Another part of this change is that
libbpf now doesn't try to link map load and goto *rX, as
1) this is absolutely not reliable 2) for some use cases this
is impossible (namely, when more than one jump table can be used
in the same gotox instruction).
* Added insn_successors() support (Alexei, Eduard). This includes
getting rid of the ugly bpf_insn_set_iter_xlated_offset()
interface (Eduard).
* Removed hack for the unreachable instruction, as new LLVM thank to
Eduard doesn't generate it.
* Set mem_size for direct map access properly instead of hacking.
Remove off>0 check. (Alexei)
* Do not allocate new memory for min_index/max_index (Alexei, Eduard)
* Information required during check_cfg is now cached to be reused
later (Alexei + general logic for supporting multiple JT per jump)
* Properly compare registers in regsafe (Alexei, Eduard)
* Remove support for JMP32 (Eduard)
* Better checks in adjust_ptr_min_max_vals (Eduard)
* More selftests were added (but still there's room for more) which
directly use gotox (Alexei)
* More checks and verbose messages added
* "unique pointers" are no more in the map
Links:
1. https://lpc.events/event/18/contributions/1941/
2. https://lwn.net/Articles/1017439/
3. https://lore.kernel.org/bpf/20250615085943.3871208-1-a.s.protopopov@xxxxxxxxx/
4. https://github.com/llvm/llvm-project/pull/149715
Anton Protopopov (11):
bpf: fix the return value of push_stack
bpf: save the start of functions in bpf_prog_aux
bpf, x86: add new map type: instructions array
selftests/bpf: add selftests for new insn_array map
bpf: support instructions arrays with constants blinding
selftests/bpf: test instructions arrays with blinding
bpf, x86: allow indirect jumps to r8...r15
bpf, x86: add support for indirect jumps
bpf: disasm: add support for BPF_JMP|BPF_JA|BPF_X
libbpf: support llvm-generated indirect jumps
selftests/bpf: add selftests for indirect jumps
arch/x86/net/bpf_jit_comp.c | 39 +-
include/linux/bpf.h | 30 +
include/linux/bpf_types.h | 1 +
include/linux/bpf_verifier.h | 20 +-
include/uapi/linux/bpf.h | 11 +
kernel/bpf/Makefile | 2 +-
kernel/bpf/bpf_insn_array.c | 350 ++++++++++
kernel/bpf/core.c | 20 +
kernel/bpf/disasm.c | 9 +
kernel/bpf/syscall.c | 22 +
kernel/bpf/verifier.c | 603 ++++++++++++++++--
.../bpf/bpftool/Documentation/bpftool-map.rst | 2 +-
tools/bpf/bpftool/map.c | 2 +-
tools/include/uapi/linux/bpf.h | 11 +
tools/lib/bpf/libbpf.c | 159 ++++-
tools/lib/bpf/libbpf_probes.c | 4 +
tools/lib/bpf/linker.c | 12 +-
tools/testing/selftests/bpf/Makefile | 4 +-
.../selftests/bpf/prog_tests/bpf_goto_x.c | 132 ++++
.../selftests/bpf/prog_tests/bpf_insn_array.c | 498 +++++++++++++++
.../testing/selftests/bpf/progs/bpf_goto_x.c | 384 +++++++++++
21 files changed, 2230 insertions(+), 85 deletions(-)
create mode 100644 kernel/bpf/bpf_insn_array.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_goto_x.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
create mode 100644 tools/testing/selftests/bpf/progs/bpf_goto_x.c
--
2.34.1
