* Motivation * Currently, sched_ext only supports a single scheduler. scx kfuncs can reference the calling scheduler simply with a global pointer, scx_root, whether being called in struct_ops, tracing, syscall or async callback. However, when sched_ext moves to support multiple scx schedulers, scx kfunc without task_struct argument won't be able to easily refer to the calling scheduler without passing additional information. The vision is to be able to view struct_ops, tracing, syscall programs and async callbacks in a file as part of a scheduler, and have the ability to refer to the scheduler in kfuncs called from any of the programs. However, right now there is not a notion that connects bpf programs in a file in the kernel. Until it is clear what that should be, as a stopgap, we would like to reuse cookie to associate bpf programs belonging to the same scheduler. * Design * The interim solution is to use bpf cookie as the scheduler id, and add the cookie as an argument to scx kfuncs without task_struct. By maintaining the cookie to scheduler mapping in sched_ext, kfuncs will be able to refer to the calling scheduler internally using cookie. For the user space loader, it should use the same cookie when attaching bpf programs. In bpf programs, they will call bpf_get_attach_cookie() to get the cookie and pass it to scx kfuncs. The following patches are the first part of the kernel-side plumbing: to support cookie for link-based struct_ops attachment. struct_ops already uses bpf trampoline. The only problem is the trampoline is being populated during map_update when the cookie is not known yet. This patchset moves the trampoline preparation from map_update to link_create for link-based struct_ops attachment. Then, we extend the UAPI to take the cookie and propagate it all the way to the struct_ops link_create. Finally, bpf_get_attach_cookie() is allowed to struct_ops programs. A problem that has not been addressed in this RFC is that by moving trampoline preparation to link_create, creating multiple links using the same struct_ops map becomes problematic. A simple solution may be just disable if there is no obvious use case for it. * Alternative * Martin has suggested a way that involves less kernel changes: Using struct_ops map id as the sched_ext scheduler id. Instead of further complicating struct_ops, sched_ext can create the struct_ops map id to scheduler mapping in reg(). The id will also be stored in a global variable in the file referenced by different types of programs. The rest is similar to the cookie approach: kfuncs without task_struct will need to add an argument, and they will lookup the scheduler by the id. * Previous approach * The previous solution was to set a "this pointer" pointing to the struct_ops struct during struct_ops map_update. Then, kfuncs can get a pointer to the scheduler with a __prog argument. However, this approach is broken as it would only work for struct_ops programs. Some thought after Martin brought up the idea I think we should use the map id instead of cookie. Especially if we are going to disable creating multiple links per map, the cookie effectively becomes the same as the map id. Amery Hung (4): bpf: Factor out bpf_struct_ops_prepare_attach() bpf: Support cookie for linked-based struct_ops attachment libbpf: Support link-based struct_ops attach with options selftests/bpf: Test bpf_get_attach_cookie() in struct_ops program include/uapi/linux/bpf.h | 3 + kernel/bpf/bpf_struct_ops.c | 199 ++++++++++++------ kernel/bpf/helpers.c | 18 ++ tools/include/uapi/linux/bpf.h | 3 + tools/lib/bpf/bpf.c | 5 + tools/lib/bpf/bpf.h | 3 + tools/lib/bpf/libbpf.c | 14 +- tools/lib/bpf/libbpf.h | 10 + tools/lib/bpf/libbpf.map | 1 + .../bpf/prog_tests/test_struct_ops_cookie.c | 42 ++++ .../selftests/bpf/progs/struct_ops_cookie.c | 48 +++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 1 + 12 files changed, 280 insertions(+), 67 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_cookie.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_cookie.c -- 2.47.1
