On Thu, May 1, 2025 at 4:52 PM Andrii Nakryiko <andrii@xxxxxxxxxx> wrote: > > BTF dedup has a strong assumption that compiler with deduplicate identical > types within any given compilation unit (i.e., .c file). This property > is used when establishing equilvalence of two subgraphs of types. > > Unfortunately, this property doesn't always holds in practice. We've > seen cases of having truly identical structs, unions, array definitions, > and, most recently, even pointers to the same type being duplicated > within CU. > > Previously, we mitigated this on a case-by-case basis, adding a few > simple heuristics for validating that two BTF types (having two > different type IDs) are structurally the same. But this approach scales > poorly, and we can have more weird cases come up in the future. > > So let's take a half-step back, and implement a bit more generic > structural equivalence check, recursively. We still limit it to > reasonable depth to avoid long reference loops. Depth-wise limiting of > potentially cyclical graph isn't great, but as I mentioned below doesn't > seem to be detrimental performance-wise. We can always improve this in > the future with per-type visited markers, if necessary. > > Performance-wise this doesn't seem too affect vmlinux BTF dedup, which > makes sense because this logic kicks in not so frequently and only if we > already established a canonical candidate type match, but suddenly find > a different (but probably identical) type. > > On the other hand, this seems to help to reduce duplication across many > kernel modules. In my local test, I had 639 kernel module built. Overall > .BTF sections size goes down from 41MB bytes down to 5MB (!), which is Forgot to mention that vmlinux BTF size itself didn't change at all. > pretty impressive for such a straightforward piece of logic added. But > it would be nice to validate independently just in case my bash and > Python-fu is broken. > > Signed-off-by: Andrii Nakryiko <andrii@xxxxxxxxxx> > --- > tools/lib/bpf/btf.c | 137 ++++++++++++++++++++++++++++---------------- > 1 file changed, 89 insertions(+), 48 deletions(-) > [...]
