On Tue, 2025-04-15 at 11:22 -0600, Andreas Dilger wrote: [...] > Negative dentries are only useful if there are fewer than the number > of entries in that directory. I agree with this, yes. > If the negative dentry count exceeds the actual entry count, Yes, but finding this number is going to be hard. We can't iterate a directory to count them in the fast path and a directory i_size is extremely filesystem and format dependent. However, since we only need a rough count, perhaps having the filesystem export its average directory entry size and simply dividing by that would give a good enough approximation to the number? > it would be more efficient to just cache all of the positive dentries > and mark the directory with a "full dentry list" flag that indicates > all of the names are already present in dcache and any miss is > authoritative. In essence that gives an "infinite" negative lookup > cache instead of explicitly storing all of the possible negative > entries. Practically, I think directories with that flag would probably automatically retain positive child dentries as an addition to our retain_dentry() logic and automatically kill negative ones. This behaviour, though, would remove them from the shrinkers, so probably there would have to be a global count of the number of unshrinkable children this gives us and have that factor into the superblock shrinkers somehow. Probably add the parent to the lru list but make dentry_lru_isolate() always skip until the tipping point for shrinking filled directories is reached? > For directories like ~/bin, /usr/bin, /usr/lib64, etc. (or any > directory) where negative lookups are frequent, it should be possible > to determine this threshold automatically. Once the negative dentry > count exceeds the size of the directory by some factor (e.g. > directory size / 16, or the actual entry count if the filesystem > knows this, it doesn't have to be exactly correct) then a readdir > could load all of the names to fully populate the dcache and set the > "full dentry list" flag on the directory would allow dropping all > negative dentries in that directory. All this supposes we have some per directory count of the negative dentries. I think there'd be push back on adding this to struct dentry and making it an exact count in the fast path. The next logical place to evaluate it would be the shrinkers but then that wouldn't solve Matthew's use case where the shrinkers don't get activated. I suppose some flag that userspace could add to directories it identifies as hot might be the next best thing? > The VFS/VM should avoid dropping directories/dentries from cache in > this case, since it is saving more memory (and avoiding filesystem > IO) to keep them pinned rather than dropping them from cache. There > might need to be a matching "part of full dentry list" flag on the > positive dentries to avoid dcache shrinking of those entries (which > would invalidate the premise that the parent holds all of the > possible entries in that directory), if checking the parent's flag is > too expensive. As I said above, I think simply checking the parent flags in retain_dentry should do. Since you don't need it to be exact and the parent should have a positive refcount, it should be possible to do a READ_ONCE rather than locking. Regards, James
Attachment:
signature.asc
Description: This is a digitally signed message part
