On Fri, Sep 12, 2025 at 11:43:38AM -0400, Stephen Gallagher wrote: > On Fri, Sep 12, 2025 at 5:05 AM Karolina Surma <ksurma@xxxxxxxxxx> wrote: > > > > Since all the builds will most probably be trigerred by me, a simple > > koji query filtering my builds after a given timestamp will be a good > > enough solution to make sure we caught them all. Well, we already have a releng script that handles merging side tags correctly. It will check the dest tag and not tag something if there's a newer build already (instead of just blindly tagging it in and overriding a newer maintainer build that was there). But as long as anything handles these corner cases, ok... > > Kevin, to your original question: if we create individual updates for > > each package, what, except for my work behind the scenes (to determine > > built packages and trigger an update for them) can go wrong? Can Bodhi > > handle ~3700 individual updates appearing in the span of ~2 days? If so, > > we'd most likely go this route. > > What is the best way forward from your point of view? > > > > ~3700 individual updates hitting Bodhi over a short period is almost > certainly going to cause problems with the load, similar to trying to > manually create a single update with all the same packages. It will definitely cause ci load... since all of those will need to pass gating tests (if any). > To my mind, we have three realistic options here: > > Option 1) You perform all of the builds, we identify them and tag them > manually into Fedora, bypassing Bodhi entirely. > Pros: Low resource usage, can be done with or without a side-tag as > preferred. (Though a side-tag means it's easier to keep track of which > packages were built specifically for this and it allows maintainers to > submit their own builds into the side-tag if needed, rather than > relying only on Karolina to build everything.) Well, or use a side tag and merge via script. Then if maintainers have a 'newer' build already in, it just doesn't tag the side tag one. No need to tell people to build in a sidetag or anything, they just keep doing what they are doing. > Cons: It bypasses all testing, so we won't know until the next compose > attempt whether things still work. Right, although, I think Adam might have some way to test a side tag seperately in openqa... > Option 2) Build everything into a side-tag, we try submitting the side > tag as a Bodhi update > Pros: The set of changes gets tested (though it probably fails and > needs to waive at least a few cases). It exercises a path ELN wants to > take and it provides Fedora data on whether Bodhi can handle such a > large update set. Even if Bodhi fails, we still have the option to > just migrate everything from the side-tag directly into stable. > Cons: If the load generated is too high, we could cause a DoS on Bodhi. > > Option 3) Don't do this mass-rebuild at all. > Pros: No resource usage, no risk to infrastructure. > Cons: Low initial end-user performance, as we'll ship Python packages > that need to byte-compile every module the first time they are used. Right, and many things will be rebuilt before final, but of course not all. > > > Does anyone see an Option 4 somewhere? I don't off hand. kevin -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue