On Wed, Jul 02, 2025 at 10:54:39AM -0700, Luck, Tony wrote: > On Wed, Jul 02, 2025 at 10:22:50AM -0700, Breno Leitao wrote: > > On Wed, Jul 02, 2025 at 09:31:30AM -0700, Luck, Tony wrote: > > > On Wed, Jul 02, 2025 at 08:39:51AM -0700, Breno Leitao wrote: > > > So unless someone feels it would be better to create a new TAINT > > > flag (TAINT_FATAL_GHES? TAINT_FIRMWARE_REPORTED_FATAL_ERRROR?) > > > then this seems OK to me. > > > > Thanks. That brings another topic. I am seeing crashes and warnings that > > are only happening after recoverable errors. I.e, there is a GHES > > recoverable error, and then machine crashes minutes later. A classical > > example is when the PCI downstream port disappear, and recovers later, > > re-enumerating everything, which is simply chaotic. > > > > I would like to be able to correlate the crash/warning with a machine > > that had a recoverable error. At scale, this improves the kernel > > monitoring by a lot. > > > > So, if we go toward using TAINT_FATAL_GHES, can we have two flavors? > > TAINT_FATAL_GHES_RECOVERABLE and TAINT_FATAL_GHES_FATAL? > > Do you really want to TAINT for recoverable errors? If most errors > are successfully recovered, then a TAINT indication that a recovery > happened a week ago would be misleading. > > Maybe better to save a timestamp for when the most recent recoverable > error occurred, then compare that against the current time in panic() > path and print warning if the recoverable error was "recent" (for > some TBD value of "recent"). Thanks for your insight. I believe it would be simpler to just add support for a TAINT that the hardware got an error while this kernel was booted. That is what I would like to indicate to the user. The user shouldn't not correlate that to the crash or panic. As you said, the hardware error could have happened weeks ago and completely unrelated. Tainting the kernel that a hardware error happen must NOT imply that the kernel crashed because of the hardware error. Something similar happens with proprietary module taint (PROPRIETARY_MODULE). The kernel is tainted when an proprietary module is loaded, but, it does not imply that the crash came because of the external module being loaded. It is just an extra information that will help the investigation later. In summary, I don't think we should solve the problem of correlation here, given it is not straightforward. I just want to tag that the hardware got an error while the kernel was running, and the operator can use this information the way they want. Am I on the right track? Thanks for the discussion, --breno
