On 4/23/2025 10:04 AM, Jonathan Cameron wrote:
> On Wed, 26 Mar 2025 20:47:05 -0500
> Terry Bowman <terry.bowman@xxxxxxx> wrote:
>
>> The AER service driver includes a CXL-specific kfifo, intended to forward
>> CXL errors to the CXL driver. However, the forwarding functionality is
>> currently unimplemented. Update the AER driver to enable error forwarding
>> to the CXL driver.
>>
>> Modify the AER service driver's handle_error_source(), which is called from
>> process_aer_err_devices(), to distinguish between PCIe and CXL errors.
>>
>> Rename and update is_internal_error() to is_cxl_error(). Ensuring it
>> checks both the 'struct aer_info::is_cxl' flag and the AER internal error
>> masks.
>>
>> If the error is a standard PCIe error then continue calling pcie_aer_handle_error()
>> as done in the current AER driver.
>>
>> If the error is a CXL-related error then forward it to the CXL driver for
>> handling using the kfifo mechanism.
>>
>> Introduce a new function forward_cxl_error(), which constructs a CXL
>> protocol error context using cxl_create_prot_err_info(). This context is
>> then passed to the CXL driver via kfifo using a 'struct work_struct'.
>>
>> Signed-off-by: Terry Bowman <terry.bowman@xxxxxxx>
> Hi Terry,
>
> Finally got back to this. I'm not following how some of the reference
> counting in here is working. It might be fine but there is a lot
> taking then dropping device references - some of which are taken again later.
>
>> @@ -1082,10 +1094,44 @@ static void cxl_rch_enable_rcec(struct pci_dev *rcec)
>> pci_info(rcec, "CXL: Internal errors unmasked");
>> }
>>
>> +static void forward_cxl_error(struct pci_dev *_pdev, struct aer_err_info *info)
>> +{
>> + int severity = info->severity;
> So far this variable isn't really justified. Maybe it makes sense later in the
> series?
This is used below in call to cxl_create_prot_err_info().
>> + struct cxl_prot_err_work_data wd;
>> + struct cxl_prot_error_info *err_info = &wd.err_info;
> Similarly. Why not just use this directly in the call below?
The reference assignment is made so that err_info can be initialized above and is then
used here to assign and later passed below as part of the work structure.
>> + struct pci_dev *pdev __free(pci_dev_put) = pci_dev_get(_pdev);
> Can you talk me through the reference counting? You take one pci device reference
> here....
This will add reference count to the PCI device (not the CXL device) to prevent it
from being destroyed until scope exit cleanup.
>> +
>> + if (!cxl_create_prot_err_info) {
>> + pci_err(pdev, "Failed. CXL-AER interface not initialized.");
>> + return;
>> + }
>> +
>> + if (cxl_create_prot_err_info(pdev, severity, err_info)) {
> ...but the implementation of this also takes once internally. Can we skip that
> internal one and document that it is always take by the caller?
Yes, it can. I will have to verify other the 5 or 6 calls do the same before calling.
I was wanting to make the reference incr as early as possible immediately after error
detection and also not forcing callers to do the same setup everywhere beforehand. I
see your point and will consolidate them.
>> + pci_err(pdev, "Failed to create CXL protocol error information");
>> + return;
>> + }
>> +
>> + struct device *cxl_dev __free(put_device) = get_device(err_info->dev);
> Also this one. A reference was acquired and dropped in cxl_create_prot_err_info()
> followed by retaking it here. How do we know it is still about by this call
> and once we pull it off the kfifo later?
Yes, this is a problem I realized after sending the series.
The device reference incr could be changed for all the devices to the non-cleanup
variety. Then would add the reference incr in the caller after calling cxl_create_prot_err_info().
I need to look at the other calls to to cxl_create_prot_err_info() as well.
In addition, I think we should consider adding the CXL RAS status into the struct cxl_prot_err_info.
This would eliminate the need for further accesses to the CXL device after being dequeued from the
fifo. Thoughts?
>> +
>> + if (!kfifo_put(&cxl_prot_err_fifo, wd)) {
>> + pr_err_ratelimited("CXL kfifo overflow\n");
>> + return;
>> + }
>> +
>> + schedule_work(cxl_prot_err_work);
>> +}
>> +
> Thanks,
>
> Jonathan
>
Thanks for reviewing Jonathan.
-Terry
