On Tue, Apr 29, 2025 at 7:55 AM Caleb Sander Mateos
<csander@xxxxxxxxxxxxxxx> wrote:
>
> On Mon, Apr 28, 2025 at 9:05 PM Ming Lei <ming.lei@xxxxxxxxxx> wrote:
> >
> > On Mon, Apr 28, 2025 at 08:12:52AM -0700, Caleb Sander Mateos wrote:
> > > On Mon, Apr 28, 2025 at 7:28 AM Caleb Sander Mateos
> > > <csander@xxxxxxxxxxxxxxx> wrote:
> > > >
> > > > On Sun, Apr 27, 2025 at 6:05 AM Ming Lei <ming.lei@xxxxxxxxxx> wrote:
> > > > >
> > > > > On Sat, Apr 26, 2025 at 10:58:00PM -0600, Caleb Sander Mateos wrote:
> > > > > > In preparation for calling it from outside ublk_dispatch_req(), factor
> > > > > > out the code responsible for setting up an incoming ublk I/O request.
> > > > > >
> > > > > > Signed-off-by: Caleb Sander Mateos <csander@xxxxxxxxxxxxxxx>
> > > > > > ---
> > > > > > drivers/block/ublk_drv.c | 53 ++++++++++++++++++++++------------------
> > > > > > 1 file changed, 29 insertions(+), 24 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
> > > > > > index 01fc92051754..90a38a82f8cc 100644
> > > > > > --- a/drivers/block/ublk_drv.c
> > > > > > +++ b/drivers/block/ublk_drv.c
> > > > > > @@ -1151,17 +1151,44 @@ static inline void __ublk_abort_rq(struct ublk_queue *ubq,
> > > > > > blk_mq_requeue_request(rq, false);
> > > > > > else
> > > > > > blk_mq_end_request(rq, BLK_STS_IOERR);
> > > > > > }
> > > > > >
> > > > > > +static void ublk_start_io(struct ublk_queue *ubq, struct request *req,
> > > > > > + struct ublk_io *io)
> > > > > > +{
> > > > > > + unsigned mapped_bytes = ublk_map_io(ubq, req, io);
> > > > > > +
> > > > > > + /* partially mapped, update io descriptor */
> > > > > > + if (unlikely(mapped_bytes != blk_rq_bytes(req))) {
> > > > > > + /*
> > > > > > + * Nothing mapped, retry until we succeed.
> > > > > > + *
> > > > > > + * We may never succeed in mapping any bytes here because
> > > > > > + * of OOM. TODO: reserve one buffer with single page pinned
> > > > > > + * for providing forward progress guarantee.
> > > > > > + */
> > > > > > + if (unlikely(!mapped_bytes)) {
> > > > > > + blk_mq_requeue_request(req, false);
> > > > > > + blk_mq_delay_kick_requeue_list(req->q,
> > > > > > + UBLK_REQUEUE_DELAY_MS);
> > > > > > + return;
> > > > > > + }
> > > > > > +
> > > > > > + ublk_get_iod(ubq, req->tag)->nr_sectors =
> > > > > > + mapped_bytes >> 9;
> > > > > > + }
> > > > > > +
> > > > > > + ublk_init_req_ref(ubq, req);
> > > > > > +}
> > > > > > +
> > > > > > static void ublk_dispatch_req(struct ublk_queue *ubq,
> > > > > > struct request *req,
> > > > > > unsigned int issue_flags)
> > > > > > {
> > > > > > int tag = req->tag;
> > > > > > struct ublk_io *io = &ubq->ios[tag];
> > > > > > - unsigned int mapped_bytes;
> > > > > >
> > > > > > pr_devel("%s: complete: qid %d tag %d io_flags %x addr %llx\n",
> > > > > > __func__, ubq->q_id, req->tag, io->flags,
> > > > > > ublk_get_iod(ubq, req->tag)->addr);
> > > > > >
> > > > > > @@ -1204,33 +1231,11 @@ static void ublk_dispatch_req(struct ublk_queue *ubq,
> > > > > > pr_devel("%s: update iod->addr: qid %d tag %d io_flags %x addr %llx\n",
> > > > > > __func__, ubq->q_id, req->tag, io->flags,
> > > > > > ublk_get_iod(ubq, req->tag)->addr);
> > > > > > }
> > > > > >
> > > > > > - mapped_bytes = ublk_map_io(ubq, req, io);
> > > > > > -
> > > > > > - /* partially mapped, update io descriptor */
> > > > > > - if (unlikely(mapped_bytes != blk_rq_bytes(req))) {
> > > > > > - /*
> > > > > > - * Nothing mapped, retry until we succeed.
> > > > > > - *
> > > > > > - * We may never succeed in mapping any bytes here because
> > > > > > - * of OOM. TODO: reserve one buffer with single page pinned
> > > > > > - * for providing forward progress guarantee.
> > > > > > - */
> > > > > > - if (unlikely(!mapped_bytes)) {
> > > > > > - blk_mq_requeue_request(req, false);
> > > > > > - blk_mq_delay_kick_requeue_list(req->q,
> > > > > > - UBLK_REQUEUE_DELAY_MS);
> > > > > > - return;
> > > > > > - }
> > > > >
> > > > > Here it needs to break ublk_dispatch_req() for not completing the
> > > > > uring_cmd, however ublk_start_io() can't support it.
> > > >
> > > > Good catch. How about I change ublk_start_io() to return a bool
> > > > indicating whether the I/O was successfully started?
> >
> > That is doable.
> >
> > >
> > > Thinking a bit more about this, is the existing behavior of returning
> > > early from ublk_dispatch_req() correct for UBLK_IO_NEED_GET_DATA? It
> >
> > The requeue isn't related with UBLK_IO_NEED_GET_DATA actually, when
> > UBLK_IO_FLAG_NEED_GET_DATA is cleared.
> >
> > It is usually caused by running out of pages, so we have to requeue until
> > ublk_map_io() can make progress.
> >
> > > makes sense for the initial ublk_dispatch_req() because the req will
> > > be requeued without consuming the ublk fetch request, allowing it to
> > > be reused for a subsequent I/O. But for UBLK_IO_NEED_GET_DATA, doesn't
> > > it mean the io_uring_cmd will never complete? I would think it would
> > > be better to return an error code in this case.
> >
> > The same request will be requeued and re-dispatched to ublk driver after
> > a short delay, so the uring_cmd won't be never complete.
>
> I am referring to the UBLK_IO_NEED_GET_DATA uring_cmd, not the FETCH
> one. Doesn't the early return in ublk_dispatch_req() mean
> ublk_complete_io_cmd() won't be called? How else can the
> UBLK_IO_NEED_GET_DATA complete?
>
> >
> > Anyway, it isn't another story, which shouldn't be added into this
> > cleanup patch.
>
> I agree it belongs in a separate patch.
I am not going to fix it in this patch set. I am not sure what the
result value for UBLK_IO_NEED_GET_DATA should be if
ublk_copy_user_pages() returns 0. I also noticed that the return value
of import_ubuf() is ignored, which means that a bad userspace address
will result in an uninitialized struct iov_iter. I think ublk_map_io()
and ublk_unmap_io() may need to have more failure modes.
Best,
Caleb
