Hi Ming, Thank you for the fast reply. To be clear, I don't want calling DELETE_DEV or STOP_DEV as I want the kernel bdev will be stay while upgrading the ublk server application. It would be nice to have a nice way to have something like FREEZE_DEV that we may use which will also make all the cmds back with ABORT result but both block and char device will be stay until a new userspace application will reconnect. Thank you ________________________________________ From: Ming Lei <ming.lei@xxxxxxxxxx> Sent: Tuesday, April 15, 2025 2:06 PM To: Yoav Cohen Cc: linux-block@xxxxxxxxxxxxxxx; axboe@xxxxxxxxx Subject: Re: ublk: Graceful Upgrade of ublk server application External email: Use caution opening links or attachments On Tue, Apr 15, 2025 at 08:15:08AM +0000, Yoav Cohen wrote: > I am seeking advice on whether it is possible to upgrade the ublksrv version without terminating the daemon abruptly. Specifically, I would like the daemon to exit gracefully, ensuring all necessary cleanups are performed. > In my current implementation, I attempted to cancel all ublk uring SQEs (specifically the COMMIT_AND_FETCH_REQ/FETCH_REQ operations) using the following approach: > io_uring_prep_cancel_fd(sqe, cdev_fd, IORING_ASYNC_CANCEL_ALL);io_uring_prep_cancel_fd(sqe, cdev_fd, IORING_ASYNC_CANCEL_ALL); That shouldn't work because COMMIT_AND_FETCH_REQ/FETCH_REQ has been issued to ublk driver already. > However, this method does not seem to be effective. In my scenario, I have a single io_uring instance that serves multiple devices and other producers, so simply stopping the polling of CQEs is not a viable solution due to potential race conditions. > Could you please provide any suggestions or guidance on how to achieve a graceful upgrade of the ublksrv version without causing disruptions? > You can delete all ublk devices handled by this single io_ring instance first by sending UBLK_CMD_DEL_DEV, then exit ublk server loop if there isn't any pending uring_cmd & target IO. And the ublk server need to stop to issue COMMIT_AND_FETCH_REQ after getting uring_cmd with UBLK_IO_RES_ABORT. I guess you want to send the control command in single pthread too? If yes, it still can work with coroutine or modern language's .async/await. This feature is actually in my todo list for libublk-rs, just not started because of not seeing real requirement. Thanks, Ming
