On Thu, Jul 31, 2025 at 2:10 AM Hannes Reinecke <hare@xxxxxxx> wrote:
>
> On 7/30/25 22:08, Olga Kornievskaia wrote:
> > Revert kvec msg iterator before trying to process a TLS alert
> > when possible.
> >
> > In nvmet_tcp_try_recv_data(), it's assumed that no msg control
> > message buffer is set prior to sock_recvmsg(). Hannes suggested
> > that upon detecting that TLS control message is received log a
> > message and error out. Left comments in the code for the future
> > improvements.
> >
> > Fixes: a1c5dd8355b1 ("nvmet-tcp: control messages for recvmsg()")
> > Suggested-by: Hannes Reinecke <hare@xxxxxxx>
> > Reviewed-by: Hannes Reinecky <hare@xxxxxxx>
> > Signed-off-by: Olga Kornievskaia <okorniev@xxxxxxxxxx>
> > ---
> > drivers/nvme/target/tcp.c | 30 +++++++++++++++++++-----------
> > 1 file changed, 19 insertions(+), 11 deletions(-)
> >
> > diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
> > index 688033b88d38..055e420d3f2e 100644
> > --- a/drivers/nvme/target/tcp.c
> > +++ b/drivers/nvme/target/tcp.c
> > @@ -1161,6 +1161,7 @@ static int nvmet_tcp_try_recv_pdu(struct nvmet_tcp_queue *queue)
> > if (unlikely(len < 0))
> > return len;
> > if (queue->tls_pskid) {
> > + iov_iter_revert(&msg.msg_iter, len);
> > ret = nvmet_tcp_tls_record_ok(queue, &msg, cbuf);
> > if (ret < 0)
> > return ret;
> > @@ -1217,19 +1218,28 @@ static void nvmet_tcp_prep_recv_ddgst(struct nvmet_tcp_cmd *cmd)
> > static int nvmet_tcp_try_recv_data(struct nvmet_tcp_queue *queue)
> > {
> > struct nvmet_tcp_cmd *cmd = queue->cmd;
> > - int len, ret;
> > + int len;
> >
> > while (msg_data_left(&cmd->recv_msg)) {
> > + /* to detect that we received a TlS alert, we assumed that
> > + * cmg->recv_msg's control buffer is not setup. kTLS will
> > + * return an error when no control buffer is set and
> > + * non-tls-data payload is received.
> > + */
> > len = sock_recvmsg(cmd->queue->sock, &cmd->recv_msg,
> > cmd->recv_msg.msg_flags);
> > + if (cmd->recv_msg.msg_flags & MSG_CTRUNC) {
> > + if (len == 0 || len == -EIO) {
> > + pr_err("queue %d: unhandled control message\n",
> > + queue->idx);
> > + /* note that unconsumed TLS control message such
> > + * as TLS alert is still on the socket.
> > + */
>
> Hmm. Will it get cleared when we close the socket?
If the socket is closed then any data on that socket would be freed.
> Or shouldn't we rather introduce proper cmsg handling?
That would be what I have originally proposed (I know that was on the
private list). But yes, we can setup a dedicated kvec to receive the
TLS control message once its been detected and then call
nvme_tcp_tls_record_ok().
Let me know if proper cmsg handling is what's desired for this patch.
> (If we do, we'll need it to do on the host side, too)
I can see that the host doesn't have any TLS alert handling now. If
the only place where (TLS) traffic is read from is in host/tcp.c
nvme_tcp_init_connection(), then that's seems like an easy case
because it uses a kvec to back the kernel_recvmsg() msg structure. If
the ctype is tls alert, you can call tls_alert_recv() and pass in the
"iov". -- assuming patch#4 already went in by that time)
>
> > + return -EAGAIN;
> > + }
> > + }
> > if (len <= 0)
> > return len;
> > - if (queue->tls_pskid) {
> > - ret = nvmet_tcp_tls_record_ok(cmd->queue,
> > - &cmd->recv_msg, cmd->recv_cbuf);
> > - if (ret < 0)
> > - return ret;
> > - }
> >
> > cmd->pdu_recv += len;
> > cmd->rbytes_done += len;
> > @@ -1267,6 +1277,7 @@ static int nvmet_tcp_try_recv_ddgst(struct nvmet_tcp_queue *queue)
> > if (unlikely(len < 0))
> > return len;
> > if (queue->tls_pskid) {
> > + iov_iter_revert(&msg.msg_iter, len);
> > ret = nvmet_tcp_tls_record_ok(queue, &msg, cbuf);
> > if (ret < 0)
> > return ret;
> > @@ -1453,10 +1464,6 @@ static int nvmet_tcp_alloc_cmd(struct nvmet_tcp_queue *queue,
> > if (!c->r2t_pdu)
> > goto out_free_data;
> >
> > - if (queue->state == NVMET_TCP_Q_TLS_HANDSHAKE) {
> > - c->recv_msg.msg_control = c->recv_cbuf;
> > - c->recv_msg.msg_controllen = sizeof(c->recv_cbuf);
> > - }
>
> As you delete this you can also remove the definition of 'recv_msg'
> from nvmet_tcp_cmd structure.
You mean 'recv_cbuf', right? recv_msg would still be needed by the code.
I can send v2 with that change. Whether or not cmsg handling is needed
in v2 I'd need a confirmation on. Given I'm working on compile only
mode, I'd rather keep changes to minimal.
> > c->recv_msg.msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL;
> >
> > list_add_tail(&c->entry, &queue->free_list);
> > @@ -1736,6 +1743,7 @@ static int nvmet_tcp_try_peek_pdu(struct nvmet_tcp_queue *queue)
> > return len;
> > }
> >
> > + iov_iter_revert(&msg.msg_iter, len);
> > ret = nvmet_tcp_tls_record_ok(queue, &msg, cbuf);
> > if (ret < 0)
> > return ret;
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke Kernel Storage Architect
> hare@xxxxxxx +49 911 74053 688
> SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
> HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
>
