I just tried it with 3 keepalive daemons and one nfs daemon, it
doesn't really work because all three hosts have the virtual IP
assigned, preventing my client from mounting. So this doesn't really
work as a workaround, it seems.
Thanks, Adam.
I just tried it with 3 keepalive daemons and one nfs daemon, it
doesn't really work because all three hosts have the virtual IP
assigned, preventing my client from mounting. So this doesn't really
work as a workaround, it seems. I feel like the proper solution would
be to include keepalive in the list of
RESCHEDULE_FROM_OFFLINE_HOSTS_TYPES.
Zitat von Adam King <adking@xxxxxxxxxx>:
> Which daemons get moved around like that is controlled by
>
https://github.com/ceph/ceph/blob/main/src/pybind/mgr/cephadm/utils.py#L30
,
> which appears to only include nfs and haproxy, so maybe this keepalive
only
> case was missed in that sense. I do think that you could alter the
> placement of the ingress service to just match all the hosts and it ought
> to work though. The only reason we require specifying a set of hosts
with a
> count field lower than the number of hosts matching the placement for nfs
> is that ganesha has no transparent state migration. Keepalive on the
other
> hand should work fine with "extra" daemons deployed in order to be highly
> available.
>
> On Tue, Mar 25, 2025 at 10:06 AM Malte Stroem <malte.stroem@xxxxxxxxx>
> wrote:
>
>> Hi Eugen,
>>
>> yes, for me it's kind of "test-setting" for small setups.
>>
>> Doc says:
>>
>> Setting --ingress-mode keepalive-only deploys a simplified ingress
>> service that provides a virtual IP with the nfs server directly binding
>> to that virtual IP and leaves out any sort of load balancing or traffic
>> redirection. This setup will restrict users to deploying only 1 nfs
>> daemon as multiple cannot bind to the same port on the virtual IP.
>>
>> Best,
>> Malte
>>
>>
>> On 25.03.25 13:46, Eugen Block wrote:
>> > Yeah, it seems to work without the "keepalive-only" flag, at least
from
>> > a first test. So keepalive-only is not working properly, it seems?
>> > Should I create a tracker for that or am I misunderstanding its
purpose?
>> >
>> > Zitat von Malte Stroem <malte.stroem@xxxxxxxxx>:
>> >
>> >> Hi Eugen,
>> >>
>> >> try omitting
>> >>
>> >> --ingress-mode keepalive-only
>> >>
>> >> like this
>> >>
>> >> ceph nfs cluster create ebl-nfs-cephfs "1 ceph01 ceph02 ceph03" --
>> >> ingress --virtual_ip "192.168.168.114/24"
>> >>
>> >> Best,
>> >> Malte
>> >>
>> >> On 25.03.25 13:25, Eugen Block wrote:
>> >>> Thanks for your quick response. The specs I pasted are actually the
>> >>> result of deploying a nfs cluster like this:
>> >>>
>> >>> ceph nfs cluster create ebl-nfs-cephfs "1 ceph01 ceph02 ceph03" --
>> >>> ingress --virtual_ip 192.168.168.114 --ingress-mode keepalive-only
>> >>>
>> >>> I can try redeploying it via dashboard, but I don't have a lot of
>> >>> confidence that it will work differently with a failover.
>> >>>
>> >>> Zitat von Malte Stroem <malte.stroem@xxxxxxxxx>:
>> >>>
>> >>>> Hi Eugen,
>> >>>>
>> >>>> try deploying the NFS service like this:
>> >>>>
>> >>>> https://docs.ceph.com/en/latest/mgr/nfs/
>> >>>>
>> >>>> Some had only success deploying it via the dashboard.
>> >>>>
>> >>>> Best,
>> >>>> Malte
>> >>>>
>> >>>> On 25.03.25 13:02, Eugen Block wrote:
>> >>>>> Hi,
>> >>>>>
>> >>>>> I'm re-evaluating NFS again, testing on a virtual cluster with
>> >>>>> 18.2.4. For now, I don't need haproxy so I use "keepalive_only:
>> >>>>> true" as described in the docs [0]. I first create the ingress
>> >>>>> service, wait for it to start, then create the nfs cluster. I've
>> >>>>> added the specs at the bottom.
>> >>>>>
>> >>>>> I can mount the export with the virtual ip. Then I just shut down
>> >>>>> the VM where the nfs service was running, the orchestrator
>> >>>>> successfully starts a nfs daemon elsewhere, but the keepalive
>> >>>>> daemon is not failed over. So mounting or accessing the export is
>> >>>>> impossible, of course. And after I power up the offline host
again,
>> >>>>> nothing is "repaired", keepalive and nfs run on different servers
>> >>>>> until I intervene manually. This doesn't seem to work as expected,
>> >>>>> is this a known issue (couldn't find anything on tracker)? I have
>> >>>>> my doubts, but maybe it works better with haproxy? Or am I missing
>> >>>>> something in my configuration?
>> >>>>> I haven't tried with a newer release yet. I'd appreciate any
>> comments.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Eugen
>> >>>>>
>> >>>>> ---snip---
>> >>>>> service_type: ingress
>> >>>>> service_id: nfs.ebl-nfs-cephfs
>> >>>>> service_name: ingress.nfs.ebl-nfs-cephfs
>> >>>>> placement:
>> >>>>> count: 1
>> >>>>> hosts:
>> >>>>> - ceph01
>> >>>>> - ceph02
>> >>>>> - ceph03
>> >>>>> spec:
>> >>>>> backend_service: nfs.ebl-nfs-cephfs
>> >>>>> first_virtual_router_id: 50
>> >>>>> keepalive_only: true
>> >>>>> monitor_port: 9049
>> >>>>> virtual_ip: 192.168.168.114/24
>> >>>>>
>> >>>>>
>> >>>>> service_type: nfs
>> >>>>> service_id: ebl-nfs-cephfs
>> >>>>> service_name: nfs.ebl-nfs-cephfs
>> >>>>> placement:
>> >>>>> count: 1
>> >>>>> hosts:
>> >>>>> - ceph01
>> >>>>> - ceph02
>> >>>>> - ceph03
>> >>>>> spec:
>> >>>>> port: 2049
>> >>>>> virtual_ip: 192.168.168.114
>> >>>>> ---snip---
>> >>>>>
>> >>>>> [0] https://docs.ceph.com/en/reef/cephadm/services/nfs/#nfs-with-
>> >>>>> virtual-ip-but-no-haproxy
>> >>>>> _______________________________________________
>> >>>>> ceph-users mailing list -- ceph-users@xxxxxxx
>> >>>>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>> >>>
>> >>>
>> >>>
>> >
>> >
>> >
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@xxxxxxx
>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>>
