Hi,
I have added 42 x 18TB HDD disks ( 6 on each of the 7 servers )
My expectation was that the pools configured to use "hdd_class" will
have their capacity increased ( e.g. default.rgw.buckets.data which is
uses an EC 4+2 pool for data )
It seems it is not happening ...yet ?!
Is it because the peering is still going ?
Many thanks
Steven
Below are outputs from
ceph -s
ceph df
ceph osd df tree
cluster:
id: 0cfa836d-68b5-11f0-90bf-7cc2558e5ce8
health: HEALTH_WARN
1 OSD(s) experiencing slow operations in BlueStore
1 failed cephadm daemon(s)
569 pgs not deep-scrubbed in time
2609 pgs not scrubbed in time
39 daemons have recently crashed
services:
mon: 5 daemons, quorum
ceph-host-1,ceph-host-2,ceph-host-3,ceph-host-7,ceph-host-6 (age 2m)
mgr: ceph-host-1.lqlece(active, since 18h), standbys: ceph-host-2.suiuxi
mds: 19/19 daemons up, 7 standby
osd: 181 osds: 181 up (since 4d), 181 in (since 14h); 2770 remapped pgs
flags noautoscale
data:
volumes: 4/4 healthy
pools: 16 pools, 7137 pgs
objects: 256.82M objects, 484 TiB
usage: 742 TiB used, 1.5 PiB / 2.2 PiB avail
pgs: 575889786/1468742421 objects misplaced (39.210%)
4247 active+clean
2763 active+remapped+backfill_wait
77 active+clean+scrubbing
43 active+clean+scrubbing+deep
7 active+remapped+backfilling
io:
client: 84 B/s rd, 2.8 KiB/s wr, 0 op/s rd, 1 op/s wr
recovery: 384 MiB/s, 128 objects/s
--- RAW STORAGE ---
CLASS SIZE AVAIL USED RAW USED %RAW USED
hdd_class 1.4 PiB 814 TiB 579 TiB 579 TiB 41.54
nvme_class 293 TiB 293 TiB 216 GiB 216 GiB 0.07
ssd_class 587 TiB 424 TiB 163 TiB 163 TiB 27.80
TOTAL 2.2 PiB 1.5 PiB 742 TiB 742 TiB 32.64
--- POOLS ---
POOL ID PGS STORED OBJECTS USED %USED MAX
AVAIL
.mgr 1 1 277 MiB 71 831 MiB 0
93 TiB
.rgw.root 2 32 1.6 KiB 6 72 KiB 0
93 TiB
default.rgw.log 3 32 63 KiB 210 972 KiB 0
93 TiB
default.rgw.control 4 32 0 B 8 0 B 0
93 TiB
default.rgw.meta 5 32 1.4 KiB 8 72 KiB 0
93 TiB
default.rgw.buckets.data 6 2048 289 TiB 100.35M 434 TiB 68.04
136 TiB
default.rgw.buckets.index 7 1024 5.4 GiB 521 16 GiB 0
93 TiB
default.rgw.buckets.non-ec 8 32 551 B 1 13 KiB 0
93 TiB
metadata_fs_ssd 9 128 6.1 GiB 15.69M 18 GiB 0
93 TiB
ssd_ec_project 10 1024 108 TiB 44.46M 162 TiB 29.39
260 TiB
metadata_fs_hdd 11 128 9.9 GiB 8.38M 30 GiB 0.01
93 TiB
hdd_ec_archive 12 1024 90 TiB 87.94M 135 TiB 39.79
136 TiB
metadata_fs_nvme 13 32 260 MiB 177 780 MiB 0
93 TiB
metadata_fs_ssd_rep 14 32 17 MiB 103 51 MiB 0
93 TiB
ssd_rep_projects 15 1024 132 B 1 12 KiB 0
130 TiB
nvme_rep_projects 16 512 3.5 KiB 30 336 KiB 0
93 TiB
ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP
META AVAIL %USE VAR PGS STATUS TYPE NAME
-1 2272.93311 - 2.2 PiB 742 TiB 731 TiB 63 GiB
3.1 TiB 1.5 PiB 32.64 1.00 - root default
-7 254.54175 - 255 TiB 104 TiB 102 TiB 9.6 GiB
455 GiB 151 TiB 40.78 1.25 - host ceph-host-1
107 hdd_class 18.43300 1.00000 18 TiB 13 TiB 13 TiB 4 KiB
48 GiB 5.2 TiB 71.71 2.20 422 up osd.107
110 hdd_class 18.43300 1.00000 18 TiB 13 TiB 13 TiB 4 KiB
45 GiB 5.3 TiB 71.30 2.18 421 up osd.110
116 hdd_class 18.43300 0.95001 18 TiB 13 TiB 13 TiB 4 KiB
43 GiB 5.3 TiB 71.50 2.19 424 up osd.116
123 hdd_class 18.43300 1.00000 18 TiB 13 TiB 13 TiB 14 KiB
44 GiB 5.5 TiB 70.20 2.15 414 up osd.123
130 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 6 KiB
56 GiB 4.2 TiB 76.99 2.36 451 up osd.130
137 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 6 KiB
50 GiB 4.3 TiB 76.47 2.34 449 up osd.137
152 hdd_class 18.19040 1.00000 18 TiB 241 GiB 239 GiB 10 KiB
1.8 GiB 18 TiB 1.29 0.04 7 up osd.152
2 nvme_class 13.97260 1.00000 14 TiB 14 GiB 906 MiB 3.9 GiB
8.9 GiB 14 TiB 0.10 0.00 300 up osd.2
8 nvme_class 13.97260 1.00000 14 TiB 7.2 GiB 846 MiB 2.5 GiB
3.9 GiB 14 TiB 0.05 0.00 293 up osd.8
15 nvme_class 13.97260 1.00000 14 TiB 11 GiB 887 MiB 3.3 GiB
6.9 GiB 14 TiB 0.08 0.00 288 up osd.15
23 ssd_class 6.98630 1.00000 7.0 TiB 1.8 TiB 1.8 TiB 6 KiB
14 GiB 5.2 TiB 26.28 0.81 106 up osd.23
30 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
8.4 GiB 5.0 TiB 27.73 0.85 114 up osd.30
37 ssd_class 6.98630 1.00000 7.0 TiB 1.8 TiB 1.8 TiB 6 KiB
15 GiB 5.2 TiB 26.28 0.80 104 up osd.37
44 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
8.3 GiB 5.1 TiB 27.34 0.84 111 up osd.44
52 ssd_class 6.98630 1.00000 7.0 TiB 1.8 TiB 1.8 TiB 4 KiB
15 GiB 5.1 TiB 26.35 0.81 104 up osd.52
59 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
12 GiB 5.0 TiB 27.79 0.85 107 up osd.59
66 ssd_class 6.98630 1.00000 7.0 TiB 1.8 TiB 1.8 TiB 4 KiB
14 GiB 5.1 TiB 26.29 0.81 104 up osd.66
72 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.8 TiB 4 KiB
15 GiB 5.1 TiB 26.68 0.82 107 up osd.72
79 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
15 GiB 5.1 TiB 27.05 0.83 111 up osd.79
86 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 4 KiB
10 GiB 5.0 TiB 28.49 0.87 116 up osd.86
93 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
5.5 GiB 5.1 TiB 27.29 0.84 112 up osd.93
100 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
15 GiB 5.0 TiB 28.19 0.86 114 up osd.100
-15 345.49374 - 345 TiB 106 TiB 104 TiB 8.7 GiB
523 GiB 239 TiB 30.69 0.94 - host ceph-host-2
115 hdd_class 18.43300 1.00000 18 TiB 13 TiB 12 TiB 7 KiB
43 GiB 5.9 TiB 68.17 2.09 402 up osd.115
122 hdd_class 18.43300 1.00000 18 TiB 13 TiB 13 TiB 2 KiB
45 GiB 5.4 TiB 70.72 2.17 420 up osd.122
134 hdd_class 18.43300 1.00000 18 TiB 14 TiB 13 TiB 12 KiB
52 GiB 4.7 TiB 74.54 2.28 440 up osd.134
141 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 6 KiB
54 GiB 4.3 TiB 76.58 2.35 447 up osd.141
144 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 4 KiB
56 GiB 4.5 TiB 75.77 2.32 443 up osd.144
146 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 7 KiB
52 GiB 4.4 TiB 76.13 2.33 449 up osd.146
159 hdd_class 18.19040 1.00000 18 TiB 203 GiB 200 GiB 10 KiB
2.4 GiB 18 TiB 1.09 0.03 6 up osd.159
176 hdd_class 18.19040 1.00000 18 TiB 135 GiB 133 GiB 6 KiB
1.4 GiB 18 TiB 0.72 0.02 4 up osd.176
177 hdd_class 18.19040 1.00000 18 TiB 74 GiB 73 GiB 6 KiB
1.7 GiB 18 TiB 0.40 0.01 2 up osd.177
178 hdd_class 18.19040 1.00000 18 TiB 238 GiB 236 GiB 7 KiB
2.3 GiB 18 TiB 1.28 0.04 5 up osd.178
179 hdd_class 18.19040 1.00000 18 TiB 76 GiB 75 GiB 6 KiB
828 MiB 18 TiB 0.41 0.01 2 up osd.179
180 hdd_class 18.19040 1.00000 18 TiB 198 GiB 197 GiB 10 KiB
1.7 GiB 18 TiB 1.07 0.03 5 up osd.180
6 nvme_class 13.97260 1.00000 14 TiB 11 GiB 903 MiB 2.9 GiB
7.0 GiB 14 TiB 0.08 0.00 282 up osd.6
13 nvme_class 13.97260 1.00000 14 TiB 9.4 GiB 1.1 GiB 2.7 GiB
5.6 GiB 14 TiB 0.07 0.00 300 up osd.13
20 nvme_class 13.97260 1.00000 14 TiB 11 GiB 862 MiB 3.1 GiB
7.2 GiB 14 TiB 0.08 0.00 288 up osd.20
26 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 9 KiB
16 GiB 5.0 TiB 28.58 0.88 108 up osd.26
32 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 9 KiB
16 GiB 5.0 TiB 28.21 0.86 107 up osd.32
41 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 9 KiB
16 GiB 5.0 TiB 28.23 0.86 113 up osd.41
48 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 11 KiB
16 GiB 5.0 TiB 28.19 0.86 114 up osd.48
53 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 9 KiB
15 GiB 5.0 TiB 28.21 0.86 107 up osd.53
61 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 9 KiB
16 GiB 5.0 TiB 28.22 0.86 112 up osd.61
68 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 9 KiB
16 GiB 5.0 TiB 28.19 0.86 107 up osd.68
74 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 9 KiB
16 GiB 5.0 TiB 28.56 0.88 114 up osd.74
82 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 9 KiB
16 GiB 5.0 TiB 27.79 0.85 106 up osd.82
89 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 9 KiB
16 GiB 5.0 TiB 27.85 0.85 108 up osd.89
95 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 11 KiB
16 GiB 5.0 TiB 28.58 0.88 112 up osd.95
104 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 9 KiB
16 GiB 5.0 TiB 27.83 0.85 108 up osd.104
-13 345.49374 - 345 TiB 108 TiB 106 TiB 9.5 GiB
423 GiB 238 TiB 31.16 0.95 - host ceph-host-3
114 hdd_class 18.43300 1.00000 18 TiB 12 TiB 12 TiB 4 KiB
42 GiB 6.3 TiB 66.07 2.02 389 up osd.114
121 hdd_class 18.43300 1.00000 18 TiB 12 TiB 12 TiB 9 KiB
44 GiB 5.9 TiB 67.77 2.08 400 up osd.121
127 hdd_class 18.43300 1.00000 18 TiB 14 TiB 13 TiB 14 KiB
48 GiB 4.7 TiB 74.32 2.28 437 up osd.127
133 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 9 KiB
54 GiB 4.0 TiB 78.33 2.40 461 up osd.133
140 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 2 KiB
50 GiB 4.4 TiB 76.20 2.33 447 up osd.140
143 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 11 KiB
50 GiB 4.4 TiB 76.39 2.34 446 up osd.143
154 hdd_class 18.19040 1.00000 18 TiB 281 GiB 279 GiB 9 KiB
2.3 GiB 18 TiB 1.51 0.05 8 up osd.154
158 hdd_class 18.19040 1.00000 18 TiB 725 GiB 722 GiB 20 KiB
3.0 GiB 17 TiB 3.89 0.12 22 up osd.158
163 hdd_class 18.19040 1.00000 18 TiB 696 GiB 692 GiB 22 KiB
3.2 GiB 18 TiB 3.73 0.11 22 up osd.163
167 hdd_class 18.19040 1.00000 18 TiB 458 GiB 455 GiB 15 KiB
2.4 GiB 18 TiB 2.46 0.08 11 up osd.167
171 hdd_class 18.19040 1.00000 18 TiB 788 GiB 785 GiB 25 KiB
3.1 GiB 17 TiB 4.23 0.13 23 up osd.171
175 hdd_class 18.19040 1.00000 18 TiB 518 GiB 515 GiB 22 KiB
3.0 GiB 18 TiB 2.78 0.09 15 up osd.175
4 nvme_class 13.97260 1.00000 14 TiB 9.1 GiB 1.1 GiB 2.8 GiB
5.2 GiB 14 TiB 0.06 0.00 283 up osd.4
11 nvme_class 13.97260 1.00000 14 TiB 10 GiB 848 MiB 3.0 GiB
6.4 GiB 14 TiB 0.07 0.00 289 up osd.11
19 nvme_class 13.97260 1.00000 14 TiB 9.1 GiB 922 MiB 3.6 GiB
4.6 GiB 14 TiB 0.06 0.00 289 up osd.19
25 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
4.8 GiB 5.0 TiB 28.42 0.87 113 up osd.25
31 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
5.1 GiB 5.0 TiB 28.05 0.86 110 up osd.31
38 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
5.1 GiB 5.1 TiB 27.34 0.84 110 up osd.38
45 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
8.1 GiB 5.0 TiB 27.74 0.85 108 up osd.45
51 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
14 GiB 5.0 TiB 28.16 0.86 111 up osd.51
58 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
12 GiB 5.1 TiB 27.03 0.83 105 up osd.58
65 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
14 GiB 5.1 TiB 27.04 0.83 109 up osd.65
73 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
12 GiB 5.1 TiB 27.03 0.83 109 up osd.73
80 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
4.9 GiB 5.0 TiB 28.41 0.87 110 up osd.80
87 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
4.8 GiB 5.0 TiB 28.42 0.87 114 up osd.87
94 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.3 GiB 5.0 TiB 28.10 0.86 114 up osd.94
101 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.2 GiB 5.0 TiB 28.07 0.86 113 up osd.101
-11 345.49374 - 345 TiB 107 TiB 105 TiB 8.2 GiB
439 GiB 239 TiB 30.95 0.95 - host ceph-host-4
108 hdd_class 18.43300 1.00000 18 TiB 12 TiB 12 TiB 4 KiB
43 GiB 6.1 TiB 67.05 2.05 394 up osd.108
113 hdd_class 18.43300 1.00000 18 TiB 12 TiB 12 TiB 7 KiB
47 GiB 6.3 TiB 65.92 2.02 387 up osd.113
120 hdd_class 18.43300 1.00000 18 TiB 12 TiB 12 TiB 2 KiB
40 GiB 6.2 TiB 66.60 2.04 389 up osd.120
126 hdd_class 18.43300 0.95001 18 TiB 15 TiB 15 TiB 7 KiB
58 GiB 3.6 TiB 80.37 2.46 473 up osd.126
132 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 6 KiB
52 GiB 4.1 TiB 78.00 2.39 457 up osd.132
139 hdd_class 18.43300 1.00000 18 TiB 15 TiB 14 TiB 14 KiB
53 GiB 3.9 TiB 78.87 2.42 463 up osd.139
151 hdd_class 18.19040 1.00000 18 TiB 38 GiB 37 GiB 6 KiB
1.1 GiB 18 TiB 0.20 0.01 1 up osd.151
156 hdd_class 18.19040 1.00000 18 TiB 205 GiB 203 GiB 12 KiB
1.9 GiB 18 TiB 1.10 0.03 6 up osd.156
161 hdd_class 18.19040 1.00000 18 TiB 853 GiB 849 GiB 4 KiB
3.9 GiB 17 TiB 4.58 0.14 26 up osd.161
165 hdd_class 18.19040 1.00000 18 TiB 569 GiB 566 GiB 22 KiB
2.7 GiB 18 TiB 3.05 0.09 15 up osd.165
169 hdd_class 18.19040 1.00000 18 TiB 891 GiB 888 GiB 24 KiB
3.3 GiB 17 TiB 4.78 0.15 25 up osd.169
173 hdd_class 18.19040 1.00000 18 TiB 591 GiB 589 GiB 24 KiB
2.6 GiB 18 TiB 3.17 0.10 19 up osd.173
3 nvme_class 13.97260 1.00000 14 TiB 10 GiB 929 MiB 2.8 GiB
6.3 GiB 14 TiB 0.07 0.00 283 up osd.3
10 nvme_class 13.97260 1.00000 14 TiB 8.7 GiB 872 MiB 2.6 GiB
5.3 GiB 14 TiB 0.06 0.00 271 up osd.10
17 nvme_class 13.97260 1.00000 14 TiB 9.5 GiB 904 MiB 2.8 GiB
5.8 GiB 14 TiB 0.07 0.00 298 up osd.17
24 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
8.3 GiB 5.0 TiB 28.11 0.86 112 up osd.24
33 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
5.1 GiB 5.0 TiB 28.41 0.87 113 up osd.33
40 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
4.8 GiB 5.0 TiB 28.44 0.87 115 up osd.40
47 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
14 GiB 5.1 TiB 27.03 0.83 106 up osd.47
55 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
15 GiB 5.0 TiB 28.22 0.86 108 up osd.55
60 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
14 GiB 5.1 TiB 27.06 0.83 111 up osd.60
67 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
5.3 GiB 5.1 TiB 27.28 0.84 110 up osd.67
75 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
15 GiB 5.1 TiB 27.06 0.83 109 up osd.75
83 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
16 GiB 5.1 TiB 27.05 0.83 105 up osd.83
88 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
5.0 GiB 5.0 TiB 28.42 0.87 109 up osd.88
98 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
4.7 GiB 5.0 TiB 28.41 0.87 114 up osd.98
103 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
4.7 GiB 5.0 TiB 28.07 0.86 107 up osd.103
-3 290.92255 - 291 TiB 104 TiB 103 TiB 8.7 GiB
452 GiB 187 TiB 35.81 1.10 - host ceph-host-5
105 hdd_class 18.43300 1.00000 18 TiB 13 TiB 13 TiB 11 KiB
53 GiB 5.3 TiB 71.10 2.18 417 up osd.105
109 hdd_class 18.43300 0.95001 18 TiB 13 TiB 13 TiB 7 KiB
47 GiB 5.1 TiB 72.56 2.22 428 up osd.109
117 hdd_class 18.43300 0.95001 18 TiB 13 TiB 13 TiB 6 KiB
47 GiB 5.3 TiB 71.49 2.19 422 up osd.117
124 hdd_class 18.43300 1.00000 18 TiB 13 TiB 13 TiB 11 KiB
52 GiB 5.2 TiB 71.87 2.20 424 up osd.124
129 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 2 KiB
50 GiB 4.4 TiB 75.91 2.33 449 up osd.129
136 hdd_class 18.43300 1.00000 18 TiB 14 TiB 13 TiB 11 KiB
49 GiB 4.9 TiB 73.30 2.25 432 up osd.136
181 hdd_class 18.19040 1.00000 18 TiB 109 GiB 108 GiB 7 KiB
846 MiB 18 TiB 0.59 0.02 3 up osd.181
182 hdd_class 18.19040 1.00000 18 TiB 356 GiB 354 GiB 12 KiB
2.6 GiB 18 TiB 1.91 0.06 11 up osd.182
183 hdd_class 18.19040 1.00000 18 TiB 155 GiB 154 GiB 10 KiB
1.8 GiB 18 TiB 0.83 0.03 5 up osd.183
1 nvme_class 13.97260 1.00000 14 TiB 12 GiB 875 MiB 3.2 GiB
7.7 GiB 14 TiB 0.08 0.00 290 up osd.1
7 nvme_class 13.97260 1.00000 14 TiB 7.4 GiB 1.2 GiB 2.6 GiB
3.7 GiB 14 TiB 0.05 0.00 285 up osd.7
14 nvme_class 13.97260 1.00000 14 TiB 11 GiB 837 MiB 3.0 GiB
7.6 GiB 14 TiB 0.08 0.00 290 up osd.14
21 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
5.2 GiB 5.1 TiB 27.24 0.83 106 up osd.21
28 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
15 GiB 5.1 TiB 27.08 0.83 105 up osd.28
35 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
15 GiB 5.1 TiB 27.05 0.83 104 up osd.35
42 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
8.9 GiB 5.0 TiB 27.72 0.85 107 up osd.42
49 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
15 GiB 5.0 TiB 28.19 0.86 112 up osd.49
56 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.2 GiB 5.0 TiB 28.10 0.86 107 up osd.56
63 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.4 GiB 5.0 TiB 28.10 0.86 111 up osd.63
70 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
4.4 GiB 5.0 TiB 28.01 0.86 112 up osd.70
77 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
15 GiB 5.1 TiB 27.06 0.83 109 up osd.77
84 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
15 GiB 5.1 TiB 27.06 0.83 104 up osd.84
90 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
4.9 GiB 5.0 TiB 28.42 0.87 114 up osd.90
96 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
15 GiB 5.1 TiB 27.06 0.83 104 up osd.96
-9 345.49374 - 345 TiB 107 TiB 106 TiB 9.7 GiB
446 GiB 238 TiB 31.02 0.95 - host ceph-host-6
111 hdd_class 18.43300 1.00000 18 TiB 12 TiB 12 TiB 6 KiB
45 GiB 6.4 TiB 65.30 2.00 384 up osd.111
118 hdd_class 18.43300 1.00000 18 TiB 12 TiB 12 TiB 11 KiB
52 GiB 6.0 TiB 67.51 2.07 397 up osd.118
128 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 12 KiB
50 GiB 4.4 TiB 75.96 2.33 445 up osd.128
135 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 11 KiB
48 GiB 4.7 TiB 74.68 2.29 440 up osd.135
142 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 9 KiB
56 GiB 4.1 TiB 77.54 2.38 454 up osd.142
145 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 9 KiB
55 GiB 4.4 TiB 75.98 2.33 445 up osd.145
153 hdd_class 18.19040 1.00000 18 TiB 169 GiB 167 GiB 7 KiB
1.7 GiB 18 TiB 0.90 0.03 5 up osd.153
157 hdd_class 18.19040 1.00000 18 TiB 488 GiB 486 GiB 14 KiB
2.7 GiB 18 TiB 2.62 0.08 15 up osd.157
162 hdd_class 18.19040 1.00000 18 TiB 440 GiB 437 GiB 17 KiB
2.6 GiB 18 TiB 2.36 0.07 11 up osd.162
166 hdd_class 18.19040 1.00000 18 TiB 852 GiB 849 GiB 24 KiB
3.3 GiB 17 TiB 4.57 0.14 25 up osd.166
170 hdd_class 18.19040 1.00000 18 TiB 473 GiB 470 GiB 22 KiB
2.5 GiB 18 TiB 2.54 0.08 14 up osd.170
174 hdd_class 18.19040 1.00000 18 TiB 673 GiB 670 GiB 20 KiB
3.0 GiB 18 TiB 3.61 0.11 21 up osd.174
5 nvme_class 13.97260 1.00000 14 TiB 9.1 GiB 846 MiB 2.9 GiB
5.3 GiB 14 TiB 0.06 0.00 276 up osd.5
12 nvme_class 13.97260 1.00000 14 TiB 12 GiB 873 MiB 3.5 GiB
7.8 GiB 14 TiB 0.08 0.00 296 up osd.12
18 nvme_class 13.97260 1.00000 14 TiB 15 GiB 857 MiB 3.3 GiB
11 GiB 14 TiB 0.10 0.00 288 up osd.18
27 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
8.5 GiB 5.0 TiB 28.44 0.87 114 up osd.27
34 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.3 GiB 5.0 TiB 28.11 0.86 113 up osd.34
39 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.4 GiB 5.0 TiB 28.09 0.86 108 up osd.39
46 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
5.5 GiB 5.1 TiB 27.65 0.85 107 up osd.46
54 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
8.0 GiB 5.1 TiB 27.70 0.85 107 up osd.54
62 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
8.5 GiB 5.0 TiB 28.49 0.87 109 up osd.62
69 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
8.2 GiB 5.0 TiB 28.08 0.86 113 up osd.69
76 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
12 GiB 5.0 TiB 27.78 0.85 111 up osd.76
81 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
5.1 GiB 5.0 TiB 28.46 0.87 115 up osd.81
91 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
15 GiB 5.0 TiB 27.82 0.85 108 up osd.91
97 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 7 KiB
5.0 GiB 5.0 TiB 28.47 0.87 109 up osd.97
102 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.3 GiB 5.0 TiB 28.11 0.86 108 up osd.102
-5 345.49374 - 345 TiB 106 TiB 105 TiB 8.8 GiB
435 GiB 239 TiB 30.73 0.94 - host ceph-host-7
106 hdd_class 18.43300 1.00000 18 TiB 13 TiB 12 TiB 2 KiB
47 GiB 5.8 TiB 68.70 2.10 406 up osd.106
112 hdd_class 18.43300 1.00000 18 TiB 13 TiB 12 TiB 6 KiB
51 GiB 5.8 TiB 68.47 2.10 403 up osd.112
119 hdd_class 18.43300 1.00000 18 TiB 13 TiB 13 TiB 11 KiB
54 GiB 5.7 TiB 69.31 2.12 407 up osd.119
125 hdd_class 18.43300 1.00000 18 TiB 13 TiB 12 TiB 4 KiB
45 GiB 5.7 TiB 68.98 2.11 410 up osd.125
131 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 11 KiB
57 GiB 4.2 TiB 77.12 2.36 454 up osd.131
138 hdd_class 18.43300 1.00000 18 TiB 14 TiB 14 TiB 9 KiB
58 GiB 4.1 TiB 77.85 2.38 456 up osd.138
150 hdd_class 18.19040 1.00000 18 TiB 133 GiB 131 GiB 7 KiB
1.5 GiB 18 TiB 0.71 0.02 4 up osd.150
155 hdd_class 18.19040 1.00000 18 TiB 48 GiB 46 GiB 9 KiB
2.0 GiB 18 TiB 0.26 0.01 2 up osd.155
160 hdd_class 18.19040 1.00000 18 TiB 873 GiB 869 GiB 4 KiB
3.6 GiB 17 TiB 4.69 0.14 27 up osd.160
164 hdd_class 18.19040 1.00000 18 TiB 850 GiB 847 GiB 4 KiB
3.2 GiB 17 TiB 4.57 0.14 25 up osd.164
168 hdd_class 18.19040 1.00000 18 TiB 696 GiB 693 GiB 19 KiB
3.2 GiB 18 TiB 3.74 0.11 17 up osd.168
172 hdd_class 18.19040 1.00000 18 TiB 852 GiB 848 GiB 2 KiB
3.7 GiB 17 TiB 4.57 0.14 25 up osd.172
0 nvme_class 13.97260 1.00000 14 TiB 10 GiB 858 MiB 2.8 GiB
6.3 GiB 14 TiB 0.07 0.00 293 up osd.0
9 nvme_class 13.97260 1.00000 14 TiB 9.3 GiB 847 MiB 2.9 GiB
5.6 GiB 14 TiB 0.07 0.00 281 up osd.9
16 nvme_class 13.97260 1.00000 14 TiB 10 GiB 897 MiB 3.1 GiB
6.3 GiB 14 TiB 0.07 0.00 288 up osd.16
22 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
5.2 GiB 5.1 TiB 27.28 0.84 107 up osd.22
29 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
4.9 GiB 5.0 TiB 28.42 0.87 112 up osd.29
36 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.3 GiB 5.0 TiB 28.08 0.86 109 up osd.36
43 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
5.1 GiB 5.0 TiB 28.42 0.87 115 up osd.43
50 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
5.6 GiB 5.1 TiB 27.28 0.84 111 up osd.50
57 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
5.3 GiB 5.1 TiB 27.32 0.84 112 up osd.57
64 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
4.9 GiB 5.0 TiB 28.45 0.87 113 up osd.64
71 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.4 GiB 5.0 TiB 28.46 0.87 109 up osd.71
78 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 4 KiB
14 GiB 5.1 TiB 27.08 0.83 105 up osd.78
85 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
8.9 GiB 5.0 TiB 28.50 0.87 114 up osd.85
92 ssd_class 6.98630 1.00000 7.0 TiB 2.0 TiB 2.0 TiB 6 KiB
5.1 GiB 5.0 TiB 28.41 0.87 109 up osd.92
99 ssd_class 6.98630 1.00000 7.0 TiB 1.9 TiB 1.9 TiB 6 KiB
12 GiB 5.0 TiB 27.77 0.85 112 up osd.99
TOTAL 2.2 PiB 742 TiB 731 TiB 63 GiB
3.1 TiB 1.5 PiB 32.64
MIN/MAX VAR: 0.00/2.46 STDDEV: 26.17
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
