ZFS wont remove old vdev

Post Reply
zemerdon
Site Admin
Posts: 239
Joined: Mon Jan 23, 2023 8:13 pm

ZFS wont remove old vdev

Post by zemerdon »

ZFS status before fix

Code: Select all

  pool: zfs-raidz3
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Mon Jun 10 15:55:29 2024
        31.0T scanned at 25.2G/s, 1.19T issued at 988M/s, 31.0T total
        174G resilvered, 3.84% done, 08:46:25 to go
config:

        NAME                                    STATE     READ WRITE CKSUM
        zfs-raidz3                              DEGRADED     0     0     0
          raidz3-0                              DEGRADED     0     0     0
            ata-WDC_WUH721414ALE6L4_81G1YDBV    ONLINE       0     0     0
            ata-WDC_WUH721414ALE6L4_9MG6WHWA    ONLINE       0     0     0
            ata-WDC_WUH721414ALE6L4_9MH183EJ    ONLINE       0     0     0
            replacing-3                         DEGRADED     0     0     0
              old                               OFFLINE      0     0     0
              ata-WDC_WUH721414ALE6L4_QGGJ0DAT  ONLINE       0     0     0  (resilvering)
            ata-WDC_WUH721414ALE6L4_Y5HK756C    ONLINE       0     0     0  (awaiting resilver)
            ata-WDC_WUH721414ALE6L4_Y5KTHK8C    ONLINE       0     0     0  (resilvering)
            ata-WDC_WUH721414ALE6L4_Y6GT6PWD    ONLINE       0     0     0
zbd output to find disk identifiers

Code: Select all

root@NAS:/home/zemerdon# zdb
zfs-raidz2:
    version: 5000
    name: 'zfs-raidz2'
    state: 0
    txg: 2573542
    pool_guid: 7967553646632217524
    errata: 0
    hostid: 1260664462
    hostname: 'NAS'
    com.delphix:has_per_vdev_zaps
    vdev_children: 1
    vdev_tree:
        type: 'root'
        id: 0
        guid: 7967553646632217524
        create_txg: 4
        children[0]:
            type: 'raidz'
            id: 0
            guid: 12116301508942199509
            nparity: 2
            metaslab_array: 256
            metaslab_shift: 34
            ashift: 12
            asize: 48000493682688
            is_log: 0
            create_txg: 4
            com.delphix:vdev_zap_top: 129
            children[0]:
                type: 'disk'
                id: 0
                guid: 5852806432283670735
                path: '/dev/disk/by-id/ata-WDC_WD120EFBX-68B0EN0_5QKSYWVB-part1'
                devid: 'ata-WDC_WD120EFBX-68B0EN0_5QKSYWVB-part1'
                phys_path: 'pci-0000:00:17.0-ata-5.0'
                whole_disk: 1
                DTL: 11686
                create_txg: 4
                com.delphix:vdev_zap_leaf: 130
            children[1]:
                type: 'disk'
                id: 1
                guid: 15643486114837770365
                path: '/dev/disk/by-id/ata-WDC_WD120EFBX-68B0EN0_5QKT3BPB-part1'
                devid: 'ata-WDC_WD120EFBX-68B0EN0_5QKT3BPB-part1'
                phys_path: 'pci-0000:00:17.0-ata-6.0'
                whole_disk: 1
                DTL: 11685
                create_txg: 4
                com.delphix:vdev_zap_leaf: 131
            children[2]:
                type: 'disk'
                id: 2
                guid: 1841540730203153095
                path: '/dev/disk/by-id/ata-WDC_WD120EFBX-68B0EN0_5QKT3USB-part1'
                devid: 'ata-WDC_WD120EFBX-68B0EN0_5QKT3USB-part1'
                phys_path: 'pci-0000:19:00.0-sas-phy1-lun-0'
                whole_disk: 1
                DTL: 11684
                create_txg: 4
                com.delphix:vdev_zap_leaf: 132
            children[3]:
                type: 'disk'
                id: 3
                guid: 14280781180547498456
                path: '/dev/disk/by-id/ata-WDC_WD120EFBX-68B0EN0_5QKT8B9B-part1'
                devid: 'ata-WDC_WD120EFBX-68B0EN0_5QKT8B9B-part1'
                phys_path: 'pci-0000:00:17.0-ata-8.0'
                whole_disk: 1
                DTL: 11683
                create_txg: 4
                com.delphix:vdev_zap_leaf: 133
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
zfs-raidz3:
    version: 5000
    name: 'zfs-raidz3'
    state: 0
    txg: 1602256
    pool_guid: 12544808713034463427
    errata: 0
    hostid: 1260664462
    hostname: 'NAS'
    com.delphix:has_per_vdev_zaps
    vdev_children: 1
    vdev_tree:
        type: 'root'
        id: 0
        guid: 12544808713034463427
        create_txg: 4
        children[0]:
            type: 'raidz'
            id: 0
            guid: 12989879130817016843
            nparity: 3
            metaslab_array: 256
            metaslab_shift: 34
            ashift: 12
            asize: 98003531071488
            is_log: 0
            create_txg: 4
            com.delphix:vdev_zap_top: 129
            children[0]:
                type: 'disk'
                id: 0
                guid: 1737205048579848810
                path: '/dev/disk/by-id/ata-WDC_WUH721414ALE6L4_81G1YDBV-part1'
                devid: 'ata-WDC_WUH721414ALE6L4_81G1YDBV-part1'
                phys_path: 'pci-0000:19:00.0-sas-exp0x5001e677c0a55fff-phy19-lun-0'
                whole_disk: 1
                DTL: 7968
                create_txg: 4
                com.delphix:vdev_zap_leaf: 130
            children[1]:
                type: 'disk'
                id: 1
                guid: 8650379791404448481
                path: '/dev/disk/by-id/ata-WDC_WUH721414ALE6L4_9MG6WHWA-part1'
                devid: 'ata-WDC_WUH721414ALE6L4_9MG6WHWA-part1'
                phys_path: 'pci-0000:19:00.0-sas-exp0x5001e677c0a55fff-phy14-lun-0'
                whole_disk: 1
                DTL: 7967
                create_txg: 4
                com.delphix:vdev_zap_leaf: 131
            children[2]:
                type: 'disk'
                id: 2
                guid: 2194983220020267303
                path: '/dev/disk/by-id/ata-WDC_WUH721414ALE6L4_9MH183EJ-part1'
                devid: 'ata-WDC_WUH721414ALE6L4_9MH183EJ-part1'
                phys_path: 'pci-0000:19:00.0-sas-exp0x5001e677c0a55fff-phy18-lun-0'
                whole_disk: 1
                DTL: 7964
                create_txg: 4
                com.delphix:vdev_zap_leaf: 132
            children[3]:
                type: 'replacing'
                id: 3
                guid: 11212340955531754205
                whole_disk: 0
                create_txg: 4
                children[0]:
                    type: 'disk'
                    id: 0
                    guid: 7480485599434468757
                    path: '/dev/disk/by-id/ata-WDC_WUH721414ALE6L4_QGGJ0DAT-part1/old'
                    phys_path: 'pci-0000:19:00.0-sas-exp0x5001e677c0a55fff-phy16-lun-0'
                    whole_disk: 1
                    DTL: 7961
                    create_txg: 4
                    com.delphix:vdev_zap_leaf: 133
                    offline: 1
                children[1]:
                    type: 'disk'
                    id: 1
                    guid: 4478615572646404198
                    path: '/dev/disk/by-id/ata-WDC_WUH721414ALE6L4_QGGJ0DAT-part1'
                    devid: 'ata-WDC_WUH721414ALE6L4_QGGJ0DAT-part1'
                    phys_path: 'pci-0000:19:00.0-sas-phy3-lun-0'
                    whole_disk: 1
                    DTL: 910
                    create_txg: 4
                    com.delphix:vdev_zap_leaf: 908
                    resilver_txg: 1584677
            children[4]:
                type: 'disk'
                id: 4
                guid: 2118638491481543425
                path: '/dev/disk/by-id/ata-WDC_WUH721414ALE6L4_Y5HK756C-part1'
                devid: 'ata-WDC_WUH721414ALE6L4_Y5HK756C-part1'
                phys_path: 'pci-0000:19:00.0-sas-exp0x5001e677c0a55fff-phy12-lun-0'
                whole_disk: 1
                DTL: 7960
                create_txg: 4
                com.delphix:vdev_zap_leaf: 134
                com.datto:resilver_defer
            children[5]:
                type: 'disk'
                id: 5
                guid: 10703400751132370287
                path: '/dev/disk/by-id/ata-WDC_WUH721414ALE6L4_Y5KTHK8C-part1'
                devid: 'ata-WDC_WUH721414ALE6L4_Y5KTHK8C-part1'
                phys_path: 'pci-0000:19:00.0-sas-exp0x5001e677c0a55fff-phy13-lun-0'
                whole_disk: 1
                DTL: 7959
                create_txg: 4
                com.delphix:vdev_zap_leaf: 135
            children[6]:
                type: 'disk'
                id: 6
                guid: 4797841653521964094
                path: '/dev/disk/by-id/ata-WDC_WUH721414ALE6L4_Y6GT6PWD-part1'
                devid: 'ata-WDC_WUH721414ALE6L4_Y6GT6PWD-part1'
                phys_path: 'pci-0000:19:00.0-sas-exp0x5001e677c0a55fff-phy17-lun-0'
                whole_disk: 1
                DTL: 7958
                create_txg: 4
                com.delphix:vdev_zap_leaf: 136
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
root@NAS:/home/zemerdon# 
command to remove UNAVAIL vdev

Code: Select all

zemerdon@NAS:~$ sudo zpool detach zfs-raidz3 7480485599434468757
ZFS status after successful vdev removal

Code: Select all

zemerdon@NAS:~$ zpool status
  pool: zfs-raidz2
 state: ONLINE
  scan: scrub repaired 0B in 14:11:03 with 0 errors on Sun Jun  9 14:35:04 2024
config:

        NAME                                    STATE     READ WRITE CKSUM
        zfs-raidz2                              ONLINE       0     0     0
          raidz2-0                              ONLINE       0     0     0
            ata-WDC_WD120EFBX-68B0EN0_5QKSYWVB  ONLINE       0     0     0
            ata-WDC_WD120EFBX-68B0EN0_5QKT3BPB  ONLINE       0     0     0
            ata-WDC_WD120EFBX-68B0EN0_5QKT3USB  ONLINE       0     0     0
            ata-WDC_WD120EFBX-68B0EN0_5QKT8B9B  ONLINE       0     0     0

errors: No known data errors

  pool: zfs-raidz3
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Mon Jun 10 15:55:29 2024
        31.0T scanned at 20.8G/s, 1.53T issued at 1.03G/s, 31.0T total
        224G resilvered, 4.94% done, 08:08:21 to go
config:

        NAME                                  STATE     READ WRITE CKSUM
        zfs-raidz3                            ONLINE       0     0     0
          raidz3-0                            ONLINE       0     0     0
            ata-WDC_WUH721414ALE6L4_81G1YDBV  ONLINE       0     0     0
            ata-WDC_WUH721414ALE6L4_9MG6WHWA  ONLINE       0     0     0
            ata-WDC_WUH721414ALE6L4_9MH183EJ  ONLINE       0     0     0
            ata-WDC_WUH721414ALE6L4_QGGJ0DAT  ONLINE       0     0     0  (resilvering)
            ata-WDC_WUH721414ALE6L4_Y5HK756C  ONLINE       0     0     0  (awaiting resilver)
            ata-WDC_WUH721414ALE6L4_Y5KTHK8C  ONLINE       0     0     0  (resilvering)
            ata-WDC_WUH721414ALE6L4_Y6GT6PWD  ONLINE       0     0     0
Post Reply