[OpenIndiana-discuss] in desperate need of fsck.zfs

Lucas Van Tol catseyev9 at hotmail.com
Mon Jul 23 13:01:26 UTC 2012


Maybe the disks are faild by fmd?   It will often prevent a system from using disks it thinks have failed; even if its a new disk in the same slot.   Check with 'fmadm faulty' anc see if you can convince fmd they are repaired/replaced/acquitted.   That may make them visible to the pool again, although that might not help much at this point.

-Lucas Van Tol

Ray Arachelian <ray at arachelian.com> wrote:

I think it's high time we get an fsck.zfs tool.  While attempting to
copy data from one zpool to another, I ran across a broken file that
caused rsync to pause and all access to the zpool to lock up any command
that touched it.  I then ran a scrub, and after a few days, the machine
crashed.  I brought it back up and it continued the scrub just fine
until this morning.

At this point, it's claiming too many I/O errors, but iostat doesn't
show this (except for one transport failure), and yet, I cannot get it
to clear, even with zpool clear -F!  What do I do to fix this?

I'd be happy to mount the volume on this zpool read only and force an on
error return so I can move the readable data off it... but, it's
claiming there's too many errors.

I suspect the issue is that since this is a USB enclosure that at some
point the bus was saturated and wouldn't respond, or something like that.
Any help in getting these volumes mounted would be greatly appreciated.

Yes, I know, I'm running on crappy drives over USB, I was trying to
migrate off of them when this happened. :(


root at harvest:/export/home/ray# zpool status -v data
  pool: data
 state: UNAVAIL
status: One or more devices are faulted in response to IO failures.
action: Make sure the affected devices are connected, then run 'zpool
clear'.
   see: http://www.sun.com/msg/ZFS-8000-JQ
  scan: scrub in progress since Wed Jul 18 22:57:35 2012
    3.10T scanned out of 6.08T at 41.6K/s, (scan is slow, no estimated time)
    0 repaired, 50.93% done
config:

        NAME         STATE     READ WRITE CKSUM
        data         UNAVAIL      0     0     0  insufficient replicas
          raidz1-0   UNAVAIL      0     0     0  insufficient replicas
            c22t0d4  FAULTED     18     0     0  too many errors
            c22t0d5  FAULTED     15     0     0  too many errors
            c22t0d6  FAULTED     12     0     0  too many errors
            c22t0d7  FAULTED      9     0     0  too many errors

errors: Permanent errors have been detected in the following files:

        <metadata>:<0x0>
        <metadata>:<0x1>
        <metadata>:<0x1b>
        <metadata>:<0x2d>
        <metadata>:<0x4d>
        <metadata>:<0x8d>
        <metadata>:<0x8f>
        <metadata>:<0x90>
        data:<0x0>
        data:<0x211632>
        data:<0xfffffffffffffffe>
        data:<0xffffffffffffffff>
root at harvest:/export/home/ray# zpool clear data
cannot clear errors for data: I/O error
You have mail in /var/mail/root
root at harvest:/export/home/ray# uname -a
SunOS harvest 5.11 oi_151a i86pc i386 i86pc Solaris
root at harvest:/export/home/ray# cat /etc/release
                      OpenIndiana Development oi_151a X86
        Copyright 2010 Oracle and/or its affiliates. All rights reserved.
                        Use is subject to license terms.
                           Assembled 01 September 2011


Vendor: WDC WD20 Product: EARS-00MVWB0     Revision:  Serial No:
Size: 2000.40GB <2000398934016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 41 Predictive Failure Analysis: 0
c22t0d5          Soft Errors: 0 Hard Errors: 0 Transport Errors: 1
Vendor: WDC WD20 Product: EARS-00MVWB0     Revision:  Serial No:
Size: 2000.40GB <2000398934016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 41 Predictive Failure Analysis: 0
c22t0d6          Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: WDC WD20 Product: EARS-00S8B1      Revision:  Serial No:
Size: 2000.40GB <2000398934016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 41 Predictive Failure Analysis: 0
c22t0d7          Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: WDC WD20 Product: EARS-00S8B1      Revision:  Serial No:
Size: 2000.40GB <2000398934016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 41 Predictive Failure Analysis: 0

Here are the crashes:

1st one was on 7/19:
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc
pcplusmp scsi_vhci zfs ip hook neti sockfs arp usba uhci sd fctl
stmf_sbd stmf lofs idm sata random crypto cpc fcip ufs logindmux ptm
nsmb sppp nfs ]
> ::status
debugging crash dump vmcore.53 (64-bit) from harvest
operating system: 5.11 oi_151a (i86pc)
image uuid: 5a02d959-1733-608a-9a13-cd014e93a6c2
panic message:
assertion failed: 0 == zap_update(scn->scn_dp->dp_meta_objset,
DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_SCAN, sizeof (uint64_t),
SCAN_PHYS_NUMINTS, &scn->scn_phys, tx), file: ../../common/fs/zfs/dsl_scan.
dump content: kernel pages only
> ::stack
vpanic()
assfail+0x7e(fffffffff7a5c708, fffffffff7a5c7e0, 1c1)
dsl_scan_sync_state+0x64(ffffff01cc6ff780, ffffff01f2b76b48)
dsl_scan_sync+0x2d3(ffffff01cf7bf680, ffffff01f2b76b48)
spa_sync+0x3e7(ffffff01ce0ccb00, 2f83d02)
txg_sync_thread+0x247(ffffff01cf7bf680)
thread_start+8()
> ::messagebuf
> ::msgbuf
MESSAGE
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,6 (sd57) online
sd58 at scsa2usb13: target 0 lun 7
sd58 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,7
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,7 (sd58) online
pseudo-device: audio0
audio0 is /pseudo/audio at 0
pseudo-device: stmf_sbd0
stmf_sbd0 is /pseudo/stmf_sbd at 0
PCI Express-device: pci8086,244e at 1e, pci_pci1
pci_pci1 is /pci at 0,0/pci8086,244e at 1e
NOTICE: e1000g0 registered
pcplusmp: pci8086,107c (e1000g) instance 0 irq 0x14 vector 0x60 ioapic
0x2 intin 0x14 is bound to cpu 0
pseudo-device: devinfo0
devinfo0 is /pseudo/devinfo at 0
pseudo-device: zfs0
zfs0 is /pseudo/zfs at 0
pcplusmp: ide (ata) instance 3 irq 0xe vector 0x40 ioapic 0x2 intin 0xe
is bound to cpu 1
iscsi0 at root
iscsi0 is /iscsi
NOTICE: e1000g0 link up, 1000 Mbps, full duplex
pcplusmp: ide (ata) instance 4 irq 0xf vector 0x40 ioapic 0x2 intin 0xf
is bound to cpu 0
NOTICE: IRQ19 is being shared by drivers with different interrupt levels.
This may result in reduced system performance.
This Solaris instance has UUID 5a02d959-1733-608a-9a13-cd014e93a6c2
dump on /dev/zvol/dsk/rpool/dump size 2039 MB
PCI Express-device: pci1458,a002 at 1b, audiohd1
audiohd1 is /pci at 0,0/pci1458,a002 at 1b
pseudo-device: pm0
pm0 is /pseudo/pm at 0
xsvc0 at root: space 0 offset 0
xsvc0 is /xsvc at 0,0
device pci8086,29c2 at 2(display#0) keeps up device sd at 0,0(disk#33), but
the former is not power managed
pseudo-device: power0
power0 is /pseudo/power at 0
pseudo-device: srn0
srn0 is /pseudo/srn at 0
pcplusmp: ide (ata) instance 3 irq 0xe vector 0x40 ioapic 0x2 intin 0xe
is bound to cpu 0
pcplusmp: asy (asy) instance 0 irq 0x4 vector 0xb0 ioapic 0x2 intin 0x4
is bound to cpu 1
ISA-device: asy0
asy0 is /pci at 0,0/isa at 1f/asy at 1,3f8
pseudo-device: pseudo1
pseudo1 is /pseudo/zconsnex at 1
acpinex: sb at 0, acpinex1
acpinex1 is /fw/sb at 0
pcplusmp: lp (ecpp) instance 0 irq 0x7 vector 0x43 ioapic 0x2 intin 0x7
is bound to cpu 1
ISA-device: ecpp0
ecpp0 is /pci at 0,0/isa at 1f/lp at 1,378
ISA-device: pit_beep0
pit_beep0 is /pci at 0,0/isa at 1f/pit_beep
pseudo-device: llc10
llc10 is /pseudo/llc1 at 0
pseudo-device: lofi0
lofi0 is /pseudo/lofi at 0
pseudo-device: ramdisk1024
ramdisk1024 is /pseudo/ramdisk at 1024
pseudo-device: ucode0
ucode0 is /pseudo/ucode at 0
pcplusmp: ide (ata) instance 4 irq 0xf vector 0x40 ioapic 0x2 intin 0xf
is bound to cpu 0
pseudo-device: nvidia255
nvidia255 is /pseudo/nvidia at 255
pseudo-device: fcp0
fcp0 is /pseudo/fcp at 0
pseudo-device: fcsm0
fcsm0 is /pseudo/fcsm at 0
pseudo-device: dcpc0
dcpc0 is /pseudo/dcpc at 0
pseudo-device: dtrace0
dtrace0 is /pseudo/dtrace at 0
pseudo-device: fasttrap0
fasttrap0 is /pseudo/fasttrap at 0
pseudo-device: fbt0
fbt0 is /pseudo/fbt at 0
pseudo-device: lockstat0
lockstat0 is /pseudo/lockstat at 0
pseudo-device: profile0
profile0 is /pseudo/profile at 0
pseudo-device: sdt0
sdt0 is /pseudo/sdt at 0
pseudo-device: systrace0
systrace0 is /pseudo/systrace at 0
pseudo-device: stmf0
stmf0 is /pseudo/stmf at 0
pseudo-device: fct0
fct0 is /pseudo/fct at 0
pseudo-device: fssnap0
fssnap0 is /pseudo/fssnap at 0
pseudo-device: pool0
pool0 is /pseudo/pool at 0
pseudo-device: bpf0
bpf0 is /pseudo/bpf at 0
pseudo-device: winlock0
winlock0 is /pseudo/winlock at 0
pseudo-device: nsmb0
nsmb0 is /pseudo/nsmb at 0
IP Filter: v4.1.9, running.
pcplusmp: ide (ata) instance 3 irq 0xe vector 0x40 ioapic 0x2 intin 0xe
is bound to cpu 1
pcplusmp: ide (ata) instance 4 irq 0xf vector 0x40 ioapic 0x2 intin 0xf
is bound to cpu 0
pcplusmp: ide (ata) instance 3 irq 0xe vector 0x40 ioapic 0x2 intin 0xe
is bound to cpu 1
pcplusmp: ide (ata) instance 4 irq 0xf vector 0x40 ioapic 0x2 intin 0xf
is bound to cpu 0
/pseudo/zconsnex at 1/zcons at 0 (zcons0) online
pseudo-device: devinfo0
devinfo0 is /pseudo/devinfo at 0
pcplusmp: ide (ata) instance 3 irq 0xe vector 0x40 ioapic 0x2 intin 0xe
is bound to cpu 1
pcplusmp: ide (ata) instance 4 irq 0xf vector 0x40 ioapic 0x2 intin 0xf
is bound to cpu 0
pseudo-device: bpf0
bpf0 is /pseudo/bpf at 0
pseudo-device: lofi0
lofi0 is /pseudo/lofi at 0
pseudo-device: nsmb0
nsmb0 is /pseudo/nsmb at 0
WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,6 (sd57):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,7 (sd58):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,2 (sd53):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,4 (sd55):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,5 (sd56):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,3 (sd54):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,1 (sd45):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,0 (sd44):
        Command failed to complete...Device is gone


panic[cpu0]/thread=ffffff00087a2c40:
assertion failed: 0 == zap_update(scn->scn_dp->dp_meta_objset,
DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_SCAN, sizeof (uint64_t),
SCAN_PHYS_NUMINTS, &scn->scn_phys, tx), file:
../../common/fs/zfs/dsl_scan.c,
line: 449


ffffff00087a2a50 genunix:assfail+7e ()
ffffff00087a2a70 zfs:dsl_scan_sync_state+64 ()
ffffff00087a2ad0 zfs:dsl_scan_sync+2d3 ()
ffffff00087a2b80 zfs:spa_sync+3e7 ()
ffffff00087a2c20 zfs:txg_sync_thread+247 ()
ffffff00087a2c30 unix:thread_start+8 ()

syncing file systems...
 done
dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel
> ::cpuinfo -v
 ID ADDR             FLG NRUN BSPL PRI RNRN KRNRN SWITCH
THREAD           PROC
  0 fffffffffbc3ac50  1b    0    0  60   no    no t-0
ffffff00087a2c40 sched
                       |
            RUNNING <--+
              READY
             EXISTS
             ENABLE

 ID ADDR             FLG NRUN BSPL PRI RNRN KRNRN SWITCH
THREAD           PROC
  1 ffffff01cecd1540  1f    2    0  -1   no    no t-0
ffffff0007c7ac40 (idle)
                       |    |
            RUNNING <--+    +-->  PRI THREAD           PROC
              READY                99 ffffff000811ec40 zpool-rpool
           QUIESCED                60 ffffff00097afc40 sched
             EXISTS
             ENABLE

> ::panicinfo
             cpu                0
          thread ffffff00087a2c40
         message
assertion failed: 0 == zap_update(scn->scn_dp->dp_meta_objset,
DMU_POOL_DIRECTORY_OBJECT, DMU_POOL_SCAN, sizeof (uint64_t),
SCAN_PHYS_NUMINTS, &scn->scn_phys, tx), file:
../../common/fs/zfs/dsl_scan.c,
line: 449
             rdi fffffffffbbe32c0
             rsi ffffff00087a29e0
             rdx fffffffff7a5c7e0
             rcx              1c1
              r8 ffffff00087a2930
              r9 ffffff00087a2a20
             rax                0
             rbx              1c1
             rbp ffffff00087a2a10
             r10 c7b4df67293fadc7
             r10 c7b4df67293fadc7
             r11  3fffffc07375225
             r12 fffffffff7a5c7e0
             r13 fffffffff7a5c708
             r14 ffffff01f2b76b48
             r15 ffffff01ce0cd0c8
          fsbase                0
          gsbase fffffffffbc304a0
              ds               4b
              es               4b
              fs                0
              gs              1c3
          trapno                0
             err                0
             rip fffffffffb86c870
              cs               30
          rflags              246
             rsp ffffff00087a2928
              ss               38
          gdt_hi                0
          gdt_lo         e00001ef
          idt_hi                0
          idt_lo         d0000fff
             ldt                0
            task               70
             cr0         8005003b
             cr2          81238a0
             cr3          4400000
             cr4            406f8
>



Here's the 2nd crash, this one from this morning.


root at harvest:/var/crash/harvest# printf
"::status\n::stack\n::msgbuf\n::cpuinfo -v\n::panicinfo\n" | mdb -k
unix.54 vmcore.54 | tee output.54.txt
debugging crash dump vmcore.54 (64-bit) from harvest
operating system: 5.11 oi_151a (i86pc)
image uuid: 1f9ce548-7e5d-c4dd-b1b6-dbd2b383a6d1
panic message:
assertion failed: 0 == zap_update(scn->scn_dp->dp_meta_objset,
DMU_POOL_DIRECTOR
Y_OBJECT, DMU_POOL_SCAN, sizeof (uint64_t), SCAN_PHYS_NUMINTS,
&scn->scn_phys, t
x), file: ../../common/fs/zfs/dsl_scan.
dump content: kernel pages only
vpanic()
assfail+0x7e(fffffffff7a5c708, fffffffff7a5c7e0, 1c1)
dsl_scan_sync_state+0x64(ffffff01cf49d800, ffffff01f6c0b6f0)
dsl_scan_sync+0x2d3(ffffff01ef09cc80, ffffff01f6c0b6f0)
spa_sync+0x3e7(ffffff01cdd94b00, 2f88331)
txg_sync_thread+0x247(ffffff01ef09cc80)
thread_start+8()
MESSAGE
pseudo-device: fcp0
fcp0 is /pseudo/fcp at 0
pseudo-device: fcsm0
fcsm0 is /pseudo/fcsm at 0
pseudo-device: dcpc0
dcpc0 is /pseudo/dcpc at 0
pseudo-device: dtrace0
dtrace0 is /pseudo/dtrace at 0
pseudo-device: fasttrap0
fasttrap0 is /pseudo/fasttrap at 0
pseudo-device: fbt0
fbt0 is /pseudo/fbt at 0
pseudo-device: lockstat0
lockstat0 is /pseudo/lockstat at 0
pseudo-device: profile0
profile0 is /pseudo/profile at 0
pseudo-device: sdt0
sdt0 is /pseudo/sdt at 0
pseudo-device: systrace0
systrace0 is /pseudo/systrace at 0
pseudo-device: stmf0
stmf0 is /pseudo/stmf at 0
pseudo-device: fct0
fct0 is /pseudo/fct at 0
pseudo-device: fssnap0
fssnap0 is /pseudo/fssnap at 0
pseudo-device: pool0
pool0 is /pseudo/pool at 0
pseudo-device: bpf0
bpf0 is /pseudo/bpf at 0
pseudo-device: winlock0
winlock0 is /pseudo/winlock at 0
pseudo-device: nsmb0
nsmb0 is /pseudo/nsmb at 0
IP Filter: v4.1.9, running.
pcplusmp: ide (ata) instance 3 irq 0xe vector 0x40 ioapic 0x2 intin 0xe
is bound
 to cpu 1
pcplusmp: ide (ata) instance 4 irq 0xf vector 0x40 ioapic 0x2 intin 0xf
is bound
 to cpu 0
/pseudo/zconsnex at 1/zcons at 0 (zcons0) online
pcplusmp: ide (ata) instance 3 irq 0xe vector 0x40 ioapic 0x2 intin 0xe
is bound
 to cpu 1
pcplusmp: ide (ata) instance 4 irq 0xf vector 0x40 ioapic 0x2 intin 0xf
is bound
 to cpu 0
pseudo-device: devinfo0
devinfo0 is /pseudo/devinfo at 0
pseudo-device: bpf0
bpf0 is /pseudo/bpf at 0
pseudo-device: lofi0
lofi0 is /pseudo/lofi at 0
pseudo-device: nsmb0
nsmb0 is /pseudo/nsmb at 0
xsvc0 at root: space 0 offset 0
xsvc0 is /xsvc at 0,0
WARNING: i915_set_status_page set status page addr 0x0082b000

pcplusmp: lp (ecpp) instance 0 irq 0x7 vector 0x40 ioapic 0x2 intin 0x7
is bound
 to cpu 1
ISA-device: ecpp0
ecpp0 is /pci at 0,0/isa at 1f/lp at 1,378
pcplusmp: asy (asy) instance 0 irq 0x4 vector 0xb0 ioapic 0x2 intin 0x4
is bound
 to cpu 0
ISA-device: asy0
asy0 is /pci at 0,0/isa at 1f/asy at 1,3f8
pseudo-device: ucode0
ucode0 is /pseudo/ucode at 0
pcplusmp: ide (ata) instance 3 irq 0xe vector 0x43 ioapic 0x2 intin 0xe
is bound
 to cpu 1
pseudo-device: llc10
llc10 is /pseudo/llc1 at 0
pseudo-device: ramdisk1024
ramdisk1024 is /pseudo/ramdisk at 1024
pseudo-device: nvidia255
nvidia255 is /pseudo/nvidia at 255
pseudo-device: fcp0
fcp0 is /pseudo/fcp at 0
pseudo-device: fcsm0
fcsm0 is /pseudo/fcsm at 0
pseudo-device: dcpc0
dcpc0 is /pseudo/dcpc at 0
pseudo-device: fbt0
fbt0 is /pseudo/fbt at 0
pseudo-device: lockstat0
lockstat0 is /pseudo/lockstat at 0
pseudo-device: profile0
profile0 is /pseudo/profile at 0
pseudo-device: sdt0
sdt0 is /pseudo/sdt at 0
pseudo-device: systrace0
systrace0 is /pseudo/systrace at 0
pseudo-device: stmf0
stmf0 is /pseudo/stmf at 0
pseudo-device: fct0
fct0 is /pseudo/fct at 0
pseudo-device: fssnap0
fssnap0 is /pseudo/fssnap at 0
pseudo-device: winlock0
winlock0 is /pseudo/winlock at 0
pseudo-device: pm0
pm0 is /pseudo/pm at 0
pcplusmp: ide (ata) instance 4 irq 0xf vector 0x43 ioapic 0x2 intin 0xf
is bound
 to cpu 0
device pci8086,29c2 at 2(display#0) keeps up device sd at 0,0(disk#33), but
the former
 is not power managed
USB 2.0 device (usb152d,551) operating at hi speed (USB 2.x) on USB 2.0
root hub
: storage at 5, scsa2usb13 at bus address 3
        JMicron USB to ATA/ATAPI Bridge 1F08FFFFFFFF
scsa2usb13 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5 (scsa2usb13) online
sd44 at scsa2usb13: target 0 lun 0
sd44 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,0
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,0 (sd44) online
sd45 at scsa2usb13: target 0 lun 1
sd45 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,1
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,1 (sd45) online
sd53 at scsa2usb13: target 0 lun 2
sd53 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,2
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,2 (sd53) online
sd54 at scsa2usb13: target 0 lun 3
sd54 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,3
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,3 (sd54) online
sd55 at scsa2usb13: target 0 lun 4
sd55 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,4
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,4 (sd55) online
sd56 at scsa2usb13: target 0 lun 5
sd56 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,5
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,5 (sd56) online
sd57 at scsa2usb13: target 0 lun 6
sd57 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,6
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,6 (sd57) online
sd58 at scsa2usb13: target 0 lun 7
sd58 is /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,7
/pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,7 (sd58) online
WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,6 (sd57):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,7 (sd58):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,4 (sd55):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,5 (sd56):
        Command failed to complete...Device is gone

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,4 (sd55):
        drive offline

WARNING: /pci at 0,0/pci1458,5006 at 1d,7/storage at 5/disk at 0,5 (sd56):
        drive offline


panic[cpu1]/thread=ffffff0009ff5c40:
assertion failed: 0 == zap_update(scn->scn_dp->dp_meta_objset,
DMU_POOL_DIRECTOR
Y_OBJECT, DMU_POOL_SCAN, sizeof (uint64_t), SCAN_PHYS_NUMINTS,
&scn->scn_phys, t
x), file: ../../common/fs/zfs/dsl_scan.c, line: 449


ffffff0009ff5a50 genunix:assfail+7e ()
ffffff0009ff5a70 zfs:dsl_scan_sync_state+64 ()
ffffff0009ff5ad0 zfs:dsl_scan_sync+2d3 ()
ffffff0009ff5b80 zfs:spa_sync+3e7 ()
ffffff0009ff5c20 zfs:txg_sync_thread+247 ()
ffffff0009ff5c30 unix:thread_start+8 ()

syncing file systems...
 done
dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel
 ID ADDR             FLG NRUN BSPL PRI RNRN KRNRN SWITCH
THREAD           PROC
  0 fffffffffbc304a0  1f    3    0  59  yes    no t-0    ffffff01df075b40
hald-addon-stora
                       |    |
            RUNNING <--+    +-->  PRI THREAD           PROC
              READY                99 ffffff000811ec40 zpool-rpool
           QUIESCED                60 ffffff0009c1dc40 sched
             EXISTS                59 ffffff01cf893c00 syslogd
             ENABLE

 ID ADDR             FLG NRUN BSPL PRI RNRN KRNRN SWITCH
THREAD           PROC
  1 fffffffffbc3ac50  1b    3    0  60   no    no t-0
ffffff0009ff5c40 sched
                       |    |
            RUNNING <--+    +-->  PRI THREAD           PROC
              READY                99 ffffff0007b8bc40 zpool-rpool
             EXISTS                59 ffffff01df06c420 zpool
             ENABLE                 0 ffffff0007d19c40 zpool-rpool

             cpu                1
          thread ffffff0009ff5c40
         message
assertion failed: 0 == zap_update(scn->scn_dp->dp_meta_objset,
DMU_POOL_DIRECTOR
Y_OBJECT, DMU_POOL_SCAN, sizeof (uint64_t), SCAN_PHYS_NUMINTS,
&scn->scn_phys, t
x), file: ../../common/fs/zfs/dsl_scan.c, line: 449
             rdi fffffffffbbe32c0
             rsi ffffff0009ff59e0
             rdx fffffffff7a5c7e0
             rcx              1c1
              r8 ffffff0009ff5930
              r9 ffffff0009ff5a20
             rax                0
             rbx              1c1
             rbp ffffff0009ff5a10
             r10 36fc75617844180f
             r10 36fc75617844180f
             r11  3fffffc07d64f73
             r12 fffffffff7a5c7e0
             r13 fffffffff7a5c708
             r14 ffffff01f6c0b6f0
             r15 ffffff01cdd950c8
          fsbase                0
          gsbase ffffff01ce9ba000
              ds               4b
              es               4b
              fs                0
              gs              1c3
          trapno                0
             err                0
             rip fffffffffb86c870
              cs               30
          rflags              246
             rsp ffffff0009ff5928
              ss               38
          gdt_hi                0
          gdt_lo              1ef
          idt_hi                0
          idt_lo         80000fff
             ldt                0
            task               70
             cr0         8005003b
             cr2         feb23228
             cr3          4400000
             cr4            406f8
root at harvest:/var/crash/harvest#




_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss at openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



More information about the OpenIndiana-discuss mailing list