[OpenIndiana-discuss] zpool errors a bit confusing

[Harry Putnam]

I don't have enough experience to know if the `zpool status -v' errors
I'm seeing are serious or what.  So, went to the recommended site:
illumos.org/msg/ZFS-800-8A

Which left me with the impression that I must be seeing something
unusual. 

It appears from the `zpool status' output on that site that the problem
referred to is actually in a file[s], whereas what I'm seeing appears to
be in directories.  zfs filesytem directories with [...]/.zfs. 

I will post further along the actual `zpool status -v' output for the
two problem pools but first I wanted to try to explain a bit of what
is going on.... it may be way off base compared to what experienced
folks do.

I notice that one of the cited directories in each pool that is
mentioned as a problem, is the mountpoint of the poolname.  I did do
something that may be ill-advised before noticing the errors. And may
even be the cause of them.

The pools I created all seemed to automatically find mountpoints on
'/'.  I guess that is default behavior. 

That is, if I create a pool: p/some/dir.
I then see /p  appear on '/'.

But, since I also wanted some of the first portion of the system like:
p1/HERE to be mounted on '/HERE' (That is, on '/') too. And as you can
surmise, with very much of that and '/' will be really cluttered.

OK, I decided I would rather have the pools all have their mountpoints
in one directory on '/' instead of having a pile of pools on '/'

so instead of /p1 /p2 /p3 /p4 [...] I would have

/p/p1 /p/p2 [...]
 
So '/p' would be the only point on '/' and all pools would be under /p

Maybe that is a dingy or even harmful thing to do.  I just set the
mountpoint of each pool to be mounted under /p.

If that is a bad thing, then it may be the cause of these messages.
However it only appears on two out of four pools and all of them have
there mountpoints under /p/

Below is the output from two pools (p3 p4) that show the problem.  I
snipped away the body of the `zpool' output in the second (p4) since
it was the same. I did include the actual directories mentioned in p4.

`zpool status -v pool' shows only directories as the 'corrupted' data.
Each shows the initial directory of that zfs system but p4 also shows  
the terminal directory of that zfs system.

Further, the zpool output speaks of replacing files, but the only data
mentioned are directories..... so does it mean the entire directory
content?

The `zpool status -v' output is followed by a few brief questions.
-------        ---------       ---=---       ---------      -------- 

zpool status -v p3
  pool: p3
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: http://illumos.org/msg/ZFS-8000-8A
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        p3          ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c4t6d0  ONLINE       0     0     0
            c4t7d0  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        /p/p3/
-------        ---------       ---=---       ---------      -------- 
pool p4 has same output except disks and list of effected files
-------        ---------       ---=---       ---------      -------- 

zpool status -v p4

[...]

   NAME        STATE     READ WRITE CKSUM
   p4          ONLINE       0     0     0
     mirror-0  ONLINE       0     0     0
       c4t8d0  ONLINE       0     0     0
       c4t9d0  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

      /p/p4/
      /eBk/eImgMus/
-------        ---------       ---=---       ---------      -------- 


NOTES:
In both cases there are no files in /p/p3 or /p/p4.

There are 23034 music and photo files/directories in p4/eBk/eImgMus/

What do I need to do to get to the bottom of the problem?  It is
really likely that all the files under p4/eBk/eImgMus/ are corrupted?