[OpenIndiana-discuss] Access to ZFS viz CIFS from windows regularly hangs.

[John McEntee]

Ok, and update on my progress as I have not fixed it yet.

The domain controller the OpenIndiana server is connected to  had problems
with the file replication, I cleared the cache and it fixed that problem.
The occurrence of the CIFS server on OpenIndiana dropped dramatically, but
still exist. I found and ran the Microsoft environment IT health check tool,
that had problems contacting another domain controller, stopping the tool
from running (WMI errors), if fixed those and now the toll runs fine with no
significant errors. (complains about the network config of couple of
pre-production exchange servers). This seems to have reduced the hangs even
more.

This week most the users are on a training course so it is not being heavily
used either, but I have a couple of users doing a tcpdump/snoop/wireshark to
a ringbuffer so I can hopefully get a packet trace.

What I have noticed is an ulimit -a returns

core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 10
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 29995
virtual memory          (kbytes, -v) unlimited


Is the open files going to cause CIFS a problem, 256 seems a bit low, could
easily hit that limit if it is shared amongst all the users.


I also have 2 scripts running every minute via cron (from a linux box). One
copies a 10 MB file via cifs the other via nfs, neither have been delayed
for more than 5 seconds, normally completes with 1 second. These are done to
their own zfs file systems at the moment, but I can foresee having to point
them at the production filesystem if it does not detect the freezes that the
users report.

John

-----Original Message-----
From: Mike La Spina [mailto:mike.laspina at laspina.ca] 
Sent: 14 June 2012 04:43
To: Discussion list for OpenIndiana
Subject: Re: [OpenIndiana-discuss] Access to ZFS viz CIFS fromwindows
regularlyhangs.


> Does the suspend event only occur on SMB clients or does it impact the
other storage clients when triggered by the Windows clients?

It does not seem to effect the vmware hosted machines via nfs. Next time it
hangs I will try a nfs transfer to it.
- If this is correct its a further indication of an AD/SMB issue, but not
verified at this point.

> Any domain controller event errors?

Yes there are, I will go and resolve this first before I go any further.

- Highly suspect this is where you need to focus.
- This error is suspicious and does look like a issue on the domain. 
- Jun 12 11:26:07 ringwood smbd[6032]: [ID 702911 daemon.notice]
smbd_dc_update: stirling-dynamics.com: located red
- Jun 12 11:34:17 ringwood smbd[6032]: [ID 702911 daemon.error]
smbrdr_exchange[4]: failed (INVALID_HANDLE)
- I would look further back in time and see if it correlates with the
suspended access event. That would define a clear resolution path.

> dmsg output?

Attached - is this the correct etiquette?

- Jun 11 21:38:18 ringwood fmd: [ID 377184 daemon.error] SUNW-MSG-ID:
SMF-8000-YX, TYPE: defect, VER: 1, SEVERITY: major
- Jun 11 21:38:18 ringwood EVENT-TIME: Mon Jun 11 21:38:18 BST 2012
- Jun 11 21:38:18 ringwood PLATFORM: S5520HC, CSN: ............,
HOSTNAME: ringwood
- Jun 11 21:38:18 ringwood SOURCE: software-diagnosis, REV: 0.1
- Jun 11 21:38:18 ringwood EVENT-ID:
cc9f2029-a779-cbd2-e425-8ffbaa19f639
- Jun 11 21:38:18 ringwood DESC: A service failed - a method is failing in a
retryable manner but too often.
- Jun 11 21:38:18 ringwood   Refer to http://sun.com/msg/SMF-8000-YX for
more information.
- Jun 11 21:38:18 ringwood AUTO-RESPONSE: The service has been placed into
the maintenance state.
- Jun 11 21:38:18 ringwood IMPACT: svc:/application/time-slider:default
is unavailable.

- The time slider snapshot service failed? Or was it stopped manually?

> fmdump -eV output?

Also attached.
- Nothing remarkable

> uname -a output?

SunOS ringwood 5.11 oi_148 i86pc i386 i86pc Solaris

> Have you attempted a packet capture of the event?
> snoop -o smb-client.cap <clientip>

Not yet, It could be caputureing for 4 hours before it happens, I will
resolve the AD domain issue first.
- Good approach. 4 hours of packet tracing is hard to digest! It would
certainly need to be truncated down to the trigger event. 

- Mike



_______________________________________________________________________

The contents of this e-mail and any attachment(s) are strictly confidential
and are solely for the person(s) at the e-mail address(es) above. If you are
not an addressee, you may not disclose, distribute, copy or use this e-mail,
and we request that you send an e-mail to admin at stirling-dynamics.com and
delete this e-mail.  Stirling Dynamics Ltd. accepts no legal liability for
the contents of this e-mail including any errors, interception or
interference, as internet communications are not secure.  Any views or
opinions presented are solely those of the author and do not necessarily
represent those of Stirling Dynamics Ltd. Registered In England No. 2092114
Registered
Office: 26 Regent Street, Clifton, Bristol. BS8 4HG VAT no. GB 464 6551
29
_______________________________________________________________________

This e-mail has been scanned for all viruses MessageLabs.

_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss at openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss


_______________________________________________________________________

The contents of this e-mail and any attachment(s) are strictly confidential and are solely for the person(s) at the e-mail address(es) above. If you are not an addressee, you may not disclose, distribute, copy or use this e-mail, and we request that you send an e-mail to admin at stirling-dynamics.com and delete this e-mail.  Stirling Dynamics Ltd. accepts no legal liability for the contents of this e-mail including any errors, interception or interference, as internet communications are not secure.  Any views or opinions presented are solely those of the author and do not necessarily represent those of Stirling Dynamics Ltd. Registered In England No. 2092114 Registered Office: 26 Regent Street, Clifton, Bristol. BS8 4HG
VAT no. GB 464 6551 29
_______________________________________________________________________

This e-mail has been scanned for all viruses MessageLabs.