[OpenIndiana-discuss] Group Membership Limitation

[andre at ak47.co.za]

Hi 

As background: 

I have had my OI v151a machine joined to my AD
for a long long time now. I have been assigning permissions to my ZFS
file systems using the users and groups from AD. 

Recently I had to
migrate from my existing domain to a much larger domain and so I joined
my OI machine to the new domain and I was able to assign permissions
using users and groups from the new domain as per normal. Everything
looked really good. 

Reality: 

When I tried to connect to one of my
shares windows reported back an error. Upon closer inspection of the
logs on OI, I saw a PANIC and then the SAMBA daemon restarts. 

I have
been researching this problem and everything points to a limitation of
Solaris group memberships. According to what I read Solaris does not
cater for group membership of more than 16 to 32 groups. I am not sure
on which flavours of Solaris these differ. Linux used to have the same
limitation but this has been increased to 64K from kernel 2.6.3+. I
found an old bug report from a then Sun engineer documenting this. I
would have assumed that this would have been fixed by now, 8 years
later. From my experience however I can tell that it hasn't.


https://blogs.oracle.com/peteh/entry/increasing_unix_group_membership_easy


The topic is also touched on these two forums:


https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1075670
http://samba.2283325.n4.nabble.com/sys-setgroups-in-samba-3-3-X-fails-cause-a-panic-td2483755.html


I wanted to rule out the new domain join etc, so I rolled back to my
previous boot environment on the old domain and created 8 groups (test1,
test2, test3...test8) and added my own account to these additional
groups. When I tried to access my share the same thing happened on the
old BE and old domain. So it is definitely related to group memberships
and the Solaris limitation. 

Question is: is there a way to increase
the group membership limitation as described by the Sun engineer or can
this be fixed? 

==> /var/samba/log/log.atlza-70kkn1j <==
[2013/09/13
13:12:02.972159, 0] lib/util.c:1468()
 PANIC (pid 983): sys_setgroups
failed
[2013/09/13 13:12:02.973476, 0] lib/util.c:1572()
 BACKTRACE: 23
stack frames:
 #0 /usr/sbin/smbd'log_stack_trace+0x29 [0x83bbb91]
 #1
/usr/sbin/smbd'smb_panic+0x5f [0x83bba6b]
 #2
/usr/sbin/smbd'set_unix_security_ctx+0x47 [0x819c27b]
 #3
/usr/sbin/smbd'set_sec_ctx+0xb5 [0x819c349]
 #4
/usr/sbin/smbd'change_to_user+0x2ee [0x818c77a]
 #5
/usr/sbin/smbd'make_connection_snum+0xadc [0x81b2d44]
 #6
/usr/sbin/smbd'make_connection+0x4d1 [0x81b390d]
 #7
/usr/sbin/smbd'reply_tcon_and_X+0x23f [0x816878f]
 #8
/usr/sbin/smbd'switch_message+0x3d8 [0x81af1f4]
 #9
/usr/sbin/smbd'construct_reply+0xde [0x81af3c2]
 #10
/usr/sbin/smbd'process_smb+0x135 [0x81af5e9]
 #11
/usr/sbin/smbd'smbd_server_connection_read_handler+0x9c [0x81b0468]
 #12
/usr/sbin/smbd'smbd_server_connection_handler+0x45 [0x81b04b9]
 #13
/usr/sbin/smbd'run_events+0x27d [0x83cc9e5]
 #14
/usr/sbin/smbd'smbd_server_connection_loop_once+0x117 [0x81aea63]
 #15
/usr/sbin/smbd'smbd_process+0x6c4 [0x81b0df8]
 #16
/usr/sbin/smbd'smbd_accept_connection+0x209 [0x86da851]
 #17
/usr/sbin/smbd'run_events+0x27d [0x83cc9e5]
 #18
/usr/sbin/smbd's3_event_loop_once+0x111 [0x83ccbe9]
 #19
/usr/sbin/smbd'_tevent_loop_once+0x79 [0x83cd6b1]
 #20
/usr/sbin/smbd'smbd_parent_loop+0x82 [0x86db28a]
 #21
/usr/sbin/smbd'main+0xbea [0x86dc492]
 #22 /usr/sbin/smbd'_start+0x7d
[0x813469d]
[2013/09/13 13:12:02.974395, 0] lib/fault.c:326()
 dumping
core in /var/samba/log/cores/smbd

==> /var/samba/log/log.smbd
<==
[2013/09/13 13:12:02.979519, 1] smbd/server.c:267()
 Scheduled
cleanup of brl and lock database after unclean shutdown
[2013/09/13
13:12:03, 0] smbd/server.c:1143()
 smbd version 3.5.21 started.

Copyright Andrew Tridgell and the Samba Team 1992-2010

==>
/var/svc/log/network-samba:default.log <==
[ Sep 13 13:12:02 Stopping
because process dumped core. ]
[ Sep 13 13:12:03 Executing stop method
("/usr/bin/kill `cat /var/samba/locks/smbd.pid`"). ]
[ Sep 13 13:12:03
Method "stop" exited with status 0. ]
[ Sep 13 13:12:03 Executing start
method ("/usr/sbin/smbd -D"). ]
[ Sep 13 13:12:03 Method "start" exited
with status 0. ]