[OpenIndiana-discuss] Serial console for OI/Solaris on Intel MFSYS blades

[Jim Klimov]

Hello all,

   I have a peculiar problem: the Intel MFSYS blade servers, in short ;)

   They have a number of interesting features, due to which are deployed
at a number of our customers, but with somewhat under-cooked firmware
features wherever I've seen. The one bugging me this week is a Remote
Serial Console implementation.

   This blade chassis has up to 6 server blades, a management module,
1 or 2 storage controller modules driving the 14 disks (the SCMs do
the RAID and publish LUNs to server blades), and 1 or 2 networking
modules (GbE switches). Overall, quite a bit of redundancy would
make it a good box, if its code was all working properly ;)

   So, the management module takes the servers' redirected text console
IO and publishes that as a "SOLCLI" (Serial-Over-LAN CLI) session via
SSH on dedicated ports like 2201-2206 for the 6 blades. I found next
to zero specific info on the serial emulation, but it seems to be
either auto-bauding or (as the Linux blades successfully use it) a
"115200,8,n,1,-" link. I'd bet auto-bauding, because "9600,8,n,1,-"
also seems to work. Terminal emulation is unknown, but somewhat like
VT100 (however, GRUB menu fails to render properly, unless I specify
"--dumb" for a simplified rendering mode).

   The problem is, the redirected console gets lost after Solaris 10
or OI boots on the blade. It works somewhat for BIOS and GRUB (but
even there it sometimes loses connections or farts out invalid
characters), and if I boot with "-B console=ttya" - sercon usually
makes it through the OS build version banner and prints a character
or two of the OS initialization (console logs from SMF startups).

   Afterwards nothing comes in out of the ttya console: echos to it
hang (sometimes the first character is printed), cats from it hang
and in half a minute cause restart of the SSH session to the console.
The ttymon greeting never comes up, either.

   However, when the OS is rebooted, it does fully print the line
"rebooting..." on the sercon.

   I guess that some of the OS components working with the serial
port (ttymon, stty, pm, saf) might be setting "invalid" options
on the port which cause the Intel Remote Console fail into coma.
Likewise, when such component is unloaded and un-initialized
before reboot, console becomes responsive again.

   Does this ring a bell to someone? What can be (un-)set or at
least debugged in this case? I learned a lot of new stuff about
pmadm, sttyadm, ttymon, stty settings and console-login SMF
services, but this did not get me a working serial console
on this box. Same settings work for other IPMI-enabled boxes
(mostly various Sun Fire models; they don't have any other
Intel-branded servers, so I don't know if their other models
share such bugginess).


   Side note: Linux installations had few hiccups using serial
consoles on the same blades with just this line in /etc/inittab:
   co::respawn:/sbin/mgetty -s 115200 -r /dev/ttyS0
Using "difficult" terminal emulations or fast-scrolling outputs
can also cause the sercon collapse into a long coma to reboot,
or at least break the SSH session to it; simply running "vi"
was notorious for that effect.

Thanks in advance,
//Jim Klimov