Household Enterprise Computing » CLARiiON FC Array Information

June 11, 2004

CLARiiON FC Array Information

Filed under: Storage — octo @ 9:11 am

I recently came across one of these units, and thought I’d share my experiences
learning about using one. My perspective is as the second-hand buyer
hobbyist, so my collected information will be of more use than some official
documentation to this crowd. For example, I do not assume you magically
have access to the official configuration software frequently mentioned
in the vendor documentation.

For information on the particular hardware that I have, see
this page.

This unit is frequently sold as an OEM’d item, and thus can be found under a
variety of brand names. So far, I’ve been able to uncover the following:

EMC CLARiiON FC5400/FC5500
SGI Origin FibreVault
Owner’s Guide
Administrator’s Guide
Dell PowerVault 650F(DPE) & 630F(DAE) (FC5600/FC5700, but similar)
Installation and Service Guide

The RAID system, which I’ll simply refer to as a 5400 from now on, is found
as a number of separate boxes. They include:

DPE - Contains two (SP) controllers, two power supplies, and space for 10 hard drives
DAE - Contains two power supplies, and space for 10 hard drives
SPS - Proprietary UPS, required if you want to enable write cache

Getting to the FCLI

One of the first problems you are likely to run into, as a second-hand
user, is a complete lack of the “official software” for configuring and
managing the device. Thankfully, you do not really need this software.

The 5400’s SP (controller) boards each have two serial ports. One is labeled
“SPS” and the other one has a picture of a terminal on it. Your first step is
to plug a terminal into that port with a picture of a terminal. (By “terminal”, all I mean is something that lets you, as a human user, do RS-232 I/O. You can either use a real serial terminal, or a program on your desktop like minicom or HyperTerminal.) Then, set your terminal to 9600 bps, 8 bits, no parity, 1 stop bit (commonly referred to as simply “9600 8N1″). Then, fire up the 5400
unit. If all goes well, you should see startup messages. If you see nothing,
then you need to stick a null-modem adaptor on your cable.

If you are lucky, this will get you to a prompt that says “fcli>” and you are
good to go, and can skip the rest of this section. Otherwise, the messages
will likely stop after printing “Entering Serial Application Mode”.
Type: {}{}
This is what I believe to be the magic debugger break sequence. It will
show you the contents of all the SP’s registers, and give you a prompt.
Type: menu
You should now get a menu with lots of low-level scarry-looking options.
Select: 9 (Nonvol Menu)
Select: 12 (Serial Mode)
Choose: 1 (fcli mode)
This toggles whether the FLARE code starts in FCLI mode, or Serial Application Mode
Select: 0 (Exit)
Select: 12 (FLARE Menu)
Select: 5 (Start FLARE)
Now the FLARE firmware should start up again, but this time you should get an
“fcli>” prompt. From here, you can access all the commands needed to
manage the array.
(shamelessly paraphrased from here.)

I’ve taken the liberty of putting together a document showing all of the
FCLI commands, and their associated online help. You can find it here:
5400 FCLI Commands

If you are using a Dell PowerVault 650F (CLARiiON FC5700, so the SP will
probably claim to be a “5600″), and wish to break out of “Serial Application
Mode”, the magic break sequence is “@$@$”. I cannot give
detail the other steps for the 5600 platform, but I’ve been told that they
are similar.
The FCLI commands for the 5600 SP are detailed here:
5600 FCLI Commands

The Supplemental Power Supply (SPS)

The next thing you will want to to, is attach an SPS. What is an SPS?
Essentially, it is a special proprietary UPS designed to talk with the SP.
Without a fully functional and charged SPS, you cannot enable write cache.
On one hand, enabling write cache without an SPS is a pretty bad idea.

To connect your SPS, you use a special cable that has an RJ-12 plug on one
end, and a standard DB9 plug on the other. The pinout for this cable is
as follows:

RJ-12 plug
(top view)
 /-------+
 |-------|--6 ---- 3 (on DB9)
 |-------|--5 ---- 2 (on DB9)
||-------|--4
||-------|--3
 |-------|--2
 |-------|--1 ---- 5 (on DB9)
 -------+

On the other hand, as a second-hand user, it is highly unlikely that you will
actually be able to get your hands on a real SPS. So, we do the next best
thing.

Basically, the goal is to figure out the protocol the SPS uses to
talk to the SP, and emulate it. This way, you can have a self-programmed
microcontroller or computer with a couple serial ports, acting as a go-between.
The idea is that you would have bought a standard UPS, and have a way to find
out its status. You then report this information to the “SPS” port on the SP.

From some independent tinkering, I’ve actually been able to
experimentally learn a great deal about how the SP talks to its SPS.
I also finally got my hands on a real SPS, and did some sniffing which verified
some of my guesses and corrected others. In any case, I now have a pretty good
understanding of how the communication works.
The communication uses standard RS-232 serial over the “SPS” port on the
back of the SP. The SP then periodically sends out a query, to which the
SPS responds. Depending on this response, any manner of SPS status can be
indicated. It also appears as though the SP is capable of controlling the SPS
to a limited degree. When I unplugged the SPS, the SP flushed the write cache.
Then, it sent a command to the SPS, after which the SPS promptly shut itself off.
I suspect this is to preserve SPS batter power, just in case, as it is really not
important for the SP to remain online once the cache is flushed.

SPS Emulation

Due to popular demand, I have finally finished an initial release of my SPS
emulator daemon. While it probably has a lot of room for improvement, it
does get the basic job done. It consists of a daemon that sits on your
serial port and manages communications with the SP, and also has a control
program and shutdown script, so you can make it interact with whatever software
you use to monitor the status of a normal UPS.

I’ve written and tested this code on FreeBSD, though it should probably and run
just fine on other ‘nixes. If not, feel free to send me the modifications
needed to make the source more portable. Also, refer to the README file
for installation and usage instructions

spsd-0.2.tar.gz
(note: updated to
incorporate support for the BBU test procedure, in response to the
BATTEST message)
spsd-0.1.tar.gz
(initial release)

SPS Protocol

SP Query:
  ASCII: "COND?" + 0A
  HEX:   43 4F 4E 44 3F 0A

SPS Responses:
 SPS removed:
  ASCII: nothing
  HEX:   nothing

 SPS not ready, battery charging:
  ASCII: "0" + 0A
  HEX:   30 0A

 SPS battery on-line, main power off-line, running on battery:
  ASCII: "1" + 0A
  HEX:   31 0A

 SPS Enabled, battery fully charged, system cache enabled:
  ASCII: "2" + 0A
  HEX:   32 0A

 SPS testing battery
  ASCII: "128" + 0A
  HEX:    31 32 38 0A

 SPS faulted: (this one discovered experimentally, not seen from actual SPS)
  ASCII: "10" + 0A
  HEX:   31 30 0A

SP Command:
 Cache flushed, shut down SPS:
  ASCII: "STOP" + 0A
  HEX:   83 84 79 80 0A

SPS Response:
 Ok, going dead now:
  ASCII: "" + 0A
  HEX:   0A

SP Command:
 BBU test routine:
  ASCII: "BATTEST" + 0A

  Procedure:
   Reply with a simple 0A response to the "BATTEST" query.
   Then, send a "128" in response to the next several "COND?" messages,
   then send "0" in response to the next several "COND?" messages after
   that.  Finally, revert to the normal behavior of sending "2" in
   response to "COND?".

(I’m sorry if my BATTEST description looks confusing, but it was recently
discovered, and is more of a procedure than a simple response. Thus, it
doesn’t fit nicely into the formatting with which I described the rest
of the protocol.)