CLARiiON FC Array Information
Note: This content was originally written in 2004
I recently came across one of these units, and thought I’d share my experiences learning about using one. My perspective is as the second-hand buyer hobbyist, so my collected information will be of more use than some official documentation to this crowd. For example, I do not assume you magically have access to the official configuration software frequently mentioned in the vendor documentation.
This unit is frequently sold as an OEM’d item, and thus can be found under a variety of brand names. So far, I’ve been able to uncover the following:
- EMC CLARiiON FC5400/FC5500
- SGI Origin FibreVault
- Dell PowerVault 650F(DPE) & 630F(DAE) (FC5600/FC5700, but similar)
The RAID system, which I’ll simply refer to as a 5400 from now on, is found as a number of separate boxes. They include:
- DPE - Contains two (SP) controllers, two power supplies, and space for 10 hard drives
- DAE - Contains two power supplies, and space for 10 hard drives
- SPS - Proprietary UPS, required if you want to enable write cache
Getting to the FCLI
One of the first problems you are likely to run into, as a second-hand user, is a complete lack of the "official software" for configuring and managing the device. Thankfully, you do not really need this software.
The 5400’s SP (controller) boards each have two serial ports. One is labeled "SPS" and the other one has a picture of a terminal on it. Your first step is to plug a terminal into that port with a picture of a terminal. (By "terminal", all I mean is something that lets you, as a human user, do RS-232 I/O. You can either use a real serial terminal, or a program on your desktop like minicom or HyperTerminal.) Then, set your terminal to 9600 bps, 8 bits, no parity, 1 stop bit (commonly referred to as simply "9600 8N1"). Then, fire up the 5400 unit. If all goes well, you should see startup messages. If you see nothing, then you need to stick a null-modem adapter on your cable.
If you are lucky, this will get you to a prompt that says "fcli>" and you are good to go, and can skip the rest of this section. Otherwise, the messages will likely stop after printing "Entering Serial Application Mode".
This is what I believe to be the magic debugger break sequence. It will show you the contents of all the SP’s registers, and give you a prompt.
You should now get a menu with lots of low-level scary-looking options.
Select: 9 (Nonvol Menu)
Select: 12 (Serial Mode)
Choose: 1 (fcli mode)
This toggles whether the FLARE code starts in FCLI mode, or Serial Application Mode
Select: 0 (Exit)
Select: 12 (FLARE Menu)
Select: 5 (Start FLARE)
Now the FLARE firmware should start up again, but this time you should get an "fcli>" prompt. From here, you can access all the commands needed to manage the array.
(shamelessly paraphrased from here.)
I’ve taken the liberty of putting together a document showing all of the FCLI commands, and their associated online help. You can find it here:
If you are using a Dell PowerVault 650F (CLARiiON FC5700, so the SP will probably claim to be a "5600"), and wish to break out of "Serial Application Mode", the magic break sequence is "@$@$". I cannot give detail the other steps for the 5600 platform, but I’ve been told that they are similar. The FCLI commands for the 5600 SP are detailed here:
The Supplemental Power Supply (SPS)
The next thing you will want to to, is attach an SPS. What is an SPS? Essentially, it is a special proprietary UPS designed to talk with the SP. Without a fully functional and charged SPS, you cannot enable write cache. On one hand, enabling write cache without an SPS is a pretty bad idea.
To connect your SPS, you use a special cable that has an RJ-12 plug on one end, and a standard DB9 plug on the other. The pinout for this cable is as follows:
RJ-12 plug (top view) /-------+ |-------|--6 ---- 3 (on DB9) |-------|--5 ---- 2 (on DB9) ||-------|--4 ||-------|--3 |-------|--2 |-------|--1 ---- 5 (on DB9) -------+
On the other hand, as a second-hand user, it is highly unlikely that you will actually be able to get your hands on a real SPS. So, we do the next best thing.
Basically, the goal is to figure out the protocol the SPS uses to talk to the SP, and emulate it. This way, you can have a self-programmed microcontroller or computer with a couple serial ports, acting as a go-between. The idea is that you would have bought a standard UPS, and have a way to find out its status. You then report this information to the "SPS" port on the SP.
From some independent tinkering, I’ve actually been able to experimentally learn a great deal about how the SP talks to its SPS. I also finally got my hands on a real SPS, and did some sniffing which verified some of my guesses and corrected others. In any case, I now have a pretty good understanding of how the communication works. The communication uses standard RS-232 serial over the "SPS" port on the back of the SP. The SP then periodically sends out a query, to which the SPS responds. Depending on this response, any manner of SPS status can be indicated. It also appears as though the SP is capable of controlling the SPS to a limited degree. When I unplugged the SPS, the SP flushed the write cache. Then, it sent a command to the SPS, after which the SPS promptly shut itself off. I suspect this is to preserve SPS batter power, just in case, as it is really not important for the SP to remain online once the cache is flushed.
Due to popular demand, I have finally finished an initial release of my SPS emulator daemon. While it probably has a lot of room for improvement, it does get the basic job done. It consists of a daemon that sits on your serial port and manages communications with the SP, and also has a control program and shutdown script, so you can make it interact with whatever software you use to monitor the status of a normal UPS.
I’ve written and tested this code on FreeBSD, though it should probably and run just fine on other ‘nixes. If not, feel free to send me the modifications needed to make the source more portable. Also, refer to the README file for installation and usage instructions
- (note: updated to incorporate support for the BBU test procedure, in response to the BATTEST message)
- (initial release)
SP Query: ASCII: "COND?" + 0A HEX: 43 4F 4E 44 3F 0A SPS Responses: SPS removed: ASCII: nothing HEX: nothing SPS not ready, battery charging: ASCII: "0" + 0A HEX: 30 0A SPS battery on-line, main power off-line, running on battery: ASCII: "1" + 0A HEX: 31 0A SPS Enabled, battery fully charged, system cache enabled: ASCII: "2" + 0A HEX: 32 0A SPS testing battery ASCII: "128" + 0A HEX: 31 32 38 0A SPS faulted: (this one discovered experimentally, not seen from actual SPS) ASCII: "10" + 0A HEX: 31 30 0A
SP Command: Cache flushed, shut down SPS: ASCII: "STOP" + 0A HEX: 83 84 79 80 0A SPS Response: Ok, going dead now: ASCII: "" + 0A HEX: 0A SP Command: BBU test routine: ASCII: "BATTEST" + 0A Procedure: Reply with a simple 0A response to the "BATTEST" query. Then, send a "128" in response to the next several "COND?" messages, then send "0" in response to the next several "COND?" messages after that. Finally, revert to the normal behavior of sending "2" in response to "COND?".
(I’m sorry if my BATTEST description looks confusing, but it was recently discovered, and is more of a procedure than a simple response. Thus, it doesn’t fit nicely into the formatting with which I described the rest of the protocol.)
- EMC Fibre Channel Storage System (CX-Series) Configuration Planning Guide
- EMC Installation Roadmap for CX-Series and FC-Series Storage Systems
- EMC FC-Series and C-Series Storage-System and EMC ControlCenter Navisphere Event Codes
- EMC ControlCenter Navisphere Analyzer Administrator’s Guide
- EMC ControlCenter Navisphere Command Line Interface (CLI) Reference
- EMC ControlCenter Navisphere Manager Administrator’s Guide
- EMC CLARiiON Open Systems Configuration Guide