Jason Antman’s Blog

There’s a known issue with the SSH implementation in the iLO firmware for HP Proliant servers (specifically G2 and G3) and OpenSSH 5.1p1. There was a thread on the OpenSSH developers list that referenced this problem and suggested a solution, but it doesn’t seem to be a sure fix.

This problem is present on my DL360 G2’s which are running the 1.84 2006-05-05 version of the iLO firmware (iLO 1.84 pass9) with the P26 2004.05.01 version of the system firmware. I also see the issue on a DL380G3 running iLO 1.92 2008.04.24 and system firmware P29 2004.09.15. The only way that I can reliably get into the iLO is by SSHing from a box with an older version of SSH, such as 4.2p1.

Most of the things that I could find online referenced unsetting the LANG environment variable:

unset LANG

and then SSHing with agent forwarding disabled:

ssh -a hostname-ilo

Unfortunately this combination doesn’t seem to do it for me.

I happened to stumble by this post to the debian-ssh mailing list, which suggested that shortening the new OpenSSH version string fixed the problem.

I was able to confirm that the version string is, in fact, the sole problem. I downloaded the source of OpenSSH 5.2p1 and, with the following small patch to version.h, managed to get SSH working to the iLO perfectly:

--- openssh-patched/version.h   2009-06-12 00:35:48.000000000 -0400
+++ openssh-5.2p1/version.h     2009-02-22 19:09:26.000000000 -0500
@@ -1,6 +1,6 @@
 /* $OpenBSD: version.h,v 1.55 2009/02/23 00:06:15 djm Exp $ */
 
-#define SSH_VERSION    "OpenSSH"
+#define SSH_VERSION    "OpenSSH_5.2"
 
-#define SSH_PORTABLE   ""
+#define SSH_PORTABLE   "p1"
 #define SSH_RELEASE    SSH_VERSION SSH_PORTABLE

I patched version.h, ran `./configure`, `make`, and then copied the compiled ssh binary to /usr/bin/ilossh, so that my original ssh binary would be intact, and the ilossh binary would be left alone by RPM upgrades.

Tech HowTos ilo, openssh, proliant, ssh

Not much of an “upgrade” for anyone who’s in IT, but jasonantman.com is currently being upgraded from old desktops used as servers to a pile of generation-1 (G1) HP/Compaq Proliants. I know that there are utilities for Linux to manage the servers, specifically control fan speed and monitor hardware-level health for Linux. However, the most recent download on HP’s site is for SLES9. All of my boxes will be running openSuSE 10.2, and the SLES9 version wouldn’t install on them.

After an hour long phone call to HP support, I ended up speaking with Paulo, the third support person I was transferred to. #1 read off the web site, #2 knew what Linux was, but Paulo (#3) actually told me that he was experimenting with installing HPASM (HP’s server administration/management utility) on an older Proliant as well. He spent about half an hour walking me through it. Here’s what I found:

The most compatible version of HPASM (I guess it’s some hidden feature for people who know it) is the version for the DL380 G4. Paulo instructed me to download this RPM from their site. I did, choosing the SLES10 (x86) download (hpasm-7.7.0-115.sles10.i586.rpm). This installed fine. Running hpasm status from the command line asks us to activate it first. Do the activation. Now, running hpasm status still asks us to activate. Paulo confirmed this as happening on his machine too. Try /etc/init.d/hpasm status and you should see that all of the modules are working.

Now, the install is complete. I’m not sure if the SNMP works, but it should as long as your snmpd is running. The hpasm activate command modifies snmpd.conf appropriately. and you will be queried for the currect configuration information.

To give it a test, run hplog -f or hplog -p and you should see fan and power status, respectively.

Paulo also told me that I could download the hpadu package (also DL380 G4 / SLES10) to get array diagnostics, He warned me that some of the install scripts in HPADU look for the web management homepage, which we haven’t installed. To get around this, install the HPADU RPM file (hpadu-7.70-12.linux.rpm) rpm -ivh --force --nodeps --noscripts hpadu-7.70-12.linux.rpm. Be aware, though, that this package is supposed to be web-based. It installs to /opt/hp/hpadu.

The web interface, luckily for me, is written in PHP. It is pretty complex so it might take me a while to figure out the workings, but when I do, I’ll post as much info as I can on how to make a CLI interface, or where one exists if I can find it.

Also, I’ll most likely develop a Python check script to use with Nagios to monitor most of the hpasm-enabled components.

For the use of anyone else, here are some of the links that HP Support sent me after the call:

Link for users guide for Proliant Support Pack, which includes documentation on HPASM from the CLI:

http://h18000.www1.hp.com/support/files/server/us/WebDoc/720/psp-users-guide.pdf

Product manuals:

http://h20180.www2.hp.com/apps/Nav?h_pagetype=s-003&h_lang=en&h_cc=us&h_product=241435&h_page=hpcom&h_client=z-a-r1002-3&cc=us&lang=en

ML370 G1

http://h20000.www2.hp.com/bc/docs/support/UCR/SupportManual/TPM_143091-004/TPM_143091-004.pdf

Tech HowTos compaq, hp, hpadu, hpasm, linux, management, proliant, psp