unbelievable oddities: HP NC510F (NetXen 10ge NIC)


I just gotta type this down, even if I should have been sleeping for hours already.
Did You ever see a mouse break a highend network adapter? I didn’t, until today.

A few days ago I got an awesome 10gig ethernet nic (HP NC510F, which is a rebranded NetXen model).
I had to update the nic’s firrmware as the original one
was lame (had many features disabled), and didn’t work with the OSS driver module that came with my 2.6.29 gentoo install on that box.
The error message:

Oct 13 13:17:25 localhost kernel: nx_nic: Flash version[2.3.57] is incompatible with the driver.

in dmesg output and the missing ethN device were good points after all.
I downloaded the firmware and all other software from HP at http://h18000.www1.hp.com/products/servers/networking/nc510f/index.html and then found out I would be best off with a standard linux over gentoo, so I did a “quick” CentOS 5 install, which, including a fancy webbrowser and all the other graphical stuff took a better part of an hour. (The “part 2” links at the above site holds the install instructions.)

Oh well, that was the easy stuff – installation was easy too:

yum -y update & reboot
yum -y install kernel-devel rpm-build (reboot first so you save the nx_nic driver trying to link against the “running” kernel while it d/l newer sources)
yum groupinstall “Development Tools”
rpm -i the source rpms
cd /usr/src/redhat
rpmbuild -bb SPECS/nx_nic.spec

… or something
do the same for the LSA module, too

rpm -i the resulting rpms.

Then (missing from manual) add a modprobe.conf.local entry setting a device name for the driver.

Next: Updating – not…

/opt/netxen is where all the vendor tools reside, namely the one to flash the firmware.

Ok, manual says:

nxflash –list (get device number)
nxflash -i -a

JUST THAT DOESNT WORK:
example output:

Oct 13 13:08:26 localhost kernel: nx_nic: Flash Version: Firmware[2.3.57], BIOS[255.255.65535]
Oct 13 13:08:27 localhost kernel: nx_nic: Incompatible Flash BIOS[255.255.65535] for file FW[nxromimg.bin] version[3.4.339:0], BIOS[1.0.2]
Oct 13 13:08:28 localhost kernel: nx_nic: Flash version[2.3.57] is incompatible with the driver.
Oct 13 13:08:28 localhost kernel: nx_nic: Loading the firmware from flash.
Oct 13 13:08:40 localhost kernel: nx_nic[eth%d]: Receive Peg initialization not complete: 0x0.
Oct 13 13:08:40 localhost kernel: nx_nic: probe of 0000:01:00.0 failed with error -5
Oct 13 13:08:40 localhost kernel: nx_lsa: Loading Module

A few frustrated tries –somehow the nic kept disappearing – later I noticed there was a specific rom image for the HP flavoured nics!

See the “xfp_hp_romimage”

[root@localhost ~]# ls /opt/netxen/
8FA5E8BA_cut_thru_romimage hmez_romimage nxflash_info_ethtool.sh showvpd
cm_cut_thru_b1_romimage mm_cut_thru_b1_romimage nxudiag xfp_hp_romimage
cx4_hp_romimage nxflash nx_xport.ko
hmez_cut_thru_b1_romimage nxflash.bin phantomcore_p3

Now, when I tried to flash that I noticed the card was gone yet again.

  • I switched from using a PEG riser card to putting it directly into the slot (16lane I think)
  • I switched from the 16lane slot to the 8lane
  • The card was even gone from lspci a few times, even when turning the system off and on.

After some scratching head I noticed I, too, had been adding and removing my razer copperhead mouse.
The mouse which actually really had caused the NetXen nic to disappear.

I’ve been doing this PC stuff for 16 years now and this is the 2nd-oddest thing ever to happen to me.

Last, after flashing the HP rom image, some diag output with bumped up firmware version. Right now I don’t have the Xenpak module inserted because I don’t got another 10GE device anyway.

So much looking forward to turn off that “legacy” mode

[root@localhost ~]# /opt/netxen/nxudiag -i eth2
NETXEN Interface is eth2
Interface eth2 is UP.
************* Board Info *************
Board Ser# : xxxxxx
Chip Rev : C0
Board Type : 0xb (HP NC510F 10G Ethernet Adapter)
Core clock : 300 MHz
Mem clock : 166 MHz
Crystal freq : 33 MHz
Firmware in : LEGACY mode
DDR size : 128 MB, ECC Enabled
QDR size : 4 MB, ECC Enabled
Peg ICACHE : OK, Enabled
Peg DCACHE : OK, Enabled
Firmware ver.: 3.4.339
Driver ver. : 4.0.505
MAC Addr 0 : 00:17:A4:xX:xx:xx
**************************************
Card Temperature = 37 degrees C , Status = ‘NORMAL’

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s