Asrock E3C224DI-14S IPMI not reachable

Storage Server: Documentation missing

There's definately some documentation missing on the IPMI settings. I managed to lock myself out of the IPMI (also know as 'Integrated Light Out', or ILO) management interface. Not sure how I managed to do that, but in the quest to find out how to restore devine powers, I noticed quite a lot of people suffered from this. And, the solution is quite simple, when you know it. As usual...

Two configurations

The cause of error probably was me updating the network configuration, using the dashboard, instead of using the BIOS update.

IPMI configuration at BIOS
Please note, in the BIOS, you have eth0 and eth1. Eth0 usually is the first interface, so when you assume this would be the IPMI interface, you assumed as I.

Link to Network configuration on the dashboard

Which is completely and utterly wrong... Eth1, which has the label IPMI for a reason, is the correct one, and is found as Port 8 (above the 2 USB ports).

what's with the eth0/eth1, then?

It turns out there's a nifty, quite undocumented feature, of IPMI fallback. the BIOS eth0/NCSI item can be used as IPMI fallback - in case eth1 is not connected... I found this because after the lock out, I could actually use the IPMI when all cables were plugged in. The NCSI port is port 6, or LAN1 (designated as such in the manual), or eth0 (as seen in the BIOS BMC configuration). For completeness sake, LAN2 is not mentioned in the BIOS, only in the Megarac SP configuration (and hard to find).

My recommendation

Stay away from the megarac SP network configration items. Use the BIOS, which takes precedence over the Mearace SP settings anyway, only, and only configure IPMI/eth1 for a fixed ip-adddress. You can always use arp -a to find out the DCHP-assigned ip address to the other interfaces. You can find the MAC-addresses of your LAN1 and LAN2 interfaces at the BMC configuration section of the BIOS, under BMC MAC Restore Tool:

Hope this helps anyone.

Storage server

Storage server

Hardware

Aiming at 2 VDEVs of 5 or 6 disks each, I'd need a motherboard capable of running 12 disks.I used a SuperMicro board in the ESXi build, mainly because virtualization using bare metal hypervisors was quite new to me. However, these boards have quite a steep price.

There's a new motherboard by SuperMicro, that screams NAS, but that has not yet hit the shops.

So, I ended up with:

• 2.5" Toshiba 3TB harddisks (MQ03ABB300)
• 3.5" Toshiba 3TB harddisks
• Asrock E3C224D4i-14S motherboard, supplied with 14 (!) SATA ports
• 32GB 1600MHz ECC ram (Kingston KVR16E11K4/32)
• Define R5 by Fractal Design
• Pentium G3258 by Intel
• G-360 Seasonic power supply
• EVO850 by Samsung

Assembly

It starts off with placing the processor and memory on the motherboard. This is best done outside the case:

One of the reasons I love Fractal cases is the disks cages; not only can you replace the cage with hot plug ones, you may also relocate one or both cages. And the disk frames just slide out - no tools needed.

And, there's room for two SSD's at the back of the motherboard:

The case assemply starts with adding the power supply.

Then, place the motherboard, and attach the power cables. Some cable management is in order, but will be done after all disks have been installed and hooked up.

Time to put these components to the test. Download Memtestx86, or the commercial version, and let it run for a while.

OK, that'll do pig, that'll do.

To be continued with part two of the storage server: Firmware

Things to do after you cloned a Virtual Machine

Clean up a cloned VM

After you made a clone of your (base) VM, you will need to do some stuff.

MAC-address

First of all, I suspect you have a different MAC-address than the original machine. VMWare does that, as long as you have your MAC address assigned automatically. VirtualBox will ask you whether to re-initialize the MAC-address while cloning.
The problem is the udev process, responsable for handling devices. This uses confuguration files, located in/etc/udev/rules.d directory. The file
70-persistent-net.rules will have an entry, based on your original machine. An entry looks like:

# PCI device 0x15ad:0x07b0 (vmxnet3) (custom name provided by external tool) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:71:3c:be", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

You see the MAC-address, as wel as the assigned link (eth0). I already altered this line to reflect the correct MAC-address, and link name. The easiest way turns out to be to remove this file completely; it will be re-generated when absent:

cd /etc/udev/rules.d/ rm -f 70-persistent-net.rules reboot now reboot

eth0 or eth1

Another problem may be the fact your assigned link names went wrong; you may have extra rules in your udev file for eth1. And, your existing eth0 may still carry the wrong MAC-address, which may cause errors like

ifup eth0 eth0 does not seem to be present,delaying initialization.

Check if your MAC-addresses are still lingering around with:

/sbin/ifconfig | grep "^eth" eth0 Link encap:Ethernet HWaddr 00:0C:29:71:3C:BE

This is a correct output - MAC-addresses match. If not, change the /etc/sysconfig/network-scripts/ifcfg-eth0 file to reflect the correct MAC-address:

DEVICE=eth0 TYPE=Ethernet ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=none HWADDR=00:0C:29:71:3C:BE

Depending on the number of NIC's, ifcfg-eth1 may require some tweaking, as well. You should now successfully be able to start networking services. 

hostname

Change the hostname; edit /etc/sysconfig/network, and adapt /etc/hosts

Now, here's an idea...

Gaining control

Or rather - regaining control. Over my own data, and what's done with it.

Currently, I use several services, of which I know they are monitored. Several of these services fall under US legislation, although I'm not a US citizen. This allows several agencies to go through my documents, email and other stuff, whether I like that or not (I do not).

Of course, for some of this, I gave permission - blogging on a google platform undoubly allows google to scan, "in order to enhance services rendered". Or something similar. Using gmail: ditto. Drobox: ditto. MS Windows: Ditto.

And all of these firms store data on US territory, or are US based, which basically tells me my data is being scanned.

Now, I am aware of this, but not overly comfortable with it. I like my privacy. I like the idea of being innocent until proven otherwise.

So, how about taking matters in one's own hands? How about setting up my own email and cloud services?

Services wanted

Just freewheeling here, but how about: 

• replace gmail by dovecot 
• replace dropbox by nextcloud (successor/fork of owncloud) 
• create some virtual/cloud computing platform to replace ESXi. My own Azure, so to speak. 

I do have a previous (not documented) ESXi server build, and I run some 10 virtual machines on it, one of them being FreeNAS - because of it's native ZFS.
This works as a charm, despite the FreeNAS community being.... let's say sceptical, about the idea.
The only problem is that ESXi looses connection, and that is canof hard to re-establish.
So, I felt the urge to build a dedicated, 24/7 storage server. 
This would also have to take care of some laptop storage, and runs 24/7, so it better be energy-efficient. Of course, I'm not able, budget wise, to go for the ultimate option, SSD-only. A mix of SSD and 2.5" drives should do, and I could probably salvage some 3TB disks of the ESXi build. 

Enter Sub-project 1: storage server.