There are two things I love more in my work than everything else : how smart is the way we (NetApp) store data, and virtualization (especially VMware). When you realize how accessible are these technologies today, you understand it gets pretty easy to get your own lab running on your laptop, as long as you have a bunch of RAM available.

Ok, I must admit I’m not running  « cheap » hardware, but like a lot of my coworkers, I chose to work on a high-end MacBook Pro with 16GB of RAM and a 512GB SSD drive. SSD and RAM is the key here, remove one or the other and that’s going to be painful. I use my lab all the time, it saves me the hassle of finding a lab and setup base environment, and if saves me time, it also makes my customer happier.

Lab content

So far, here is what I’ve been able to run easily on my lab :

• Two dual-node cDOT (Clustered Data ONTAP) simulators with intercluster replication
• Two single node 7-mode simulator with SnapMirror
• A couple of old-fashion 7.3 simulators
• A Microsoft Windows Domain Controller
• An OnCommand Unified Manager 5.2 (7-mode)
• An OnCommand Unified Manager 5.2 (cDOT)
• An OnCommand Unified Manager 6.0 (cDOT, obviously)
• An OnCommand Workflow Automator Server (WFA)
• An OnCommand Report 1.1
• 7 mode transition tool (7MTT)

Not all of these runs at the same time, but I run the domain controller, an OnCommand UM and 4 ONTAP simulators without a sweat, which is more than enough for what I have to do.

As much as possible, you would try to consolidate multiple products running on Windows into the same Windows server, even your Active directory controller. This is not something you will want to do in a production environment of course, but it will save you a significant amount of RAM.

Since some products are using the same TCP IP ports, you don’t always have a choice, but just know that most of the time you can tweak configuration to use alternate ports. This is what I did with 7MTT (I’ll try to cover this in part II later this year).

If you’re feeling adventurous, you can also run a couple ESX servers in VMs.

Networking

One of the key thing in the configuration of your lab is networking. As you can see above, you can end up with a respectable amount of VMs created, running or not, each with its own IP.

Most of the workstation virtualization solution out there provides three ways of networking :

• NAT : Runs a private network inside your current system, which is acting like a gateway when your lab needs access to the outside/internet

• Bridge : A VM connected through bridge networking is connected on the network the same way the current system is, if your network provides DHCP, it will get an IP that way.

• Host only : With this method, the VM is privately connected to your current OS and will not be able to access internet or any other outside network.

My first advice here : use NAT, because :

• You will need to access internet, eventually,
• You will need more IP than you think,
• You will need IP you remember easily (i.e. you assigned yourself),
• You will need to run everywhere and not have your lab rely on some IP assigned by your corporate network that are no longer available at home for example (Yes, you’re working from home some times).

Another piece on advice : forget about using a class C network limited to 255 IPs, use a full class B or even class A.

Example using a class C network :

• 192.168.0.2 – local VMware managed NAT interface
• 192.168.0.10 – first cDOT cluster management network
• 192.168.0.11 – node 1 management
• 192.168.0.12 – node 2 management
• 192.168.0.20 – second cDOT cluster management network
• 192.168.0.21 – node 1 management
• 192.168.0.22 – node 2 management
• Oh wait, I need intercluster now… well ok, let’s use 192.168.0.x3 and x4
• Ok, mmm, lets create some LIFs… damn only 6 IP left in that range, I’ll never remember which one belongs to which SVM…

Believe me, this is going to be a nightmare to manage. I ended up using a class B network 10.78.x.x (you can also pick something else than your year of birth…)

• 10.78.0.x – lab management
• 10.78.1.x – First cluster
• 10.78.2.x – Second cluster
• 10.78.3.x – 7-mode stuffs

That gives you a lot of available IP and room for your creativity to define easy to remember ones.

All it takes is put 255.255.0.0 in the netmask field instead of the traditional 255.255.255.0.

VMware Fusion configuration

The standard version of VMware is limited in term of network configuration options. So I decided to spend a few more bucks and upgrade to Pro which provides the following Network preferences. You get to choose the network range and netmask for the DHCP server as well as a couple of options to let VMs on this network access external networking or allow the Mac to connect to this network.

I’m sure there are other options here, you might not need Pro version to define new networks and adjust DHCP settings, I just thought having easy access to this setting is worth the price.

[EDIT] You might consider using UBER Network Fuser for the mac instead of spending a few more on Fusion Pro.

[EDIT2] The standard simulator VM comes with 4 network ports, you might want 6 or more in some situations, but for some reason, VMware Fusion refuses to boot the VM once you added the ports, generating an exception about pci port numbers. You can use the steps described here to workaround this. All it takes is editing the .vmx file manually and add the following lines :
pciBridge0.present = « TRUE »
pciBridge0.pciSlotNumber = « 16 »

Before you enable DHCP server, think about your soon-to-be Active Directory server, maybe you will want the DHCP service to be run by the domain, it is up to you.

In the screenshot below I created a new network with the following settings :

It takes an additional configuration step to set what range of IP is going to be served via DHCP. To do this, edit the following file :

/Library/Preferences/VMware Fusion/vmnet3/dhcpd.conf

Replace « vmnet3 » with whatever network name has been assigned in the above window.

Change the line that looks like the following to whatever suits your need :

range 10.78.128.0 10.78.255.254;

While you’re here, note the IP specified as « option routers », this will be the gateway and the DNS server you need to setup when you configure network manually on your VMs or in your Active Directory server.

Windows 2008 active directory installation

Since your DNS infrastructure is going to rely on your Windows domain, that’s probably a good idea to install the domain controller first and setup active directory and DNS.

You should ultimately rely on the DNS IP provided by VMware Fusion to resolve names outside of your lab, internet addresses, etc. If you don’t, you will not find a fixed DNS server reachable from the different locations you will run lab projects.

If you use AD server as a DNS server for your lab, you need it to be running all the time otherwise you wouldn’t be able to resolve any name. One alternative to that is to configure the IP provided by VMware Fusion as a secondary DNS in your VMs. You might experience timeouts if the first DNS (AD) is not running, but at least it will work.

Data ONTAP simulators installation

I’m not covering the installation of the simulator itself here, just remember a couple of things :

• Don’t forget to change the network of all the interfaces of the VM to your lab-dedicated network created earlier. It will not be the default usually. If you experience some connectivity issues, check that first.
• Don’t forget to change the serial number of the nodes, as described in the documentation. You can only use two different serial numbers, but don’t be afraid of using the same pairs for a different cluster. Don’t forget simulator licenses are tied to the serial number.
• You can’t create clusters bigger than 2 nodes with the simulator,
• CFO (cluster fail over) is not supported (disks can’t be shared).
• When the cluster setup asks you for DNS settings, give it the address of your Active Directory server, so that registering the simulator on the domain should work right away. You can set VMWare Fusion gateway IP as a secondary DNS.
• With VMware Fusion 6, there is a problem with the way the simulator has VMware Tools configured, whereas VMware didn’t care about that issue before, it now throws an annoying warning each time you suspend or resume a simulator. A bug report has been sent to NetApp.

OnCommand Unified Manager

You have two options when it comes to OCUM installation. Windows, or Linux ? Even if I’m not a Windows guy, I almost recommend installing it on your Active Directory server and avoid the overhead of another VM. If you need multiple OCUM instances (one for cDOT and one for 7-mode for example) then that’s probably a good idea to have one on your AD and one on a new Linux system you could use for other stuffs.

I managed to install OCUM 5.2 on a CentOS 6 VM, it is a bit tricky but it definitively works. More on this in a future blog entry I guess ! [EDIT See here]

Note that OCUM 6 comes pre-installed in a bundled VM. You might want to reduce RAM for this one, which is 12GB by default…

About consolidating VMs

Let’s get that list of products in my Lab and see how we can consolidate them :

Windows Server 2008
Active Directory
OnCommand Unified Manager 5.2 for 7-mode
WFA
7MTT

Linux CentosOS 6
OnCommand Unified Manager for cDOT
ONTAP 7.3 simulators

OnCommand Unified Manager 6.0 (virtual appliance)

OnCommand Report 1.1

Wrapping up

If you only read this paragraph, here is the take out of the lengthy article :

• Run your lab on a NATed network, and use a /16 or even /8 netmask. Use whatever IP VMware provides as a gateway and DNS server.
• Consolidate products in the smallest amount of VM possible, tweak configuration files if necessary
• Use your active directory server as DNS server and configure DNS forwarder to the IP provided by VMware Fusion. You can set that same IP as a secondary DNS address in the VMs. That way if you don’t run the domain, your environment will be able to use DNS anyway, at the price of an additional timeout thought.
• Never shut down VM, just let VMware pause when you quit. If for some reasons you need resources, close out windows, it will sleep the VM by default. I never had any problem doing this, simulators just catch up nicely when waking up. This way, when you need access to an ONTAP simulator, it is just a few seconds away.

I hope you enjoyed this first article and you learnt some things, please leave your comments below, I’m always interested in feedback!