May 19, 2012

Setup Guide for Multiple NetApp ONTAP 8.1 Simulators

I have written previously on deploying the ONTAP simulator on a vSphere host, and they seem to be some of my more popular posts.  Now that the ONTAP 8.1 simulator has been released, I thought I would do an updated post that is a little more comprehensive than my previous posts.

In my most recent post on this topic, I ran through the configuration I use to optimize usable space and getting necessary license keys installed.  I wanted to build on that for the 8.1 simulator, and specifically about getting multiple simulators to work with OnCommand.  If you install multiple copies of the simulator and then try to add them into the NetApp Management Console, you will get an error similar to this:

Even though they are separate virtual machines and have different IP addresses and hostnames, their simulator system id’s are identical so the NMC thinks it’s a duplicate host you are trying to add.  Luckily it is possible to change the serial number and system id so we can get multiple simulators added in and be able to utilize things like Protection Manager and Provisioning Manager.  The easiest way to do this is to change these values before you run the setup so you don’t have to reassign the disks.

To begin, I still utilize VMware converter to bring it into my vSphere environment.  There are other methods out there but this one has worked well for me.  I won’t screenshot the entire process as it’s mostly just taking the defaults and deciding what name you want to use etc, but here is the summary screen:

Summary screen for VMware Converter

I change the disk type to be thin provisioned instead of thick, I change the NICs to be the relevant networks for my lab, and finally I didn’t set it to power on just because I wanted to make sure I boot it into maintenance mode to change the serial number as soon as it boots up.  When it boots press a key other than enter to break the boot, then run the following commands to make this simulator unique (screenshot below):

set bootarg.nvram.sysid=1111111101
set SYS_SERIAL_NUM=1111111101
boot

Interrupt boot process to change serial number

These commands were documented in this post on the NetApp Communities, and I’ve followed the same pattern – 8 1′s with a unique two character string at the end that matches the hostname (e.g. STO-FAS1 is 01 STO-FAS2 is 02 etc)

After you hit enter to boot the simulator, you need to go into maintenance mode to wipe the config and setup the simulator, you need to hit control-C to bring up the boot menu and then option 4 for wiping the configuration:

Boot menu options

Once you run through the wizard, this is the config I apply (I usually use a text snippet to insert the following into  my SSH session):

options security.passwd.rules.enable off
snap reserve -A aggr0 0
snap sched -A aggr0 0
options autosupport.enable off
aggr options aggr0 raidsize 28
disk assign all
license add DZDACHD
license add PZKEAZL
yes
license add NAZOMKC
license add ANLEAZL
license add BSLRLTG
license add NQBYFJJ
license add ELNRLTG
license add MTVVGAF
license add BQOEAZL
license add RKBAFSN
license add HNGEAZL
license add BCJEAZL
license add DFVXFJJ
license add XJQIVFK
license add DNDCBQH
license add JQAACHD
license add ZYICXLC
license add PVOIVFK
license add PDXMQMI
license add RQAYBFE
license add ZOFNMID
license add ZOPRKAM
license add RIQTKCL
ndmpd on
options nfs.export.auto-update off

I turn off the password rules since this is just my lab and I typically use a very easy password (yep, you probably already guessed it) for most of the lab stuff that can only be accessed from inside.

Next step is to add some more disks in the simulator, this info was found on this thread over on the NetApp Communities.  Not all of the commands worked for me, in part 2 step 3 I wasn’t able to successfully enter those commands.  It mentions that was a glitch in the way the program was complied so I’m just guessing that it may now be resolved and not necessary now as without it I still was able to add disks without issue.

priv set advanced
useradmin diaguser unlock
useradmin diaguser password

Enter a password to use for the diaguser (again, it’s my lab so I use a relaxed password)

systemshell

Login with the user diag and the password you just created

setenv PATH "${PATH}:/sim/bin"
cd /sim/dev
sudo makedisks.main -n 14 -t 23 -a 2
sudo makedisks.main -n 14 -t 23 -a 3
exit
useradmin diaguser lock
priv set admin
reboot

After the system comes back up, login and we can assign the new disks

disk assign all

If you run into an error where you get a bad disk label, that can be easily fixed by:

aggr status -f

This output will give you the list of the failed disks, make note of the disk ids – they should be similar to v6.32

priv set advanced
disk unfail -s v6.32

At this point you should have a total of 56 disks on the simulator, with an aggregate raid group size of 28. I added 52 of the disks into my aggregate to leave 1 as a spare so I don’t constantly get errors about low spare count (you can disable the option to warn you about low spares but unfortunately that only works on systems with 16 disks or less).

aggr add aggr0 52

Now you should have the first simulator completed, I took a VM snapshot at this point to be able to revert to after I do my testing.  Next up is configuring the second simulator, obviously the VMware Converter steps are basically identical except for the VM name so I won’t repeat that part.  The only thing to be aware of is when you first power on the VM, press a key other than enter just like you did previously so we can set a different serial number:

 

Configuring unique serial number and system id on second simulator instance

Run through the rest of the configuration steps above and now you should be able to add both simulators into the NetApp Management Console:

Both simulators showing within the NetApp Management Console. Click for larger image.

Hopefully that helps you get started with the ONTAP 8.1 simulator, I’ll have some more posts coming up that will build off of this.

Popularity: 35% [?]

Home Lab Tip: TextExpander

TextExpander is a tool I’ve had installed on my Macbook Pro for awhile now, but never really utilized it all that often (other than the built-in spelling corrections I only had 1 or two custom expansions configured).  Recently I started using it more and I think it can be very helpful in home lab environments where you are constantly configuring things over and over with the same (or similar) settings.  I’ve configured a few NetApp simulators in my home lab and rather than remember what settings I apply each time to the simulator and configure them, I have it all configured within TextExpander so that when I type /ntapsim it will replace that line with everything I want set.  In my case it will then run the following:

options security.passwd.rules.enable off
snap reserve -A aggr0 0
snap sched -A aggr0 0
options autosupport.enable off
aggr options aggr0 raidsize 28
disk assign all
license add DZDACHD
license add PZKEAZL
yes
license add NAZOMKC
license add ANLEAZL
license add BSLRLTG
license add NQBYFJJ
license add ELNRLTG
license add MTVVGAF
license add BQOEAZL
license add RKBAFSN
license add HNGEAZL
license add BCJEAZL
license add DFVXFJJ
license add XJQIVFK
license add DNDCBQH
license add JQAACHD
license add ZYICXLC
license add PVOIVFK
license add PDXMQMI
license add RQAYBFE
license add ZOFNMID
license add ZOPRKAM
license add RIQTKCL
aggr add aggr0 24
ndmpd on
options nfs.export.auto-update off

You get the idea, but that will automatically disable AutoSupport, disable the aggregate snapshot reserve and increase the default RAID group size to squeeze a little extra usable space out of the sim, and apply all the simulator license keys for me.  I also have snippets set up for applying a default Cisco switch/router config for me which has been extremely helpful when initially configuring everything in a GNS lab.  I also have one for ESXi hosts that will tweak a couple of settings for my lab such as allowing ESXi to use less RAM.  In that case, the specific tweak was mentioned here and I just chose to insert it via the echo command instead of using vi manually.

That’s just some lab examples of what I use it for, but in reality there are tons of uses for this – almost every file on my machine is date stamped in the format yyyy-mm-dd and I have a quick two character snippet setup to automatically insert the current date in that format for when I’m creating a new folder/file.  Other examples that I use are for inserting my work phone number (I can never remember it), my address, an email signature, and a few for web addresses that I frequent.

If you aren’t willing to shell out the $35 for TextExpander (which, in my opinion, is a little high for a utility app – though it is incredibly useful), there are other ways to accomplish this.  Built in to OSX (and iOS 5 for that matter) is native text expansion, the downside is that it isn’t quite as easy to quickly add in new snippets, or modify old ones.  However, it’s free and as long as you are running OSX 10.6 or later – you already have it.  Just go to System Preferences –> Language & Text to create your snippets.

Anyone using TextExpander and have some other snippet examples?  I’d love to hear them.

Popularity: 6% [?]

ESX Whitebox for Home Lab

In a previous post I mentioned some of the upgrades I’ve been doing for my home lab, I finally got around to posting about my ESX upgrade.  In the past I’ve always used the cheapest components and have not been concerned with ongoing power costs.  Because power was a concern for me now, I needed something that supported more than 8 GB of RAM, as I don’t want to have more than two ESX hosts (anytime soon).  I ended up going with a SuperMicro X8SIL-F board, and based almost all of my components on an excellent blog post done by @rootwyrm.

ipmiThe board actually supports 32 GB RAM which I really liked, and also has remote management (minor plus for me) which also supports virtual media (big plus for me).  Now I don’t need to put a CD/DVD drive on it and since it has an internal USB port and I will be running ESXi on this system that will function just fine for the ESXi install location.

I used some memory I already had for the time being, but I still need to order some as it only has 8 GB currently.  Unfortunately it seems to be very picky on RAM and I likely will be using 4 GB sticks and having 16 GB RAM per host.

My final list of equipment was:

  1. SuperMicro X8SIL-F – $189.99
  2. Intel Xeon Quad Core X3450 – $244.98 (FT Compatible)
  3. Lian Li – $109.99
  4. Corsair VX450 Power Supply – $61.88
  5. USB Stick – $16.99
  6. Case fans – $18.88

Total: $642.71, as I mentioned above – this doesn’t include the cost of RAM.  Still a pretty well priced(and very capable!) system for a home lab.

So far I’ve been pretty impressed, unfortunately the Synology doesn’t support VLAN tags like I thought it did so I will have to have 1 interface specifically for IP storage and 1 interface for user/server traffic with no redundancy.  Looking around it seems like most of the storage options that are similar to the Synology also don’t support tags.  I also need to add a NIC yet, both onboard NICs on the motherboard are fully supported but I would like to add a few more to use for playing around with dvSwitches.

Update 2011-05-04: After a bit of searching, I finally found some 8GB RAM sticks that work in this system .  The cost for my RAM was $696.10 and I also added a Intel PCIe dual port NIC for $77.98 bringing the total for this system to $1416.79 I added all of the info to my home lab page as well.

My favorite thing about this setup?  The power usage, see the pictures below (lens flare courtesy of iPhone camera – at no extra charge)

IMG_2120

This is with the ESXi whitebox turned off, 262 watts are being used by everything else on the Kill-A-Watt

IMG_2121

This is while the ESXi whitebox is booting up, roughly 77 watts higher

IMG_2122

This is after I have about 8 GB RAM worth of VMs powered on the ESXi host. About 89 watts.

Having paid the electricity bill on 2 NetApp DS14 shelves of disk, this is a welcome treat.  I haven’t used the Kill-A-Watt on just the Synology yet but I know it is extremely low.  The 262 watts in the first picture is for almost everything listed here: the Synology, PIX 515, 3 Cisco 2600′s, 1 Cisco 2509, 1 HP ProCurve 1810G, a Cisco 3550, and 2 Cisco 2950′s – not too bad.

Another big benefit I didn’t mention is this thing is extremely quiet, if it wasn’t for the lights on the case I wouldn’t even know when it’s running.  In all honestly my MacBook Pro is louder then this thing (and it’s not a close contest).

Old ESX whitebox system when idle

Just for giggles I also measured my “old” ESX whitebox to see what results I’d get, and they were a little better than expected but still a bit more than 2x the power draw of the new system.  If my quick bar napkin math is correct it’s about a $5-10/month savings.

Featured image photo by Horia Varlan from Flickr

Update 2011-04-04: I finally purchased RAM and an extra dual port NIC for the system.  After factoring in the 32GB RAM kit I purchased and the dual port PCIe Intel NIC my total cost for the system is: $1404.69

Popularity: 18% [?]

A New Year, and New Upgrades to Home Lab

HP ProCurve 1810G-24

Now that 2011 is upon us, I’ve been focusing on updating my home lab so that I can prepare for the VMware VCAP exams as well as the Cisco CCNP exams, my goal is to finish both of these by the end of 2011.  To accomplish this I needed to make some upgrades in my home lab.  The first upgrade I needed to make was switching, about 2 weeks ago the only gigabit switch I had in my lab (a Linksys SRW2024) decided to take a permanent vacation and I set out to find a replacement.  Since there is a switching exam for the CCNP my first thought was to replace it with something that would also help me on that exam, however I quickly found that a 24 port GbE switch just wasn’t going to happen given my lab budget. I did some research and found that a lot of people with home labs are using the HP ProCurve 1810g, so I ordered a 24 port model.  I haven’t used HP switches much but I liked that it is fan less (noise is a concern for my lab, I can hear it running upstairs if I turn on all the equipment) and also that it comes with a lifetime warranty.

Synology DS1511+

Next on my list was storage, right now I have a NetApp DS14 shelf with 14x72Gb FC disks.  Remember how I said I can hear my lab upstairs?  Well this guy is the main reason, not to mention the fact that it is getting too expensive to run – especially when you consider how much usable space I’m getting out of it and that this particular system cannot run dedupe.  I had already been looking to upgrade my storage as I have a Drobo that I use with my Mac that has been giving me a lot of problems, I looked into most of the SMB offerings out there like the Iomega, QNAP, and Synology and in the end I picked the Synology DS1511+ I also picked up 5 2TB Seagate Barracuda XT drives to go in it, this should give me enough space to move everything from my Drobo as well as give me storage for my lab virtual machines.  The Synology can do both NFS and iSCSI for my VMware environment, and for the moment I am planning on keeping my NetApp/Brocade equipment for the times I want to mock something up with FC.

Image courtesy of anthonyimages via Flickr

After my wallet heals a little, I still need to buy a few more things, a couple of whitebox servers to use for ESXi hosts (I’m going with whitebox servers over something like a HP ML110 or a Dell T110 due to noise).  I’m hoping to build one that supports 16Gb RAM, but may need to settle for 8Gb depending on cost, 2 of the virtual machines I will be running are the NetApp ONTAP 8.0.1 simulator as well as the EMC Celerra Uber VSA, together those two consume a good chunk of RAM by themselves – not to mention all the additional VMs I’m planning on running.

I also will need a couple of Cisco 3550 switches to use for L3 switching, and an additional 2950 switch to use for an access switch. I have a few Cisco 2600 routers (although none that are XM version) but I’m not sure if I have enough to properly do a routing lab, right now I’ve been using GNS3/Dynamips for my studying.  I plan on making another (or multiple) posts on training material/equipment for the various CCNP exams as I go through my studies.

Popularity: 7% [?]

Configuring ASDM on Cisco PIX 515

Not being a day to day Cisco guy I thought I’d throw this up on my blog, mostly for my future reference.  I currently am running PIX OS 7.0(1) and wanted to run ASDM to make configuring my client VPN access easier.

The steps for installing ASDM are pretty easy:

PRD-FW1# copy tftp flash

Enter your tftp server and asdm file name

PRD-FW1(config)# asdm image flash:/asdm-524.bin

PRD-FW1(config)# wr mem

Then to allow the http access:

PRD-FW1(config)# http server enable

PRD-FW1(config)# http 0 0 inside

I initially installed ASDM version 5.24, it copied to flash fine but when I tried to run the following command it gave me an error:

PRD-FW1(config)# asdm image flash:asdm-524.bin

It returned an error saying “Device Manager image set, but not a valid image file flash:asdm-524.bin” If I ran a show run it did show the asdm image, and I did get the webpage to sucessfully load when accessing it from internally but nothing would happen.  My next step was to take the MS approach to troubleshooting and try a reload before pulling my hair out but that didn’t solve anything.

At this point I thought maybe I had an issue with the asdm-524.bin file and I decided to try asdm-512.bin which gave me the same exact problem.  After some searching I realized the ASDM version is apparently tied to the PIX OS version and I needed to use a version like asdm-502.bin

Now I was able to sucessfully run the “asdm image flash:asdm-502.bin” command, but my next problem was that the ASDM Launcher wouldn’t do anything after I logged in and the webpage would just sit at a screen that said “Do not close this window”.

To fix that I found I needed to have a much older version of Java, I needed version 6 update 7 x86 version After I only had that version on my laptop I was able to connect to ASDM successfully.

Popularity: 13% [?]

Configuring PIX 515 with Comcast

In my last blog post I was updating my PIX 515 software to a more recent version. My next step was to get it working on Comcast.

c515-1> enable
c515-1# conf t
Configure e0 (outside interface) to get DHCP address from Comcast
c515-1(config)# int e0
c515-1(config-if)# ip address dhcp setroute
c515-1(config-if)# no shut
Configure e1 (inside interface) as 10.0.10.1
c515-1(config-if)# int e1
c515-1(config-if)# ip address  10.0.10.1 255.255.255.0
Configure NAT (Perform NAT on all addresses in the 10.0.10.x subnet)
c515-1(config)# nat (inside) 1 10.0.10.0 255.255.255.0
Configure Global Address
c515-1(config)# global (outside) 1 interface

I still have a(n extremely) basic config, e0 is the interface connected to my cable modem which is receiving it’s IP address via DHCP from Comcast. My internal interface is e1 which is statically set to 10.0.10.1. Next I need to get DHCP configured on the 515, put my current wireless solution (Apple Airport Extreme) into bridge mode and eventually have it be replaced by a Cisco AP. I’ll also be adding different vlans into my network for separating some things out but for now I wanted to make blog posts on the individual steps.

Popularity: 4% [?]

Cisco PIX 515 for Home Lab

Installing a 3des License

3des license is now available for free from Cisco, and since my PIX didn’t have a 3des license I headed over to this link to get it for free.

Upgrading IOS to Latest Version

Memory Upgrade

Since I have a PIX 515 (not a 515e) I needed to upgrade the flash from monitor mode.  The current version on it was 6.3 and I wanted it to be at least v7 as I wanted to be able to use it as a transparent firewall.

The first step was to upgrade the memory as the 515 only had 32mb, so I added a PIX-515-MEM-32 to get to 64mb DRAM

IOS Upgrade

pix515# write net <tftp server ip>:pixconfig

Now reload the PIX and send a break to enter monitor mode

monitor> interface 1
monitor> address <ip address>
monitor> server <tftp server address>
monitor> file pix704.bin
monitor> tftp

Once the new PIX image is booted, run the following:

pix515> enable
pix515# copy tftp flash

After that I did the same exact process to get to version 8.0(4)

Popularity: 4% [?]