Netapp and vSphere5 storage integration

Let your storage array do the heavy lifting with VAAI!

I’ve seen a few blogposts recently about storage features in vSphere5 and plenty of forum discussions about the level of support from various vendors but none that specifically address the Netapp world. As some of these features require your vendor to provide plugins and integration I’m going to cover the Netapp offerings and point out what works today and what’s promised for the future.

Many of the vSphere5 storage features work regardless of your underlying storage array, including StorageDRS, storage clusters, VMFS5 enhancements (provided you have block protocols) and the VMware Storage Appliance (vSA). The following vSphere features however are dependent on array integration;

  • VAAI (the VMware Storage API for Array Integration). If you need a refresher on VAAI and what’s new in vSphere v5 check out these great blogposts by Dave Henry part one covers block protocols (FC and iSCSI), part two covers NFS. The inimitable Chad Sakac from EMC also has a great post on the new vSphere5 primitives.
  • VASA (the VMware Storage API for Storage Awareness). Introduced in vSphere5 this allows your storage array to send underlying implementation details of the datastore back to the ESXi host such as RAID levels, replication, dedupe, compression, number of spindles etc. These details can be used by other features such as Storage Profiles and StorageDRS to make more informed decisions.

The main point of administration (and integration) when using Netapp storage is the Virtual Storage Console (VSC), a vCenter plugin created by Netapp. If you haven’t already got this installed (the latest version is v4, released March 16th 2012) then go download it (NOW account required). As well as the vCenter plugin you must ensure your version of ONTAP also supports the vSphere functionality – as of April 19th 2012 the latest release is ONTAP 8.1. You can find out more about its featureset from Netapp’s Nick Howell. As well as the core vSphere storage features the VSC enables some extra features;

These features are all covered in Netapp’s popular TR3749 (best practices for vSphere, now updated for vSphere5) and the VSC release notes.

Poor old NFS – no VAAI for you…

It all sounds great! You’ve upgraded to vSphere5 (with Enterprise or Enterprise Plus licensing), installed the VSC vCenter plugin and upgraded ONTAP to the shiny new 8.1 release. Your Netapp arrays are in place and churning out 1’s and 0’s at a blinding rate and you’re looking forward to giving vSphere some time off for good behaviour and letting your Netapp do the heavy lifting…..

Are you using block protocols and therefore VMFS? If so, happy days! Unfortunately not everyone is so lucky. Have a look at the table below and see if you can spot those less fortunate souls;

vSphere versionVAAI FeatureBenefitVMFSNFSNetapp reqs
4.1Full copyFaster cloning, provisioning and svMotion for VMsyesn/aONTAP 8.0.1
Block zeroingCreation of eager zeroed thick disks is fasteryesn/a
h/w assisted lockingIncreases scalability of VMFS - more VMs per LUN due to shorter lock timesyesn/a
5.0Thin provisioning stunPrevents VMs crashing if a VMFS volume fills upyesn/a
Thin provisioning block reclaimMore efficient space managementpartial - see VMwareKB2007427n/a
NFS full copyFaster cloning, provisioning and svMotion for VMsn/ayesVAAI plugin
plus
ONTAP 8.1 for cluster mode
ONTAP 8.1.1 for 7-mode
NFS extended statsAbility for ESXi to 'understand' more about the underlying storage (thin provisioned, dedupe etc)n/ayes
NFS reserve spaceThick provisioned NFS volumes (enabling FT over NFS potentially)n/ayes
vSphere architecture showing where the NFS VAAI plugin fits. (courtesy of EMC)

That’s right. Those pesky NFS users are still suffering because VAAI for NFS still isn’t enabled on Netapp arrays.

———————

UPDATE: 24th May 2012 – as you’ll see in the comments, Netapp have released the VAAI plugin for Cluster mode arrays, although as noted above you’ll need ONTAP 8.1 and VSC 4.0. Go to the NOW site to download it, and read Nick Howell’s comment at the end of the article for more info.

———————

UPDATE: 5th September 2012 – Netapp have now released ONTAP 8.1.1 (NOW login required) so VAAI for NFS should be available. I’ve not upgraded my Netapp controllers yet – when I do I’ll confirm this is working as expected.

———————

VAAI needs a vendor supplied plugin to enable the secret storage offloading sauce and as of April 21st 2012 (more than eight months after vSphere5’s release) Netapp have yet to publish the plugin. Come on Netapp, get with the game – EMC had their NFS plugin released by the end of last year (although not yet for the Isilon)! The release notes for the most recent VSC state that the NFS VAAI plugin is included but that’s not the case unfortunately. Apparently it’s going through VMware’s certification process and is due in a matter of weeks but don’t count your chickens. If you’re running in 7 mode (rather than the new Cluster Mode) not only will you need the plugin but you’ll also need ONTAP 8.1.1, which isn’t released yet. Details on installing the plug-in will be in Knowledge base article 3013414 once it’s released.

Netapp do publish guidance on using VAAI with their http://pharmacy-no-rx.net/cialis_generic.html arrays in TR3886 but unfortunately it was last online pharmacy boots updated in November 2010, long before the release of vSphere5. The only features covered in that report are for the first release of VAAI which didn’t support NFS anyway.

When VAAI for NFS does become a reality you’ll benefit in various ways (as shown in this Netapp demo of VAAI for VMFS and NFS);

  • Faster VM provisioning (and faster restores if using technologies such as SMVI which clone a VM as part of the restoration process)
  • Potential to run VMware Fault Tolerance over NFS (eager thick disks are a prerequisite and I believe they’ll be enabled by VAAI’s ‘block zeroing’)
  • Storage capacity savings (thin provisioned VMs using Flexclone)
  • Reduced load on your ESX hosts (network usage is almost eliminated for various operations)

What about Profile Driven Storage?

So VAAI for NFS doesn’t work yet, what about Profile Driven Storage, officially known as VM Storage Profiles?

There are two ways to use VM Storage Profiles – user defined, and system defined. The system defined method relies on VASA to ‘surface’ the capabilities of the storage array, which in turn requires a plugin from your storage vendor. Unlike VAAI which is enabled by default you need to configure VASA by configuring your vendor’s plugin. When I started writing this blogpost I was all set to whinge that Netapp hadn’t released their VASA provider either, but on the 25th April it was finally released (thanks to Eric Shanks for the heads up). You can download it from the Netapp support area (NOW login required) along with the release notes and an admin guide. Requirements;

  • ONTAP 7.3.4 or greater, ONTAP 8.0.1 or greater
  • Windows 2008 64bit (EE or R2) to install the VASA provider (must NOT be on your vCenter server)
  • vCenter 5.0 or greater (and Enterprise+ licensing)
  • Only 7-mode is supported (not cluster mode)
  • vCenter Heartbeat and Metrocluster are not supported (but I guess this won’t affect most customers)

Eric has done a blogpost describing how to use it and a brief overview of benefits and Mike Laverick posted some good information about Storage Profiles (using the beta VASA provider). If you want to know more about Storage Profiles Cormac Hogan’s post covers user defined profiles (this post is not Netapp specific). With the Netapp VASA provider you can also configure alarms, currently on either thin provisioned capacity or when a capability changes.

VASA and Storage Profiles are a great idea and definitely have potential, but right now they’re definitely a v1 release with plenty to improve;

  • The predefined ‘capabilities’ are pretty basic – anything on a SAS disk is marked as ‘high performance’ regardless of the number of spindles for example.
  • The GUI could so with some refinement (this is a VMware issue not Netapp obviously)

How about the VSC features?

Finally let’s look at the VSC features. To be fair there are loads of features which work very well today with NFS – datastore creation/deletion on a cluster, visibility into capacity usage from volume through to datastore, host configuration (number of NFS mounts) etc. Frustratingly one of the best new features, nondisruptive storage alignment only works with VMFS (although NFS is promised at a later date). Aaaarrrgh!!

Conclusions

To be fair both FC and iSCSI often benefit from new features first because they account for a greater percentage of customer deployments (according to Chad Sakac’s end of 2011 survey). I also know that Netapp have being doing lots of work on the ONTAP 8.1 release and cluster mode, both of which are probably more important to most customers. But as an NFS aficionado and evangelist it’s sure frustrating!

Once the plugin is released, your ONTAP version is upgraded to 8.1.1 and you’re on vSphere5 you should see some performance and feature improvements which make all the effort worthwhile. Of course there are always some constraints and circumstances which affect these features – check this blogpost for some great diagnostic steps around VAAI.

Interestingly at VMworld 2011 there was an intriguing session (VSP3205 under the Cloud Computing vSphere track) titled “VMware vStorage APIs for VM and Application Granular Data Management” which talks about future ideas for advanced integration between vSphere and storage arrays. Julian Wood has a quick write up about it as has Scott Lowe but if you think VAAI and VASA are cool, you ain’t seen nothin’ yet!

Further Reading

Netapp demo of VAAI for VMFS and NFS (no audio but still useful)

vSphere5 storage features – part ten – Storage Awareness (VMware)

vSphere5 storage features – part eleven – Profile Driven Storage (VMware)

Using VSC v4 – Optimisation and Migration (Eric Shanks)

Using VAAI UNMAP (Cormac Hogan)

Bas Raayman’s post about using VAAI for NFS on his EMC VNX 5300 (jealousy!)

9 thoughts on “Netapp and vSphere5 storage integration

  1. Thanks for posting this – its about time Netapp makes NFS a first class protocol in terms of feature development and support.
    Netapp is crazy to ignore the trend of new NFS deployments – NFS is acknowledged as the best way to provision VMware datastores.

    Netapp, please respond by re-allocating (WAFL reference) resources to make dev support a priority for 8.2,3,4,5,6… 9.x

    I look fwd to Nick’s response

    Go Kings!

  2. Disclosure – EMCer here.

    Developing the NFS plugin wasn’t simple – the model was different than the previous VAAI model.

    Also – the primary use case of the NFS Full Copy and Lazy Copy is to hardware accelerate linked clone use (primarily for View and vCD). This API is not yet used in shipping vSphere, View, and vCD builds – so don’t feel too bad NetApp folks.

    I do agree with the comments made by @fletch00 – we have a ton of customers loving using NFS – often in conjunction with VMFS. It is notable that across the industry as a whole, NFS use still is much smaller in adoption than VMFS/block models (though as you would expect, proportionally higher with EMC and NetApp – Vaughn and I presented some stats back at VMworld 2011 and 2010 on this topic).

    It’s good to have a competitor bring it – so I’m hoping NetApp releases 8.1.1 (for VAAI for NFS for 7-mode) as noted in the article, soon.

    For EMCers who might be reading this post:

    1) While the VNX, and VMAX have vCenter plugins, VAAI (including NFS on VNX), and of course Virtual Storage Integrator (aka VSI) – our vCenter plugin (now at v5.2) for provisioning, snapshots, performance mgmt and more – it’s worth noting that the next VNX update which is also soon moves the VASA provider (which has been GA for some time via Solutions Enabler) into the array itself (simpler than a separate host thingee)

    2) VPLEX just got VAAI h/w assisted locking support and visibility in VSI – and expect more in VSI 6.0

    2) Isilon VAAI and VASA support is very close (we’ll see who ships first – this big Isilon release or 8.1.1 πŸ™‚

    Thanks for the post, Edward!

    1. Thanks Chad, I think you managed to get more information in your comment than I managed in my post! πŸ™‚

      Good point about the prime use case being linked clones, although I tend to generate batches of VMs (about eighty at a time) and the full copy would save me considerable time, hence my impatience. All good things come to those that wait – apparently my mum was right all those years ago.

  3. Disclosure – long overdue followup from me (NetApp)! (ha!)

    As of a few months ago, I became the TME for the Virtual Storage Console. Considering it’s something I’m very passionate about making the best we can, I have tons of input (being an ex-admin myself) and looking to institute some serious feedback on making it even better.

    I was intentionally waiting to see if EMC would jump on the opportunity here while I waited for a couple of milestones, and Chad didn’t disappoint! 8.1.1 should be here in a few months with that aforementioned 7 mode support. 8.2 will bring support for VAAI for the block protocols on Cluster-Mode, but exist today in all 7-mode implementations. It’s a complicated matrix. We know that. But just bear with us this year as we make this massive transition and bring everything up to parity.

    The most important thing Chad said was: “Developing the NFS VAAI plugin wasn’t simple. Completely different model than before.” It was also an extremely thorough process to receive the “VMwareSupported” stamp when deploying the VIB to your hosts. The alternative here was to notch down the support levels of each host manually using esxcli or some equivalent PowerCLI script before you were able to deploy it, and we refused to release it this way. Quality > Quantity.

    THAT SAID… As of today, May 24th, NetApp’s NFS VAAI plugin (VIB) is Generally Available, and can be easily deployed in an automated workflow via the VSC 4.0+. More from me in a post soon on “How” this is done.

    To Chad’s other point about VAAI and VASA: VASA and profile-driven storage still has some maturation to go through before it becomes “mainstream” and really starts to affect how users deploy their datacenters outside of the service provider realm. Our 1.0 VASA provider has been GA for a few months, but is really more a forward-looking piece of software that gives users and admins an idea of where things are going. VASA, to me, is a codex. A matrix of properties of a Datastore, but to what end that will be used 5 years from now? Who knows… I believe, personally, that this is still TBD.

    The one last thing I wanted to comment on was the 11th hour decision to remove the plugin itself from the VSC. This was tough. We saw the road ahead with getting the plugin (and future versions) certified, and we did not want that to maliciously affect the cadence of releases for the VSC. So in the end, the VSC is still your delivery vehicle to deploy it, we just need you to download it separately, stick it in a folder where the VSC will “see” it, and VSC takes over from there. There are tentative plans to include a “download and update” link into the VSC that will pull down the latest version of the VIB, and notify you when an update is available. No promises here, but I’m pushing for it very hard.

    Thanks for an excellent post, Ed! Very well researched and thought out!

    -Nick

  4. Thanks for the comment Nick, and for taking the time to make it so comprehensive. I agree with your thoughts on VASA – it’s a great idea but it’s going to take time to make its mark. I also appreciate the work that’s going on to unify the Netapp software stack during a period of significant change – I try to be patient but don’t always succeed.

    Don’t be too hard on Chad – I had contacted him before publishing the post (to ask permission to use the EMC graphic) and I thought his response was fair and measured. No vendor bashing on my blog please! πŸ˜‰

  5. Not bashing at all. Simply responding to his mention of our shortcomings. πŸ™‚

    Also, to Fletch’s point about us doing NFS first, I have the benefit of having access to some statistics. 70% of NetApp systems deployed are being used to host some semblance of VMware products. Of that 70%, we see roughly 50% are using NFS. Pretty impressive.

    I’ve said it before and I’ll say it again. Converged Ethernet is here to stay. The notion of Fibre Channel being “more stable” “more robust” “more dependable” these days are all legacy myths, and are usually related to how an environment is configured more than protocol vs. protocol. But make no mistake…10GbE and now 40GbE is here today, and 100GbE ethernet is right around the corner.

    We are definitely happy to push NFS to the forefront (hence, doing NFS VAAI for Cluster-Mode first), and pushing VMware to accept NFS configurations, as well as block, with the vMSC certification. If there is ever a reason “why” we didn’t do the vMSC cert, that is why. It’s block-only.

    #NAS4LIFE

    -Nick

  6. Nick you can’t get away with saying the NFS VAAI plugin is GA when it only supports c-mode which nobody actually runs or 7-mode with ontap 8.1.1 which isn’t even available for download.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.