With the launch of the new vCloud Suite along with new VMware certification tracks there’s no shortage of technologies to learn so I’ve been building up my home lab in anticipation of some long hours burning the midnight oil. While doing this I’ve been mulling over a simple (I thought) question;
Why buy hardware to build home labs? Can’t we use ‘the cloud’ for our lab requirements?
I spent a while investigating the current marketplace and while some areas are well covered some are just getting started.
A typical IT ‘stack’
As an infrastructure guy I’m interested in the lower half of the IT stack, principally from the hypervisor downwards (I expect that some infrastructure professionals will need to focus on the top part of the stack in the future, but that’s a different post). There are a plenty of cloud services where you can quickly spin up traditional guest OS or application instances (any IaaS/PaaS/SaaS provider, for example Turnkey Linux do some great OSS stuff) but a more limited number that let you provision the lower half of the stack in a virtual lab;
- At the network layer Cisco’s learning labs offer cloud labs tailored to the Cisco exams (primarily CCNA and CCNP) and are sold as bundles of time per certification track. In October last year Juniper launched the Junosphere Labs, an online environment that you can use for testing or training.
- For storage EMC provide labs and this year their internal E-Lab is going virtual and a private cloud is in the works (thanks to vSpecialist Burak Uysal for the info). Scott Drummunds has a great post illustrating what these labs offer – it’s pretty impressive (and includes some VMware functionality). These labs let partners test and learn the EMC product portfolio by setting up ‘virtual’ storage arrays and is something that you’d probably struggle to do in most labs. Other storage vendors such as Netapp offer virtual storage appliances (or simulators) but you’ll need to use a separate IaaS service to run them – there’s no public cloud offering.
- At the hypervisor layer (although more application and guest OS focused) there’s Microsoft’s Technet labs. These have been available for years and for free (are you listening VMware? ) and let you play with many of Microsoft’s applications, including Hyper-V, in a live, online lab (Vladan has a good article here, and you can try Windows 2012 labs too). At the latest TechEd2012 conference the labs were made available online for two months afterwards and they were also available at the recent Microsoft Management Summit. As Hyper-V can virtualise itself but can’t run nested VMs the labs are limited to looking at the Hyper-V configuration. I tried these labs and was very impressed – they’re free, easy and quick to use (even if they do require IE).
- According to this post on Linked-In, HP are also looking at the option of publicly available virtual labs although I couldn’t find any information on what they’ll include.
While not strictly cloud labs (depending on your definition of a cloud service) you could rent space and/or infrastructure in someone else’s datacenter – recently I’ve seen companies start to specialize in offering prebuilt ‘lab’ environments which you can rent for training/testing purposes;
Many large companies will have their own lab facilities and some global companies might offer them internally via private clouds but until recently there were no public cloud services which let you experiment with the hypervisor layer. The well known blogger David Davis had similar thoughts last year and investigated cloud providers who provide ESXi as a VM and was unable to find any. There’s no technical reason why not – vSphere has been able to virtualise itself and run nested VMs for years and although performance might suffer that’s often a secondary concern for a lab environment. It’s also not officially supported but if it’s for training and test/dev rather than production is that a barrier?
As the complexity of virtual infrastructures increases it’s becoming harder to manage using conventional monitoring tools which were built with a more static environment in mind. In March 2011 VMware released the vCenter Operations product (vCOPS) to address this pain point. I’ve been running the 60 day trial at my company and now that the trial’s ending it’s time to share my thoughts.
What is vCOPS?
To quote the product page at VMware;
VMware vCenter Operations uses patented analytics and powerful visualizations to automate performance, capacity and configuration management. It collects and analyzes performance data, correlates abnormalities and identifies the root cause of building performance problems. VMware vCenter Operations provides capacity management to optimize resource usage and policy-based configuration management to assure compliance and eliminate sprawl and configuration drift. (emphasis my own)
The key differentiator is this promise to learn and understand the context of multiple metrics (CPU, memory, storage and network) and provide root cause analysis without you needing to manually define thresholds, benchmarks etc. Bear in mind that vCOPS is an infrastructure monitoring solution rather than application layer (which is more the domain on VMware’s AppSpeed, Quest’s solutions or ManageEngine’s Application Manager). I’m not the first blogger to cover this product so here’s some reading to get you up to speed;
While technically a ‘v1′ release the product comes from VMware’s purchase of Integrien (in August 2010) where it was originally marketed as VMAlive. Integrien have been working on the patented algorithms for several years so while the integration and VMware branding are new the guts of the product are not. VMware have published some YouTube videos or you can listen to VM Communities podcast #119 to get an overview of what vCOPS can offer. Read more…
Written by some of the top scripters in the VMware community the PowerCLI Reference book is really what it’s title states- a reference. What it does (and does very well) is present both a ‘cookbook’ of useful scripts and explain how and why they work. While it does explain some concepts along the way it’s not really pitched as an introductory guide or as the best way to learn PowerCLI (Hal Rottenberg’s book might be better if this is what you’re after). The book is split into five main sections (see the full table of contents);
- Install, configure and manage the vSphere environment. This section deals with vCenter automation, host deployment along with automated storage and networking provisioning.
- Managing the VM lifecycle. Deals with creating, customising, and configuring VMs and vApps.
- Securing vSphere. Covers backups, DR, security hardening and compliance.
- Monitoring and reporting. Generating reports, statistical data, monitoring and auditing.
- Scripting tools and features. Covers automation in general, the APIs (Get-View etc), Onyx, and common tools such as PowerGUI and PowerWF Studio. This chapter also covers adding a GUI to your scripts which is very useful for scripts that others need to use.
As you can see from the above list (and the fact it’s over 700 pages) it covers a lot of material but despite this I’m impressed with the technical depth on each – I picked areas where my knowledge is strongest (though not in the same league as these guys) and still found myself learning something new everytime. For example I’ve used the VIX API while creating a scripted deployment for my test and dev environments at work and thought I knew it reasonably well. To my surprise the book delved into the inner workings of the cmdlets themselves and explained how they in turn called some guest OS scripts which ship with PowerCLI. There was also had a good script for specifying a VM folder location via script, something I’d not implemented before as I couldn’t think of an easy way to specify the path. The index lists the pages where each cmdlet is used so it’s easy to look up the cmdlet you’re interested in and see code examples.
The scripts are downloadable from the book’s very own website and the authors have even put together a module containing all the code along with instructions for how to use it. This is a major bonus – you get nearly 80 prewritten functions you can integrate into your own scripts! These are useful for day to day administration, not just esoteric or niche functions. It’s worth checking this site out even if you’ve got the book – there are forums to discuss the scripts and at the moment they’re running a competition where to be in with a chance of winning you just have to take a photo of the book with a well known landmark in the background (ala ‘the orange HA book’ by Frank Denneman and Duncan Epping). I’m not sure how popular this will be as it’s a beast of a book to carry around, but that just means you’re chances of winning are that bit better!
It’s available in colour paperback or Kindle version (which is newly available again).
Disclosure – I’ve met both Jonathan Medd and Al Renouf at the VMware User Group on several occasions and was sent a copy of the book to review. There was no obligation to write a positive review and I’ve said it as I see it. I’d have bought the book anyway!
- Identify vCenter Server log file names and locations
- Identify ESX/ESXi log files names and locations
- Identify tools used to view vSphere log files
Skills and Abilities
- Generate vCenter Server and ESX/ESXi log bundles
- Use vicfg‐syslog to configure centralized logging on ESX/ESXi Hosts
- Test centralized logging configuration
- Configure the vMA appliance as a log host
- Use vilogger to enable/disable log collection on the vMA appliance
- Use vilogger to configure log rotation and retention
- Analyze log entries to obtain configuration information
- Analyze log entries to identify and resolve issues
Tools & learning resources
I’m covering the troubleshooting objectives last while preparing for the VCAP-DCA – it seems like the logical thing to do. Learn all the material then play with it, break it, fix it, recreate it etc. Practice makes perfect! I’ve been using the Trainsignal’s Troubleshooting for vSphere course but the official VMware Troubleshooting course has been getting good feedback.
vCenter log files
- %ALLUSERSPROFILE%\Application Data\VMware\VMware VirtualCenter\Logs (W2k3)
- C:\ProgramData\VMware\VMware VirtualCenter\Logs (W2k8)
- sms.log Storage Management Service
- vpxd-xxxx.log vCenter logs
- vpxd-xxxx.log.gz are archived logs. You have to unzip them to see contents.
You can change the logging level (which defaults to ‘normal’) by going to vCenter Server Settings -> Logging Options. This VMwareKB describes how to enable trivia logging in vCenter (even if vCenter isn’t running) although this may have a performance impact and should only be used temporarily while diagnosing issues.
There are numerous ways to do this; Read more…