This is a quick post to say that over the last few days I’ve been busily posting the last few objectives for the VCAP-DCA exam, and after quite a few months I’m finally done! You can download a PDF of the completed notes or refer to the VCAP-DCA index page for the online versions.
Enjoy!
Host Profiles are a new feature to vSphere 4 but are only available to Enterprise+ licencees. As my company haven’t yet found a need for Enterprise+ features I’d not really worked with them before so this section was new to me. Interestingly the main reference given in the blueprint is the Datacenter Administration Guide which has very little about host profiles. The ESX/ESXi configuration guides have a small section on host profiles but not much, so the best reference is probably the VMware Host Profiles – Technical Overview whitepaper.
Skills and Abilities
- Use Profile Editor to edit and/or disable policies
- Create sub‐profiles
- Use Host Profiles to deploy vDS
Tools & learning resources
Host Profiles (VCP revision)
Basically host profiles are the equivalent of Microsoft’s Group Policy, but for VMware hosts.
- Two primary uses
- Ease deployment challenges (faster, more consistent)
- Ongoing configuration control and audit reporting
- Policy options (determining how a configuration setting is applied)
- Use a fixed configuration
- Ask the user how to configure it
- Use an intelligent policy (using one or multiple criterion)
- Disregard a setting
- Works in a similar fashion to Update Manager;
- Create a baseline from a reference host.
- Attach the host profile to the hosts or clusters you want to configure
- Remediate (configure) the hosts or clusters
- Review compliance status
- Unlike VUM it can’t remediate all the hosts in a cluster automatically (it won’t put them into maintenance mode for you etc). You can attach a profile to the cluster but you have to apply to each host manually (this is largely because the host profile may require user input).
- Can only be used on vSphere hosts (not VI 3.x)
- Must be created using a reference host, or imported from a previously created host profile.
- Can be exported (in VMware Profile Format, *.vcf, which is XML content). Host Profiles are not shared using vCentre Linked Mode, you have to export/import to other vCentre instances.
NOTE: Administrator passwords aren’t exported as a security measure.
- An ESX reference host can be applied to either ESX or ESXi. An ESXi reference host can ONLY be applied to another ESXi host.
- When updating a host using a host profile you have to manually put the host in maintenance mode first. This is a significant issue for some people (although if you’re licenced for host profiles you’ve also got licences for vMotion and DRS so moving VMs off the host is potentially easier). Note that you need to enter maintenance mode even for trivial settings such as setting the time, timezone etc. Any setting which normally requires a reboot (changing service console memory for example) will still need a reboot.
- You must have both host profile privileges (create, delete, edit etc) AND privileges to configure the area in question (Networking, Storage etc) for the operation to be allowed.
REAL WORLD: When building a new ESX/ESXi host it will have a 60 day eval period with all features enabled so even if you don’t have Enterprise+ licencing you can use host profiles for initial configuration.
Read more…
Knowledge
- Understand the DRS slot‐size algorithm and its impact on migration recommendations
- Identify tools needed for monitoring capacity planning
- Identify performance metrics related to resource contention and saturation
Skills and Abilities
- Predict when additional ESX/ESXi Host, network or storage resources will be required by observing an existing environment
- Determine when to expand or contract provisioned Virtual Machine resources based upon observed Virtual Machine utilization
- Interpret performance metrics from vCenter to properly size the environment
Tools
Again there is a considerable overlap between this objective and the others in section three – the goal of understanding the DRS slot-size is an exact duplicate from section 3.3!
DRS slot size algorithm and its impact on migration recommendations
This was covered in section 3.3. You can always reread the DRS deepdive at Yellow Bricks.
Identify tools needed for monitoring capacity planning
- vCenter Performance Charts
- vCenter Storage views
- esxtop (particularly in batch or reply mode)
- Perfmon
- Third party tools (not likely in VCAP-DCA exam though)
Consider SCSI reservations per LUN, number of VMs per LUN. Adaptive vs predictive LUN sizing.
Predict when additional ESX/ESXi Host, network or storage resources will be required by observing an existing environment
Refer to section 3.1 for the metrics to check. Ballpark;
- Memory – how much is in the host compared to active memory used? Factor in reservations etc
- Network – any dropped packets? Might imply greater bandwidth required…
- CPU – check for long term patterns using Performance Charts.
- I/O – high latency or lack of capacity are the main indicators to look for
Interpret performance metrics from vCenter to properly size environment
Be aware what the various metrics actually show you. For example what’s the difference between Host Memory and Guest Memory in the screenshot below?? The answers can be found in VMworld session TA8129 Beginners guide to performance management.
vCenter statistics
vCenter and ESXTOP present statistics differently. While ESXTOP tends to display a more useful figure (%CPU ready for example) the value presented in vCenter needs to be calculated depending on the time interval.
Remember that vCenter summary statistics can sometimes mislead – memory per host looks fine in the screenshot above but you might find NUMA locality is low (for example).
Knowledge
- Recall vicfg-* commands related to listing storage configuration
- Recall vSphere 4 storage maximums
- Identify logs used to troubleshoot storage issues
- Describe the VMFS file system
Skills and Abilities
- Use vicfg-* and esxcli to troubleshoot multipathing and PSA‐related issues
- Use vicfg-module to troubleshoot VMkernel storage module configurations
- Use vicfg-* and esxcli to troubleshoot iSCSI related issues
- Troubleshoot NFS mounting and permission issues
- Use esxtop/resxtop and vscsiStats to identify storage performance issues
- Configure and troubleshoot VMFS datastores using vmkfstools
- Troubleshoot snapshot and resignaturing issues
Tools
- Product Documentation
- vSphere Client
- vicfg-* , esxcli, resxtop/esxtop,vscsiStats, vmkfstools
There’s obviously a large overlap between diagnosing performance issues and tuning storage performance, so check section 3.1 in tandem with this objective.
Recall vicfg-* commands related to listing storage configuration
- vicfg-scsidevs
- vmkiscsi-tool
- vicfg-mpath
- vicfg-iscsi
- esxcli corestorage | nmp | swiscsi
- vicfg-nas
- showmount -e
- esxtop/resxtop
- look for CONS/s – this indicates SCSI reservation conflicts and might indicate too many VMs in a LUN. This field isn’t displayed by default (press ‘f’ then ‘f’ again to add it)
- vscsiStats
- vmkfstools
- vicfg-module
Knowledge
- Identify resxtop/esxtop metrics related to memory and CPU
- Identify vCenter Server Performance Chart metrics related to memory and CPU
Skills and Abilities
- Troubleshoot ESX/ESXi Host and Virtual Machine CPU performance issues using appropriate metrics
- Troubleshoot ESX/ESXi Host and Virtual Machine memory performance issues using appropriate metrics
- Use Hot‐Add functionality to resolve identified Virtual Machine CPU and memory performance issues
Tools & learning resources
This is another objective that’s hard to quantify – experience will be the main requirement! There are some great general purpose resources out there;
Note that resxtop (built in to the vMA) does not offer the ‘replay’ mode available in ESX classic. Source: VMworld session MA6580, vMA Tips and Tricks. Read more…
Knowledge
- Identify virtual switch entries in a Virtual Machine’s configuration file
- Identify virtual switch entries in the ESX/ESXi Host configuration file
- Identify CLI commands and tools used to troubleshoot vSphere networking configurations
- Identify logs used to troubleshoot network issues
Skills and Abilities
- Utilize net-dvs to troubleshoot vNetwork Distributed Switch configurations
- Utilize vicfg-* commands to troubleshoot ESX/ESXi network configurations
- Configure a network packet analyzer in a vSphere environment
- Troubleshoot Private VLANs
- Troubleshoot Service Console and vmkernel network configuration issues
- Troubleshooting related issues
- Use esxtop/resxtop to identify network performance problems
- Use CDP and/or network hints to identify connectivity issues
- Analyze troubleshooting data to determine if the root cause for a given network problem originates in the physical infrastructure or vSphere environment
Tools & learning resources
Identify virtual switch entries in a VMs configuration file
Contains both vSS and vDS entries;
In the example VM below it has three vNICs on two separate vDSs. When troubleshooting you may need to coordinate the values here with the net-dvs output on the host;
- NetworkName will show “” when on a vDS.
- The .VMX will show the dvPortID, dvPortGroupID and port.connectid used by the VM – all three values can be matched against the net-dvs output and used to check the port configuration details – load balancing, VLAN, packet statistics, security etc
NOTE: Entries are not grouped together in the .VMX file so check the whole file to ensure you see all relevant entries.
Identify virtual switch entries in the ESX/i host configuration file
The host configuration file (same file for both ESX and ESXi);
Like the .VMX file it contains entries for both switch types although there are only minimal entries for the vDS. Most vDS configuration is held in a separate database and can be viewed using net-dvs (see section 6.3.7).
Command line tools for network troubleshooting
The usual suspects;
- vicfg-nics
- vicfg-vmknic
- vicfg-vswitch (-b) for CDP
- vicfg-vswif
- vicfg-route
- cat /etc/resolv.conf, /etc/hosts
- net-dvs
- ping and vmkping
Knowledge
- Identify CLI commands and tools used to troubleshoot management issues
Skills and Abilities
- Troubleshoot vCenter Server service and database connection issues
- Troubleshoot the ESX Service Console firewall
- Troubleshoot ESX/ESXi server management and connectivity issues
- Determine the root cause of vSphere management or connectivity issue
Tools
Identify CLI tools used to troubleshoot management issues
- vicfg-vswitch
- vicfg-vmknic
- vicfg-vswif
- vpxd.exe -s
There are a few more covered later in this objective for restarting management agents on ESX/i hosts. This VMware article on resolution paths is a great place to start learning about troubleshooting.
Troubleshoot vCenter Server service and database connection issues
- Check the VMware vCenter service is started and the account it’s configured to run as. Check that account isn’t locked out.
- Start vCentre using vpxd.exe;
- ‘vpxd.exe –s’ to start it as an application rather than a service. This will show error messages in plain text rather than the cryptic service codes.
- ‘vpxd.exe –p’ refreshes the password hash used to connect to the database. Used after replacing the default SSL certificates (VMwareKB1003070)
- How to set SQL as a service dependency – blog post
- With a lab setup and SQL Express the database often grows to the 4GB limit, at which point the vCenter service will fail. Follow VMwareKB1025914for details of how to cleardown data in the vCenter database.
- Check the ODBC connectivity using the ‘Test’ button. Check the SQL security logs to see failed authentication attempts.
VMwareKB1003979 gives a good overview of the previous processes.
Read more…
This is one objective where you definitely have to get hands on – there’s no way you’ll learn esxtop otherwise. Ideally you’ll have a real infrastructure to play with as you want hosts with memory contention, ballooning, swapping, NUMA optimisations etc so you can play with and understand the features.
Knowledge
- Identify hot keys and fields used with resxtop/esxtop
- Identify fields used with vscsiStats
Skills and Abilities
- Configure esxtop/resxtop custom profiles
- Determine use cases for and apply esxtop/resxtop Interactive, Batch and Replay modes
- Use vscsiStats to gather storage performance data
- Use esxtop/resxtop to collect performance data
- Given esxtop/resxtop output, identify relative performance data for capacity planning purposes
Tools & learning resources
Using resxtop
Two ways of invoking;
- resxtop –server <esxi host>
- resxtop –server <vCenter server> –vihost <esxi host>
Knowledge
- Explain DRS affinity and anti‐affinity rules
- Identify required hardware components to support DPM
- Identify EVC requirements, baselines and components
- Understand the DRS slot‐size algorithm and its impact on migration recommendations
Skills and Abilities
- Properly configure BIOS and management settings to support DPM
- Test DPM to verify proper configuration
- Configure appropriate DPM Threshold to meet business requirements
- Configure EVC using appropriate baseline
- Change the EVC mode on an existing DRS cluster
- Create DRS and DPM alarms
- Configure applicable power management settings for ESX Hosts
- Properly size virtual machines and clusters for optimal DRS efficiency
- Properly apply virtual machine automation levels based upon application requirements
Tools & learning resources
Advanced DRS
- Read the DRS deepdive at Yellow Bricks.
- Use the (new to vSphere) DRS Faults and DRS History tabs to investigate issues with DRS
- By default DRS recalculates every 5 minutes (including DPM recommendations), but it also does so when resource settings are changed (reservations, adding/removing hosts etc).For a full list of actions which trigger DRS calculations see Frank Denneman’s HA/DRS book.
- It’s perfectly possible to turn on DRS even though all prerequisite functionality isn’t enabled – for example if vMotion isn’t enabled you won’t be prompted (at least until you try to migrate a VM)!
Affinity and anti-affinity rules
There are two types of affinity/anti-affinity rules;
- VM-VM (new in vSphere v4.0)
- VM-Host (new to vSphere 4.1)
The VM-VM affinity is pretty straightforward. Simply select a group of two or more VMs and decide if they should be kept together (affinity) or apart (anti-affinity). Typical use cases;
- Webservers acting in a web farm (set anti-affinity to keep them on separate hosts for redundancy)
- A webserver and associated application server (set affinity to optimise networking by keeping them on the same host)
VM-Host affinity is a new feature (with vSphere 4.1) which lets you ‘pin’ one or more VMs to a particular host or group of hosts. Use cases I can think of;
- Pin the vCenter server to a couple of known hosts in a large cluster
- Pin VMs for licence compliance (think Oracle, although apparently they don’t recognise this new feature as being valid – see the comments in this post)
- Microsoft clustering (see section 4.3 for more details on how to configure this)
- Multi-tenancy (cloud infrastructures)
- Blade environments (ensure VMs run on different chassis in case of backplane failure)
- Stretched clusters (spread between sites. See this Netapp post for Metrocluster details)
To implement them;
- Define ‘pools’ of hosts.
- Define ‘pools’ of VMs.
- Create a rule pairing one VM group with one host group.
- Specify either affinity (keep together) or anti-affinity (keep apart).
- Specify either ‘should’ or ‘must’ (preference or mandatory)
Read more…
This objective is focused on the VMs rather than the hosts but there’s still a large overlap between this objective and the previous one.
Knowledge
- Compare and contrast virtual and physical hardware resources
- Identify VMware memory management techniques
- Identify VMware CPU load balancing techniques
- Identify pre‐requisites for Hot Add features
Skills and Abilities
- Calculate available resources
- Properly size a Virtual Machine based on application workload
- Configure large memory pages
- Understand appropriate use cases for CPU affinity
Tools & learning resources
Identify memory management techniques
The theory – read the following blogposts;
The following memory mechanisms were covered in section 3.1 so I won’t duplicate;
- transparent page sharing
- ballooning (via VMTools)
- memory compression (vSphere 4.1 onwards)
- virtual swap files
- NUMA
There are also various mechanisms for controlling memory allocations to VMs;
- reservations and limitations
- shares – disk, CPU and memory
- resource pools (in clusters)
Disable unnecessary devices in the VM settings (floppy drive, USB controllers, extra NICs etc) as they have a memory overhead.
Loading ...
Recent Comments