Saturday, 16 June 2007

Some of ESX3 New Commands

Part of this article has been compiled by B2V consultants & trainers and is based upon their personal experiences with the VMware ESX Server 3 product and is updated frequently. (Check the full article at http://b2v.co.uk/content/view/31/42/)

This will briefly discuss on some commands new to ESX3.

esxcfg-
This is a new unified tool that is used to configure a large number of items in ESX Server 3.0.

esxcfg-advcfg
The esxcfg-advcfg command is interesting as there is not a huge amount of help about this command. However, we can figure out that it is meant to do advanced configuration and we can figure out some settings that can be made. The -g switch is used to "get" settings; the -s switch is used to "set" settings.

[root@esx1 vmware]# esxcfg-advcfg -g /Misc/BlueScreenTimeout
Value of BlueScreenTimeout is 0

[root@esx1 vmware]# esxcfg-advcfg -g /Misc/HostName
Value of HostName is esx1.vmlab.net

[root@esx1 vmware]# esxcfg-advcfg -g /VMFS3/ZeroedThickVirtualDisks
Value of ZeroedThickVirtualDisks is 1

[root@esx1 vmware]# esxcfg-advcfg –g /Disk/SupportSparseLUN
Value of SupportSparseLUN is 1

The question is, how much is configurable? To figure out what is configurable, we recommend that you look in the directory /proc/vmware/config in the service console and you will see the directories :

BufferCache
Cpu
Disk
FileSystem
Irq
LVM
Mem
Migrate
Misc
Net
NFS
Numa
Scsi
User
VMFS3

From these directories and the files within, you can work out the paths to be supplied to the esxcfg-advcfg command as parameters. Remember case sensitivity.

Usage: esxcfg-advcfg []

-g--get Get the value of the config option
-s--set Set the value of the config option
-d--default Reset Config option to default
-q--quiet Suppress output
-k--set-kernel Set a VMkernel load time option value.
-j--get-kernel Get a VMkernel load time option value.
-h--help Show this message.
-r--restore Restore all advanced options from the configuration file. (FOR INTERNAL USE ONLY).

esxcfg-firewall
The service console in ESX 3 now has a firewall enabled by default. We use this command to view and configure the firewall rules.
The most popular switch will be the -q switch to query the firewall for its settings.

[root@esxhost1 root]# esxcfg-firewall -q


The -s switch will allow you to enable or disable network services that may traverse the firewall successfully. The list of known services are shown below - very case sensitive!....

nfsClient
ftpServer
ntpClient
dellom
nisClient
vncServer
tmpLicenseClient
swISCSIClient
CIMHttpsServer
sshClient
snmpd
tmpAAMClient
vpxHeartbeats
smbClient
hpim
tmpHostVmdbServer
tmpHostdSOAPServer
ftpClient
sshServer
ibmdirector
CIMHttpServer
telnetClient

The -q switch queries the state of a particular known service.

The -l switch loads the firewall and enables the IP tables.

The -u switch unloads the firewall and disables the IP tables.

We use the -e switch to enable a particular known service, so if we wanted to enable ssh outbound connections from the service console we would simply enter

[root@esxhost1 root]# esxcfg-firewall -e sshClient

We use the -d switch to disable a service. In the following example, we prevent outbound connections

[root@esxhost1 root]# esxcfg-firewall -d smbClient

esxcfg-module
This command produces an output similar to vmkload_mod -list

[root@lithium06 tools-isoimages]# esxcfg-module -l

Module Type Enabled Loaded
vmkapimod vmkapimod true true
vmklinux linux true true
cciss.o scsi true false
tg3.o nic true false
qla2300_7xx.o fc true false

Although if you compare the output with the old command, things don't exactly match up. Not sure why just yet....

esxcfg-rescan
As vmkfstools -rescan

esxcfg-upgrade
esxcfg-upgrade -h --help
-g --convert-grub
-f --convert-fstab
-r --upgrade-pre-vmkernel
-o --upgrade-post-vmkernel

The -g option may only be used with the -r option.

esxcfg-vswitch
This command allows you to list, add, modify or delete virtual Ethernet switches on an ESX host. The simplest option with this command is the -l option to list the virtual switches defined on the host.

[root@esx root]# esxcfg-vswitch -l

If you are having problems with your ESX server after an in-place upgrade, this tool is invaluable in resolving the problems with service console networking.
The output of this command is initially a little intimidating. It is best to keep in mind the network topology:

Service Console IP Interface (vswif0) ---- connected to ----> Service Console Port on vSwitch
----- uplinked to ----> vmnic

Where a vmnic is a physical Ethernet adapter in the ESX server.

To add a virtual Ethernet switch, we use "-a" with the command

[root@esx root]# esxcfg-vswitch -a vSwitch3

esxcfg-auth
Configures the service console authentication options including NIS, LDAP, Kerberos and Active Directory.

esxcfg-info
Produces an enormous amount of information about the ESX host. You really need to pipe this to a file for closer examination!

[root@esx root]# esxcfg-info >esxinfo.txt

esxcfg-mpath
Manages multi-pathing just as the vmkmultipath utility did in previous versions of ESX Server.

[root@lithium06 tools-isoimages]# esxcfg-mpath -l

Disk vmhba0:0:0 /dev/cciss/c0d0 (69459MB) has 1 paths and policy of Fixed
Local 2:1.0 vmhba0:0:0 On active preferred

Disk vmhba1:0:0 (0MB) has 1 paths and policy of Most Recently Used
FC 10:1.0 210000e08b846a72<->5006016930221397 vmhba1:0:0
On active preferred

Disk vmhba1:0:6 /dev/sda (9216MB) has 1 paths and policy of Most Recently Used
FC 10:1.0 210000e08b846a72<->5006016930221397 vmhba1:0:6
On active preferred

Disk vmhba1:0:21 /dev/sdb (10240MB) has 1 paths and policy of Most Recently Used
FC 10:1.0 210000e08b846a72<->5006016930221397 vmhba1:0:21
On active preferred

esxcfg-resgrp
Used to manage the new ESX feature called resource groups. This command can add, remove or modify existing resource groups.

esxcfg-hbadevs
The esxcfg-vmhbadevs command is used to list the equivalent Linux device names for the visible disk devices that the VMkernel references using vmhba notation.

[root@esx1 root]# esxcfg-vmhbadevs
vmhba0:0:0 /dev/sda vmhba0:0:1 /dev/sdb
vmhba0:0:2 /dev/sdcvmhba0:0:3 /dev/sdd
vmhba2:0:0 /dev/sdevmhba2:1:0 /dev/sdf

If we use this command with the –m switch, then we only list the LUNs which contain VMFS partitions. Alongside the Linux device name, a long unique hexadecimal value is listed. This is the VMFS volume signature assigned by the new logical volume manager (LVM).

[root@esx1 root]# esxcfg-vmhbadevs -m
vmhba0:0:0:1 /dev/sda1 45407607-fbc43ced-94cb-00145e231ce3
vmhba0:0:2:1 /dev/sdc1 455b08a8-8af7fee3-daa9-00145e231e35
vmhba2:0:0:3 /dev/sde3 4559c75f-831d8f3e-bc81-00145e231e35

You can view these volumes in the directory /vmfs/volumes/

esxcfg-boot
Used to configure the GRUB options presented at boot time. One thing to note is that the new esxcfg commands will not run if you boot just into Linux. If you just want to query the boot settings, you can use the -q switch but this must be qualified with the keyword boot or vmkmod.

[root@lithium06 tools-isoimages]# esxcfg-boot -q boot
272 2:;7:;10:; UUID=847199e4-d3c7-11da-8ef8-930e3d734c03 /vmlinuz-2.4.21-37.0.2.ELvmnix /initrd-2.4.21-37.0.2.ELvmnix.img

[root@lithium06 tools-isoimages]# esxcfg-boot -q vmkmod
vmkapimod vmkapimod
vmklinux linux
cciss.o scsi
tg3.o nic
qla2300_7xx.o fc

Not yet tested other options....

esxcfg-init
Should not be run manually!

esxcfg-nas
Used to configure access to Network Attached Storage (NAS).

esxcfg-nas [

-a--add Add a new NAS filesystem to /vmfs volumes. Requires --host and --share options.
-o--host Set the host name or ip address for a NAS mount.
-s--share Set the name of the NAS share on the remote system.
-d--delete Unmount and delete a filesystem.
-l--list List the currently mounted NAS file systems.
-r--restore Restore all NAS mounts from the configuration file. (FOR INTERNAL USE ONLY).
-h--help Show this message.

esxcfg-route
If we add an IP address to the VMkernel by adding a VMkernel port, then we can fully configure that IP stack by also assigning a default gateway. We can view (no parameters) and set (1st parameter) the VMkernel IP default gateway with the esxcfg-route command as shown here.

[root@esx1 etc]# esxcfg-route
VMkernel default gateway is 100.100.100.254

[root@esx1 etc]# esxcfg-route 100.100.100.1
VMkernel default gateway set to 100.100.100.1

esxcfg-vmknic
Used to view and set configure the VMkernel ports on virtual Ethernet switches. A VMkernel port is a special type of port group on a virtual Ethernet switch which is used to assign an IP address to the VMkernel. The VMkernel only needs an IP address for VMotion, software-initiated iSCSI or NFS access.
If you need to create a VMkernel port at the command line, then you need to create a port group first and then enable it as a VMkernel port. There doesn’t appear to be a way of enabling a VMkernel port for VMotion from the command line.

[root@esx1 root]# esxcfg-vswitch -A VMotion vSwitch0

[root@esx1 root]# esxcfg-vmknic -a -i 100.100.100.121 -n 255.255.255.0 VMotion

In the following example, we list the VMkernel ports, then delete one of them and then list them again.

[root@esx1 etc]# esxcfg-vmknic -l
Port Group IP Address Netmask Broadcast MAC Address MTU EnabledNFS access 100.100.100.21 255.255.255.0 100.100.100.255 00:50:56:62:ca:f6 1514 trueVMotion 100.100.100.121 255.255.255.0 100.100.100.255 00:50:56:6d:7c:7d 1514 true

[root@esx1 etc]# esxcfg-vmknic -d VMotion[root@esx1 etc]# esxcfg-vmknic -l
Port Group IP Address Netmask Broadcast MAC Address MTU EnabledNFS access 100.100.100.21 255.255.255.0 100.100.100.255 00:50:56:62:ca:f6 1514 true


esxcfg-vmknic [[]]
-a--add Add a VMkernel NIC to the system, requires IP parameters and portgroup name.
-d--del Delete VMkernel NIC on given portgroup.
-e--enable Enable the given NIC if disabled.
-D--disable Disable the given NIC if enabled.
-l--list List VMkernel NICs.
-i--ip The IP address for this VMkernel NIC. Setting an IP address requires that the --netmask option be given in same command.
-n--netmask The IP netmask for this VMkernel NIC. Setting the IP netmask requires that the --ip option be given in the same command.
-r--restore Restore VMkernel TCP/IP interfaces from Configuration file (FOR INTERNAL USE ONLY).
-h--help Show this message.

esxcfg-dumppart
Used to configure the VMkernel crash dump partition. The old ESX 2.x utility for this function (vmkdump) is still present on an ESX 3 server, but appears just to be for extracting dump files.

esxcfg-dumppart []

-l--list List the partitions available for Dump Partitions. WARNING: This will scan all LUNs on the system.
-t--get-active Get the active Dump Partition for this system, returns the internal name of the partition vmhbaX:X:X:X) or 'none'.
-c--get-config Get the configured Dump Partition for this system, returns the internal name of the partition vmhbaX:X:X:X) or 'none'.
-s--set Set the Dump Partition for this system and activate it, either vmhbaX:X:X:X or 'none' to deactivate the active dump partition.
-f--find Find usable Dump partitions and list in order of preference.
-S--smart-activate Activate the configured dump partition or find the first appropriate partition and use it (same order as -f).
-a--activate Activate the configured dump partition.
-d--deactivate Deactivate the active dump partition.
-h--help Show this message.

esxcfg-linuxnet
esxcfg-linuxnet
--setup
--remove
-h --help

The --setup option cannot be combined with the --remove option.

esxcfg-nics
This tool can be used to view and configure the speed and duplex settings of the physical network cards in the ESX Server. So this tool can replace the MUI Network Connections/Physical Adapters, the mii-tool and modules.conf for network card management,
In the following example, we run the list option to view all physical NICs and their properties.

[root@esx-v3 etc]# esxcfg-nics -l

Name PCI Driver Link Speed Duplex Descriptionvmnic2 01:01.00 tg3 Up 1000Mbps Full Broadcom Corporation NetXtreme BCM5703 Gigabit Ethernetvmnic0 01:02.00 tg3 Up 100Mbps Full Broadcom Corporation NC7781 Gigabit Server Adapter (PCI-X, 10,100,1000-T)vmnic1 04:02.00 tg3 Up 1000Mbps Full Broadcom Corporation NC7781 Gigabit Server Adapter (PCI-X, 10,100,1000-T)

This command has the following optional parameters:

esxcfg-nics [nic]

-s--speed Set the speed of this NIC to one of 10/100/1000/10000. Requires a NIC parameter.
-d--duplex Set the duplex of this NIC to one of 'full' or 'half'. Requires a NIC parameter.
-a--auto Set speed and duplexity automatically. Requires a NIC parameter.
-l--list Print the list of NICs and their settings.
-r--restore Restore the nics configured speed/duplex settings (INTERNAL ONLY)
-h--help Display this message.

esxcfg-swiscsi
ESX version 3.0 supports both hardware and software iSCSI. For hardware iSCSI, we can use host bus adapters which perform the TCP offload and so the vmkernel can just pass SCSI commands to them as normal. The iSCSI hba can then wrap the SCSI command in TCP/IP and forward to the iSCSI target.

However, in software iSCSI (swiscsi), the wrapping of SCSI commands in TCP/IP is performed by the VMkernel and a regular physical network card can be used to communicate with the iSCSI target. This is exposed in the VI Client as a host bus adapter called vmhba40.
This will place a significant load on the VMkernel and wouldn't be that great an idea, but the feature is in ESX 3.0! So we use this tool esxcfg-swiscsi to configure it. The software iSCSI initiator in the VMkernel has a dependency upon the service console, therefore both the service console and VMkernel must have an IP route to the iSCSI target.

I have found that you need this command to scan for a new iSCSI target, as the VI Client rescan of the vmhba40 adapter doesn't appear to successfully discover targets.
My suggestion for getting the software iSCSI to work is as follows:

1. Add a VMkernel port to a vSwitch that has an uplink and route to iSCSI target
2. Ensure service console IP interface has a route to the same iSCSI target
3. Using either the VI Client security profile or the esxcfg-firewall, open a service console port for iSCSI (TCP:3260)
4. In the VI Client, enable the vmhab40 software iSCSI adapter and wait for the reconfiguration task to change from "In Progress" to "Completed"
5. Reboot the ESX host. This step will result in the VMkernel module for iSCSI being loaded at next boot.
6. In the VI Client, configure the vmhba40 adapter with an iSCSI target IP address
7. At the service console command line, run esxcfg-swiscsi -e
8. At the service console command line, run esxcfg-swiscsi -d
9. At the service console command line, run esxcfg-swiscsi -e
10. At the service console command line, run esxcfg-swiscsi -s
11. In the VI Client, perform a rescan of the vmhba adapters and your iSCSI target should become visible.

The command line options for this command are:

-e, --enable Enable sw iscsi
-d, --disable Disable sw iscsi
-q, --query Check if sw iscsi is on/off
-s, --scan Scan for disk available through sw iscsi interface
-k, --kill Try to forcibly remove iscsi sw stack
-r, --restore Restore sw iscsi configuration from file (FOR INTERNAL USE ONLY)
-h, --help Show this message

esxcfg-vswif
Manages the Ethernet interfaces of the service console

/etc/vmware/esx.conf
An all new configuration file for ESX Server 3.0. This file replaces the functionality of the following configuration files found in earlier versions of ESX.

/etc/vmware/hwconfig
/etc/vmware/devnames.conf
/etc/vmware/vmkmodule.conf
/etc/vmware/netmap.conf
/etc/vmware/vmkconfig

hostd
This is the daemon that replaces vmware-serverd. We can restart this with

service mgmt-vmware restart

vpxa
This is the name of the VirtualCenter server agent that runs in the service console of ESX 3.0 servers (was called vmware-ccagent in ESX 2.x). This can be stopped, started or restarted with the service command

service vmware-vpxa restart

/etc/vmware/vpxa.cfg
This is the XML configuration file for the VirtualCenter Server Agent in the service console. Here is a typical vpxa.cfg file.

[root@esx1 vmware]# cat vpxa.cfg



false


error



false


10



root
100.100.100.11 30 100.100.100.172
902

/var/log/vmware/vpx

vpxd
This is the process name of the Windows service that is the core service running on the VirtualCenter server.

vmkfstools
Used to manipulate virtual disks at the service console command line. It is used most often for import and export operations, where a virtual disk is converted from monolithic format to sparse format (previously called COW format).

There is a great switch with the command -X which can be used to extend the size of your virtual disk; e.g. if you had a 10GB virtual disk and wanted to expand it to 20GB, you could use this command. The VM would need to be powered off for this to work.

vmkfstools -X 20GB /vmfs/volumes/storage1/vm.vmdk

Note that the -X switch specifies the NEW SIZE of the virtual disk and NOT how much you are extending it by.

If you have used the -X switch before in an older version of ESX server (earlier than 3.0) it was possible to specify a small disk size; thereby making the virtual disk smaller. This was dangerous but useful if your partition within the disk did not consume 100% of the disk size. However, this is not possible with vmkfstools command found in ESX Server version 3.x

AAM
Automated Availability Manager that now runs in the service console when you create a VMware High Availability (VMware HA) cluster. The VMware HA feature was previously known as DAS (Distributed Availability Services) but we don't mention that anymore.
This software maintains an in-memory database on active nodes in the cluster and uses heartbeats to co-ordinate the active and passive nodes. It is suggested that you configure service console with 2 ethernet interfaces to remove any single point of failure.

This is a piece of licensed Legato software which itself has been renamed to EMC AutoStart.
This component has a very high dependency upon fully functional host name resolution. So before you enable VMware HA, check your /etc/hosts file, and your /etc/resolv.conf file to ensure accuracy. The log file for VMware HA can be found in the service console in the directory
/opt/LGTOaam512/

To avoid split brain scenarios, an ESX server can determine if it has become isolated from other servers and we can configure that servers' isolation response. If the AAM component loses contact with the other nodes in the HA cluster, it attempts to contact the configured default gateway for service console using ICMP echo request (PING). If this fails, then the ESX host is isolated. If your default gateway suppresses ICMP echo requests, then we can configure an alternate IP address called the das.isolationaddress.

No comments: