Multi Architect

Issue : VM fails to power on, power off or modify. If tried to migrate to another ESXi host the VM gets disconnected. Error from vCenter : VM will be shown in vCenter as inaccessible.  Snippets from the logs from ESXi host : [root@ESXi01:/vmfs/volumes/570248ff-86524429-7f05-848f691451f9/VMname] cat VMname.vmx | grep vmdk cat: can't open 'VMname.vmx': Device or resource busy Troubleshooting : The logs shows that the VM is busy or locked due to some reason. Check the lock status using vmx [root@ESXi01:/vmfs/volumes/570248ff-86524429-7f05-848f691451f9/VMname] vmkfstools -D VMname.vmx Lock [type 10c00001 offset 151009280 v 326, hb offset 3198976 gen 61, mode 1, owner 58109c0d-10ae7368-fd7b- 848f69156ba8 mtime 15841905 num 0 gblnum 0 gblgen 0 gblbrk 0] Addr <4, 326, 119>, gen 129, links 1, type reg, flags 0, uid 0, gid 0, mode 100755 len 5230, nb 1 tbz 0, cow 0, newSinceEpoch 1, zla 2, bs 8192 The highli

Issue : ESXi hosts disconnects from vCenter and may not even connect directly using vSphere client. The VMs will continue to run. SSH to host will work. Execution of esxcfg-scsidevs -m command will hang. LUN disappears in one or more hosts in an ESXi cluster.  Errors from vCenter : A general system error occurred: Invalid response code: 503 Service Unavailable. Unable to communicate with the remote host, since it is disconnected. Cannot contact the specified host. The host may not be available on the network, a network configuration problem may exist, or the management services on this host may not be responding. Snippets from the vmkernal logs from ESXi host : Check for non-responsive luns [root@ESXi01:~] cat /var/log/vmkernel.log  | grep -i responsive cpuxx:yyyyyyy ALERT: hostd detected to be non-responsive Check vmkernel.log where device status is 0x18 which corresponds to Reservation conflict [root@ ESXi01 :~] cat /var/log/vmkernel.log |

Issue : ESXi host disconnects from vCenter. And if tried to reconnect, the activity fails at 89% Error from vCenter : A general system error occurred: internal error Processing data from vCenter agent on ESXi01 Snippets from the vpxd logs from vCenter : =============================================================== 2016-03-29T12:28:37.644+05:30 error vpxd[14460] [Originator@6876 sub=HttpConnectionPool-000001] [ConnectComplete] Connect failed to <cs p:000000001099ab00, TCP:IP Address:443>; cnx: (null), error: class Vmacore::Ssl::SSLVerifyException(SSL Exception: Verification parameters: --> PeerThumbprint: Thumbprint --> ExpectedThumbprint:  --> ExpectedPeerName: IP/Hostname --> The remote host certificate has these problems: -->  --> * The host certificate chain is incomplete. -->  --> * Host name does not match the subject name(s) in certificate. -->  --> * unable to get local issuer ce

Issue When VMware vSphere Update Manager 5 tries to remediate a host, the remediation fails at 25% with the below error: fault.com.vmware.vcIntegrity.VcIntegrityFault.summary

Issue We faced an issue today morning with one of our VMs hosted in VMware. The VM related options were greyed out. Root cause There was a snapshot job running in the background (not visible from vCenter), which prevented any administration task in the VM. This task was stuck at 0%. This activity cannot be cancelled from vCenter or from console as it was initiated by a system user called vpxuser . Workaround Login to the SSH console of the ESXi host holding the VM using putty. Identify the vmid of the affected VM (In our case the vmid was 391) using the command vim-cmd vmsvc/getallvms Check the tasks running in background for this particular VM using the command vim-cmd vmsvc/get.tasklist 391 See if you can cancel the task using the command vim-cmd vimsvc/task_cancel <taskname> [Task name will be something like hatask-391-vim.virtualmachine.createsnapshot-1234567] . In our case this was not working as the task was initiated by a system user. But

Issue While performing vMotion, the operation fails at 14% with the below error : vMotion migration [-1062731490:1419235061251156] failed to create a connection with remote host <Destination vMotion IP>: The ESX hosts failed to connect over the VMotion network Migration [-1062731490:1419235061251156] failed to connect to remote host < Destination vMotion IP > from host <source IP>: Network unreachable The vMotion failed because the destination host did not receive data from the source host on the vMotion network. Please check your vMotion network settings and physical network configuration and ensure they are correct. Resolution I've already penned a post on VMware vMotion failure at 14% . This blog is an extended version of that post. If none of the steps mentioned in my previous post helped you, then you are in the right page. Check whether vMotion is selected for multiple vmkernel NICs in ESXi host. !!!! If yes, make only one NIC available for vM

Quick Reference guide from VMware Run, %RUN: This value represents the percentage of absolute time the virtual machine was running on the system. If the virtual machine is unresponsive,  %RUN  may indicate that the guest operating system is busy conducting an operation.

What is vSAN ? It is a hypervisor-converged storage solution built by aggregating the local storage attached to the ESXi hosts managed by a vCenter.  Recommended iSCSI configuration? A separate vSwitch, and a separate network other than VMtraffic network for iSCSI traffic. Dedicated physical NICs should be connected to vSwitch configured for iSCSI traffic. What is iSCSI port binding ? Port binding is used in iSCSI when multiple VMkernel ports for iSCSI reside in the same broadcast domain and IP subnet, to allow multiple paths to an iSCSI array that broadcasts a single IP address.

Issue While performing vMotion, the operation fails at 14% with the below error : A general system error occurred: Migrtion to host <Destination ESXi IP> failed with erro Connection closed by remote host, possibly due to timeout (0xbad003f). Migrate virtual machine:A general system error occurred: Migration to host <Destination ESXi IP> failed with error Connection closed by remote host, possibly due to timeout (0xbad003f). vMotion migration [-1062729272:1406020861428172] (19-71629048648008) failed to receive...  Scenarios Scenario 1: Your management network and vmotion network are in the same subnet using the same physical NIC.

S cenario You have a single vmdk file of 200 GB and it has two logical volumes C & D with 100 GB each. Suppose you need to add another 100 GB to the D drive making it 200 GB. What would you do ? The steps are simple and straight forward.

Symptom:  Eth0 interface will not be present for a Centos VM after cloning. Only the loopback networking interface will be available. If you try to turn up the interface manually (using the command ifup eth0 or ifup-eth0 ), you will receive the below error. Device eth0 does not seem to be present, delaying initialisation  Root Cause: When you clone a Centos VM from a template, a new NIC card will be created for the cloned VM. In other terms, a new MAC address will be generated for the NIC of the cloned machine. This change happens only in VMware perspective and no modification is made in Centos. Therefore the kernel will be still searching for the NIC with old MAC address and hence fails.

Steps for cloning VM using SSH: SSH to the ESXi host Identify the path of the source (say,  /vmfs/volumes/datastore1/SourceVM/ where  sourceVM is the name of the source VM) Create a new folder in the desired datastore  mkdir  /vmfs/volumes/datastore1/DesintationVM   where DestinationVM is the name of the new VM Clone the sourceVM vmdk to the newly created folder DestinationVM vmkfstools -i /vmfs/volumes/datastore1/SourceVM/sourceVM.vmdk /vmfs/volumes/datastore1/DesintationVM/DesintationVM.vmdk Once cloning is completed, proceed with the creation of the new VM using vSphere client In the option where you need to provision the harddisk for the new VM, choose 'Use an existing virtual disk' Browse and point the newly cloned vmdk file

Symptom: The host and VMs in a vcenter server appears to be disconnected. Issue: The host and its corresponding vms could appear as disconnected due to issue with any of the following services.

                 I don't know if this is a subject for a blog. But since the resolution of this issue appeared to be so silly and simple, thought of sharing that with you. Issue VM in an ESXi appears to be unresponsive at times. We are receiving ping response but not able to connect via RDP or console. The ultimate method of resolving this issue was a VM reset until I found the root cause. Resolution After many hours of troubleshooting, the issue got resolved.

Issue You will find software iscsi adapter missing in ESXi Resolution Log in to the vSphere Client Select the configuration tab Select Storage Adapters Click on the Add option You will be given the option to add Software iSCSI adapter

Issue When you Power On a VM, you will be greeted with the below error message: ' Failed to start the virtual machine. Module DevicePowerOn power on failed. Unable to create virtual SCSI device for scsi0:0, '/vmfs/volumes/...........vmdk' Failed to open disk scsi0:0: Unsupported or invalid disk type 7. Ensure that the disk has been imported.' Cause This will occur when you

                E SXi is a hypervisor which is intended to work on a server platform. But that doesn't mean that we cannot install hypervisor on a workstation. Here, I'm trying to install VMware ESXi 5.5 on Dell Optiplex 990 machine. I will not be discussing about the installation steps as I believe it is pretty much straight forward and most of you are very much expert in it. Will be just focusing on the tips that would help us install ESXi on a workstaion. I've booted my DELL Optiplex 990 using the VMware ESXi 5.5 bootable disk and everything

Symptoms VMs will be shown as inaccessible in the vCenter Reason A VM can become inaccessible due to any of the below reason:

F ocus : How can we optimize the performance of a virtual machine ? Below are a few points you can consider during your virtual machine deployment time:

                          W hen you try to Power On a virtual machine, you will receive the below error: “ Failed to start the virtual machine. Module DevicePowerOn power on failed. Unable to create virtual SCSI device for scsi… ”