VM options greyed out | VMware

Issue

We faced an issue today morning with one of our VMs hosted in VMware. The VM related options were greyed out.

Root cause

There was a snapshot job running in the background (not visible from vCenter), which prevented any administration task in the VM. This task was stuck at 0%. This activity cannot be cancelled from vCenter or from console as it was initiated by a system user called vpxuser.

Workaround

Login to the SSH console of the ESXi host holding the VM using putty.

Identify the vmid of the affected VM (In our case the vmid was 391) using the command
vim-cmd vmsvc/getallvms

Check the tasks running in background for this particular VM using the command
vim-cmd vmsvc/get.tasklist 391

See if you can cancel the task using the command
vim-cmd vimsvc/task_cancel <taskname> [Task name will be something like hatask-391-vim.virtualmachine.createsnapshot-1234567] .

In our case this was not working as the task was initiated by a system user. But in scenarios were the snapshot or any VM related tasks initiated by user hangs, this command would help.

Identify the PID assigned to the VM using the command
ps | grep vmx | grep <VMname>

Kill the process using the command
kill<pid>

The parent process will be killed and the VM will be now in powered off state. After powering back the server, the options will be available.