I am having some problems recently with my clustered Hyper-V environment and need to know where to look in order to find the real problem here. We have 13 hosts in the cluster that uses a combination of CSV (Microsoft iSCSI servers) and SMB 3.0 San storage (Not managed by VMM). Regularly we have been seeing our Linux boxes fail with the following errors. (These are Administrative events from one of the hosts that is having the most failures)
I will be happy to provide any more information however I don't really know where to look. I have run a validation report and it looks good as far as I can tell. I haven't included any of the storage because I don't want to bring the clustered roles down.
Error 6/23/2016 11:13:04 AM Hyper-V-Integration 4000 None
Hyper-V Heartbeat failed to connect to virtual machine 'AppServer' because the version does not match the version expected by Hyper-V (Virtual machine ID GUID). Framework version: Negotiated (0.0) - Expected (3.0); Message version: Negotiated (0.0) - Expected (3.1). To fix this problem, you must upgrade the integration services. To upgrade, connect to the virtual machine and select Insert Integration Services Setup Disk from the Action menu.
Warning 6/21/2016 9:42:27 PM Hyper-V-Worker 23014 None
Device 'Microsoft Synthetic Display Controller' in 'AppServer' is loaded but has a different version from the server. Server version 3.3 Client version 3.2 (Virtual machine ID GUID). The device will work, but this is an unsupported configuration. This means that technical support will not be provided until this problem is resolved. To fix this problem, upgrade the integration services. To upgrade, connect to the virtual machine and select Insert Integration Services Setup Disk from the Action menu.
Warning 6/21/2016 9:42:19 PM Hyper-V-VmSwitch 27 (1019)
Networking driver in AppServer loaded but has a different version from the server. Server version 5.0 Client version 3.2 (Virtual machine ID GUID). The device will work, but this is an unsupported configuration. This means that technical support will not be provided until this problem is resolved. To fix this problem, upgrade the integration services. To upgrade, connect to the virtual machine and select Insert Integration Services Setup Disk from the Action menu.
Warning 6/21/2016 9:42:19 PM storvsp 6 None
A storage device in 'AppServer' loaded but has a different version from the server. Server version 6.0 Client version 4.2 (Virtual machine ID GUID). The device will work, but this is an unsupported configuration. This means that technical support will not be provided until this problem is resolved. To fix this problem, upgrade the integration services. To upgrade, connect to the virtual machine and select Insert Integration Services Setup Disk from the Action menu.
Warning 6/21/2016 9:42:19 PM storvsp 6 None
A storage device in 'AppServer' loaded but has a different version from the server. Server version 6.0 Client version 4.2 (Virtual machine ID GUID). The device will work, but this is an unsupported configuration. This means that technical support will not be provided until this problem is resolved. To fix this problem, upgrade the integration services. To upgrade, connect to the virtual machine and select Insert Integration Services Setup Disk from the Action menu.
Error 6/20/2016 3:29:18 AM FailoverClustering 1205 Resource Control Manager
The Cluster service failed to bring clustered role 'AppServer' completely online or offline. One or more resources may be in a failed state. This may impact the availability of the clustered role.
Error 6/20/2016 3:29:18 AM FailoverClustering 1069 Resource Control Manager
Cluster resource 'Virtual Machine AppServer' of type 'Virtual Machine' in clustered role 'AppServer' failed. The error code was '0x20' ('The process cannot access the file because it is being used by another process.').
Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it. Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.
Error 6/20/2016 3:29:18 AM Hyper-V-High-Availability 21502 None
'Virtual Machine AppServer' failed to start.
'AppServer' could not initialize. (Virtual machine ID GUID)
'AppServer' failed to create or access memory contents file \\SMB3SAN\VMInfo\AppServer\AppServer\Virtual Machines\GUID\GUID.bin. (Virtual machine ID GUID)
Error 6/20/2016 3:29:18 AM Hyper-V-High-Availability 21107 None
'Virtual Machine AppServer' failed to start.
Error 6/20/2016 3:29:18 AM Hyper-V-Worker 3040 None
'AppServer' could not initialize. (Virtual machine ID GUID)
Error 6/20/2016 3:29:18 AM Hyper-V-Worker 3130 None
'AppServer' failed to create or access memory contents file \\SMB3SAN\VMInfo\AppServer\AppServer\Virtual Machines\GUID\GUID.bin. (Virtual machine ID GUID)
Error 6/20/2016 3:29:16 AM FailoverClustering 1069 Resource Control Manager
Cluster resource 'Virtual Machine AppServer' of type 'Virtual Machine' in clustered role 'AppServer' failed.
Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it. Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.
Error 6/20/2016 3:29:16 AM Hyper-V-VMMS 20114 None
The Virtual Machine Management Service failed to verify the running state of the virtual machine 'GUID': The system cannot find the drive specified. (0x8007000F).