I have 2016 Functional level cluster with Server 2019 (basically in a process of replacing 2016 host with 2019)

If VM is running on 2019 host I can poweroff, quick migrate to 2016 host, power on & all is good

But Live migration always gives me above error

All I am getting in Event Data is (very descriptive?!):

Live migration of 'Virtual Machine Test' failed.

Nothing else, no reason.

If VM is running on 2016 host I CAN do live migration to 2019 fine! (albeit with errors reported in this thread, but I do NOT have VMM being used!)

vm\service\ethernet\vmethernetswitchutilities.cpp(124)\vmms.exe!00007FF7EA3C2030: (caller: 00007FF7EA40EC65) ReturnHr(138) tid(2980) 80070002 The system cannot find the file specified.
    Msg:[vm\service\ethernet\vmethernetswitchutilities.cpp(78)\vmms.exe!00007FF7EA423BE0: (caller: 00007FF7EA328FEE) Exception(7525) tid(2980) 80070002 The system cannot find the file specified.
] 

Both host are IDENTICAL hardware on same firmware level of every component!

There is NOTHING relating to even attempting migration in local host Hyper-V VMMS/Admin/Operational logs

In Hyper-V High Availability/Admin I get same error but with Even ID 21111

Seb

i have 2 node PowerEdge R740xd S2D enabled. Currently facing an issue unable to change owner node for cluster shared volume in Failover Cluster Manager. 

Hi All,

This weekend we had a healthy 2 node cluster get restarted. When the machines got brought up, The cluster itself would not start up, nor would the 1 role. Both the Cluster and the role had the error "The Cluster IP Address is already in use". Yes, it was in use when the cluster itself was up and running, but now its not running. To get the actual cluster working, I changed the IP of Cluster Core Resources, restarted the cluster, and boom, the cluster itself came online. The Role is a different story as it is a SQL AG cluster. 

I figure it would not be that hard to change the IP's but since this is a pre-production box, I dont want to use that workaround for when this goes into production.

Server A NIC IP : 10.172.193.89

Server A Cluster IP: 10.172.193.90

Server A Role/SQL AG IP: 10.172.193.91

Server B NIC IP : 10.172.195.89

Server B Cluster IP: 10.172.195.90

Server B Role/SQL AG IP: 10.172.195.91

Within the actual Cluster events I get the Errors:

Cluster IP address resource 'IP Address 10.172.193.91' cannot be brought online because a duplicate IP address '10.172.193.91' was detected on the network.  Please ensure all IP addresses are unique.

Cluster resource 'IP Address 10.172.193.91' of type 'IP Address' in clustered role 'CCTX-AG-01' failed. The error code was '0x13c1' ('The cluster IP address is already in use.').

Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.

Lastly,

I disabled the NIC on the VM and tried to ping the same IP's just in case someone built other machines without me knowing, but I could not ping or communicate with those IP's. As soon as I Enabled the nic again, I could ping the IP's accordingly.

Thanks,

Jordan

We are setting up a GPO firewall for SQL Servers in a cluster and we are having issues with our failover cluster failing because the Microsoft Failover Cluster Virtual Adapter (MFCVA) is changing its IPv6 address everytime it reboots.

We are trying to set up the firewall rule to allow all communications from server A on its IPv6 MFCVA-  to -server B on its IPv6 MFCVA.  

Any ideas on how to essentially get the cluster working through a firewall when the cluster is trying to communicate over IPv6? 

Thanks in advance!!!

Trying to write a monitoring script for NLB Cluster status, which has 2 nodes.

I have 2 VMS (Win Srv 2016 Std): CB-1 and CB-2

When I run this command on these vms:

I get the output I need.

But if I try the same from a remote server (same network and domain) I get an:

Accessisdenied.(ExceptionfromHRESULT:0x80070005(E_ACCESSDENIED))+CategoryInfo:+FullyQualifiedErrorId:AccessDenied,Microsoft.NetworkLoadBalancingClusters.PowerShell.GetNlbClusterNode

UAC already DISABLED

FIREWALL already DISABLED

WINRM already RUNNING

It's a clean installation in a demo server, so i can exclude any kind of systems problems

Why is that? It is ONLY the "Get-NlbClusterNODE" command that gives me access is denied. "Get-NlbCluster" for an example, works just fine.

# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ #$domainuser="dom\administrator"$domainpassword="xxxxxxx"|ConvertTo-SecureString-AsPlainText-Force$domaincredentials=New-ObjectSystem.Management.Automation.PSCredential($domainuser,$domainpassword)# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ #Invoke-Command-ComputerNamecb-1.dom.local-Credential$domaincredentials-ScriptBlock{Get-NlbClusterNode-HostNamecb-1.dom.local}

Hello,

In our 2012R2 failover cluster we have windows 2008R2 virtual machines that restart without a message. In our cluster we reseive the following error:

Cluster resource 'VM Name' of type 'Virtual Machine' in clustered role 'VM  Resources' failed.

Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.

Source: Micorsoft-Windows-FailoverClustering , event id: 1069

When i look in de cluster logging i see the following messages:

INFO  [RCM [RES] VMName embedded failure notification, code=0 _isEmbeddedFailure=false _embeddedFailureAction=2

Anybody got a idea what the problem can be??

Thanks!!

I'm having an issue with Windows Failover Cluster with a Windows Server 2012 R2  machine. I have two cluster nodes (nodeA and nodeB). My issue is that when nodeA is the owner node, and I open failover cluster manager  <clusterName> >> roles >> <fileserver role> >> shares tab it will hang and say that it is loading, but this will occur infinitely. Although when I go to nodeB (not the owner node) and I go to shares it will show me all of the shares that I have. Next when I go to <clusterName> >> Nodes  >> click on Roles tab the information says "There were errors retrieving file shares."

Now when I switch the nobeB to the owner node, I cannot view the shares on that machine but can now view them on nodeA. 

We alse have a test network where I have recreated the machines, environment and the failover cluster to as close as the production network as I can except everything works great in the test network 

Hi,

I got the following error when I have created the Cluster. All validation tests are passed and Cluster nodes (Windows Server 2016 core) are updated.

d

Any help would be appreciated.

Thanks

Building out a File server CLuster.

1.I currently have a NIC team to be used for file host

2. I have 2 sep NICs for ISCSI.

Should i create another team and use that as cluster only and potentially just IPV6 or is it best practice to just have the first NIC team as cluster and client and heartbeat will just go through that?

As the title suggests, I have a pair of server 2016 hosts which are currently operating as a 2-node failover cluster.

Due to relocation of resources, I'm looking to break up the nodes into two standalone servers. One of the servers is to be decommissioned from Hyper-V use, the other to remain as an operational stand-alone Hyper-V host.

Is there a known process for un-clustering nodes back to stand along hyper hosts.

I have a number of sizable VM's (storage size wise) which currently sit on the C:\ClusterStorage\ location, I'm trying to figure out if I need to create a new LUN on my SAN and migrate the VMs onto this new LUN, or whether I could utilize the existing disk LUN currently used by the CSV volume?

Is there scope for me to powerdown the VM's, uncluster the node, convert the CSV volume to a normal SAN based disk and power back up the VMs on the now standalone host?
*EDIT*
I know there's the option to 'remove from cluster shared volumes' but I'm not sure what the impact is of doing this is..

Hi Experts, 

I've been doing some research on how to achieve this goal and what's the best practice.

We are planning to do a high availability cluster for our server running the following services.

1. Active Directory
2. DNS and DHCP
3. File Server

Currently we have one fully operational Windows server 2016 running in Dell R530. Since we have 2 set of dell server, we want to configure a HA cluster for downtime protection. And  we want to set it up in a way were we have a main server that will be doing all the workload and a backup server that will replace the main if it fail without downtime.

But most common reference I found related to our goal involved a shared network. Now what I wanted to know are:

• Why is it recommended to have a shared storage 
• Is it possible to configure HA without shared storage
• If possible, what are the risk of not having a shared storage

Thank you in advance experts.

Hi Experts,

I would like to ask your insights about why is it bad to cluster domain and what are the risk . I have read some forums and pages concerning about this, but it seems I can't get a clear picture of it.

I understand that DC doesn't need to be clustered for failover environment. but what if there are services in our environment that needed to be clustered such as File Server. 

Thanks in advance.

Hi all,

Is it possible to cluster file server between two active directory domain controller.

As of now our server is still in standalone. We will soon add another active directory in our domain for fault tolerance if the server fail. 

Our current server runs the following services which we want to add redundancy that's why we want to add new server.

• Active Directory
• DNS
• DHCP
• File Server

In my research, Active Directory and DNS High Availability will be achieved once we add another domain controller in our current domain. And in DHCP there's a feature called DHCP Clustering.

But regarding File Server, I haven't found any clear ideas on how to achieve this.

Thanks in advance for your advises.

I have a Failover Cluster running on two Server 2012 R2 Datacenter nodes hosting our Hyper-V environment.  Recently, we have run into an issue where the VMs won’t migrate to the opposite node unless the VM is rebooted or the Saved State data is deleted.  The VMs are stored either on an SOFS volume on a separate FO Cluster or a CSV volume both nodes are connected to.  The problem occurs to VMs in either storage location.

Testing I’ve done is below.  Note that I only list one direction, but the behavior is the same moving in the opposite direction, as well:

- Live Migration: if a VM is on Node1 and I tell it to Live Migrate to Node2, it begins the process in the console and for a split second shows Node2.  It immediately flips back to Node1.  If the VM has rebooted since the last migration, it will go ahead and migrate to Node2.  It will not migrate back until the VM has been rebooted again.  The Event Log shows IDs 1205 and 1069.  1069 states “Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.”  All resources show Online in Powershell.

- Quick Migration: I initiate a Quick Migration and the VM will move from Node1 to Node2, but will fail to start on Node2.  Checking the Event Log I see Event IDs 1205 and 1069.  1069 states “Cluster resource 'Virtual Machine IDF' of type 'Virtual Machine' in clustered role 'IDF' failed. The error code was '0xc0370027' ('Cannot restore this virtual machine because the saved state data cannot be read. Delete the saved state data and then try to start the virtual machine.').”  After deleting the Saved State Data, the VM will start right up and can be Live or Quick Migrated once.

- Shutdown VM and Quick Migration: I have not had an occasion of this method fail so far.

- Rebooting the Nodes has had no discernable effect on the situation.

- I’ve shut down a VM and moved its storage from SOFS to the CSV and still have the same issues as above.  I moved the VHDX, the config file, and saved state data (which was empty while the VM was powered down) to the CSV.

Items from the FO Cluster Validation Report:
1. The following virtual machines have referenced paths that do not appear accessible to all nodes of the cluster. Ensure all storage paths in use by virtual machines are accessible by all nodes of the cluster.
Virtual Machines Storage Paths That Cannot Be Accessed By All Nodes 
Virtual Machine       Storage Path      Nodes That Cannot Access the Storage Path 
VM1                       \\sofs\vms         Node1

I’m not sure what to make of this error as most of the VMs live on this SOFS share and are running on Nodes1 and 2.  If Node1 really couldn’t access the share, none of the VMs would run on Node1.

2. Validating cluster resource File Share Witness (2) (\\sofs\HVQuorum).
This resource is configured to run in a separate monitor. By default, resources are configured to run in a shared monitor. This setting can be changed manually to keep it from affecting or being affected by other resources. It can also be set automatically by the failover cluster. If a resource fails it will be restarted in a separate monitor to try to reduce the impact on other resources if it fails again. This value can be changed by opening the resource properties and selecting the 'Advanced Policies' tab. There is a check-box 'run this resource in a separate Resource Monitor'.

I checked on this and the check-box is indeed unchecked and both Nodes report the same setting (or lack thereof).

3. Validating cluster resource Virtual Machine VM2.
This resource is configured to run in a separate monitor. By default, resources are configured to run in a shared monitor. This setting can be changed manually to keep it from affecting or being affected by other resources. It can also be set automatically by the failover cluster. If a resource fails it will be restarted in a separate monitor to try to reduce the impact on other resources if it fails again. This value can be changed by opening the resource properties and selecting the 'Advanced Policies' tab. There is a check-box 'run this resource in a separate Resource Monitor'.

Validating cluster resource Virtual Machine VM3.
This resource is configured to run in a separate monitor. By default, resources are configured to run in a shared monitor. This setting can be changed manually to keep it from affecting or being affected by other resources. It can also be set automatically by the failover cluster. If a resource fails it will be restarted in a separate monitor to try to reduce the impact on other resources if it fails again. This value can be changed by opening the resource properties and selecting the 'Advanced Policies' tab. There is a check-box 'run this resource in a separate Resource Monitor'.

I can’t find a place to see this check-box for the VMs.  The properties on the roles don’t contain the ‘Advanced Policies’ tab.

All other portions of the Validation Report are clean.

So far, I haven’t found any answers in several days of Google searching and trying different tactics.  I’m hoping someone here has run into a similar situation and can help steer me in the right direction to get this resolved.  The goal is to be able to Live Migrate freely so I can reboot the Nodes one at a time for Microsoft Updates without having to bring down all the VMs in the process.

I currently have a dell VRTX running two blades (server 2012r2 on each).  The blades are setup for failover clustering and are sharing storage.  I have hyper-v installed on the vrtx.  I'm  quickly running out of storage and need to add some new hard drives into the shared storage.  I've been told that this will "break" my cluster and I will have to rebuild everything from scratch.  

Can anyone give me some insight on this?  I was hoping it would be as easy as popping the new drives in and allocating them to the shared storage.  Thanks in advance.

I have just installed Server 2012 Datacenter edition with 4 NICs. I setup HyperV and was thinking of setting up NIC teaming for the VMs, but whenever I launch the NIC teaming option from Server Manager I get the right server listed, but it is listed as Host Unmanageable.

Am I missing a feature or something like NLB? I tried installing that and it also didnt work after that.

...

Hello,

I have two Windows 2012 Datacenter Core installed on two box. I'm using Windows 2012 and Windows 8 PC to manage those two servers that are in a cluster. Everything is working fine. But when I came in yesterday, I get "Keberos security error" in Server Manager only on the cluster object, but not the servers. See image:

First line is the first physical host and second is the second physical host. Third line is for the cluster. Now, I can't create drives any more. Please advise.

Looks like MS needs to verify my account before i can attach an image.

In my domain, i was experimenting with printer clustering and i created the cluster "pr-cl2". The (only one) node that the cluster was running crashed. Now i have this cluster on my domain which is not accessible and i cannot delete it because the node is no longer active. I deleted all the records i could find from the dns but i can still see it in my domain. All the answers i found are not working because i cannot connect to the node. What should i do?

Thanks in advance.

Good afternoon,

Need help with what seems to be a simple task, but continues to fails. We’re trying to build a Windows 2016 Failover Cluster, which continues to fail. Windows 2012 R2 Failover Cluster is successful, same domain, accounts. Here are the details on each configuration.  Will be glad to provide additional information that could help.

Thanks,  -jim

Windows 2016 Failover Cluster

AD – Windows 2016 domain

FFL – Windows 2012 R2 Forest Functional Level

DFL – Windows 2016 Domain Functional Level

2 servers, Windows 2016 Datacenter

Event Viewer – FailoverClustering DiagnosticVerbose log enabled

Results: Cluster Validation passes, select build cluster from test details. Build immediately fails, very little details in the cluster.log (see details below)

Same results via GUI or with PS New-cluster cmdlet

Windows 2012 R2 Failover Cluster

AD – Windows 2016 domain

FFL – Windows 2012 R2 Forest Functional Level

DFL – Windows 2016 Domain Functional Level

2 server, Windows 2012 R2 Standard

Results: Build completes successfully, ton of details in the cluster.log

Some additional points/details…

- Create Cluster Wizard report shows the 'bind to domain controller . more data is available.' Error (see details below).

- Prestaged the CNO, no difference with or without.

- We've also tried the build with and without the 'Deny Access to this computer from the Network' policy set. Still fails.

- Cluster DiagnosticsVerbose logs are not showing much details/errors.

- Tried alternate pair of Win2016 servers in two domains of forest, same error.

- Seems to be a permissions error in AD since the failure happens right after the cluster build dialog that states 'Find a suitable domain controller for node <nodename>'

Cluster.log from failed Windows 2016 build…

00002a78.00002b2c::2018/03/20-14:54:06.249 DBG   Cluster node cleanup thread started.

00002a78.00002b2c::2018/03/20-14:54:06.249 DBG   Starting cluster node cleanup...

00002a78.00002b2c::2018/03/20-14:54:06.249 DBG   Disabling the cluster service...

00002a78.00002b2c::2018/03/20-14:54:06.251 DBG   Releasing clustered storages...

00002a78.00002b2c::2018/03/20-14:54:06.252 DBG   Getting clustered disks...

00002a78.00002b2c::2018/03/20-14:54:06.252 DBG   Waiting for clusdsk to finish its cleanup...

00002a78.00002b2c::2018/03/20-14:54:06.253 DBG   Clearing the clusdisk database...

00002a78.00002b2c::2018/03/20-14:54:06.254 DBG   Waiting for clusdsk to finish its cleanup...

00002a78.00002b2c::2018/03/20-14:54:06.255 DBG   Relinquishing clustered disks...

00002a78.00002b2c::2018/03/20-14:54:06.255 DBG   Opening disk handle by index...

00002a78.00002b2c::2018/03/20-14:54:06.258 DBG   Getting disk ID from layout...

00002a78.00002b2c::2018/03/20-14:54:06.258 DBG   Reset CSV state ...

00002a78.00002b2c::2018/03/20-14:54:06.259 DBG   Relinquish disk if clustered...

00002a78.00002b2c::2018/03/20-14:54:06.261 DBG   Opening disk handle by index...

00002a78.00002b2c::2018/03/20-14:54:06.263 DBG   Getting disk ID from layout...

00002a78.00002b2c::2018/03/20-14:54:06.264 DBG   Reset CSV state ...

00002a78.00002b2c::2018/03/20-14:54:06.264 DBG   Relinquish disk if clustered...

00002a78.00002b2c::2018/03/20-14:54:06.266 DBG   Opening disk handle by index...

00002a78.00002b2c::2018/03/20-14:54:06.271 DBG   Resetting cluster registry entries...

00002a78.00002b2c::2018/03/20-14:54:06.273 DBG   Resetting NLBSFlags value ...

00002a78.00002b2c::2018/03/20-14:54:06.278 DBG   Unloading the cluster Windows registry hive...

00002a78.00002b2c::2018/03/20-14:54:06.279 DBG   Getting the cluster Windows registry hive file path...

00002a78.00002b2c::2018/03/20-14:54:06.280 DBG   Getting the cluster Windows registry hive file path...

00002a78.00002b2c::2018/03/20-14:54:06.281 DBG   Getting the cluster Windows registry hive file path...