In certain situations the Exchange DAG witness can become a problem, this can occur during an upgrade or can happen if there is a network interruption. Although the DAG will continue to operate the witness is considered failed, which can cause odd behaviour in a failover scenario, as there is no quorum available for the database resource (i.e. to decide which side is going to be active and which passive).
It is recommended to run the PowerShell console as Administrator.
Confirming the Issue
To confirm the issue run the following PowerShell command from the Exchange Management Shell:
Get-DatabaseAvailabilityGroup -Identity "DAG01" -Status | format-list Name, *Witness*, Servers
Now in my output, I was seeing “InvalidConfiguration” under the WitnessShareInUse field.
Notice the the WitnessShareInUse is showing “InvalidConfiguration”.
If you just try to set it back up again with:
Set-DatabaseAvailabilityGroup -Identity "DAG1603" -WitnessServer "exchserver1.domain.com” -WitnessDirectory "C:\DAGFileShareWitnesses\DAG1603.domain.com"
You get this error:
Resolution
To resolve it you just need to set a new WitnessServer, in our setup we have two Witness servers each taking half of the pairs of the servers in each DAG. So instead, if you run:
Set-DatabaseAvailabilityGroup -Identity "DAG1603" -WitnessServer "wtgc-exch-wit-b.internal.sanger.ac.uk” -WitnessDirectory "C:\DAGFileShareWitnesses\DAG1603.internal.sanger.
Notice, this is a different WitnessServer specified wtgc-exch-wit-b.internal.sanger.ac.uk instead of wtgc-exch-wit-a.internal.sanger.ac.uk.
Now to confirm its okay, re-run:
Get-DatabaseAvailabilityGroup -Identity "DAG1603" -Status | format-list Name, *Witness*, Servers |
That’s better, its showing primary now.
If you also run this, you’ll see a quorum resource is now specified:
Get-ClusterQuorum | fl * |
When it is operating correctly, the QuorumResource field will show a value, when it is not, this will be blank (typically).
Alternate Witness Server
Now, if like me you have the alternate witness server specified you may need to change this as well, for example, in our case each DAG uses two witness servers, so we have A and B servers specified as the WitnessServer and the AlternateWitnessServer, now it doesn’t matter which is which, as long as they are both different servers for maximum availability.
Run the commands above to set a new directory setting specifying the different server for the alternate as per the original/primary.