Error message 18750

If a companion server issues error message 18750, check the @@cmpstate of the servers. If the primary companion is in normal companion mode, but the secondary companion is in secondary failover mode, the cluster is in an inconsistent state requiring manual recovery. The inconsistent state may be caused by the failure of an sp_companion 'prepare_failback' command on the secondary companion. You can determine whether this happened by examining the HACMP log (located in /tmp/hacmp.out) on the secondary node. To recover:

1. Shut down both the primary and the secondary companions.

2. Restart the secondary companion.

3. Repair all databases marked “suspect.” To determine which databases are suspect, issue:

   select name, status from sysdatabases
   

   Databases marked suspect have a status value of 320.

4. Allow updates to system tables:

   sp_configure "allow updates", 1
   

5. For each suspect failed-over database, perform the following:

   1> update sysdatabases set status=status-256 where name='database_name'
   2> go
   1> dbcc traceon(3604)
   2> go
   1> dbcc dbrecover(database_name)
   2> go
   

6. From the secondary companion, issue:

   sp_companion primary_companion_name, prepare_failback
   

   Make sure that this command executes successfully.

7. Restart the HACMP services on the primary node.