Lesson learned, never run BETEST on Windows Server 2008

One thing I would like tell everyone that never run BETEST on Windows Server 2008. I was working on backup failing issue. Exchange writers were failing when running vssadmin list writers.

C:\Users\Admin>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'Microsoft Exchange Writer'

   Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
   Writer Instance Id: {3e9411d9-6bbb-455a-8150-2ba5925c2a98}
   State: [8] Failed
   Last error: Inconsistent shadow copy

I am using Net App Backup (Snap Manager for Microsoft Exchange). Backup is failing when testing the vss hardware provider, in this case it's Net App hardware provider.
Deleted the Net App vss provider from registry and rebooted the server.
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\Providers\{ddd3d232-a96f-4ac5-8f7b-250fd91fd102}
Ran the command vssadmin list providers to check if the NetApp Hardware provider is not there:

C:\Users\Admin>vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Provider name: 'Microsoft Software Shadow Copy provider 1.0'
   Provider type: System
   Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
   Version: 1.0.0.7

Tried backup again and it failed again with the same vss error.Now I am fighting to get the Exchange writer state in Stable and without any error. To check if the exchange writers are find I was running BETEST which was failing. The point is, you should never ever run BETEST on Windows Server 2008, because BETEST is for Windows Server 2003 not for 2008. In place of BETEST there is tool call DiskShadow.

Ran the DiskShadow on the server and checked the exchange writer state by running vssadmin list writers and its showing stable without any error:

Writer name: 'Microsoft Exchange Writer'
   Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
   Writer Instance Id: {e87db07c-b876-4ef7-affe-5b8ae968cf30}
   State: [1] Stable
   Last error: No error

Ran the backup and it failed again with error in Log.
[18:16:12.038]  [ExchangeServer] Asynchronous call DoSnapshotSet timed out!

[18:16:42.570]  [ExchangeServer] The call for DoSnapshotSet was cancelled.
[ExchangeServer] Failed to perform VSS snapshot, aborting...
[ExchangeServer] Backup is aborted.

Imported the delete registry of NetApp vss provider. Ran VSSREPORT Tool on the server and found that there is one registry key "819e04d3-809b-4657-aee0-778ca24470da" of Net App Hardware provider which is missing. That key is at the below location on the server.
HKEY_CLASSES_ROOT\CLSID\819e04d3-809b-4657-aee0-778ca24470da.

Now there are two option to get that key 1: Remove and Re-install Snap Manager for Exchange and Snap Drive 2: Repair Snap Manager for Exchange and Snap Drive. Ran the repair on both the application and ran the backup and it completed successfully.

Reference articles:
http://technet.microsoft.com/en-us/library/cc772172(WS.10).aspx
http://technet.microsoft.com/en-us/library/ee221016(WS.10).aspx
http://blogs.technet.com/b/josebda/archive/2007/11/30/diskshadow-the-new-in-box-vss-requester-in-windows-server-2008.aspx
http://blogs.technet.com/b/enterprise_admin/archive/2009/11/19/diskshadow-xcopy-backup-of-hyper-v.aspx


Cheers,

Leave a Reply

EXCHANGE RANGER