Lesson learned, never run BETEST on Windows Server 2008

One thing I would like tell everyone that never run BETEST on Windows Server 2008. I was working on backup failing issue. Exchange writers were failing when running vssadmin list writers.

C:\Users\Admin>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Writer name: 'Microsoft Exchange Writer'

   Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
   Writer Instance Id: {3e9411d9-6bbb-455a-8150-2ba5925c2a98}
   State: [8] Failed
   Last error: Inconsistent shadow copy

I am using Net App Backup (Snap Manager for Microsoft Exchange). Backup is failing when testing the vss hardware provider, in this case it's Net App hardware provider.
Deleted the Net App vss provider from registry and rebooted the server.
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\Providers\{ddd3d232-a96f-4ac5-8f7b-250fd91fd102}
Ran the command vssadmin list providers to check if the NetApp Hardware provider is not there:

C:\Users\Admin>vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Provider name: 'Microsoft Software Shadow Copy provider 1.0'
   Provider type: System
   Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
   Version: 1.0.0.7

Tried backup again and it failed again with the same vss error.Now I am fighting to get the Exchange writer state in Stable and without any error. To check if the exchange writers are find I was running BETEST which was failing. The point is, you should never ever run BETEST on Windows Server 2008, because BETEST is for Windows Server 2003 not for 2008. In place of BETEST there is tool call DiskShadow.

Ran the DiskShadow on the server and checked the exchange writer state by running vssadmin list writers and its showing stable without any error:

Writer name: 'Microsoft Exchange Writer'
   Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
   Writer Instance Id: {e87db07c-b876-4ef7-affe-5b8ae968cf30}
   State: [1] Stable
   Last error: No error

Ran the backup and it failed again with error in Log.
[18:16:12.038]  [ExchangeServer] Asynchronous call DoSnapshotSet timed out!

[18:16:42.570]  [ExchangeServer] The call for DoSnapshotSet was cancelled.
[ExchangeServer] Failed to perform VSS snapshot, aborting...
[ExchangeServer] Backup is aborted.

Imported the delete registry of NetApp vss provider. Ran VSSREPORT Tool on the server and found that there is one registry key "819e04d3-809b-4657-aee0-778ca24470da" of Net App Hardware provider which is missing. That key is at the below location on the server.
HKEY_CLASSES_ROOT\CLSID\819e04d3-809b-4657-aee0-778ca24470da.

Now there are two option to get that key 1: Remove and Re-install Snap Manager for Exchange and Snap Drive 2: Repair Snap Manager for Exchange and Snap Drive. Ran the repair on both the application and ran the backup and it completed successfully.

Reference articles:
http://technet.microsoft.com/en-us/library/cc772172(WS.10).aspx
http://technet.microsoft.com/en-us/library/ee221016(WS.10).aspx
http://blogs.technet.com/b/josebda/archive/2007/11/30/diskshadow-the-new-in-box-vss-requester-in-windows-server-2008.aspx
http://blogs.technet.com/b/enterprise_admin/archive/2009/11/19/diskshadow-xcopy-backup-of-hyper-v.aspx


Cheers,

Hyper-V Features in Windows Server 8


In Windows Server 8, Hyper-V features are enhanced, improving virtual machine performance and providing a scalable virtualization base for cloud deployments. This table provides a comparison of Hyper-V key features between Windows Server 2008 R2 and Windows Server 8.
Hyper-V Features
Windows Server 2008 R2
Windows Server 8
Host Memory
1 TB
2 TB
Logical Processors
64 (Max)
160 (Max)
Guest VM Memory
64 GB (Max)
512 GB (Max)
Guest Virtual Processors
4 per VM (Max)
32 per VM (Max)
Guest NUMA
N
Y
Host Failover Cluster
Y (16 nodes)
Y (63 nodes)
VM Support - Failover Cluster
1000 (Max)
4000 (Max)
Live Migration
Y (serial)
Y (concurrent)
Live Migration (no cluster or shared storage)
N
Y
Live Storage Migration
N
Y

Cheers,

A fix for the interoperability issues between Exchange 2007 and 2010 EMC and IE9 is now available

Finally there is a fix!
We are happy to report that a fix for the Exchange Management Console (EMC) issues when Internet Explorer 9 is installed is now available. To be specific, we have talked about this in a previous blog post:
How does this fix need to be applied?
In order to install the fix, a released version of IE9 needs to be installed on the machine first. Then:
  • MS11-081: Cumulative Security Update for Internet Explorer: October 11, 2011 needs to be installed. This can be obtained from Windows Update or - if you need to download it for local network installation, the packages can be obtained here. Please note that the packages for client and server OSes might be different, depending on what you need. The installation of this package is REQUIRED for proper operation of the EMC hotfix.
  • In order to obtain the actual hotfix that resolves the interoperability problem with EMC, you will need to call Microsoft support and request a hotfix. The hotfix package is currently not available for public download, but can be obtained from support engineers, who can get it from internal hotfix servers. When you talk to support, the hotfix that you need to request is for the KB 2624899. Please note that this article is not publicly available at this time either.

Cheers,

Windows Server Backup failing with error code 2155348010

I was working with one customer yesterday on backup issue. Customer was taking backup using Windows Server Backup of Exchange Server 2010 but it was failing with error code "2155348010". Checked the application log and found the event 517.

He informed that he is taking backup on USB drive and in Windows Server Backup failure result it was talking about I/O failure. We were taking backup of two entire drive, C drive and E drive.(Windows Server Backup is volume backup)
Restarted the Exchange Server and started taking the backup, but this time we took only system state backup on the server drive and not on the USB drive. Backup started fine without any error.

Started the backup of Exchange Database drive "E" and this time we were taking backup on the USB drive and backup completed successfully.
When you select multiple drive for the backup (using windows server backup) and you are backing up on the USB drive, the chances are that they will fail due to I/O read and write failure.

Note: This is one of the reason why Microsoft doesn't recommend to take backup on USB drive.

If you do not have a choice other than running the backup on USB drive than there one thing you can do. Keep in mind its a temporary option.
1: Take the backup of the database on the server drive.
2: Once the backup is complete, copy the backup file paste it on the USB drive and delete it from the server.

Cheers,

EXCHANGE RANGER