Beware of windows.old when upgrading to Windows 8

I have just taken the plunge and upgraded from Windows 7 to Windows 8 (partly because of the £25 upgrade offer). After the upgrade I noticed a severe lack of space on my otherwise roomy C: drive.

A quick check in Windows Explorer reveals a windows.old directory which contains all of my old Windows 7 ‘Windows’ directory, Program Files, Users and PerfLogs directories – and its just short of 20Gb. I’ll be shifting it to an external backup drive shortly…


Including Custom Databases in the Backup BizTalk Server Job

I’m sure its common knowledge that other databases can be backed-up using the Backup BizTalk Server Job, meaning that the transaction consistency that is that is written the the BizTalk databases can be included in your own custom databases.

I thought it was pretty easy to include custom databases, simply by adding an entry into the adm_OtherBackupDatabases table in the BizTalkMgmtDb database. In the screenshot below, I’ve added the BizTalk RosettaNet Accelerator (BTARN) databases:

I thought it was that simple, until I tried it for the first time in real life and was greeted with the following error from SQL Agent (notice the error in bold):

Executed as user: NT AUTHORITYSYSTEM. Processed 17520 pages for database ‘BizTalkDTADb’, file ‘BizTalkDTADb’ on file 1. [SQLSTATE 01000] (Message 4035)  Processed 2 pages for database ‘BizTalkDTADb’, file ‘BizTalkDTADb_log’ on file 1. [SQLSTATE 01000] (Message 4035)  BACKUP DATABASE successfully processed 17522 pages in 10.454 seconds (13.093 MB/sec). [SQLSTATE 01000] (Message 3014)  Processed 2264 pages for database ‘BizTalkMgmtDb’, file ‘BizTalkMgmtDb’ on file 1. [SQLSTATE 01000] (Message 4035)  Processed 3 pages for database ‘BizTalkMgmtDb’, file ‘BizTalkMgmtDb_log’ on file 1. [SQLSTATE 01000] (Message 4035)  BACKUP DATABASE successfully processed 2267 pages in 2.284 seconds (7.751 MB/sec). [SQLSTATE 01000] (Message 3014)  Processed 3280 pages for database ‘BizTalkMsgBoxDb’, file ‘BizTalkMsgBoxDb’ on file 1. [SQLSTATE 01000] (Message 4035)  Processed 2 pages for database ‘BizTalkMsgBoxDb’, file ‘BizTalkMsgBoxDb_log’ on file 1. [SQLSTATE 01000] (Message 4035)  BACKUP DATABASE successfully processed 3282 pages in 2.156 seconds (11.890 MB/sec). [SQLSTATE 01000] (Message 3014)  Processed 256 pages for database ‘BizTalkRuleEngineDb’, file ‘BizTalkRuleEngineDb’ on file 1. [SQLSTATE 01000] (Message 4035)  Processed 1 pages for database ‘BizTalkRuleEngineDb’, file ‘BizTalkRuleEngineDb_log’ on file 1. [SQLSTATE 01000] (Message 4035) Could not find stored procedure ‘BTARNARCHIVE.dbo.sp_BackupBizTalkFull’. [SQLSTATE 42000] (Error 2812) BACKUP DATABASE successfully processed 257 pages in 0.326 seconds (6.158 MB/sec). [SQLSTATE 01000] (Error 3014).  The step failed.

‘Could not find stored procedure ‘BTARNARCHIVE.dbo.sp_BackupBizTalkFull’. Oh really?

It turns out that it isn’t as simple as adding a custom database to the adm_OtherBackupDatabases table – a number of stored procedures and tables also need to be added to your custom database to get the backup job to successfully work, however these can be easily added by executing the two SQL scripts Backup_Setup_All_Procs.sql and Backup_Setup_All_Tables.sql against your custom databases (the scripts can be found in the ‘[BizTalk Installation Directory]Schema’ directory). You may want to force a full backup to ensure you have full backups across all of your databases:

UPDATE [BizTalkMgmtDb].[dbo].[adm_ForceFullBackup] SET ForceFull = 1

Re-run your backup job and you will see your custom databases included in the backup.

The requirement to include additional stored procedures and tables in custom databases is well worth remembering, especially if you are deploying (or re-deploying) custom databases that don’t include these artifacts by default.

The procedure is detailed in more depth on MSDN.

BTARN ‘Service Content’ Error in the RNDisAssembler Component

We came across the following error late last night which was a bit of a show-stopper. We were trying to load a custom PIP (specifically a PIDX OrderChange), but kept hitting this issue time and time again:

Event Type: Error
Event Source: BizTalk Accelerator for RosettaNet
Event Category: None
Event ID: 4096
Source module:
Correlation information:
Receive pipeline rejected incoming message
due to the following RNIF exception:
UNP.SCON.VALERR : A failure occurred while validating the service content.
Event Type: ErrorEvent Source: BizTalk Accelerator for RosettaNetEvent Category: NoneEvent ID: 4096Date: 18/08/2010Time: 10:33:47User: N/AComputer: [COMPUTER NAME]Description:Source module:RNDisAssembler
Correlation information:


Receive pipeline rejected incoming message due to the following RNIF exception: UNP.SCON.VALERR : A failure occurred while validating the service content.

At first I didn’t quite understand the cause of the error – the PIDX OrderChange message contained within the Service Content was valid (as far as I was aware), all of the other messages within the payload looked correct and it was 3am….

It turns out that the RNDisassembler does in-fact attempt to validate the message contained within the Service Content against a deployed schema just like the standard XmlDisassembler. The message that our trading partner was sending did not validate and hence the RosettaNet Accelerator threw this error message; once we had corrected the schema and redeployed, the error went away.

This is certainly one to be aware of if you are developing custom PIP’s to use with the RosettaNet Accelerator: ensure that the message in the Service Content validates against your custom PIP schema!

SCOM can cause Unchecked Growth in SSODB

Ok, that’s a bit of an attention grabbing headline, so let me clarify that statement: SCOM can cause unchecked growth in SSODB if you’re not regularly backing up the SSODB transaction log.

We encountered this one today – a client’s SSODB ran out of space overnight, causing the BizTalk environment to shut-down. On further investigation, it would appear that every time SCOM checks the health of Enterprise Single Sign-On, an entry is recorded in the SSOX_AuditXpLookup table:

ESSO appears to be clever enough to manage the size of this table, truncating it every 30 minutes, however this doesn’t help if you’re not managing the size of the database transaction log through backups. To quote the SQL Server documentation:

If log records were never deleted from the transaction log, it would eventually fill all the disk space that is available to the physical log files. Log truncation automatically frees space in the logical log for reuse by the transaction log.

All the more reason to enable and run the supplied Backup BizTalk Server job to help maintain the health of your BizTalk environment.

If you’re looking for more information on the Backup BizTalk Server Job, take a look at my series of posts on the topic:

Hat-tip to this MSDN Forums posting that helped us identify the issue.

BizTalk Server 2010 Beta Released!

Update 2 (24th May 2010): The beta is back, download it now at the link below.

Update: Well, it was there about four hours ago, but the link no longer works…. I’m pleased to say that it isn’t just me who got all excited. I’ll update this post if the link becomes active again. In the meantime, I did manage to download a copy, so will post my experiences here.

Looks like the beta of BizTalk Server 2010 was released today – three words: Windows 7 Support!

Hat-tip: Nick M.

Further MS-DTC Issues – Check the Startup Order of Clustered Services

We’ve just encountered an obscure MSDTC/SQL Server issue that I thought would be beneficial to the wide-community.

Following a failover of our (Windows Server 2008) cluster, we started to encounter unexpected errors when BizTalk attempted to perform any tasks that required a distributed transaction, even though everything appeared to be running correctly: MS-DTC was running and we could DTCPing the BizTalk Server from the SQL Server (and vice-a-versa), so no issues with DTC; SQL Server was also running as we could connect to the instance via Management Studio and BizTalk could read the Management Database etc., yet DTC operations still failed with the following error:

Enlist operation failed: 0x8004d01c(XACT_E_CONNECTION_DOWN). SQL Server could not register with Microsoft Distributed Transaction Coordinator (MS DTC) as a resource manager for this transaction. The transaction may have been stopped by the client or the resource manager.
A severe error occurred on the current command.  The results, if any, should be discarded. (Microsoft SQL Server, Error: 8510)

It turns out that when clustered services are brought online, they must be started in a specific order, with the DTC service being started before SQL Server. If they are not brought online in this order, SQL Server fails to register itself and DTC transactions cannot be initiated, even though both the DTC and SQL service’s are running and everything looks correct.

Thanks to this forum post for providing us with the much needed pointer:

Team Explorer 2010 ‘narks’ Enterprise Single Sign-On

Update 20th July 2010: There is now a Hotfix from Microsoft that resolves this issue, see; thanks for the pointer Daniel.

If you’re developing BizTalk projects and using TFS 2010, you’ll probably need to install Team Explorer 2010 to leverage some of the new functionality not accessible through the Visual Studio 2008 TFS hooks. Team Explorer 2010 is a stripped down version of the Visual Studio 2010 environment used solely to access Team Foundation Server services.

Be aware that if you do install Team Explorer 2010, you’ll nark your Enterprise Single Sign-On installation, meaning you can’t make any changes to BizTalk – when ESSO attempts to start, you’ll probably receive the following error in the Application Event Log:

Could not create SSOSQL. To fix the problem, reinstall SSO or try 'regasm SSOSQL.dll' from a Visual Studio command prompt.
Error Code: 0x80131700
To correct the missing SSOSQL assembly, simply run the following command in a Visual Studio command prompt as suggested:
regasm "C:Program FilesCommon FilesEnterprise Single Sign-OnSSOSQL.dll"
ESSO should now start as expected.