Searching *Inside* Stored Procedures

I’ve recently inherited a large BizTalk system and needed a quick way to determine which custom stored procedures accessed which tables/views and called other sprocs. Thanks to my co-worker Shaun, I now have the power of the INFORMATION_SCHEMA views at my disposal!

Information Schema Views

Information schema views provide an internal, system table-independent view of SQL Server metadata. They provide information on tables, views, stored procedures to name but a few objects. The views included in SQL Server 2005 comply with the ANSI SQL-92 standard definition, so any queries can (theoretically) be taken and executed on Oracle, or DB2, etc. More information can be found online at http://msdn2.microsoft.com/en-us/library/ms186778.aspx

Searching within Stored Procedures

To search for – or within – stored procedures, you need to work with the INFORMATION_SCHEMA.ROUTINES view, the following sample returns all stored procedures where the name contains ‘BizTalkServerApplication’:

–– Change to the BizTalk Message Box database.
USE BizTalkMsgBoxDb
GO
–– Search for sprocs with ‘BizTalkServerApplication’ in the name.
SELECT ROUTINE_NAME,
CONVERT(VARCHAR(8), created, 3) + ‘ ‘ + CONVERT(VARCHAR(8), created, 108) AS ‘Created Date’,
CONVERT(VARCHAR(8), last_altered, 3) + ‘ ‘ + CONVERT(VARCHAR(8), last_altered, 108) AS ‘Last Altered On’
FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_NAME LIKE ‘%BizTalkServerApplication%’
ORDER BY 1

To search within stored procedures, simply change your WHERE clause to use the ROUTINE_DEFINITION column (Note: the ROUTINE_DEFINITION column only includes the first 4000 characters of the T-SQL statements that created the stored procedure). The following sample returns all stored procedures that contain the text ‘SuspendedQ’:

–– Change to the BizTalk Message Box database.
USE BizTalkMsgBoxDb
GO
–– Search for sprocs that contain the text ‘SuspendedQ’ in the name.
SELECT ROUTINE_NAME,
CONVERT(VARCHAR(8), created, 3) + ‘ ‘ + CONVERT(VARCHAR(8), created, 108) AS ‘Created Date’,
CONVERT(VARCHAR(8), last_altered, 3) + ‘ ‘ + CONVERT(VARCHAR(8), last_altered, 108) AS ‘Last Altered On’
FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_DEFINITION LIKE ‘%SuspendedQ%’
ORDER BY 1

BizTalk 2006 (incl. R2) Management Pack for Operations Manager 2007 Released

Looks like the BizTalk 2006 & BizTalk 2006 R2 Management Pack for Operations Manager 2007 has been released and is available in the Management Pack Catalog.

To quote the catalogue entry, the management pack:

…is a entirely new … providing comprehensive discovery and monitoring of BizTalk Server components and applications. In addition to general support for BizTalk Server 2006, BizTalk Server 2006 R2, this management pack provides coverage for new BizTalk Server 2006 R2 features, such as the native EDI runtime and RFID.

I think its great that we now have native EDI and RFID monitoring ‘out of the box’ – no more need to cludge something together to monitor these two aspects of the product. I’d personally like to see this MP backported to MOM 2005, but I doubt that will happen!

Why Archive and Purge when you can just Purge?

In BizTalk 2004 SP2, the BizTalk Team gave us the Archive and Purge SQL Server Maintenance Job for managing the size of the Tracking Database. This was a great tool and really took away some of the admin headaches in maintaining this particular database.

The job allows administrators to archive olde tracking data and verify the integrity of the backup before purging from the live database (for detailed instructions, see the MSDN website). This is a good, pro-active practice for the health of any BizTalk environment, in that:

  • BizTalkDTADb database growth is constantly checked, allowing the TDDS service to run effectively, thereby maintaining a healthy Message Box.
  • Maintaining the size of the BizTalkDTADb data and log files ensures that the database doesn’t just eat all of your available disk space.
  • The tracking data can be restored in a dedicated OLAP environment, allowing reports to be run without affecting the live BizTalk environment.

However, imagine a scenario where you don’t want the archived data – you simply want to purge. Where that’s the case, you can either manually run a truncate on the BizTalkDTADb (see my posts detailing how to do this under BizTalk 2004 or BizTalk 2006 – there are some subtle differences), or run the hidden admin gem dtasp_PurgeTrackingDatabase (a stored procedure used by the dtasp_ArchiveAndPurgeTrackingDatabase which just purges) on a scheduled basis, so you no longer need to worry about manually purging.

Configuring the dtasp_PurgeTrackingDatabase Stored Procedure

The Purge stored procedure is used in a very similar manner to dtasp_ArchiveAndPurgeTrackingDatabase, taking 4 parameters:

  • Live Hours – Any completed instance older than the live hours + live days…
  • Live Days – …will be deleted along with all associated data.
  • Hard Days – all data older than this will be deleted. The time interval specified here should be greater than the live window of data.
  • Last Backup – UTC Datetime of the last backup. When set to NULL, data is not purged from the database.

As an example of its usage, if you wanted to purge any tracking data older than two hours and hard-delete any data older than one day, you would use the follow T-SQL:


–– Change to the BizTalk Tracking database.
USE BizTalkDTADb
GO

–– Prune tracking data older than two hours.
DECLARE @dtLastBackup DATETIME
SET @dtLastBackup = GetUTCDate()
EXEC dtasp_PurgeTrackingDatabase 2, 0, 1, @dtLastBackup

The @dtLastBackup parameter is used to ensure that records that have not been backed-up are not deleted by the purge procedure, so we set it to the current UTC date/time to ensure that whatever live hours/days you specify, records are deleted. I’m not too sure why the development team included this as a parameter: the procedure is a wrapper that calls the dtasp_PurgeTrackingDatabase_Internal (which is also called by the dtasp_ArchiveAndPurgeTrackingDatabase procedure) so it could have been included in that wrapper given that it is always defaulted to the current UTC date/time during purges!

One other thing to note is that the wrapper script also modifies the prune before date (the date that is built by the live hours/days specified): this date date is tweaked to remove ten minutes to ensure redundancy in the remaining data.  In the example used above, rather than keeping two hours of data, there will in fact be two hours and ten minutes once the code has been run.

Using the dtasp_PurgeTrackingDatabase Stored Procedure

To start using this procedure, I would suggest that you first truncate your database, then configure the SQL Server Agent job to run every X hours/days depending on your environmental requirements. I suggest initially truncating your database to ensure the first run of the job doesn’t take several hours!

Further details about the dtasp_PurgeTrackingDatabase stored procedure can be found online at MSDN.

Diagnosing Performance Problems in SQL Server

SQL Server is the heart of a BizTalk environment, so a performant database is a must for any self-respecting BizTalk setup. But, if you consider that SQL Server is causing you performance issues, where do you start to look?

Thankfully, there is a two part series on this exact topic from Graham Kent, a SQL Server Support Team escalation engineer, where is he discusses PSSDIAG/SQLDIAG and the information that it produces to help diagnose a whole raft of possible SQL Server performance problems. Links are below, enjoy!

Truncating the BizTalk 2006 Tracking Database

In Truncating the BizTalk 2004 Tracking  Database I discussed how to truncate the tracking database in BizTalk 2004. Over on the BizTalk Gurus forums, user Nick Busy wanted to do the same thing for BizTalk Server 2006 – he’s kindly allowed me to repost his instructions for the community on this blog:

0. Before start, ensure you have got the database admin priveleges on the database

1. Stop all BizTalk Server Host Instances

2. Full backup BizTalkDTADb database (just in case)

3. Make scripts to create views (MANDATORY)

dbo.dtav_ServiceFacts
dbo.dtav_MessageFacts
dbo.dtav_FindMessageFacts

4. Run SQL script:

use BizTalkDTADb
GO

— Drop the Views (before you perform this, ensure you take copies of these views!)
— unfortunately, it’s necessary for SQL 2000, but you can skip it for SQL 2005
Drop View dbo.dtav_ServiceFacts
Drop View dbo.dtav_MessageFacts
Drop View dbo.dtav_FindMessageFacts
Go

— Truncate the necessary Tables
Truncate Table dta_CallChain
Truncate Table dta_DebugTrace
Truncate Table dta_MessageInOutEvents

Truncate Table dta_ServiceInstanceExceptions
Truncate Table dta_ServiceInstances

Truncate Table Tracking_Fragments1
Truncate Table Tracking_Parts1
Truncate Table Tracking_Spool1

Truncate Table dta_MessageFieldValues

— end of the script

5. Update statistics on BizTalkDTADb database

— update statistics
exec sp_updatestats

6. Run the saved scripts (see step 3) to recreate the dropped views from your own environment.

7. Shrink BizTalkDTADb database (sometimes it doesn’t work from GUI, so using sql command will help)

— shrink database
dbcc shrinkdatabase (BizTalkDTADb, 10)

8. Start BizTalk Server Host Instances

9. Configure and enable SQL Agent job “DTA Purge and Archive” (to avoid over-growing the database in the future)

P.S. The script above does not truncate Rule Engine related tables.

Thanks Nick, much appreciated.

XmlAssembler TargetCharset Property Error in BizTalk 2006 & 2006 R2

Update: Tomas Restrepo added a comment to this post in which he explains why this particular ‘feature’ is present – I think its worth repeating here:

…the problem was that the Target Charset property of the assembler actually gets written as two separate elements when serialized to the pipeline XML file, and both must be present for the assembler to actually figure out the encoding to use… So what happens is that the metadata in the actual design time properties for the assembler component only actually represents one of those elements in the serialized format, and this is what the per-instance pipeline configuration dialog uses to ask the user to enter the values. So in essence, the dialog only allows you to edit one of the values and not the other, so you always end up with the component incorrectly configured.

Thanks Tomas!

While researching a post about message encoding in BizTalk, I can across this Microsoft Knowledge Base article regarding the TargetCharset property on the XmlAssembler component – it would appear that if the property is set through the Admin Console, it won’t take effect. This is strange, because this bug appears to have been fixed in BizTalk 2004 SP1, but somehow made it back into all versions of BizTalk 2006 & 2006 R2. Furthermore for BizTalk 2006, Microsoft only offer a work-around rather than a Hotfix which is frustrating.

So, what happens when you attempt to set the TargetCharset property? The default encoding when using the XmlTransmit pipeline – without the TargetEncoding property set – is a UTF-8 encoded Xml message (click on the image for a full-size version):

UTF-8 Encoded Message

If you change the TargetCharset property on the XmlAssembler component of the default XmlTransmit pipeline in the BizTalk Admin console (as shown below), the change is not picked-up and the same UTF-8 encoded message (as above) is returned; in this case, we’re trying to change it to UTF-16:

XmlAssembler-AdminConsole-TargetCharsetProperty

To change the encoding, you must create a custom pipeline and use the XmlAssembler with the TargetCharset property set appropriately, as follows:

XmlAssembler Custom Pipline Component TargetCharset Property

Using this new custom pipeline with the TargetCharset property correctly configured to output UTF-16 encoded messages, we acheive our desired encoding, as follows (click on the image for a full-size version):

UTF-16 Encoded Message

Encoding can also be acheived by setting the XMLNORM.TargetCharset property of the message to be output in an Orchestration Message Assignment shape as follows:

<MessageName>(XMLNORM.TargetCharset) = "UTF-16";

Note, this testing was performed on BizTalk 2006 R2.