Rogue Service-Windows Created for Response Subscriptions in BizTalk 2006

Yesterday I came across an interesting production issue – a service-window that BizTalk Server 2006 had incorrectly created on a response message subscription due to the way it handles daylight savings time.

Background to the Problem

The orchestration in question consumes a third-party web-service via the SOAP adapter. The request had been successfully sent, however the orchestration was dehydrated with the web-service response message in the ‘Queued (scheduled for later delivery)’ state. As described on Technet, this status indicates that the message is waiting to be sent by a send port that has a service window set.

Now that’s strange: the Send Port in question is dynamic, so no chances of an overly enthusiastic BizTalk Admin applying a service window, or a service window being applied in code (you can’t configure a service window on a dynamic port). So how did we get a service window on the response message subscription?

After some digging , I came across Microsoft Hotfix 939797 (http://support.microsoft.com/kb/939797/en-us) which addresses a problem whereby message delivery may be delayed until 11:59 PM in BizTalk Server 2006.

A Bug in the BizTalk 2006 MessageBox Logic?

There appears to be a bug in the BizTalk MessageBox logic which only manifests itself during months where daylight saving time is in effect. To paraphrase the Hotfix: When BizTalk creates a subscription for a request-response message, it converts Universal Time Coordinated (UTC) time to the local time. To do this, it calculates the difference between the current local time and the current UTC time and then adds this value to the service window of the send port. If this operation is delayed for any period of time, the BizTalk engine incorrectly interprets this as a service window and applies it to the response subscription, even though a service window has not been specifically applied on the port.

As a result, the subscription does not activate until 23:59, at which time the response message is rehydrated and the orchestration is allowed to continue. Why 23:59 I don’t know, but looking at the subscriptions table in the BizTalkMsgBoxDb databases, we did in-fact have a rogue service window applied to the response subscription, as shown in the red-box below:

When I checked the next morning, the orchestration had rehydrated and was running; the response message was no longer queued awaiting later delivery, as expected.

This bug may also explain some of the other behaviour we have seen: business processes appear to hang and our Operations Team restart them, but when our Dev’s do further investigations there are no suspended instances as we would expect had there been an error.

Wrap-Up

If you’re experiencing queued response messages within orchestrations, or rogue service-windows as described above, this Hotfix may resolve your issue. Be aware though that the Hotfix is only to be used on BizTalk Server 2006 – it is included in BizTalk Server 2006 R2 and BizTalk Server 2009 by default.

Duplicate SOAP Subscription on Dynamic Request-Response Port

Joy of joys, more subscriptions problems to debug this week with some unexpected results.

The first sign of problems were persistence errors:

Microsoft.XLANGs.Core.PersistenceException: Exception occurred when persisting state to the database. ---> Microsoft.BizTalk.XLANGs.BTXEngine.PersistenceItemException: A batch item failed persistence Item-ID dbbd66a3-d748-4051-a238-fea9509efcf4 OperationType MAIO_CommitBatch Status -1061151949 ErrorInfo The message found multiple request response subscriptions. A message can only be routed to a single request response subscription. . ---> Microsoft.BizTalk.XLANGs.BTXEngine.PublishMessageException: Failed to publish (send) a message in the batch. This is usually because there is no one expecting to receive this message.  The error was The message found multiple request response subscriptions. A message can only be routed to a single request response subscription.  with status -1061151949.

With the actual error in bold above – multiple request-response subscriptions were found which aren’t allowed (I’m not running 2006 R2 and don’t have KB923632 installed).

My solution employs an orchestration which consumes a web-service over a dynamic solicit-response port; the port itself has been created using early-binding (creating one of those ugly port names).

Subscriptions Problems

Digging into the dynamic port subscriptions, I was interested to see that a subscription is created for each adapter you have registered in your BizTalk Group. This makes sense as the transport type isn’t known up-front:

Digging into those subscriptions, I discovered two subscriptions that are exactly the same for the SOAP adapter – no wonder we have multiple request-response subscriptions…..

Apart from hacking the database to remove this unwanted subscription entry, I could find no way of updating the subs. on this dynamic port. Interestingly however, creating a new dynamic port from the Admin Console and binding that to the orchestration resulted in the correct number of subscriptions and a working solution.

I’m wondering then where the bug is: I don’t think it can be when the early-bound ports are created during deployment as the subscriptions themselves aren’t created until the port is enlisted, which (AFAIK) happens in exactly the same way for manually created send-ports. The the problem is only evident on early- and not late-bound ports!

I’m now toying with the idea of KB923632 so we don’t have to worry about this issue again, but I’m keen to understand why this is a problem in the first place, so I’d also be pleased to hear from anyone who knows of a Hotfix that addresses this duplicate subscription issue. I also think its a good one to keep in the back of your mind in-case you encounter any ‘interesting’ subscription issues.

The Case for a New Tool?

One final thing on this: diagnosing subscription problems really raises the need for a ‘Subscription Finder’ tool. We currently have the Subscription Viewer in the Admin Console, which is great, but we have to know what subscriptions we are looking for in the first place. What would have been really helpful here would be a tool where I can enter the details of the promoted properties from the Failed Routing Report and the tool locates all subscriptions that would match those properties. I am looking at putting together such a tool, so watch this space.

Related articles by Zemanta

• Microsoft bullish on BizTalk Server beta
• Microsoft’s application server goes Irish

Which Host does a Dynamic Send Port Use?

Something I’ve never come across before – which Host does a Dynamic Send Port use? There is no value displayed under the Handler column for a dynamic Send Port:

Also, when you look at the configuration for a dynamic Send Port (in this case a dynamic HTTP port), there is no drop-down option for the Handler:

So, Which Host does a Dynamic Send Port use?

After some digging and testing, it became apparent that the Dynamic Send Port runs on the default Host configured for the adapter that is being used; in our case, the Dynamic Send Port was running the HTTP adapter, do a quick look in the Admin Console reveals we are running the Sender Host as our default HTTP adapter host:

In researching for this blog entry, I’ve also come across this discussion on the topic. Jan Eliasen raises the interesting point of this design – we are tied to using the same default Host for each dynamic port that uses the same adapter, which could cause separation of concern issues with bigger systems. There are also possible configuration headaches: when using the SMTP adapter for example, you are tied to using the same adapter settings for each Dynamic Send Port – e.g. SMTP server name & authentication!