Insight into BizTalk Server message tracking

I had to go through this a few times recently so I thought I’d document it:: when does BizTalk Server save tracking information to the tracking database, and how does it all link together?

Basically BizTalk will track information about a messages in the following locations:

• On a receive port right after the adapter receives the message and before the receive pipeline or any maps are executed
• On a receive port right after the pipeline and the maps are executed – this means the message type tracked after the receive port is the message type after the map execution
• On an orchestration as a message is received
• On an orchestration as a message is sent
• On a send port right before any maps or the send pipeline is executed
• On a send port right after the pipeline is executed just before the adapter sends it out

Let’s consider a typical scenario: a message is received via a receive port with the file adapter. No disassembly is made (i.e. there is only one document in the message). An orchestration subscribes to the message, does some processing, and sends out only one message. A send port bound to the orchestration sends the message out.

This diagram shows the stages where BizTalk would record information about the message to the tracking database:

Note that BizTalk records that the message has gone through those stages even if you haven’t enabled property or message body tracking on the ports or orchestration. This is recorded online by the Tracking Data Decode Service to the dta_MessageInOutEvents and other tables in the BizTalk Tracking database. If message content and context tracking is enabled on the particular service then the BizTalk SQL Agent  jobs will take care of moving this from the Message Box database to the Tracking database on a scheduled basis (every minute with the default setting).

Now that we’ve established the basics of where BizTalk records the message information, how does it link together? How do you know the sequence of what happened after BizTalk has processed the messages? If you look back at the diagram you will see that there is a direction and a different message GUIDs at each message tracking point, and there is a different Service Instance GUID on receive ports, send ports and orchestrations.  Service instances are the specific instance of the service (receive pipeline, send pipeline, orchestration) that handled that particular message. A service instance will receive a message with a message GUID and send a message with a different message GUID, with the exception of when failures occur. So a receive port service instance receives a message with message GUID 1. It then sends one or more messages out, each with their own message GUIDs. The next service instance (send port or orchestration) that subscribes to those messages will have their own service GUID, but the received message GUID is the same as the message GUID sent by the receive port. Then if this orchestration or send port service instance sends out one or more messages these will have their own message GUIDs as well, and so forth.

To make it clear, let’s look at a scenario where a receive port receives an XML message and disassembles it into two messages; then two instances of an orchestration that subscribes to the disassembled messages performs some operation on them; then the orchestration instances send out a final message via a send port. With this you will have this GUID tree structure in the tracking database:

• Receive Port Service Instance 69dee51d-21c0-4f85-ba80-58fdb08da6ec
• Receive Message 2b220b2d-a07a-4752-ad23-4cfd9f769c25 << Unparsed Interchange (ie original unchanged message)
• Send Message 56684d73-5dac-4bd8-a2ce-faba9bc89b6c
• Send Message 7553db2f-3ea6-47a1-9cf4-fdb02d6d3c37
• Orchestration Service Instance f3dfe962-3718-4995-bcfb-d3f0cdf82772
• Receive Message 56684d73-5dac-4bd8-a2ce-faba9bc89b6c
• Send Message dade4b4b-cc39-4464-b587-d8fa09772796
• Send Port Service Instance a96660f7-9665-4a05-b440-6929679eafd5
• Receive Message dade4b4b-cc39-4464-b587-d8fa09772796
• Send Message 854a04da-d1b1-4c26-94df-7e36fbe8cb3b << end of tree
• Orchestration Service Instance 380185b9-0c0e-4b21-afd5-94afebc11dfb
• Receive Message 7553db2f-3ea6-47a1-9cf4-fdb02d6d3c37
• Send Message 36baa5ba-3cb8-45aa-afe3-1d0e0b60469a
• Send Port Service Instance 68041f1b-242f-4043-8f95-a6227b82dd12
• Receive Message 36baa5ba-3cb8-45aa-afe3-1d0e0b60469a
• Send Message 67e51bf5-e530-47d9-90b5-48ffeaca8ad0 << end of tree

As you can see there is an initiating tracking entry with an unparsed interchange receive and the tree builds from there with any number for nodes. If you enable message body tracking on the receive port, orchestration or send port BizTalk will also track the message content. It is with the information from the GUID values that the ‘message flow’ option in the BizTalk Administration Console is built. If you open the BizTalk Administration Console and look at the Tracked Message Events you can see the direction in the Event Type column, the schema name (with unparsed interchange as the first one), and the service class/service name to determine the service that was handling the message. With the above scenario you will get a result like the below: a received unparsed interchange and two sends from the same pipeline service instance. Two orchestration instances receive each message and send a message out. And two send pipeline instances receive the messages from the orchestration and sends them out:

If you right click on the unparsed interchange entry and select ‘message flow’ you get the good old HAT view of the message flow, that gives us a lot of information: the ‘Instance ID’ at the top is the service instance id of the receive pipeline. Each entry on the table has an in or out event depicted by the envelope with an arrow coming in or an arrow going out. Each entry also shows the message id and has a clickable link to the next or previous step in the tree if there is one. I’ve highlighted in red the parts of interest:

From the above you can then click on one the orchestration links and the message flow will show the flow from that orchestration instance’s perspective (where you can also switch to the orchestration debugger but that’s another story):

This way you can traverse the entire tree of what happened after the original message was received. Another way of following this structure is by querying the dtav_MessageFacts view on the BizTalk Tracking database and building a common table expression query (with MUCH caution) around the three values [MessageInstance/InstanceID], [ServiceInstance/InstanceID], [Event/Direction].

I hope with this insight into message tracking in BizTalk you can now follow the tracking of messages and events through the BizTalk Administration Console with ease!

Regards,

Thiago Almeida