Friday, August 23, 2013

ESB Series – Final thoughts


This is the last post in the ESB Series which i’ve been doing;

Hopefully this has showed some ways of using the ESB Toolkit, and ways of extending it so that it is more suited for various scenarios.

This post will close the gaps i left open in earlier posts. In the end, i have hopefully demonstrated some common scenarios which are possible with the ESB Toolkit. My final thought on this subject is that the ESB toolkit is targeted to work in a more abstract/generic way, which is a good thing. After using the ESB Toolkit I think it is a nice addition to BizTalk, but I only recommend to use it when the problem demands for this type of a solution. There is a learning curve involved, it works differently as classic BizTalk, and means that you have to define your solution differently, sometimes it even makes the components more complex. In the end, it will bring flexibility, maintainability and allows re-use, however this requires an investment.

The ESB Toolkit is not a golden hammer, don’t use it per definition, think about the problem, think of the possible solutions, think of the process, possible changes and decide which approach would be the best.

ESB Toolkit Itinerary scenarios

In the first post ‘Itinerary’ i’ve showed the difference between a Receive-Send port and the same set up using an Itinerary. Below, you can see the different scenarios possible, and the comparison with a BizTalk set up.

  • Pub / sub (Port 2 Port)


This Itinerary basically, receives a messages from the Receive Pipeline, performs Tracing, Routing and send a message to a Send port. The subscription is management by BizTalk, however, the Send port is configured Dynamically.

Advantage: No Orchestration required, no Pipeline component required, just a BRE call with the Routing information.


  • Pub / Sub (Port 2 Orchestration)


Let’s say we have a backend System which needs to be called. By managing this in the Itinerary, we don’t have to manage the subscription based on the MessageType of the Orchestrations Receive port. This means we don’t have the MessageType Dependency and could work with Xml.

Advantage: Process is more flexible, the Orchestration is not depending on a Schema…thus deployment is easier.

Advantage: We can change our process by adding another Orchestration Itinerary Service, this allows us to manage our process dynamically. The orchestration has become an abstract ESB Service component which we call. So instead of defining our process in the Orchestration, we isolate the functionality and link them using the Itinerary. If the process flow changes, we don’t need to redeploy our processed, only our Itinerary, which is an Xml file without any dependencies.

  • Pub / sub (Orchestration to MessageBox)


By starting the process from an Orchestration, we can dynamically chain processes together, start Itineraries based on preconditions.

Advantage: This allows for more flexibility as we can control the process flow, for example based on backend result. In case of a backend result, we can determine that we need to start a human workflow (ItineraryA), or are able to automate the process (ItineraryB).

  • Repair and Resubmit from the ESB Management Portal Sample


This shows an example of the default Error handling capabilities, which can already be included in your custom portal

  • Resubmit from ItineraryStep X


This shows how to extend the (your)Portal so that restarting processes can be even more sophisticated.

ESB Toolkit -  top reasons for using it

I’ve tried to discuss with Tomasso Groenendijk, on what the most important features are provided by the ESB Toolkit, and discuss scenarios for which BizTalk is limited and the ESB Toolkit adds functionality. This is a quick list, and partly explained by the aforementioned examples and also some use cases mentioned below the list;

  • Components are developed to be made generic, as opposed to just for 1 process
  • Pipelines are developed to be made generic, as opposed to just for 1 process
  • ESB Toolkit allows for a higher performance in low latency because it is easier to use pipeline components
  • BAM support is out of the box (although limited)
  • Generic Error handling
  • Runtime flexibility by using business rules (which can be changed without impact on the BizTalk deployement)
  • Deployment, Orchestrations are untyped (not dependending on maps/schemas), schemas and maps can be redeployed without affecting the orchestration

Example: Flexibility
[Add generic <functionality> in each process]
BizTalk only: create a pipeline component, add this in all pipelines in all ports (redeploy interfaces)
+ESB Toolkit: add a generic ItineraryService / update Itineraries (no redeploy of interfaces required)

Example: [A new version of the map is available]
BizTalk only: redeploy the interfaces (due to dependencies)
+ESB Toolkit: only update the map + itinerary (no redeploy of interfaces)

Example: [Determine the process flow at runtime
BizTalk only: Orchestrations are fixed an not flexible
+ESB Toolkit: Itinerary can be started based on message properties, rules, and can contain dynamic mappings

Example: [Restart a process]
BizTalk only: Not possible, messages must be stored somewhere, custom plumbing is required to resubmit messages
+ESB Toolkit: Message goes to the ESB Db, message can be resubmitted (even at the point of failure)


Before using the toolkit, I would like to summarize with the following considerations;

1) The toolkit can be quite complex, there is a learning curve involved
2) There are some features not available out of the box, for this custom plumbing needs to be done

3) ESB Portal
Exception handling is sophisticated, however the ESB Portal is a Sample, therefore, the Toolkit is
most useful when you integrate the error handling within your own Dashboard.

The portal is a sample application and is out of the box not production ready, and requires customizations.

Some common changes required:

  • Cleanup db jobs (e.g. archiving)
  • Extending the stored procedures so that additional context / fault information can be stored
  • Change the Failed messages model, so that there is a notion of State (Exception can be ignored, etc)
  • UI modifications to make the portal more flexible, do not use webservices for performance with large data sets, add features such as bulk resubmit or ignore

Known challenges

Lastly, some challenges, already identified and solved, and a list of great resources;


Tomasso Groenendijk will present a combination of these posts and topics related to this on the next BTUG on the 12th of September.

I suspect, that he will also demonstrate fascinating features, and his tools to test and use maps!


Kind regards,


No comments: