ESB Series – Final thoughts

 

This is the last post in the ESB Series which i’ve been doing;

Hopefully this has showed some ways of using the ESB Toolkit, and ways of extending it so that it is more suited for various scenarios.

This post will close the gaps i left open in earlier posts. In the end, i have hopefully demonstrated some common scenarios which are possible with the ESB Toolkit. My final thought on this subject is that the ESB toolkit is targeted to work in a more abstract/generic way, which is a good thing. After using the ESB Toolkit I think it is a nice addition to BizTalk, but I only recommend to use it when the problem demands for this type of a solution. There is a learning curve involved, it works differently as classic BizTalk, and means that you have to define your solution differently, sometimes it even makes the components more complex. In the end, it will bring flexibility, maintainability and allows re-use, however this requires an investment.

The ESB Toolkit is not a golden hammer, don’t use it per definition, think about the problem, think of the possible solutions, think of the process, possible changes and decide which approach would be the best.

ESB Toolkit Itinerary scenarios

In the first post ‘Itinerary’ i’ve showed the difference between a Receive-Send port and the same set up using an Itinerary. Below, you can see the different scenarios possible, and the comparison with a BizTalk set up.

  • Pub / sub (Port 2 Port)

image

This Itinerary basically, receives a messages from the Receive Pipeline, performs Tracing, Routing and send a message to a Send port. The subscription is management by BizTalk, however, the Send port is configured Dynamically.

Advantage: No Orchestration required, no Pipeline component required, just a BRE call with the Routing information.

 

  • Pub / Sub (Port 2 Orchestration)

image

Let’s say we have a backend System which needs to be called. By managing this in the Itinerary, we don’t have to manage the subscription based on the MessageType of the Orchestrations Receive port. This means we don’t have the MessageType Dependency and could work with Xml.

Advantage: Process is more flexible, the Orchestration is not depending on a Schema…thus deployment is easier.

Advantage: We can change our process by adding another Orchestration Itinerary Service, this allows us to manage our process dynamically. The orchestration has become an abstract ESB Service component which we call. So instead of defining our process in the Orchestration, we isolate the functionality and link them using the Itinerary. If the process flow changes, we don’t need to redeploy our processed, only our Itinerary, which is an Xml file without any dependencies.

  • Pub / sub (Orchestration to MessageBox)

image

By starting the process from an Orchestration, we can dynamically chain processes together, start Itineraries based on preconditions.

Advantage: This allows for more flexibility as we can control the process flow, for example based on backend result. In case of a backend result, we can determine that we need to start a human workflow (ItineraryA), or are able to automate the process (ItineraryB).

  • Repair and Resubmit from the ESB Management Portal Sample

image

This shows an example of the default Error handling capabilities, which can already be included in your custom portal

  • Resubmit from ItineraryStep X

image

This shows how to extend the (your)Portal so that restarting processes can be even more sophisticated.

ESB Toolkit -  top reasons for using it

I’ve tried to discuss with Tomasso Groenendijk, on what the most important features are provided by the ESB Toolkit, and discuss scenarios for which BizTalk is limited and the ESB Toolkit adds functionality. This is a quick list, and partly explained by the aforementioned examples and also some use cases mentioned below the list;

  • Components are developed to be made generic, as opposed to just for 1 process
  • Pipelines are developed to be made generic, as opposed to just for 1 process
  • ESB Toolkit allows for a higher performance in low latency because it is easier to use pipeline components
  • BAM support is out of the box (although limited)
  • Generic Error handling
  • Runtime flexibility by using business rules (which can be changed without impact on the BizTalk deployement)
  • Deployment, Orchestrations are untyped (not dependending on maps/schemas), schemas and maps can be redeployed without affecting the orchestration

Example: Flexibility
[Add generic <functionality> in each process]
BizTalk only: create a pipeline component, add this in all pipelines in all ports (redeploy interfaces)
+ESB Toolkit: add a generic ItineraryService / update Itineraries (no redeploy of interfaces required)

Example: [A new version of the map is available]
BizTalk only: redeploy the interfaces (due to dependencies)
+ESB Toolkit: only update the map + itinerary (no redeploy of interfaces)

Example: [Determine the process flow at runtime
BizTalk only: Orchestrations are fixed an not flexible
+ESB Toolkit: Itinerary can be started based on message properties, rules, and can contain dynamic mappings

Example: [Restart a process]
BizTalk only: Not possible, messages must be stored somewhere, custom plumbing is required to resubmit messages
+ESB Toolkit: Message goes to the ESB Db, message can be resubmitted (even at the point of failure)

Considerations

Before using the toolkit, I would like to summarize with the following considerations;

1) The toolkit can be quite complex, there is a learning curve involved
2) There are some features not available out of the box, for this custom plumbing needs to be done

3) ESB Portal
Exception handling is sophisticated, however the ESB Portal is a Sample, therefore, the Toolkit is
most useful when you integrate the error handling within your own Dashboard.

The portal is a sample application and is out of the box not production ready, and requires customizations.

Some common changes required:

  • Cleanup db jobs (e.g. archiving)
  • Extending the stored procedures so that additional context / fault information can be stored
  • Change the Failed messages model, so that there is a notion of State (Exception can be ignored, etc)
  • UI modifications to make the portal more flexible, do not use webservices for performance with large data sets, add features such as bulk resubmit or ignore

Known challenges

Lastly, some challenges, already identified and solved, and a list of great resources;

 

Tomasso Groenendijk will present a combination of these posts and topics related to this on the next BTUG on the 12th of September.

I suspect, that he will also demonstrate fascinating features, and his tools to test and use maps!

 

Kind regards,

Sander

Comments

Post a Comment

Popular posts from this blog

Azure implementation guidelines

In My post ‘ Service Bus Management ’ I pointed out a way of implementing a DTAP strategy for managing the Service Bus environment. There’s now a great post available from Microsoft which goes into more detail about other aspects of managing the Azure subscriptions and other artefacts which is quite useful when setting up your Azure environments. Follow this link to the ‘Azure Implementation Guidelines’ HTH, Sander
Read more

Focus and innovation - recap of the last 2 years

It's been a long time since my last post. Hereby a partial explanation for that, with the note that this blog will remain active for the purpose of being a resource for historical references. My IT career started with programming, and adventures within holland, as well as overseas. When I realised programming is something you can specify, I figured that providing added value is crucial for a long term career and decided to focus on process centric roles. I started with webservice integration and eventually from that to process integration using BizTalk. The great feeling when discussing high level processes and implementing solutions within days, even highly scalable and interfacing with LOB systems is great. After several years of implementing solutions like these, I started questioning if all of this work was justified by the business case, if the solution was actually contributing, if the choices had been valid. In other words, I needed to understand the choices made. When...
Read more

BizTalk Pub/sub vs Topics based routing–discussion

My colleagues and I had a brief discussion about the subject ‘Topics’ and what it brings… Do you all agree/disagree with the following statement: topic-based pub-sub messaging doesn’t really exist in BizTalk “In my opinion; I disagree, as BizTalk offers the following 'out-of-the-box' • Message Routing (f.e build in context properties like message type, trading partner, operation etc) • Content Routing (f.e. promoted properties, message inspection in orchestrations/pipelines icw orchestration logic)” My colleague Rene Brauwers Here my 2 cents…. I agree that it exists...however the current offerings in BizTalk are tightly coupled, making the usage far from perfect. When doing content based routing on custom fields (promoted properties), you will be making the solution message-coupled; Your custom message property schema is deployed, and the message filter on these fields have a direct relationship with this property schema. Making any change to this schema will g...
Read more