This approach will help to change the traditional approach of point-to-point communication in Manufacturing Execution Systems (MES) to using BizTalk server as a middleware to Integrate several systems

1. Table of Contents
1.
Asset Bounds ...................................................................................................................................... 3
2.
Asset Introduction ............................................................................................................................ 3
3.
Advantages of BizTalk ........................................................................................................................... 4
4.
Integration Process ............................................................................................................................... 5
4.1
File or Web Service ....................................................................................................................... 6
5.
Adapter and Integration Solutions........................................................................................................ 7
6.
BIZTALK Best Practices .......................................................................................................................... 8
a.
Design Best Practices ........................................................................................................................ 8
b.
Schema Creation Best Practices........................................................................................................ 9
c.
Map Creation Best Practices ........................................................................................................... 10
d.
Orchestration Best Practices ........................................................................................................... 11
e.
Best Practices for Performance ...................................................................................................... 15
1

2. Integration Reusable Assets
- Systems Integration
Approach for MES
Date
27/1/2014
Owner
Vinod kumar kodatham
Description
This approach will help to change the traditional approach of
point-to-point communication inManufacturing Execution
Systems (MES) to using BizTalk server as a middleware to
Integrate several systems. Multiple channels of several systems
can be plugged into the centralized BizTalk server.Enterprise
Service Bus (BizTalk server)with its inbuilt BAM and ESB
capabilities will help especially for reporting Application tracking
and application failures in respective BAM and ESB portals
respectively. These are totally reusable and isolated which can be
used by any application across the industry which uses BizTalk as
the middle ware platform
OBJECT OVERVIEW
Technical Name
Description
Integration Approach for MES.
The asset will provide high level design considerations which
enable seam less integration cross platform and systemsusing
BizTalk framework which developers can modify as per the
requirement.
2

3. 1. Asset Bounds
This asset can be used for the several simple to complex Integrationscenarios that needs to scale
from low to high volumes with low latency and need of fault tolerance that has got redundancy,
scalable and message retry featuresand governance over the solution.The conceptual architecture
of a solution is done by even taking into account non-functional requirements like performance,
availability, extensibility, resilience, robustness, and scalability and ensure they will be properly
addressed during the technical design phase. This gives you out of the box features to
develop/customize all these functionalities. The asset is independent of Source and Target
System(s)/ Application.
2. Asset Introduction
This document describes the idea of systems Integration using a centralized BizTalk
application i.e. using BizTalk framework as an Integration platform for integrating systems
(process in MES System) like POM, RIM, IOM, RCM, QOM, SAP and PCS (Process
Control Systems). BizTalk framework orchestrates the integration process by maintaining the
performance, robustness and availability.This framework will describe the design decision
made and development task required to build the BizTalk application’s components.
Any Business scenario which operates with integration has the above typical interface.
1. Business has a need to transfer information from one application to a different
application or one system to a different system that are residing in a single or
different multiple servers. These applications can be either in a same server or
a different servers
2. These applications can be either an application server which uses Webservices
or a database server. Now this source data needs to be migrated to the target
system meeting the business requirements which is called as transformations.
3. In order to integrate these two different systems so that they can communicate
each other business requires a Middleware i.e. BizTalk server.
There are lots of Middleware technologies available in market but if you need to go for
the best then we say its BizTalk server as it is one of the most successful product in the
market. It’s a Microsoft product. The benefits of this tool are mention in the very next
section.
3

4. 3. Advantages of BizTalk
Figure 2: Orchestrated Data transfer Process
ESB Portal(6)
Monthly
basis(2)
BizTalk(4)
Receive Port
Exception Handling(5)
Orchestration
Process
WCF-Custom
Source
Webservice(1)
Send Port
WCF
Custom
Target Webservice(7)
e
Retrieve Source
Data(3)
Receive
Pipeline
Send
Pipeline
Message Box
While planning for the integration scenario BizTalk takes care about tasks like Gathering
information, Defining naming conventions, planning team development, setting up and working
with source control.
Advantages of using BizTalk are.
1. Recoverable Interchange. In BizTalk, an Interchange can contain two or more
messages, such as a batch. Recoverable Interchange was introduced in BizTalk Server
2006, only messages will be suspended if they fail validation, and when corrected
theycan be resumed after the error.
2. Failed message routing. This functionality is available to enable failed messages to be
subscribed by orchestration and send ports. When used appropriately, failed message
routing will be used to notify users about the failed messages or to handle error
properly and repairing the messages if required.
3. Lowest total cost of ownership (TCO). BizTalk Server reduces the cost and
complexity of managing business processes and its automation with a single, unified
solution for Enterprise Application Integration (EAI), Business-to-Business
integration (B2Bi), and Business Process Management (BPM).
4. BizTalk Integrated management and development tools will enhance productivity.
5. Using TAPI it also supports Computer Telephony Interface (CTI).
6. Powerful, familiar tools to design, develop, deploy, will enable to manage those
processes easily
4

5. 4. Integration Process
Figure 3: A typical Integration Scenario
Source(ERP,LOB,Thir
d Party Vendor etc.,)
Web Method1
Target(ERP,LOB,Third
Party Vendor etc.,)
Integration Layer
1
Web Method2
On a scheduled
basis(Daily/
Monthly etc.,)
2
Web Method1
Web Method2
Web Services
2
Web Services
2. Retrieve Data
from source
Source XML
Data Stream
1
3. Map source structure
to target structure
Send data to Target
Target XML
Data Stream
In this scenario, information will betransferred from multiple source systems like Enterprise
Resource Planning (ERP), Line of Business (LOB) systems or any other third party vendor
systemsto Target system (ERP, LOB or a third party vendor). Messages inside BizTalk are
through the XML documents and defined with the XML schemas in XSD standard. Maps are
implemented with the XSLT standard. Orchestrations or mappings are implemented to meet the
business needs while transferring data from source to target. Schemas, maps, pipelines and
orchestrations are created visually using graphical tools within Microsoft Visual Studio. The
additional functionality can be delivered by .NET assemblies that can be called from existing
modules—including, for instance, orchestrations, maps, pipelines, business rules.
BizTalk allows creating 'send' and 'receive' port through which external applications can
exchange messages .The ports can be configured to use a set of transport protocols and
application adapters that are shipped with BizTalk Server. With these adapters, BizTalk can
exchange messages over multiple channels such as files, MSMQ, HTTP (Hypertext Transfer
Protocol), SOAP, IBM MQ, etc. Application adapters, which can directly interact with packaged
applications like SAP, Oracle Apps, etc., are also available.
5

6. 4.1
File or Web Service
Data is retrieved from a third party web service or a file based where the business rules are
applied in and will transform the data to the targetsystem. There are differentways to expose and
consume Webservices from BizTalk Server.
BizTalk can understand both positional/delimited and XML message formats. Biztalk can
transform messages from one format to another.
Since BizTalk has HTTP and SOAP Adapters, it can interact with web-based applications,
especially web services. The BizTalk Server interacts with services in the following two ways:
1. BizTalk process orchestrations can be exposed as Web services. In such a case, external
clients can invoke the orchestration by making SOAP/HTTP calls.
2. BizTalk can consume Web services by using the SOAP Adapter.
6

7. 5. Adapter and Integration Solutions
Adapter for BizTalk Server seamlessly integrates BizTalk with enterprise databases,
applications, transactions, business packages, EDI, and more.
Adapters allowconnecting Microsoft BizTalk server to Enterprise Information Systems,
including homegrown applications, databases like DB2, legacy data stores, transaction systems,
e-business formats such as EDI, HIPAA and HL7, and packaged applications.
The Adapter Offering for BizTalk Server:
1. Provides easy administration and configuration through existing BizTalk Server
Administration Console
2. Complete, Proven, High-Performance Adapter Set
3. Relational and legacy database systems
4. Transaction processors
5. Custom applications
6. Object models like COM objects, Enterprise JavaBeans, and others
7. Messaging environments like IBM MQSeries,Oracle AQ,TIBCO EMS Q
8. Terminal-based systems like mainframes and AS/400s
9. Packaged applications such as Enterprise Resource Planning (ERP) systems, Supply
Chain Management (SCM) systems, and Customer Relationship Management (CRM)
systems
10. Electronic Data Interchange (EDI) systems
11. e-Business XML formats such as SWIFT, FIX, HIPAA, ebXML, BOD, and cXML
7

8. Each adapter masks the complexities of its underlying source, so developers can use industrystandard XML, SQL, or Stored Procedures to work with all types of information. Also, each
adapter is highly optimized for its target, so developers don't require specialized knowledge to
take full advantage of special features and syntax for unusual data structures, applications, and
transactions. All adapters provide standard metadata about available functions and fields, return
values, error codes, and more.
6. BIZTALK Best Practices
Some of the BizTalk design best practices to be followed are.
a. Design Best Practices
Category
Technique
Design
Group the projects as per
artifacts type.
Design
Promote only those properties
that are required for routing or
tracking. Use distinguished
property for expression shape
usage.
Reference
It is a best to group the
projects as per the kind of
artifacts it contains. Schemas
are broken into Internal, plus
either External, Import and
Export, or even separated by
adapter: SQL, Oracle,
etc. Orchestrations or
Processes should be in one
project. Avoid using
Transforms in Orchestrations.
Put maps in the Ports.
Pipelines, if you need them, in
a separate project.
Maps or Transforms in a
project.
Performance
More processing required to
find the promoted properties
from the message. These
properties are loaded in the
memory, hence having a lot of
them can put the system under
extreme loads. Also, they are
8

9. written to the database for
tracking taking up disk space
and more CPU cycles.
b. Schema Creation Best Practices
Category
Technique
Reference
Schemas
Always Use Separate Internal
and External Schemas
Always transform messages
into canonical schema—
regardless of how simple the
schema is or who the source
is. This can reduce the number
of maps you need. Suppose
you need to map three types of
inbound messages to three
types of outbound messages
(perhaps not today, but maybe
in the future). Applying this
technique, you create a map to
your canonical schema for
each inbound schema and then
a map from your canonical
schema to each outbound
schema. That’s only six maps
to build and maintain, not
nine.
Schemas
Always validate xml message
against the standard schema.
N/A
9

10. c. Map Creation Best Practices
Category
Technique
Reference
Transformation
Use Transformation on
Receive and Send Ports
when you have multiple
partners sending multiple
message formats and you
need to convert these
messages to a canonical
format.
Business reason. Embed the map
in the schedule and the partner
changes the format, not only do
you have to rebuild the map, you
have to rebuild the schedule to
use the new version of the map.
Transformation
Use transformation in your Performance.
orchestration schedule
When you need to generate
a new message in the
schedule and use the
modified (mapped)
contents of an existing
message as the base. When
you want to map multiple
parts of a message into one
outbound message (this
type of mapping cannot be
done in receive / send
port). There are
performance gains which
come from doing
mappings in receive ports
sometimes, but they are
mostly around how many
persisted messages your
scenario generates and it is
a bit complicated to
explain.
Transformation
Pure XSLT is much faster
than XSLT which uses
inline script or referenced
assemblies.
Performance
Use xslt as much as possible in
mapping.
10

11. Transformation
Use external XSLT or
serializable classes in case
of complex maps with
1000+ connections.
Maintainability.
Transformation
Do not use .NET objects
directly inside the map.
Use external assemblies
instead.
Maintainability and Performance.
Transformation
To minimize memory
consumption, we
recommend that you put
any map that uses these
functoids into a small
assembly. Then, reference
that assembly.
Performance.
The
System.Policy.Security.Evidence
object is often used in transforms
and can consume a lot of
memory. Whenever a map
contains a scripting functoid that
uses inline C# (or any other inline
language), the assembly is created
in memory. The
System.Policy.Security.Evidence
object uses the object of the actual
calling assembly. This situation
creates a rooted object that is not
deleted until the BizTalk service
is restarted.
d. Orchestration Best Practices
Category
Orchestration
Technique
Eliminate orchestrations for
messaging only patterns
Reference
When possible minimize the
use of orchestrations to
increase overall throughput
and reduce the latency of
11

12. business processes. If there is
no need to run long running
transactions and there is no
need to invoke several systems
for each request, then consider
eliminating orchestrations and
moving business logic to
Receive and Send Ports to
reduce the total amount of
roundtrips to the
BizTalkMsgBoxDb and
decrease the latency due to
database access. In this case,
implement custom pipelines
and reuse helper classes that
were previously invoked from
orchestrations. Use
orchestrations only when
strictly needed to implement
design patterns as Scatter and
Gather or Convoys.
Orchestration
Always Use Multi-Part
Message Types
If your logical port and
Receive/Send shape are using
your multi-part message type,
and you can easily change the
underlying schema in the
future without having to
disconnect the port from the
receive shape.
Orchestration
Optimize orchestration latency Best practice
by reducing the number of
persistence points.
Orchestration
Minimize orchestration
complexity to improve
performance
Best practice
Orchestration
Use the Call Orchestration
shape versus the Start
Orchestration shape to
improve performance
Best practice
Orchestration
Use XmlReader with
XLANGMessage versus using
Performance
12

13. XmlReader with
XmlDocument to improve
orchestration performance
Orchestration
minimizing the use of logical
ports bound to physical ports
Performance
Orchestration
to extract or set properties
used with business logic in an
orchestration
If you are using a map to
extract or set properties used
with business logic in an
orchestration, use
distinguished fields or
promoted properties. This
practice should be followed
because when extracting or
setting values with a map the
document is loaded into
memory however when using
distinguished fields or
promoted properties, the
orchestration engine accesses
the message context and does
not load the document into
memory.
Orchestration
aggregate several fields into
one field
If you are using a map to
aggregate several fields into
one field, use distinguished
fields or promoted properties
with an orchestration variable
to accumulate the result set.
Orchestration
Use Orchestration Design
Patterns when required
Performance and
maintainability
Orchestration
Make appropriate use of .NET
classes in orchestrations to
maximize performance
Performance and
maintainability
Orchestration
Orchestration Exception
Handler Blocks
When using Orchestration
exception Handler blocks,
include all orchestration
shapes in one or multiple
scopes (non transactional
scopes whenever possible) and
create at least the following
exception handler blocks:
13

14. An exception handler block
for handling a generic
System.Exception error.
An exception handler block
for handling a general
exception in order to catch and
handle possible unmanaged
errors, such as COM
exceptions.
Orchestration
Always Try to Design
Orchestrations with DirectBound Ports
You’ve got three DirectBound port types: Message
Box Routing, Self-Correlating,
and Orchestration-toOrchestration (also called
Partner Ports). Direct-Bound
ports are self-configured,
without sacrificing flexibility
or performance. A handy
benefit of Direct-Bound ports
is that neither the developer
nor the administrator needs to
create corresponding physical
ports in BizTalk Explorer or
BizTalk Administration
Console. So Direct-Bound
ports yield orchestrations that
are more self-contained, and
therefore more reusable and
easier to redeploy
independently.
Orchestration
Publish Your External
Schema, Not Your
Orchestration
It provides loosely coupled
mechanism for exposing the
services. This buys you more
freedom to change your
orchestration without breaking
14

15. the caller
e. Best Practices for Performance
Category
Technique
Performance
Optimize the BizTalk
Registry for Web Services.
Reference
If your BizTalk orchestration is
published as a Web service, you’ll
definitely want to tweak some of the
default ASP.NET parameters used by
BizTalk. Below are the recommended
settings.
Windows Registry Editor version 5.00
[HKEY_LOCAL_MACHINESYSTE
MCurrentControlSetServicesBTSSvc$
BTSHOSTCLR Hosting]
“MaxIOThreads”=dword:00000064
“MaxWorkerThreads”=dword:00000064
“MinIOThreads”=dword:00000019
“MinWorkerThreads”=dword:00000019
Please refer
http://msdn2.microsoft.com/enus/library/aa561380.aspx for more
details.
Performance
Configure Parameters Used Please refer below link for more
for Low Latency
information.
Performance Tuning
http://msdn.microsoft.com/enus/library/aa475435.aspx
15

16. Performance
High Availability
Create separate hosts for
orchestration, Receive and
Send Ports
Creating separate hosts allows BizTalk
to implement throttle mechanism and
help it to load balance the incoming
messages.
Cluster the resources for
high availability.
Clustering of MSDTC, SSO, Disk and
BizTalk for High availability.
http://www.microsoft.com/downloads/d
etails.aspx?displaylang=en&FamilyID=
eb437722-2828-4cbb-84c317556b4df26b
High Availability
Use database mirroring for
BizTalk Server databases
high availability.
Database mirroring of BizTalk databases
helps to achieve the High Availability
for Database servers and helps to
recover from Failover scenarios.
Appendix
Reference used for Best Practices:
http://social.technet.microsoft.com/wiki/contents/articles/5210.biztalk-server-2010-orchestration-bestpractices.aspx
16