WebSphere Connectivity & Integration: What's New in the Messaging Family?

Hursley Comes To The Nordics – May 2011

WebSphere Connectivity & Integration:
What's New in the Messaging Family?

Andy Piper
WebSphere Messaging Community Lead

WebSphere

© 2011 IBM Corporation

Hursley Comes to the Nordics – May 2011

Coming up in the next hour...

Universal Messaging

WebSphere MQ 7.0.1.x

Files, Security, Telemetry... oh my!

What's new in the community?

2 © 2011 IBM Corporation


The vision: Universal Messaging

Any skills Any platform
Any QoS

Any network Any speed

Any data Any delivery level
Any device

Universal Messaging

3 25/05/11 © 2011 IBM Corporation


Universal Messaging
Dynamic network that delivers the data you require from wherever it resides to wherever you
want it in whatever way you want it at whatever time you want it

Universal Messaging

1. Best Delivery
• Choice of service
• Resilience, Integrity, Security
• Throughput, Latency
• High availability

2. Anything Anywhere
• Any skills
• Any traffic
• Any language
• Any environment
• Any platform

3. Scale Dynamically
• Start small
• Grow incrementally
• Stretch elastically
• Scale admin



Messaging and a Smarter Planet
Connectivity for Applications Head Offices,
Data centres
Transport WebSphere MQ
REST/HTTP Intelligent Cloud Computing
WS*
Mission-Critical Connectivity & Intelligence
Messages WebSphere MQ

Regional Offices Files WebSphere MQ File Transfer Edition
Stores, Outlets
Stores, Outlets Transform, WebSphere Message Broker
Interconnected Enrich, Mediate WebSphere ESB
DataPower
WebSphere Sensor Events

Catalog WebSphere Service Registry & Repository

Intelligence WebSphere Business Events
Cognos. ILOG, SPSS
Instrumented InfoSphere Streams
Other InfoSphere and Tivoli products

Connectivity for Smart Devices Remote Systems and Devices
Systems and Devices

Transport MQ Telemetry
HTTP
Multicast

Embedded Controllers Sensors Actuators
Filtering of duplicate read events, Store- Power meters, weather data Tag printers, status lights, Load
based HVAC & lighting controls, Industrial SCADA sensors, pressure, volume, generation, HVAC and lighting,
Network Gateways (SCADA) RFID readers, Motion detectors… Valves, switches and pumps…



WebSphere MQ Delivery Roadmap
(2011)
MQ FTE 7.0.4
(11/10)
MQ FTE V7.0.3
(11/09) (10/10)
MQ FTE V7.0.2 MQ Advanced
(06/09) (08/10) Message
MQ FTE V7.0.1 Security
MQ v7.0.1.3
(08/10)
(12/08) MQ Telemetry
MQ File Transfer Edition V7 (1H10)
(08/09) SupportPacs etc
MQ V7.0.1 with
(01/09) Multi-Instance Queue Managers
MQ V7.0.0.1 with Automatic Client Reconnect
Service Def Wizard Enhanced .NET support
(05/08) XA-aware API Exit Increased z/OS Capacity
MQ V7 with z/OS Group Units of Work
Integrated Pub/Sub
Rearchitected JMS
Extended APIs V7.0.1 EAP Early Access Programs
06/08 12/08 06/09 12/09 06/10 12/10



IBM WebSphere MQ V7.0.0 - Summary
 Enhanced JMS
–Underpins many SOA/ESB solutions needing access to messaging
–Improved performance & ease-of-use
 Enhanced Publish-and-subscribe
–Ease-of-use
–New support for z/OS
 Extended verbs and behaviors for MQI programming interface
 Enhanced MQ clients for increased throughput resilience and availability

 Web 2.0 support to help create richer user experience



WebSphere MQ
V7.0.1



WebSphere MQ V7.0.1 – Major Themes

 Enhanced availability options on Distributed platforms

 Constraint relief on z/OS

 IBM portfolio exploitation, and extension of reach, for V7 features

 Ongoing performance, consumability and serviceability enhancements

 Keeping pace with industry evolution in areas such as platforms and SSL



Distributed Platforms: Multi-instance Queue Managers
 Basic failover support without HA coordinator
– Faster takeover: fewer moving parts
– Cheaper: no specialised software or administration skills needed
– All non-z/OS platforms

 Queue manager data is held in networked storage
– NAS, NFS, GPFS etc so more than one machine sees the queue manager data
– Improves storage management options: formal support for these even without failover
config
 Multiple (2) instances of a queue manager on different machines
– One is “active” instance; other is “standby” instance
– Active instance “owns” the queue manager’s files and will accept app connections
– Standby instance does not “own” the queue manager’s files and apps cannot connect
• If active instance fails, standby performs queue manager restart and becomes active

 Instances share data, so it’s the SAME queue manager



Multi-instance Queue Managers

1. Normal MQ MQ
Execution Client Client

network

192.168.0.1 192.168.0.2

Machine A Machine B
QM1 QM1
Active can fail-over Standby
instance instance

QM1

networked storage
Owns the queue manager data




2. Disaster MQ MQ
Strikes Client Client

network
Connections
broken from
clients

192.168.0.1 192.168.0.2

Machine A Machine B
QM1 QM1
Active locks freed Standby
instance instance

QM1

networked storage




3. Standby MQ MQ
Connections
Comes to Life Client Client still broken

network

192.168.0.2

Machine B
QM1
Active
instance

QM1

networked storage




4. Recovery MQ MQ
Complete Client Client Clients
reconnected.
Processing
continues.
network

192.168.0.2

Machine B
QM1
Active
instance

QM1

networked storage



Multi-instance queue managers: Details
 MQ is NOT becoming an HA coordinator
– Generally, if other resources also required, use an HA coordinator such as HACMP
– Service objects can restart applications with qmgr but limited control
– Message Broker also integrates with and exploits this MQ function
 The IP address is not taken over
– Channel config needs all possible addresses unless you use external IPAT or
intelligent router
– CONNAME(‘host1(port1),host2(port2)’) syntax extension on all platforms including
z/OS
 Support for networked storage over modern network file system protocols
– For example, NFS v4 (not v3)
– Tool shipped to validate configuration

 New options for crtmqm/strmqm/endmqm to control operations
– Cannot guarantee which instance becomes the primary
 Removes need for MC91 (withdrawn)



Automatic Client Reconnection
 Client library provides necessary reconnection logic on detection of a failure
 Hides failure from application code

QM1
QM1

Application
Application
QM2
QM2
MQ Client
MQ Client

QM3
QM3



Automatic Client Reconnection
 Tries to hide queue manager failures by restoring current state automatically
– Re-opens queues and other qmgr objects, re-establishes subscriptions
– For example, if MQPUT returns error, client reruns MQCONN/MQOPEN/MQPUT internally
– Not all MQI is seamless, but majority repaired transparently

 Uses the list of addresses in CONNAME to find queue manager
– MQSERVER environment variable also understands list
– MQSERVER=SYSTEM.DEF.SVRCONN/TCP/host1(1414),host2(1414)

 Can reconnect to the same or different Queue Manager

 Reconnection interval is backed off exponentially on each unsuccessful retry
– Total timeout is configurable – default 30 minutes.

 Enabled in application code or ini file
– Event Handler callback shows reconnection is happening if app cares



Windows Communication Foundation

 Windows Communication Foundation (WCF)
– Underpins .NET Web Services and Messaging
– Built-in Transports – e.g. MSMQ, HTTP(S), Named Pipes, TCP/IP, etc.
– Transports can be extended with “custom channels”
– Allows alternative transports (like MQ) to be slotted into WCF seamlessly

 WCF support now included in product
– Previously an Alphaworks prototype

 Built on XMS .Net classes
– Now shipped as part of the product instead of SupportPac
– Internally, it exploits V7 API, in same way as JMS implementation



Security & Monitoring

 Command and Configuration Events for Distributed platforms
– Matching already-available z/OS function
– Successful commands – MQSC or PCF – recorded as event messages
– Configuration changes report “before” and “after” definitions of objects
– Provides a record of who changed what and when
display qmgr event
AMQ8408: Display Queue Manager details.
QMNAME(V7) CMDEV(ENABLED) CONFIGEV(ENABLED) …

 SSL OCSP Support
– Now commonly used as alternative to LDAP-based CRLs
– Simpler to manage as no need to have an LDAP server
– Can use details provided in inbound certificate
– Also can be configured within queue manager or by application code



z/OS Constraint Relief
 In V7.0, the queue manager started to exploit 64-bit addressing
– New Pub/Sub features

 In V7.0.1 more Queue Manager storage moves to 64-bit
– 64-bit Queue Indices
– 64-bit Lock Manager

 Can have more open queues, more messages on indexed queues etc

 Small message storage changed
– One message per page instead of fitting multiples into single page
– May increase DASD use, especially if messages not quickly retrieved
– But typical workloads show improved performance and reduced CPU



z/OS Log Compression
 Can increase the throughput possible for persistent messages

 May reduce the size of your logs
– Dependent on your message content
– Useful if you are DASD constrained.

 RLE (run-length encoding) of “insert” log records for private queue messages
– Will not compress shared queue log records
– SMF 115 records updated to show compression rates achieved etc
 Controlled via zPARM option at queue manager level.
– COMPLOG(NONE) or COMPLOG(RLE) in CSQ6LOGP
– Can also be viewed/controlled via DISPLAY LOG / SET LOG



WebSphere MQ
SupportPacs



Most recent SupportPacs

 MA98: WebSphere MQ - Connection Endpoint Look-up Exit
– Find connection information from an LDAP server

 MH05: WebSphere MQ - Events Display Tool
– Reads and displays events from SYSTEM.ADMIN.*.EVENT queues
– Can be triggered from WMQ
– Can be written to a file

 MO06: WebSphere MQ – Message Log Summary Utility
– Summarise and filter Qmgr Message Logs

See http://pipr.co/supportpac for the full list of recent
SupportPacs
23 CSS: S D © 2011 IBM Corporation


WebSphere MQ
Low Latency
Messaging



WebSphere MQ Low Latency Messaging

 Initially built for Financial Markets customers requiring high performance,
reliable messaging in the front office for market data and trade data
 Included with WebSphere Front Office for Financial Markets
 Integrated with Partners such as Data Vendors/ISVs for internal application
communication and external application connectivity
 Production deployments in market data distribution systems, internal trade
execution systems and exchange execution systems
 Also applicable in industries with similar quality of service requirements
– Defense and Aerospace
– Telecommunications
– Logistics



Positioning of IBM Messaging Transports

WebSphere MQ Low Latency
WebSphere MQ
Messaging
Speed – Quality of Time Independent Ultra-Fast
Service “As-Soon-As-Possible” “Microseconds”

Reliability – Quality of Either non-persistent or persistent Ranges from unreliable to assured
Service “Ultra-Reliable, Bet-The-Business” “Variable, based on class of data”

Target Market Enterprise Messaging Low Latency Messaging

Financial Markets and several others with similar
Target Industries All
QoS needs

Queuing Yes No

Standardized Unique / specific
APIs (JMS, XMS and MQI) (C, Java, .NET)

Linux and Windows on x86,
Platform coverage 80 platforms Linux on pSeries & zSeries, Solaris
(x86,SPARC), HP-UX, AIX

Included in
WAS, WMB, WESB, DataPower XI50, DB2,
Interoperates with CICS, IMS, etc, etc…
WebSphere Front Office
Integrated with DataPower



Software Overview

Peer-to-peer
 Fully distributed
 No centralized management
– No centralized queue for messages
– Daemon-less

Publish-Subscribe
 Transmitter publishes on a topic (RMM) or “queue” (RUM)
 Receiver subscribes to a topic or “queue”
 Topic usually identified by a unique name.
– Can have multiple transmitters/receivers for the same Topic

Transports
–Ethernet 1GigE and 10 GigE
–Infiniband – Native verbs or IB over IP
–Shared Memory



WebSphere MQ
File Transfer
Edition



Introducing WMQ File Transfer Edition
 Adds file transfer services to WebSphere MQ to enable managed file movement
 Multi-purpose solution combining file transfer and messaging on a single backbone

 Reliable – leveraging the WebSphere MQ
transport automate
 Multi-purpose – transfers both messages and track audit
configure
files
 Auditable – logging subsystem tracks transfer at
source and destination for audit purposes
 Centralized – monitoring, control and
configuration
 No need to program – no MQ skills required
 Graphical tooling – visual configuration and
status
 Moves Massive files – no size limit, Kb, Mb, Gb…
 Command line interface – for advanced users
WebSphere MQ File Transfer Edition
 Scripting support – enables automation via
scripting of complex multi-step transfers in XML
using Apache Ant
 Flexible backbone – moves files from anywhere a b c x y z
to everywhere in network
 Integration with MQ-enabled apps and ESBs
 Automatic file character conversion
 Security of file payload using SSL
 Support a wide range of Platforms and O/S


WMQ File Transfer Edition 7.0.3 & 7.0.4

 Increased web support
–REST interfaces
–Web transfers from end-users

 Extended platform support: AIX on Power7, Linux on Power7, Oracle
Solaris 10 on Intel x86 and virtualized platforms
 WebSphere Message Broker integration
–FTEInput and FTEOutput nodes in WMB FP2
 WebSphere Advanced Message Security integration
–Easily encrypt file transfers

 Sterling Connect:Direct Bridge capability including audit trail
 7.0.4 GA June 3 2011


WebSphere MQ
Advanced
Message Security



WebSphere MQ Advanced Message Security
 End-to-End Message Security - Secures application data even
before it is passed to MQ
 Extension to base MQ – No changes to existing applications

WebSphere MQ standard security:
Message data can be encrypted in transport
(SSL) but not when it resides in the queues
Authentication is based on Operating System
identifier of local process
Application A Application Z

Supplements WMQs security features:
+ Assurance that messages have not been WebSphere MQ Advanced Message Security
altered in transit WebSphere MQ
+ Assurance that messages originated from
the expected source
+ Assurance that messages can only be
viewed by intended recipient(s)
+ Administered using queue based policies Securing the data and
created from the WMQ Explorer or the applications
command line tooling.



Advanced Message Security – Key Elements
 Message Interceptors
 Protection policy for the application queue
 Public / private key pairs
 Recipient’s public key

Sending Application (Alice) Receiving Application (Bob)
AMS_QM

APP.Q

Keystore Keystore
Alice Priv Bob Priv
Alice Pub Bob Pub
Bob Pub
Policy
APP.Q
Privacy
Recipient : Bob



 Adds message-level security to existing WebSphere MQ (V6 & V7)

 Support details:
–C MQI and Java clients
–Standard WMQ distributed platforms (apart from iSeries), including zOS
–Requires a minimum of WMQ v6.0.2.8 or WMQ v7.0.1.2

 Constraints
–Point to point messaging domain only
–On zOS MQI v6 verbs only (no async consume, message properties etc)



WebSphere MQ
Telemetry



The Internet of Things

Many smart devices
instrument our world
today

Interconnecting these
smart devices creates a
Central Nervous
System



Universal Messaging for a Smarter Planet
Enterprise
Regional Office

Offer connectivity capabilities
optimised for sensors and devices Branch
Outlet

Provide Enterprise Access to
new sources of data to support
your business
Provide access to Enterprise
data and services from new
platforms/businesses Sensor Retail
e.g. Store
Deliver data to intelligent RFID

decision-making assets
Zero-Administration for “remote”
deployments Pervasive
Device
Refinery

Petrol
Forecourt



Connectivity with MQTT

■ MQ Telemetry Transport (MQTT) is
designed with the following primary
intentions:
– Publish/subscribe messaging paradigm as
required by the majority of SCADA and
sensor applications.
– Minimise the on-the-wire footprint.
– Expect and cater for frequent network
disruption.
– Cope with slow and poor quality networks
– Expect that client applications may have
very limited processing resources available.
– Provide traditional messaging qualities of
service where the environment allows
– Publish the protocol for ease of adoption by
device vendors and third-party client
software.



Extended Reach
Basic Telemetry
WebSphere MQ Telemetry V7.0.1 extends IBM’s Client

industrial strength enterprise message system
Backend Server
• Highly scalable
– A single queue manager can handle 100,000 concurrently
connected devices
• Ships with two types of client: WebSphere
– Basic MQ
• direct connectivity from a device
– Advanced:
• Acts as a “concentrator/hub” for mini-networks of Telemetry
devices
• Can connect to multiple backend servers
• Can failover to alternate backend server
• Can buffer messages
MQ Telemetry
• Provides rich security Daemon for
Devices
– Network: SSL
– Authentication: JAAS
– Authorisation: OAM Advanced
Telemetry Client

• In addition, any third party, open source or “roll
your own” MQTT client can be used



WebSphere MQ Telemetry Implementation
 Fully integrated / interoperable with WMQ
– Publishers and subscribers can exchange messages with MQI and JMS applications
 Telemetry channels enable MQTT connections to queue manager
– Supports MQTTv3.1 protocol
 Administration using WMQ Explorer
 Initially available on Windows and Linux
 Ships with reference Java (for MIDP upwards) and C clients
 Includes WebSphere MQ Telemetry Daemon for Devices
– An advanced client which also accepts connections from other clients
– Provides ability to store and forward, network local devices.



Broader integration
Simple
Lightweight (CPU,Mem,**Net)
Data-centric
Distribution (pub/sub)
Range of QoS
=> developer/community interest!



Case Study: Smarter Healthcare

Medical organization created a remote pace-maker monitoring
solution to provide better patient care

Client Pains
 Physicians needed better monitoring
of cardiac patients
 Improve efficiency of checkups
 Meet healthcare data capture
standards

Enables higher level of patient care and peace of mind
Improves administrative efficiency and maintenance
Helps conform to standards and ease integration of data



Case Study: Improving Energy Usage

Utility company developing an Intelligent Utility Network offering for
optimizing load on electricity grids

Business Partner
 Needs robust middleware
technology to connect to remote
smart meters
 Needs to be able to rapidly scale
solution nationally

Able to offer daily energy savings of 15-20%
Enables utilities to reduce peaks and avoid punitive charges
Helps save electricity through better peak load management



WebSphere MQ
Community



We love our users!
 developerWorks
 MQSeries.net community
– http://mqseries.net/
 Global WebSphere Community GWC
– http://websphereusergroup.org

 MQTT – http://mqtt.org
 IBMer's Blog on Messaging on developerWorks
 Twitter – @IBM_WMQ and @IBM_Broker
 YouTube – http://youtube.com/mqontv

 Store and Forward – T.Rob's WMQ Security blog
– http://t-rob.net/

 Open, vibrant ecosystem
– e.g. PyMQI, mosquitto, github projects, etc. etc.
45

Thank you!

Contact: Andy Piper andy.piper@uk.ibm.com
http://twitter.com/andypiper | http://andypiper.co.uk

WebSphere


WebSphere Connectivity & Integration: What's New in the Messaging Family?

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (18)

Andere mochten auch

Andere mochten auch (11)

Ähnlich wie WebSphere Connectivity & Integration: What's New in the Messaging Family?

Ähnlich wie WebSphere Connectivity & Integration: What's New in the Messaging Family? (20)

Mehr von IBM Sverige

Mehr von IBM Sverige (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

WebSphere Connectivity & Integration: What's New in the Messaging Family?