Weitere ähnliche Inhalte Ähnlich wie WebSphere Connectivity & Integration: What's New in the Messaging Family? (20) Mehr von IBM Sverige (20) Kürzlich hochgeladen (20) WebSphere Connectivity & Integration: What's New in the Messaging Family?1. Hursley Comes To The Nordics – May 2011
WebSphere Connectivity & Integration:
What's New in the Messaging Family?
Andy Piper
WebSphere Messaging Community Lead
WebSphere
© 2011 IBM Corporation
2. Hursley Comes to the Nordics – May 2011
Coming up in the next hour...
Universal Messaging
WebSphere MQ 7.0.1.x
Files, Security, Telemetry... oh my!
What's new in the community?
2 © 2011 IBM Corporation
3. Hursley Comes to the Nordics – May 2011
The vision: Universal Messaging
Any skills Any platform
Any QoS
Any network Any speed
Any data Any delivery level
Any device
Universal Messaging
3 25/05/11 © 2011 IBM Corporation
4. Hursley Comes to the Nordics – May 2011
Universal Messaging
Dynamic network that delivers the data you require from wherever it resides to wherever you
want it in whatever way you want it at whatever time you want it
Universal Messaging
1. Best Delivery
• Choice of service
• Resilience, Integrity, Security
• Throughput, Latency
• High availability
2. Anything Anywhere
• Any skills
• Any traffic
• Any language
• Any environment
• Any platform
3. Scale Dynamically
• Start small
• Grow incrementally
• Stretch elastically
• Scale admin
4 25/05/11 © 2011 IBM Corporation
5. Hursley Comes to the Nordics – May 2011
Messaging and a Smarter Planet
Connectivity for Applications Head Offices,
Data centres
Transport WebSphere MQ
REST/HTTP Intelligent Cloud Computing
WS*
Mission-Critical Connectivity & Intelligence
Messages WebSphere MQ
Regional Offices Files WebSphere MQ File Transfer Edition
Stores, Outlets
Stores, Outlets Transform, WebSphere Message Broker
Interconnected Enrich, Mediate WebSphere ESB
DataPower
WebSphere Sensor Events
Catalog WebSphere Service Registry & Repository
Intelligence WebSphere Business Events
Cognos. ILOG, SPSS
Instrumented InfoSphere Streams
Other InfoSphere and Tivoli products
Connectivity for Smart Devices Remote Systems and Devices
Systems and Devices
Transport MQ Telemetry
HTTP
Multicast
Embedded Controllers Sensors Actuators
Filtering of duplicate read events, Store- Power meters, weather data Tag printers, status lights, Load
based HVAC & lighting controls, Industrial SCADA sensors, pressure, volume, generation, HVAC and lighting,
Network Gateways (SCADA) RFID readers, Motion detectors… Valves, switches and pumps…
5 © 2011 IBM Corporation
6. Hursley Comes to the Nordics – May 2011
WebSphere MQ Delivery Roadmap
(2011)
MQ FTE 7.0.4
(11/10)
MQ FTE V7.0.3
(11/09) (10/10)
MQ FTE V7.0.2 MQ Advanced
(06/09) (08/10) Message
MQ FTE V7.0.1 Security
MQ v7.0.1.3
(08/10)
(12/08) MQ Telemetry
MQ File Transfer Edition V7 (1H10)
(08/09) SupportPacs etc
MQ V7.0.1 with
(01/09) Multi-Instance Queue Managers
MQ V7.0.0.1 with Automatic Client Reconnect
Service Def Wizard Enhanced .NET support
(05/08) XA-aware API Exit Increased z/OS Capacity
MQ V7 with z/OS Group Units of Work
Integrated Pub/Sub
Rearchitected JMS
Extended APIs V7.0.1 EAP Early Access Programs
06/08 12/08 06/09 12/09 06/10 12/10
6 © 2011 IBM Corporation
7. Hursley Comes to the Nordics – May 2011
IBM WebSphere MQ V7.0.0 - Summary
Enhanced JMS
–Underpins many SOA/ESB solutions needing access to messaging
–Improved performance & ease-of-use
Enhanced Publish-and-subscribe
–Ease-of-use
–New support for z/OS
Extended verbs and behaviors for MQI programming interface
Enhanced MQ clients for increased throughput resilience and availability
Web 2.0 support to help create richer user experience
7 © 2011 IBM Corporation
8. Hursley Comes to the Nordics – May 2011
WebSphere MQ
V7.0.1
8 © 2011 IBM Corporation
9. Hursley Comes to the Nordics – May 2011
WebSphere MQ V7.0.1 – Major Themes
Enhanced availability options on Distributed platforms
Constraint relief on z/OS
IBM portfolio exploitation, and extension of reach, for V7 features
Ongoing performance, consumability and serviceability enhancements
Keeping pace with industry evolution in areas such as platforms and SSL
9 © 2011 IBM Corporation
10. Hursley Comes to the Nordics – May 2011
Distributed Platforms: Multi-instance Queue Managers
Basic failover support without HA coordinator
– Faster takeover: fewer moving parts
– Cheaper: no specialised software or administration skills needed
– All non-z/OS platforms
Queue manager data is held in networked storage
– NAS, NFS, GPFS etc so more than one machine sees the queue manager data
– Improves storage management options: formal support for these even without failover
config
Multiple (2) instances of a queue manager on different machines
– One is “active” instance; other is “standby” instance
– Active instance “owns” the queue manager’s files and will accept app connections
– Standby instance does not “own” the queue manager’s files and apps cannot connect
• If active instance fails, standby performs queue manager restart and becomes active
Instances share data, so it’s the SAME queue manager
10 © 2011 IBM Corporation
11. Hursley Comes to the Nordics – May 2011
Multi-instance Queue Managers
1. Normal MQ MQ
Execution Client Client
network
192.168.0.1 192.168.0.2
Machine A Machine B
QM1 QM1
Active can fail-over Standby
instance instance
QM1
networked storage
Owns the queue manager data
11 © 2011 IBM Corporation
12. Hursley Comes to the Nordics – May 2011
Multi-instance Queue Managers
2. Disaster MQ MQ
Strikes Client Client
network
Connections
broken from
clients
192.168.0.1 192.168.0.2
Machine A Machine B
QM1 QM1
Active locks freed Standby
instance instance
QM1
networked storage
12 © 2011 IBM Corporation
13. Hursley Comes to the Nordics – May 2011
Multi-instance Queue Managers
3. Standby MQ MQ
Connections
Comes to Life Client Client still broken
network
192.168.0.2
Machine B
QM1
Active
instance
QM1
networked storage
Owns the queue manager data
13 © 2011 IBM Corporation
14. Hursley Comes to the Nordics – May 2011
Multi-instance Queue Managers
4. Recovery MQ MQ
Complete Client Client Clients
reconnected.
Processing
continues.
network
192.168.0.2
Machine B
QM1
Active
instance
QM1
networked storage
Owns the queue manager data
14 © 2011 IBM Corporation
15. Hursley Comes to the Nordics – May 2011
Multi-instance queue managers: Details
MQ is NOT becoming an HA coordinator
– Generally, if other resources also required, use an HA coordinator such as HACMP
– Service objects can restart applications with qmgr but limited control
– Message Broker also integrates with and exploits this MQ function
The IP address is not taken over
– Channel config needs all possible addresses unless you use external IPAT or
intelligent router
– CONNAME(‘host1(port1),host2(port2)’) syntax extension on all platforms including
z/OS
Support for networked storage over modern network file system protocols
– For example, NFS v4 (not v3)
– Tool shipped to validate configuration
New options for crtmqm/strmqm/endmqm to control operations
– Cannot guarantee which instance becomes the primary
Removes need for MC91 (withdrawn)
15 © 2011 IBM Corporation
16. Hursley Comes to the Nordics – May 2011
Automatic Client Reconnection
Client library provides necessary reconnection logic on detection of a failure
Hides failure from application code
QM1
QM1
Application
Application
QM2
QM2
MQ Client
MQ Client
QM3
QM3
16 © 2011 IBM Corporation
17. Hursley Comes to the Nordics – May 2011
Automatic Client Reconnection
Tries to hide queue manager failures by restoring current state automatically
– Re-opens queues and other qmgr objects, re-establishes subscriptions
– For example, if MQPUT returns error, client reruns MQCONN/MQOPEN/MQPUT internally
– Not all MQI is seamless, but majority repaired transparently
Uses the list of addresses in CONNAME to find queue manager
– MQSERVER environment variable also understands list
– MQSERVER=SYSTEM.DEF.SVRCONN/TCP/host1(1414),host2(1414)
Can reconnect to the same or different Queue Manager
Reconnection interval is backed off exponentially on each unsuccessful retry
– Total timeout is configurable – default 30 minutes.
Enabled in application code or ini file
– Event Handler callback shows reconnection is happening if app cares
17 © 2011 IBM Corporation
18. Hursley Comes to the Nordics – May 2011
Windows Communication Foundation
Windows Communication Foundation (WCF)
– Underpins .NET Web Services and Messaging
– Built-in Transports – e.g. MSMQ, HTTP(S), Named Pipes, TCP/IP, etc.
– Transports can be extended with “custom channels”
– Allows alternative transports (like MQ) to be slotted into WCF seamlessly
WCF support now included in product
– Previously an Alphaworks prototype
Built on XMS .Net classes
– Now shipped as part of the product instead of SupportPac
– Internally, it exploits V7 API, in same way as JMS implementation
18 © 2011 IBM Corporation
19. Hursley Comes to the Nordics – May 2011
Security & Monitoring
Command and Configuration Events for Distributed platforms
– Matching already-available z/OS function
– Successful commands – MQSC or PCF – recorded as event messages
– Configuration changes report “before” and “after” definitions of objects
– Provides a record of who changed what and when
display qmgr event
AMQ8408: Display Queue Manager details.
QMNAME(V7) CMDEV(ENABLED) CONFIGEV(ENABLED) …
SSL OCSP Support
– Now commonly used as alternative to LDAP-based CRLs
– Simpler to manage as no need to have an LDAP server
– Can use details provided in inbound certificate
– Also can be configured within queue manager or by application code
19 © 2011 IBM Corporation
20. Hursley Comes to the Nordics – May 2011
z/OS Constraint Relief
In V7.0, the queue manager started to exploit 64-bit addressing
– New Pub/Sub features
In V7.0.1 more Queue Manager storage moves to 64-bit
– 64-bit Queue Indices
– 64-bit Lock Manager
Can have more open queues, more messages on indexed queues etc
Small message storage changed
– One message per page instead of fitting multiples into single page
– May increase DASD use, especially if messages not quickly retrieved
– But typical workloads show improved performance and reduced CPU
20 © 2011 IBM Corporation
21. Hursley Comes to the Nordics – May 2011
z/OS Log Compression
Can increase the throughput possible for persistent messages
May reduce the size of your logs
– Dependent on your message content
– Useful if you are DASD constrained.
RLE (run-length encoding) of “insert” log records for private queue messages
– Will not compress shared queue log records
– SMF 115 records updated to show compression rates achieved etc
Controlled via zPARM option at queue manager level.
– COMPLOG(NONE) or COMPLOG(RLE) in CSQ6LOGP
– Can also be viewed/controlled via DISPLAY LOG / SET LOG
21 © 2011 IBM Corporation
22. Hursley Comes to the Nordics – May 2011
WebSphere MQ
SupportPacs
22 © 2011 IBM Corporation
23. Hursley Comes to the Nordics – May 2011
Most recent SupportPacs
MA98: WebSphere MQ - Connection Endpoint Look-up Exit
– Find connection information from an LDAP server
MH05: WebSphere MQ - Events Display Tool
– Reads and displays events from SYSTEM.ADMIN.*.EVENT queues
– Can be triggered from WMQ
– Can be written to a file
MO06: WebSphere MQ – Message Log Summary Utility
– Summarise and filter Qmgr Message Logs
See http://pipr.co/supportpac for the full list of recent
SupportPacs
23 CSS: S D © 2011 IBM Corporation
24. Hursley Comes to the Nordics – May 2011
WebSphere MQ
Low Latency
Messaging
24 © 2011 IBM Corporation
25. Hursley Comes to the Nordics – May 2011
WebSphere MQ Low Latency Messaging
Initially built for Financial Markets customers requiring high performance,
reliable messaging in the front office for market data and trade data
Included with WebSphere Front Office for Financial Markets
Integrated with Partners such as Data Vendors/ISVs for internal application
communication and external application connectivity
Production deployments in market data distribution systems, internal trade
execution systems and exchange execution systems
Also applicable in industries with similar quality of service requirements
– Defense and Aerospace
– Telecommunications
– Logistics
25 © 2011 IBM Corporation
26. Hursley Comes to the Nordics – May 2011
Positioning of IBM Messaging Transports
WebSphere MQ Low Latency
WebSphere MQ
Messaging
Speed – Quality of Time Independent Ultra-Fast
Service “As-Soon-As-Possible” “Microseconds”
Reliability – Quality of Either non-persistent or persistent Ranges from unreliable to assured
Service “Ultra-Reliable, Bet-The-Business” “Variable, based on class of data”
Target Market Enterprise Messaging Low Latency Messaging
Financial Markets and several others with similar
Target Industries All
QoS needs
Queuing Yes No
Standardized Unique / specific
APIs (JMS, XMS and MQI) (C, Java, .NET)
Linux and Windows on x86,
Platform coverage 80 platforms Linux on pSeries & zSeries, Solaris
(x86,SPARC), HP-UX, AIX
Included in
WAS, WMB, WESB, DataPower XI50, DB2,
Interoperates with CICS, IMS, etc, etc…
WebSphere Front Office
Integrated with DataPower
26 © 2011 IBM Corporation
27. Hursley Comes to the Nordics – May 2011
Software Overview
Peer-to-peer
Fully distributed
No centralized management
– No centralized queue for messages
– Daemon-less
Publish-Subscribe
Transmitter publishes on a topic (RMM) or “queue” (RUM)
Receiver subscribes to a topic or “queue”
Topic usually identified by a unique name.
– Can have multiple transmitters/receivers for the same Topic
Transports
–Ethernet 1GigE and 10 GigE
–Infiniband – Native verbs or IB over IP
–Shared Memory
© 2011 IBM Corporation
28. Hursley Comes to the Nordics – May 2011
WebSphere MQ
File Transfer
Edition
28 © 2011 IBM Corporation
29. Hursley Comes to the Nordics – May 2011
Introducing WMQ File Transfer Edition
Adds file transfer services to WebSphere MQ to enable managed file movement
Multi-purpose solution combining file transfer and messaging on a single backbone
Reliable – leveraging the WebSphere MQ
transport automate
Multi-purpose – transfers both messages and track audit
configure
files
Auditable – logging subsystem tracks transfer at
source and destination for audit purposes
Centralized – monitoring, control and
configuration
No need to program – no MQ skills required
Graphical tooling – visual configuration and
status
Moves Massive files – no size limit, Kb, Mb, Gb…
Command line interface – for advanced users
WebSphere MQ File Transfer Edition
Scripting support – enables automation via
scripting of complex multi-step transfers in XML
using Apache Ant
Flexible backbone – moves files from anywhere a b c x y z
to everywhere in network
Integration with MQ-enabled apps and ESBs
Automatic file character conversion
Security of file payload using SSL
Support a wide range of Platforms and O/S
© 2011 IBM Corporation
30. Hursley Comes to the Nordics – May 2011
WMQ File Transfer Edition 7.0.3 & 7.0.4
Increased web support
–REST interfaces
–Web transfers from end-users
Extended platform support: AIX on Power7, Linux on Power7, Oracle
Solaris 10 on Intel x86 and virtualized platforms
WebSphere Message Broker integration
–FTEInput and FTEOutput nodes in WMB FP2
WebSphere Advanced Message Security integration
–Easily encrypt file transfers
Sterling Connect:Direct Bridge capability including audit trail
7.0.4 GA June 3 2011
© 2011 IBM Corporation
31. Hursley Comes to the Nordics – May 2011
WebSphere MQ
Advanced
Message Security
31 © 2011 IBM Corporation
32. Hursley Comes to the Nordics – May 2011
WebSphere MQ Advanced Message Security
End-to-End Message Security - Secures application data even
before it is passed to MQ
Extension to base MQ – No changes to existing applications
WebSphere MQ standard security:
Message data can be encrypted in transport
(SSL) but not when it resides in the queues
Authentication is based on Operating System
identifier of local process
Application A Application Z
WebSphere MQ Advanced Message Security
Supplements WMQs security features:
+ Assurance that messages have not been WebSphere MQ Advanced Message Security
altered in transit WebSphere MQ
+ Assurance that messages originated from
the expected source
+ Assurance that messages can only be
viewed by intended recipient(s)
+ Administered using queue based policies Securing the data and
created from the WMQ Explorer or the applications
command line tooling.
32 © 2011 IBM Corporation
33. Hursley Comes to the Nordics – May 2011
Advanced Message Security – Key Elements
Message Interceptors
Protection policy for the application queue
Public / private key pairs
Recipient’s public key
Sending Application (Alice) Receiving Application (Bob)
AMS_QM
APP.Q
Keystore Keystore
Alice Priv Bob Priv
Alice Pub Bob Pub
Bob Pub
Policy
APP.Q
Privacy
Recipient : Bob
© 2011 IBM Corporation
34. Hursley Comes to the Nordics – May 2011
WebSphere MQ Advanced Message Security
Adds message-level security to existing WebSphere MQ (V6 & V7)
Support details:
–C MQI and Java clients
–Standard WMQ distributed platforms (apart from iSeries), including zOS
–Requires a minimum of WMQ v6.0.2.8 or WMQ v7.0.1.2
Constraints
–Point to point messaging domain only
–On zOS MQI v6 verbs only (no async consume, message properties etc)
© 2011 IBM Corporation
35. Hursley Comes to the Nordics – May 2011
WebSphere MQ
Telemetry
35 © 2011 IBM Corporation
36. Hursley Comes to the Nordics – May 2011
The Internet of Things
Many smart devices
instrument our world
today
Interconnecting these
smart devices creates a
Central Nervous
System
36 © 2011 IBM Corporation
37. Hursley Comes to the Nordics – May 2011
Universal Messaging for a Smarter Planet
Enterprise
Regional Office
Offer connectivity capabilities
optimised for sensors and devices Branch
Outlet
Provide Enterprise Access to
new sources of data to support
your business
Provide access to Enterprise
data and services from new
platforms/businesses Sensor Retail
e.g. Store
Deliver data to intelligent RFID
decision-making assets
Zero-Administration for “remote”
deployments Pervasive
Device
Refinery
Petrol
Forecourt
37 25/05/11 © 2011 IBM Corporation
38. Hursley Comes to the Nordics – May 2011
Connectivity with MQTT
■ MQ Telemetry Transport (MQTT) is
designed with the following primary
intentions:
– Publish/subscribe messaging paradigm as
required by the majority of SCADA and
sensor applications.
– Minimise the on-the-wire footprint.
– Expect and cater for frequent network
disruption.
– Cope with slow and poor quality networks
– Expect that client applications may have
very limited processing resources available.
– Provide traditional messaging qualities of
service where the environment allows
– Publish the protocol for ease of adoption by
device vendors and third-party client
software.
© 2011 IBM Corporation
39. Hursley Comes to the Nordics – May 2011
Extended Reach
Basic Telemetry
WebSphere MQ Telemetry V7.0.1 extends IBM’s Client
industrial strength enterprise message system
Backend Server
• Highly scalable
– A single queue manager can handle 100,000 concurrently
connected devices
• Ships with two types of client: WebSphere
– Basic MQ
• direct connectivity from a device
– Advanced:
• Acts as a “concentrator/hub” for mini-networks of Telemetry
devices
• Can connect to multiple backend servers
• Can failover to alternate backend server
• Can buffer messages
MQ Telemetry
• Provides rich security Daemon for
Devices
– Network: SSL
– Authentication: JAAS
– Authorisation: OAM Advanced
Telemetry Client
• In addition, any third party, open source or “roll
your own” MQTT client can be used
39 25/05/11 © 2011 IBM Corporation
40. Hursley Comes to the Nordics – May 2011
WebSphere MQ Telemetry Implementation
Fully integrated / interoperable with WMQ
– Publishers and subscribers can exchange messages with MQI and JMS applications
Telemetry channels enable MQTT connections to queue manager
– Supports MQTTv3.1 protocol
Administration using WMQ Explorer
Initially available on Windows and Linux
Ships with reference Java (for MIDP upwards) and C clients
Includes WebSphere MQ Telemetry Daemon for Devices
– An advanced client which also accepts connections from other clients
– Provides ability to store and forward, network local devices.
© 2011 IBM Corporation
41. Hursley Comes to the Nordics – May 2011
Broader integration
Simple
Lightweight (CPU,Mem,**Net)
Data-centric
Distribution (pub/sub)
Range of QoS
=> developer/community interest!
© 2011 IBM Corporation
42. Hursley Comes to the Nordics – May 2011
Case Study: Smarter Healthcare
Medical organization created a remote pace-maker monitoring
solution to provide better patient care
Client Pains
Physicians needed better monitoring
of cardiac patients
Improve efficiency of checkups
Meet healthcare data capture
standards
Enables higher level of patient care and peace of mind
Improves administrative efficiency and maintenance
Helps conform to standards and ease integration of data
42 © 2011 IBM Corporation
43. Hursley Comes to the Nordics – May 2011
Case Study: Improving Energy Usage
Utility company developing an Intelligent Utility Network offering for
optimizing load on electricity grids
Business Partner
Needs robust middleware
technology to connect to remote
smart meters
Needs to be able to rapidly scale
solution nationally
Able to offer daily energy savings of 15-20%
Enables utilities to reduce peaks and avoid punitive charges
Helps save electricity through better peak load management
43 © 2011 IBM Corporation
44. Hursley Comes to the Nordics – May 2011
WebSphere MQ
Community
44 © 2011 IBM Corporation
45. Hursley Comes to the Nordics – May 2011
We love our users!
developerWorks
MQSeries.net community
– http://mqseries.net/
Global WebSphere Community GWC
– http://websphereusergroup.org
MQTT – http://mqtt.org
IBMer's Blog on Messaging on developerWorks
Twitter – @IBM_WMQ and @IBM_Broker
YouTube – http://youtube.com/mqontv
Store and Forward – T.Rob's WMQ Security blog
– http://t-rob.net/
Open, vibrant ecosystem
– e.g. PyMQI, mosquitto, github projects, etc. etc.
45
45 © 2011 IBM Corporation
46. Thank you!
Contact: Andy Piper andy.piper@uk.ibm.com
http://twitter.com/andypiper | http://andypiper.co.uk
WebSphere
© 2011 IBM Corporation