CIS007-3 – Comparative Integrated Systems
Curry, Edward (2005) Chapter 1 – Message Oriented Middleware
Edward Curry National University of Ireland, Galway, Ireland
From Middleware for Communications (2005) Ed. Qusay H Mahmoud, John Wiley and Sons.
(http://userpages.umbc.edu/~dgorin1/451/middleware/middleware2.htm)
By Jonathan Angel, Network Magazine
We can begin with general agreement (thank heaven for small favors) that middleware is software, not hardware. Beyond that, however, just what is it exactly? Some authors describe middleware as “the ‘slash’ in client/server,” which is clever but not particularly helpful. Middleware can also be termed “programming that sits in the middle, connecting diverse (and usually preexisting) programs together.”
Expanding either of these definitions, however, would suggest that communications software programs, network operating systems, Web browsers, Cisco Systems’ Internetwork Operating System (IOS)—and a lot more—are all middleware. This is logically true, but will not get you far as you attempt to decipher vendors’ data sheets. So rather than obsessing over an exact definition of middleware, I’ll look at how the need for this type of software arose and review the types of solutions that have evolved.
IBM WebSphere MQ
(https://en.wikipedia.org/wiki/IBM_WebSphere_MQ)
IBM MQ is a family of network software products that IBM launched in March 1992. It was originally called MQSeries, and was renamed WebSphere MQ in 2002 to join the suite of WebSphere products. In April 2014, it was renamed IBM MQ.
IBM MQ, (often referred to as “MQ”), is IBM’s Messaging solution for Enterprise and IBM’s Message Oriented Middleware offering. It allows independent and potentially non-concurrent applications on a distributed system to securely communicate with each other. MQ is available on a large number of platforms (both IBM and non-IBM), including z/OS (mainframe), OS/400 (IBM System i or AS/400), Transaction Processing Facility, UNIX (AIX, HP-UX, Solaris), HP NonStop, OpenVMS, Linux, OS 2200, and Microsoft Windows.
Message-oriented middleware
IBM MQ was the most popular system for messaging across multiple platforms in 2000, including Windows, Linux, OS/2, IBM mainframe and midrange, and Unix.
There are two parts to Message queue (MQ):
- Message: Messages are collections of binary or character (for instance ASCII or EBCDIC) data that have some meaning to a participating program. As in other communications protocols, storage, routing, and delivery information is added to the message before transmission and stripped from the message prior to delivery to the receiving application.
- Queue: Message queues are objects that store messages in an application.
A Queue Manager, although not strictly required for message-oriented middleware, is an IBM MQ prerequisite and system service that provides a logical container for the message queue, and is responsible for transferring data to other queue managers via message channels.
Programs integrated with IBM MQ use a consistent application program interface (API) across all platforms.
Support of service-oriented architecture
IBM MQ is a key component in IBM’s service-oriented architecture (SOA) strategy, providing the universal messaging backbone across 80 different platforms. The growing importance of SOA and the growth of Web Services and other connectivity mechanisms are clearly important developments. Because of the loosely coupled nature of the message queuing model, a large number of existing MQ customers feel that they are already adopting SOA principles. The M.Q Service definition support pack MA93 allows MQ applications to be catalogued as software assets that can be reused and composed as Web Services.
MQ also supports Enterprise Service Bus implementations, both proprietary and open source. For example, the Mule WMQ transport is available with Mule Enterprise Edition version 1.6 or later and, as of 2.2, contains many critical performance and reliability improvements as well as native support for WebSphere MQ-specific features.
APIs
There are many ways to access IBM MQ facilities.
Some of the APIs directly supported by IBM are:
- IBM Message Queue Interface (MQI) for C, COBOL, PL/I, Java, Rexx, RPG, and C++
- Java Message Service (JMS)
- XMS for C/C++ and .NET[4]
- .NET
- Representational State Transfer
- SOAP
Communication
The primary component of an IBM MQ installation is the Queue Manager. The queue manager handles storage, timing issues, triggering, and all other functions not directly related to actual movement of data.
Queue managers communicate with the outside world either via a direct software connection, referred to by IBM as a “bindings” connection, or via a network or “client” connection. The bindings connection is limited to programs running on the same physical host as the queue manager, whereas applications using a client connection can connect to a queue manager on any other host in the network.
Bindings connections are generally faster, but client connections allow for a more robust, easily changeable application design. For instance, with a client connection, the physical location of the queue manager is irrelevant, as long as it is reachable over the network.
Communication between queue managers relies on a channel. Each queue manager uses one or more channels to send and receive data to other queue managers. A channel is uni-directional; a second channel is required to return data. In a TCP/IP based network, a channel will send or receive data on a specific port. A sending channel has a defined destination and is associated with a specific transmission queue, the mechanism by which messages are queued awaiting transmission on the channel; a receiving channel will receive data from any other queue manager with a sending channel of the same name. When a receiving channel receives a message, it is examined to see which queue manager and queue it is destined for. In the event of a communications failure, MQ can automatically re-establish a connection when the problem is resolved.
The “listener” has the function of detecting connections from incoming channels and managing the connection of the sending to the receiving channels. It is the application’s network interface to the queue manager. In a TCP/IP network, the listener will “listen” for connections on a specific port.
Local queues represent the location where data is stored awaiting processing.
Remote queues represent a queue on another queue manager. They define the destination queue, which is one element of the routing mechanism for messages.
To transmit data to a queue on another queue manager, a message is placed on a remote queue. A remote queue is sent via the temporary storage transmission queue associated with a channel. On placing a message on a remote queue, the message will be transmitted across the remote channel. If the transmission is successful the message is removed from the transmit queue. On receiving a message, the receiving queue manager will examine the message to determine whether the message is for itself or is required to forward on to another queue manager. If it is the destination, the required queue will be checked, and if it exists, the message will be placed on this queue, if not, placed on the dead letter queue. MQ has features to manage efficient transmission of data across a variety of communication media, so for example messages can be batched together until a queue reaches a particular depth.
Although the queue is FIFO, it is ordered based on the receipt in the local queue, not the committing of the message from the sender. Messages can be prioritized, and by default, the queue is prioritized in order of arrival. Queues will only be in sequence of addition if the message is added locally. Message grouping can be used to ensure a set of messages are in a specific order, aside from that, if sequence is critical, it is the application’s responsibility to place sequence data in the message or implement a handshaking mechanism via a return queue. In reality, ordering will be maintained in straightforward configurations.
The other element of a queue manager is the log, the mechanism used to create the robustness. As a message is placed on a queue or a configuration change is made, the data is also logged. In the event of a failure, the log is used to recreate damaged objects and recreate messages. Only “persistent” messages will be recreated when a failure occurs—”non-persistent” messages are lost. Non-persistent messages can be sent across a channel set to a fast mode, in which delivery is not assured in the event of a channel failure.
MQ supports a wide variety of approaches to application development. Information can be retrieved from queues either by polling the queue to check for available data at suitable intervals, or alternatively MQ can trigger an event, allowing a client application to respond to the delivery of a message.
Accelerate growth by exploiting reliable, robust, industry-leading messaging solution
(https://www-03.ibm.com/software/products/en/ibm-mq)
IBM® MQ is robust messaging middleware that simplifies and accelerates the integration of diverse applications and business data across multiple platforms. IBM MQ facilitates the assured, secure and reliable exchange of information between applications, systems, services and file by sending and receiving message data via messaging queues, thereby simplifying the creation and maintenance of business applications. It delivers Universal Messaging with a broad set of offerings to meet enterprise-wide messaging needs, as well as connectivity for the internet of things and mobile devices.
IBM MQ Advanced for Developers is available at no-charge for development purposes for Windows and Linux platforms. There is also a no-charge, 90-day trial for all platforms.
IBM MQ provides:
- Rapid, seamless connectivity of information with a single, robust and trusted messaging backbone for dynamic heterogeneous environments.
- Secure, reliable message delivery that preserves message integrity and minimizes risk of information loss.
- High-performance and scalable message transfer to meet the demands of today’s enterprise and beyond.
- Simplified management and control for better control and usability.
- Lower cost of ownership by reducing cost of integration and accelerating time to deployment.
WebSphere MQ Primer – An Introduction to Messaging and WebSphere MQ – IBM Redbooks(https://www.redbooks.ibm.com/redpapers/pdfs/redp0021.pdf
Technical Introduction to IBM WebSphere MQ
(https://www.youtube.com/watch?v=A1uPLyWNhfY)
https://www.youtube.com/watch?v=A1uPLyWNhfY
This course is a technical overview of IBM WebSphere MQ. It provides a conceptual understanding of messaging and queuing as implemented by IBM WebSphere MQ.
Processing Message Queues with IBM Websphere MQ
(http://wiki.pentaho.com/display/EAI/Processing+Message+Queues+with+IBM+Websphere+MQ)
IBM Websphere MQ is a messaging and queuing middleware provider that can be leveraged via the Java Message Service (JMS) specification. In this manner, Pentaho Data Integration exposes IBM MQ Consumer and Producer transformation steps allowing access to produce and consume messages through IBM Websphere MQ. IBM Websphere MQ provides this functionality through IBM’s implementation of JMS, called the Websphere MQ classes for JMS.
As the MQ steps are layered atop the JMS Consumer and Producer transformation steps, there is overlap and great similarities between the documentation for JMS and IBM Websphere MQ. For the sake of usable documentation, some of the JMS documentation is repeated here.
With JMS (and as such, also with IBM Websphere MQ) it is possible to process messages in two basic methods:
Queues
A message can be sent to a specified MQ Queue and can be received by exactly one consumer. If there are no consumers available at the time the message is sent it will be kept until a consumer is available that can process the message. This is also know as a point-to-point model.
Topics
In Websphere MQ a Topic implements publish and subscribe semantics. When you publish a message it goes to all the subscribers who are interested – so zero to many subscribers will receive a copy of the message. Only subscribers who had an active subscription at the time the broker receives the message will get a copy of the message. This is also known as a publish/subscribe model.
IBM MQSeries Primer (1999) MQSeries Enterprise Application Integration Center
(http://www.dhs.state.pa.us/cs/groups/webcontent/documents/document/p_032490.pdf)
MQSeries is IBM’s award winning middleware for commercial messaging and queuing. It is used by thousands of customers in every major industry in many countries around the world. MQSeries speeds implementation of distributed applications by simplifying application development and test.
Microsoft Message Queuing
(https://en.wikipedia.org/wiki/Microsoft_Message_Queuing)
Microsoft Message Queuing or MSMQ is a message queue implementation developed by Microsoft and deployed in its Windows Server operating systems since Windows NT 4 and Windows 95. The latest Windows 8 also includes this component. In addition to its mainstream server platform support, MSMQ has been incorporated into Microsoft Embedded platforms since 1999 and the release of Windows CE 3.0.
MSMQ is essentially a messaging protocol that allows applications running on separate servers/processes to communicate in a fail-safe manner. A queue is a temporary storage location from which messages can be sent and received reliably, as and when conditions permit. This enables communication across networks and between computers, running Windows, which may not always be connected. By contrast, sockets and other network protocols assume that direct connections always exist.
MSMQ has been available to developers on Microsoft platforms since 1997,[2] and has commonly been used in enterprise software built with Visual Studio, both in the native pre-.NET incarnation (version 5 and 6), and in Visual Studio .NET. Microsoft also has incorporated MSMQ in its messaging technology framework, Windows Communication Foundation (WCF). Under WCF, MSMQ can be used for providing secure, reliable transport with a unified programming model compatible with other communications standards.
MSMQ is responsible for reliably delivering messages between applications inside and outside the enterprise. MSMQ ensures reliable delivery by placing messages that fail to reach their intended destination in a queue and then resending them once the destination is reachable. It also supports security and priority based messaging. Dead letter queues can be created for looking at messages which timed out or failed for other reasons.
MSMQ also supports transactions. It permits multiple operations on multiple queues, with all of the operations wrapped in a single transaction, thus ensuring that either all or none of the operations will take effect. Microsoft Distributed Transaction Coordinator (MSDTC) supports transactional access to MSMQ and other resources.
Message Queuing (MSMQ)
(https://msdn.microsoft.com/en-us/library/ms711472%28v=vs.85%29.aspx)
Message Queuing (MSMQ) technology enables applications running at different times to communicate across heterogeneous networks and systems that may be temporarily offline. Applications send messages to queues and read messages from queues. The following illustration shows how a queue can hold messages that are generated by multiple sending applications and read by multiple receiving applications.
Message Queuing provides guaranteed message delivery, efficient routing, security, and priority-based messaging.
It can be used to implement solutions to both asynchronous and synchronous scenarios requiring high performance. The following list shows several places where Message Queuing can be used.
- Mission-critical financial services: for example, electronic commerce.
- Embedded and hand-held applications: for example, underlying communications to and from embedded devices that route baggage through airports by means of an automatic baggage system.
- Outside sales: for example, sales automation applications for traveling sales representatives.
- Workflow: Message Queuing makes it easy to create a workflow that updates each system. A typical design pattern is to implement an agent to interact with each system. Using a workflow-agent architecture also minimizes the impact of changes in one system on the other systems. With Message Queuing, the loose coupling between systems makes upgrading individual systems simpler.
Note: Message Queuing is a message infrastructure and a development platform for creating distributed, loosely-coupled messaging applications for the Microsoft® Windows® operating system. Message Queuing applications can use the Message Queuing infrastructure to communicate across heterogeneous networks and with computers that may be offline. Message Queuing provides guaranteed message delivery, efficient routing, security, transaction support, and priority-based messaging.
Microsoft Message Queuing
(https://www.youtube.com/watch?v=RXUGcE0rCuw)
Microsoft Message Queuing or MSMQ is a message queue implementation developed by Microsoft and deployed in its Windows Server operating systems since Windows NT 4 and Windows 95. The latest Windows 8 also includes this component. In addition to its mainstream server platform support, MSMQ has been incorporated into Microsoft Embedded platforms since 1999 and the release of Windows CE 3.0.
Amazon Simple Queue Service
(https://en.wikipedia.org/wiki/Amazon_Simple_Queue_Service)
Amazon Simple Queue Service (Amazon SQS) is a distributed queue messaging service introduced by Amazon.com in late 2004. It supports programmatic sending of messages via web service applications as a way to communicate over the Internet. SQS is intended to provide a highly scalable hosted message queue that resolves issues arising from the common producer-consumer problem or connectivity between producer and consumer.
Amazon SQS can be described as commoditization of the messaging service. Well-known examples of messaging service technologies include Websphere MQ and Microsoft Message Queuing. Unlike these technologies, users do not need to maintain their own server. Amazon does it for them and sells the SQS service at a per-use rate.
Message delivery
Amazon SQS guarantees at-least-once delivery. Messages are stored on multiple servers for redundancy and to ensure availability. If a message is delivered while a server is not available, it may not be removed from that server’s queue and may be resent. As of 2007, Amazon SQS does not guarantee that the recipient will receive the messages in the order they were sent by the sender. If message ordering is important, it is required that the application place sequencing information within the messages to allow for reordering after delivery.
Messages can be of any type, and the data contained within is not restricted. Message bodies were initially limited to 8KB in size but was later raised to 64KB on 2010-07-01 and then 256KB on 2013-06-18. For larger messages, the user has a few options to get around this limitation. A large message can be split into multiple segments that are sent separately, or the message data can be stored using Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB with just a pointer to the data transmitted in the SQS message. Amazon has made an Extended Client Library available for this purpose.
The service supports both unlimited queues and message traffic.
Amazon Simple Queue Service (SQS)
(https://aws.amazon.com/sqs/)
Amazon Simple Queue Service (SQS) is a fast, reliable, scalable, fully managed message queuing service. SQS makes it simple and cost-effective to decouple the components of a cloud application. You can use SQS to transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available.
With SQS, you can offload the administrative burden of operating and scaling a highly available messaging cluster, while paying a low price for only what you use.
Amazon SQS provides significant advantages over the complexity of home-grown message queuing services or the ongoing expense of licensed or hosted messaging services.
Reliable
Amazon SQS runs within Amazon’s high-availability data centers, so queues will be available whenever applications need them. To prevent messages from being lost or becoming unavailable, all messages are stored redundantly across multiple servers and data centers.
Simple
Developers can get started with Amazon SQS by using only three APIs: SendMessage, ReceiveMessage, and DeleteMessage. Additional APIs are available to provide advanced functionality.
Scalable
Amazon SQS was designed to enable an unlimited number of messaging services to read and write an unlimited number of messages at any time.
Secure
Authentication mechanisms are provided to ensure that messages stored in Amazon SQS queues are secured against unauthorized access.
Inexpensive
No up-front or fixed expenses. The only costs of sending messages through Amazon SQS are small per-request handling fees and data transfer fees.
What is Amazon Simple Queue Service? (https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/Welcome.html)
Amazon Simple Queue Service (Amazon SQS) offers reliable and scalable hosted queues for storing messages as they travel between computers. By using Amazon SQS, you can move data between distributed components of your applications that perform different tasks without losing messages or requiring each component to be always available.
Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component. A queue is a temporary repository for messages that are awaiting processing.
Using Amazon SQS, you can decouple the components of an application so they run independently, with Amazon SQS easing message management between components. Any component of a distributed application can store messages in a fail-safe queue. Messages can contain up to 256 KB of text in any format. Any component can later retrieve the messages programmatically using the Amazon SQS API. Messages larger than 256 KB can be managed using the Amazon SQS Extended Client Library for Java, which uses Amazon S3 for storing larger payloads.
The queue acts as a buffer between the component producing and saving data, and the component receiving the data for processing. This means the queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer are only intermittently connected to the network.
Amazon SQS ensures delivery of each message at least once, and supports multiple readers and writers interacting with the same queue. A single queue can be used simultaneously by many distributed application components, with no need for those components to coordinate with each other to share the queue.
Amazon SQS is engineered to always be available and deliver messages. One of the resulting tradeoffs is that SQS does not guarantee first in, first out delivery of messages. For many distributed applications, each message can stand on its own, and as long as all messages are delivered, the order is not important. If your system requires that order be preserved, you can place sequencing information in each message, so that you can reorder the messages when the queue returns them.
Architectural Overview
There are three main actors in the overall system:
- The components of your distributed system
- Queues
- Messages in the queues
In the following diagram, your system has several components that send messages to the queue and receive messages from the queue. The diagram shows that a single queue, which has its messages (labeled A-E), is redundantly saved across multiple Amazon SQS servers.
Amazon SQS Features
Amazon SQS provides the following major features:
- Redundant infrastructure – Guarantees delivery of your messages at least once, highly concurrent access to messages, and high availability for sending and retrieving messages
- Multiple writers and readers – Multiple parts of your system can send or receive messages at the same time.
Amazon SQS locks the message during processing, keeping other parts of your system from processing the message simultaneously. - Configurable settings per queue – All of your queues don’t have to be exactly alike.
For example, one queue can be optimized for messages that require a longer processing time than others. - Variable message size – Your messages can be up to 262,144 bytes (256 KB) in size.
For even larger messages, you can store the contents of the message using the Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB, and use Amazon SQS to hold a pointer to the Amazon S3 object. Alternatively, you can split the larger message into smaller ones. - Access control – You can control who can send messages to a queue, and who can receive messages from a queue
- Delay Queues – A delay queue is one which the user sets a default delay on a queue such that delivery of all messages enqueued will be postponed for that duration of time. You can set the delay value when you create a queue with
CreateQueue, and you can update the value withSetQueueAttributes. If you update the value, the new value affects only messages enqueued after the update. - PCI compliance – Amazon SQS supports the processing, storage, and transmission of credit card data by a merchant or service provider, and has been validated as being compliant with Payment Card Industry (PCI) Data Security Standard (DSS).
Amazon Simple Queue Service – How Amazon SQS Queues Work
(https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/SQSConcepts.html)
Topics
This section describes the basic properties of Amazon SQS queues, identifiers for queues and messages, how you determine the general size of the queue, and how you manage the messages in a queue.
A queue can be empty if you haven’t sent any messages to it or if you have deleted all the messages from it.
You must assign a name to each of your queues. You can get a list of all your queues or a subset of your queues that share the same initial characters in their names (for example, you could get a list of all your queues whose names start with “T3”).
You can delete a queue at any time, whether it is empty or not. Be aware, however, that queues retain messages for a set period of time. By default, a queue retains messages for four days. However, you can configure a queue to retain messages for up to 14 days after the message has been sent.
Amazon SQS can delete your queue without notification if one of the following actions hasn’t been performed on it for 30 consecutive days: SendMessage, ReceiveMessage, DeleteMessage, GetQueueAttributes, SetQueueAttributes, AddPermission, and RemovePermission.
The following table lists the API actions to use.
| To do this… | Use this action |
|---|---|
|
Create a queue |
|
|
Get the URL of an existing queue |
|
|
List your queues |
|
|
Delete a queue |
|
Amazon Simple Queue Service – Properties of Distributed Queues
(https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/DistributedQueues.html)
Properties of Distributed Queues
The following information can help you design your application to work with Amazon SQS correctly.
Message Order
Amazon SQS makes a best effort to preserve order in messages, but due to the distributed nature of the queue, we cannot guarantee you will receive messages in the exact order you sent them. If your system requires that order be preserved, we recommend you place sequencing information in each message so you can reorder the messages upon receipt.
At-Least-Once Delivery
Amazon SQS stores copies of your messages on multiple servers for redundancy and high availability. On rare occasions, one of the servers storing a copy of a message might be unavailable when you receive or delete the message. If that occurs, the copy of the message will not be deleted on that unavailable server, and you might get that message copy again when you receive messages. Because of this, you must design your application to be idempotent (i.e., it must not be adversely affected if it processes the same message more than once).
Message Sample
The behavior of retrieving messages from the queue depends whether you are using short (standard) polling, the default behavior, or long polling.
With short polling, when you retrieve messages from the queue, Amazon SQS samples a subset of the servers (based on a weighted random distribution) and returns messages from just those servers. This means that a particular receive request might not return all your messages. Or, if you have a small number of messages in your queue (less than 1000), it means a particular request might not return any of your messages, whereas a subsequent request will. If you keep retrieving from your queues, Amazon SQS will sample all of the servers, and you will receive all of your messages.
The following figure shows short polling behavior of messages being returned after one of your system components makes a receive request. Amazon SQS samples several of the servers (in gray) and returns the messages from those servers (Message A, C, D, and B). Message E is not returned to this particular request, but it would be returned to a subsequent request.
Amazon Simple Queue Service Getting Started Guide (2014) – Amazon SQS Resources
(https://awsdocs.s3.amazonaws.com/SQS/latest/sqs-gsg.pdf)
Getting Started with Amazon SQS
(https://www.youtube.com/watch?v=-XGm2VyNV4E)
https://www.youtube.com/watch?v=-XGm2VyNV4E
Uploaded on 16 Dec 2011
Get a quick over view and hands-on practice using Amazon SQS to create a message queue that quickly and reliably sends messages from decoupled components in a distributed application.
Working with Amazon Simple Queue Service using Java
(https://www.javacodegeeks.com/2013/06/working-with-amazon-simple-queue-service-using-java.html)
Amazon Simple Queue Service or SQS is a highly scalable hosted messaging queue provided by Amazon Webservice stack. Amazon SQS can be used to completely decouple operations of different components within the system which otherwise exchange data to perform independent tasks. Amazon SQS also helps us in saving the data which would be lost in case the application is down or if one of the component becomes unavailable.
Java Message Service
(https://en.wikipedia.org/wiki/Java_Message_Service)
The Java Message Service (JMS) API is a Java Message Oriented Middleware (MOM) API for sending messages between two or more clients. JMS is a part of the Java Platform, Enterprise Edition, and is defined by a specification developed under the Java Community Process as JSR 914. It is a messaging standard that allows application components based on the Java Enterprise Edition (Java EE) to create, send, receive, and read messages. It allows the communication between different components of a distributed application to be loosely coupled, reliable, and asynchronous.
Models
The JMS API supports two models:
- Point-to-point
- Publish and subscribe
Point-to-point model
In point-to-point messaging system, messages are routed to an individual consumer which maintains a queue of “incoming” messages. This messaging type is built on the concept of message queues, senders, and receivers. Each message is addressed to a specific queue, and the receiving clients extract messages from the queues established to hold their messages. While any number of producers can send messages to the queue, each message is guaranteed to be delivered, and consumed by one consumer. Queues retain all messages sent to them until the messages are consumed or until the messages expire. If no consumers are registered to consume the messages, the queue holds them until a consumer registers to consume them.
Publish/subscribe model
The publish/subscribe model supports publishing messages to a particular message topic. Subscribers may register interest in receiving messages on a particular message topic. In this model, neither the publisher nor the subscriber knows about each other. A good analogy for this is an anonymous bulletin board
- Zero or more consumers will receive the message.
- There is a timing dependency between publishers and subscribers. The publisher has to create a message topic for clients to subscribe. The subscriber has to remain continuously active to receive messages, unless it has established a durable subscription. In that case, messages published while the subscriber is not connected will be redistributed whenever it reconnects.
JMS provides a way of separating the application from the transport layer of providing data. The same Java classes can be used to communicate with different JMS providers by using the Java Naming and Directory Interface (JNDI) information for the desired provider. The classes first use a connection factory to connect to the queue or topic, and then use populate and send or publish the messages. On the receiving side, the clients then receive or subscribe to the messages.
JavaTM Message Service Tutorial
(https://docs.oracle.com/javaee/1.3/jms/tutorial/)
The JavaTM Message Service (JMS) Tutorial is provided as a companion to the Java 2, Enterprise Edition (J2EETM) SDK, version 1.3.1.
The J2EE 1.3.1 tutorial has the following goals:
- To introduce the JMS API to new users, with simple JMS client program examples
- To show how to use the JMS API within the J2EE platform, with examples showing how to
- Consume messages asynchronously with a message-driven bean
- Produce messages from an application client
- Produce messages from a session bean
- Access an entity bean from a message-driven bean
- Produce and consume messages on more than one system
The tutorial assumes that you are familiar with the Java programming language and that you have some experience with the J2EE platform.
To run the examples, you must first install the J2EE SDK version 1.3.1. You should first install the required version of the Java 2 SDK, Standard Edition, if it is not already installed.
You may read the tutorial online:
JMS Tutorial
- 1.1 What Is Messaging?
- 1.2 What Is the JMS API?
- 1.3 When Can You Use the JMS API?
- 1.4 How Does the JMS API Work with the J2EETM Platform?
3 The JMS API Programming Model
- 3.1 Administered Objects
- 3.2 Connections
- 3.3 Sessions
- 3.4 Message Producers
- 3.5 Message Consumers
- 3.6 Messages
- 3.7 Exception Handling
4 Writing Simple JMS Client Applications
- 4.1 Setting Your Environment for Running Applications
- 4.2 A Simple Point-to-Point Example
- 4.3 A Simple Publish/Subscribe Example
- 4.4 Running JMS Client Programs on Multiple Systems
5 Creating Robust JMS Applications
6 Using the JMS API in a J2EE Application
- 6.1 Using Enterprise Beans to Produce and to Synchronously Receive Messages
- 6.2 Using Message-Driven Beans
- 6.3 Managing Distributed Transactions
- 6.4 Using the JMS API with Application Clients and Web Components
7 A Simple J2EE Application that Uses the JMS API
- 7.1 Writing and Compiling the Application Components
- 7.2 Creating and Packaging the Application
- 7.3 Deploying and Running the Application
8 A J2EE Application that Uses the JMS API with a Session Bean
- 8.1 Writing and Compiling the Application Components
- 8.2 Creating and Packaging the Application
- 8.3 Deploying and Running the Application
9 A J2EE Application that Uses the JMS API with an Entity Bean
- 9.1 Overview of the Human Resources Application
- 9.2 Writing and Compiling the Application Components
- 9.3 Creating and Packaging the Application
- 9.3.1 Starting the J2EE Server and the Deploytool
- 9.3.2 Creating a Queue
- 9.3.3 Starting the Cloudscape Database Server
- 9.3.4 Creating the J2EE Application
- 9.3.5 Packaging the Application Client
- 9.3.6 Packaging the Equipment Message-Driven Bean
- 9.3.7 Packaging the Office Message-Driven Bean
- 9.3.8 Packaging the Schedule Message-Driven Bean
- 9.3.9 Packaging the Entity Bean
- 9.3.10 Specifying the Entity Bean Deployment Settings
- 9.3.11 Specifying the JNDI Names
- 9.4 Deploying and Running the Application
10 An Application Example that Uses Two J2EE Servers
- 10.1 Overview of the Applications
- 10.2 Writing and Compiling the Application Components
- 10.3 Creating and Packaging the Application
- 10.4 Deploying and Running the Applications
- 10.5 Accessing a J2EE Application from a Remote System that Is Not Running a J2EE Server
Appendix A JMS Client Examples
Alternatively, you may download the Tutorial and use it on your own system. The download bundle is available in either zip or PDF format.
The zip file contains the HTML version of the tutorial and the source code for the examples. Unzipping the file creates a directory called jms_tutorial, in which the tutorial is installed. To access the tutorial, open the jms_tutorial/index.html file in your web browser.
Chapter 47 – Java Message Service Concepts – The Java EE 6 Tutorial
(https://docs.oracle.com/javaee/6/tutorial/doc/bncdq.html)
This chapter provides an introduction to the Java Message Service (JMS) API, a Java API that allows applications to create, send, receive, and read messages using reliable, asynchronous, loosely coupled communication. It covers the following topics:
- Overview of the JMS API
- Basic JMS API Concepts
- The JMS API Programming Model
- Creating Robust JMS Applications
- Using the JMS API in Java EE Applications
- Further Information about JMS
JMS Tutorial
(http://www.javatpoint.com/jms-tutorial)
JMS (Java Message Service) is an API that provides the facility to create, send and read messages. It provides loosely coupled, reliable and asynchronous communication.
JMS is also known as a messaging service.
Understanding Messaging
Messaging is a technique to communicate applications or software components.
JMS is mainly used to send and receive message from one application to another.
Requirement of JMS
Generally, user sends message to application. But, if we want to send message from one application to another, we need to use JMS API.
Consider a scenario, one application A is running in INDIA and another application B is running in USA. To send message from A application to B, we need to use JMS.
Advantage of JMS
1) Asynchronous: To receive the message, client is not required to send request. Message will arrive automatically to the client.
2) Reliable: It provides assurance that message is delivered.
Messaging Domains
There are two types of messaging domains in JMS.
- Point-to-Point Messaging Domain
- Publisher/Subscriber Messaging Domain
1) Point-to-Point (PTP) Messaging Domain
In PTP model, one message is delivered to one receiver only. Here, Queue is used as a message oriented middleware (MOM).
The Queue is responsible to hold the message until receiver is ready.
In PTP model, there is no timing dependency between sender and receiver.
2) Publisher/Subscriber (Pub/Sub) Messaging Domain
In Pub/Sub model, one message is delivered to all the subscribers. It is like broadcasting. Here, Topic is used as a message oriented middleware that is responsible to hold and deliver messages.
In PTP model, there is timing dependency between publisher and subscriber.
JMS Programming Model
Introducing the Java Message Service
(https://www.ibm.com/developerworks/java/tutorials/j-jms/j-jms-updated.html)
This tutorial provides an introductory overview of the Java Message Service, its functionality, and its capabilities. You will learn the basic programming techniques for creating JMS programs and work with sample code that demonstrates these techniques. Note: This tutorial has been updated to include changes to JMS version 1.1.
Java JMS Tutorial
(https://www.youtube.com/watch?v=lsAyTeUUXHk&list=PLXniY3avdn9gkpPFxC0gQdtswW5KYbZYL)
In this episode, we are going to look at what is Java messaging and how it works. Then we’ll look at sending and receiving messages. By the end, you should know all the basics for java message communication.
Transcript and code: http://www.deegeu.com/java-jms-tutorial/
Apache ActiveMQ
(http://activemq.apache.org/index.html)
Apache ActiveMQ ™ is the most popular and powerful open source messaging and Integration Patterns server.
Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache2.0 License.
Apache ActiveMQ
(https://en.wikipedia.org/wiki/Apache_ActiveMQ)
Apache ActiveMQ is an open sourcemessage broker written in Java together with a full Java Message Service (JMS) client. It provides “Enterprise Features” which in this case means fostering the communication from more than one client or server. Supported clients include Java via JMS 1.1 as well as several other “cross language” clients. The communication is managed with features such as computer clustering and ability to use any database as a JMS persistence provider besides virtual memory, cache, and journal persistency.
ActiveMQ is used in enterprise service bus implementations such as Apache ServiceMix and Mule. Other projects using ActiveMQ include Apache Camel and Apache CXF in SOA infrastructure projects.
Coinciding with the release of Apache ActiveMQ 5.3, the world’s first results for the SPECjms2007 industry standard benchmark were announced. Four results were submitted to the SPEC and accepted for publication. The results cover different topologies to analyze the scalability of Apache ActiveMQ in two dimensions.
Apache Active MQ
(https://www.youtube.com/watch?v=s-E_V5Xyg6k)
Apache ActiveMQ is the most widely used message-oriented middleware that uses messaging to connect remote applications written in Java, C/C++, Python, Perl, Ruby and many more. ActiveMQ is standards based and supports messaging protocols such as AMQP 1.0, WebSockets, Stomp, OpenWire and MQTT. Used from M2M to global enterprise deployments, this talk will cover the features, functionality and secret ninja ticks to get ActiveMQ to both perform and scale, without loosing messages.
WildFly
(https://en.wikipedia.org/wiki/WildFly)
WildFly, formerly known as JBoss AS, or simply JBoss, is an application server authored by JBoss, now developed by Red Hat. WildFly is written in Java, and implements the Java Platform, Enterprise Edition (Java EE) specification. It runs on multiple platforms.
WildFly is free and open-source software, subject to the requirements of the GNU Lesser General Public License (LGPL), version 2.1.
On 20 November 2014, JBoss Application Server was renamed WildFly. The JBoss Community and other Red Hat JBoss products like JBoss Enterprise Application Platform were not renamed.
JBoss
(https://en.wikipedia.org/wiki/JBoss_%28company%29)
Red Hat JBoss Middleware is a portfolio of enterprise-class application and integration middleware software products delivered by Red Hat, Inc. These software products are used by end users to create applications; integrate applications, data, and devices; and automate business processes. Red Hat JBoss Middleware uses an open source development model.
History
Marc Fleury started the JBoss project in 1999. JBoss Group, LLC was incorporated in 2001 in Atlanta, Georgia. JBoss became a corporation under the name JBoss, Inc. in 2004. It was a C corporation headquartered in Atlanta, GA that owned the copyright and trademarks associated with JBoss.
In early 2006 Oracle Corporation, a major distributor of database software, sought to buy JBoss Inc. for an estimated $400 million. The acquisition would have enabled Oracle to compete with rivals BEA Systems and IBM in the middleware market (Oracle eventually acquired BEA in April 2008). On April 10, 2006, however, Red Hat announced that they would buy JBoss for $420 million. The acquisition was completed in June 2006.
Red Hat® JBoss® Middleware
(https://www.redhat.com/en/technologies/jboss-middleware)
Red Hat® JBoss® Middleware is a family of lightweight, cloud-friendly, enterprise-grade products that help organizations innovate faster, in a smarter way. That’s what we call enlightened innovation.
Middleware that makes everything work better.
Having the right middleware makes the things that should work, actually work. And it makes your IT team more productive.
Learn more:
Accelerate: Applications are built, tested, and deployed faster.
Integrate: Systems and data are connected more easily.
Automate: Manual, time-consuming processes are automated.
JBoss Messaging
(https://en.wikipedia.org/wiki/JBoss_Messaging)
JBoss Messaging is the JBoss enterprise asynchronous messaging system. It supersedes JBoss MQ as the default Java Message Service (JMS) provider in JBoss Application Server (JBoss AS) 5.
JBoss Messaging 1.0 was released on 29 March 2006 as a re-engineered version of JBoss MQ intended to deliver a modular messaging engine capable of shipping with or without JBoss.
JBoss Messaging is the default JMS provider in JBoss Enterprise Application Platform 4.3, JBoss SOA Platform and JBoss Application Server 5.
JBoss Messaging is an open source project available under the Gnu LGPL licence and is led by Tim Fox with core engineers Andy Taylor, Clebert Suconic, Howard Gao and Jeff Mesnil.
On 24 August 2009, HornetQ was launched, based on the JBoss Messaging 2.0 code-base, and the JBoss Messaging project was put into bug fix mode only by JBoss.
HornetQ – putting the buzz in messaging – JBoss Community
(http://hornetq.jboss.org/)
What is HornetQ?
HornetQ is an open source project to build a multi-protocol, embeddable, very high performance, clustered, asynchronous messaging system.
HornetQ is an example of Message Oriented Middleware (MoM) .
Why should I use HornetQ?
Here are a selection of the reasons:
Open Source Software – HornetQ is released under the Apache v 2.0 Licence.
Written in Java – HornetQ runs on any platform with a Java 5 or later runtime.
JMS and above – HornetQ supports the JMS 1.1 and 2.0 APIs and also defines its own messaging API for maximum performance and flexibility. Other protocols are planned for upcoming releases.
Superb performance – HornetQ class-beating high performance journal provides persistent messaging performance at rates normally seen for non-persistent messaging. Non-persistent messaging performance rocks the boat too.
POJO-based design – HornetQ has been designed using POJO and minimal third-party dependencies. You choose how you want to use HornetQ: run it stand-alone, integrate it with JBoss Application Server or another Java server/container or embed it directly inside your own product.
Solid high availability – HornetQ offers server replication and automatic client failover to eliminate lost or duplicated messages in case of server failure.
Flexible clustering – Create clusters of HornetQ servers that know how to load balance messages. Link geographically distributed clusters over unreliable connections to form a global network. Configure routing of messages in a highly flexible way. Adapt HornetQ to your network topology, not the other way round.
Management – HornetQ provides a comprehensive management API to manage & monitor servers. It is integrated seamlessly to the servers to work in a highly available environment.
Documentation & Examples – All HornetQ features are documented and examples are provided. Read the documentation, run the examples and leverage HornetQ features to make your messaging code more robust and performant.
Message Oriented Middleware
From: https://en.wikipedia.org/wiki/Message-oriented_middleware
Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. The middleware creates a distributed communications layer that insulates the application developer from the details of the various operating systems and network interfaces. APIs that extend across diverse platforms and networks are typically provided by MOM.
MOM provides software elements that reside in all communicating components of a client/server architecture and typically support asynchronous calls between the client and server applications. MOM reduces the involvement of application developers with the complexity of the master-slave nature of the client/server mechanism.
Advantages
Central reasons for using a message-based communications protocol include its ability to store (buffer), route, or transform messages while conveying them from senders to receivers.
Asynchronicity
MOM comprises a category of inter-application communication software that generally relies on asynchronous message-passing, as opposed to a request-response architecture. In asynchronous systems, message queues provide temporary storage when the destination program is busy or not connected. In addition, most asynchronous MOM systems provide persistent storage to back up the message queue. This means that the sender and receiver do not need to connect to the network at the same time (asynchronous delivery), and problems with intermittent connectivity are solved. It also means that should the receiver application fail for any reason, the senders can continue unaffected, as the messages they send will simply accumulate in the message queue for later processing when the receiver restarts.
Routing
Many message-oriented middleware implementations depend on a message queue system. Some implementations permit routing logic to be provided by the messaging layer itself, while others depend on client applications to provide routing information or allow for a mix of both paradigms. Some implementations make use of broadcast or multicast distribution paradigms.
Transformation
In a message-based middleware system, the message received at the destination need not be identical to the message originally sent. A MOM system with built-in intelligence can transform messages en route to match the requirements of the sender or of the recipient.In conjunction with the routing and broadcast/multicast facilities, one application can send a message in its own native format, and two or more other applications may each receive a copy of the message in their own native format. Many modern MOM systems provide sophisticated message transformation (or mapping) tools which allow programmers to specify transformation rules applicable to a simple GUI drag-and-drop operation.
Disadvantages
The primary disadvantage of many message-oriented middleware systems is that they require an extra component in the architecture, the message transfer agent (message broker). As with any system, adding another component can lead to reductions in performance and reliability, and can also make the system as a whole more difficult and expensive to maintain.
In addition, many inter-application communications have an intrinsically synchronous aspect, with the sender specifically wanting to wait for a reply to a message before continuing (see real-time computing and near-real-time for extreme cases). Because message-based communication inherently functions asynchronously, it may not fit well in such situations. That said, most MOM systems have facilities to group a request and a response as a single pseudo-synchronous transaction.
Standards
Historically, there was a lack of standards governing the use of message-oriented middleware that has caused problems. Most of the major vendors have their own implementations, each with its own application programming interface (API) and management tools.
The Advanced Message Queuing Protocol (AMQP) is an approved OASIS[3] and ISO[4] standard that defines the protocol and formats used between participating application components, so implementations are interoperable. AMQP may be used with flexible routing schemes, including common messaging paradigms like point-to-point, fan-out, publish/subscribe, and request-response (note that these are intentionally omitted from v1.0 of the protocol standard itself, but rely on the particular implementation and/or underlying network protocol for routing). It also supports transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support. Java applications that use AMQP are typically written in Java JMS. Other implementations provide APIs for C#, C++, PHP, Python, Ruby, and other languages.
The MQ Telemetry Transport (MQTT) is an ISO standard (ISO/IEC PRF 20922) supported by the OASIS organization. It provides a lightweight publish/subscribe reliable messaging transport protocol on top of TCP/IP suitable for communication in M2M/IoT contexts where a small code footprint is required and/or network bandwidth is at a premium.
The Object Management Group’s Data Distribution Service (DDS) provides message-oriented Publish/Subscribe (P/S) middleware standard that aims to enable scalable, real-time, dependable, high performance and interoperable data exchanges between publishers and subscribers.[5] The standard provides interfaces to C++, C++11, C, Ada, Java and Ruby.
The eXtensible Messaging and Presence Protocol (XMPP) is a communications protocol for message-oriented middleware based on XML (Extensible Markup Language). Designed to be extensible, the protocol has also been used for publish-subscribe systems, signalling for VoIP, video, file transfer, gaming, Internet of Things applications such as the smart grid, and social networking services. Unlike most instant messaging protocols, XMPP is defined in an open standard and uses an open systems approach of development and application, by which anyone may implement an XMPP service and interoperate with other organizations’ implementations. Because XMPP is an open protocol, implementations can be developed using any software license; although many server, client, and library implementations are distributed as free and open-source software, numerous freeware and commercial software implementations also exist. The Internet Engineering Task Force (IETF) formed an XMPP working group in 2002 to formalize the core protocols as an IETF instant messaging and presence technology. The XMPP Working group produced four specifications (RFC 3920, RFC 3921, RFC 3922, RFC 3923), which were approved as Proposed Standards in 2004. In 2011, RFC 3920 and RFC 3921 were superseded by RFC 6120 and RFC 6121 respectively, with RFC 6122 specifying the XMPP address format. In addition to these core protocols standardized at the IETF, the XMPP Standards Foundation (formerly the Jabber Software Foundation) is active in developing open XMPP extensions. XMPP-based software is deployed widely across the Internet, according to the XMPP Standards Foundation, and forms the basis for the Department of Defense (DoD) Unified Capabilities Framework.[6]
The Java EE programming environment provides a standard API called JMS (Java Message Service), which is implemented by most MOM vendors and aims to hide the particular MOM API implementations; however, JMS does not define the format of the messages that are exchanged, so JMS systems are not interoperable.
A similar effort is with the actively evolving OpenMAMA project, which aims to provide a common API, particularly to C clients. However, at the moment (August 2012) it is primarily appropriate for distributing market-oriented data (e.g. stock quotes) over pub-sub middleware.
Chapter 1 Messaging Systems: An Introduction (Sun Java System Message Queue 4.3 Technical Overview)
(https://docs.oracle.com/cd/E19316-01/820-6424/aerap/index.html)
Message-Oriented Middleware (MOM)
Because businesses, institutions, and technologies change continually, the software systems that serve them must be able to accommodate such changes. Following a merger, the addition of a service, or the expansion of available services, a business can ill afford to recreate its information systems. It is at this most critical point that it needs to integrate new components or to scale existing ones as efficiently as possible. The easiest way to integrate heterogeneous components is not to recreate them as homogeneous elements but to provide a layer that allows them to communicate despite their differences. This layer, called middleware, allows software components (applications, enterprise java beans, servlets, and other components) that have been developed independently and that run on different networked platforms to interact with one another. It is when this interaction is possible that the network can become the computer.
Message-Oriented Middleware and the WebSphere MQ Introduction
(http://www.tutorialsto.com/software/engineering/middleware/message-oriented-middleware-and-the-websphere-mq-introduction.html)
First, an overview of Message-Oriented middleware
Message Queuing technology is the exchange of information between distributed applications in a technology. Message queue can reside in memory or disk storage message queue until the time they are taking applications. Through the message queue, the application can be implemented independently – they do not need to know each other’s position, or continue to implement procedures to obviate the need for waiting to receive to receive this message.
In the distributed computing environment, in order to integrate distributed applications, developers need for a heterogeneous network environment of distributed applications to provide an effective means of communication. In order to manage the need to share information, on the application of the provision of public information exchange mechanism is important.
Design methods of distributed applications include: Remote Procedure Call (PRC) – Distributed Computing Environment (DCE), one of the basis of standard components; Object Transaction Monitor (OTM) – based on the CORBA object-oriented transaction processing with industry-standard (TP) a combination of monitoring technology; message queue (MessageQueue) – Construction of loosely coupled distributed application methods.