Messaging Design
Top to bottom
Endpoints are interaction mechanisms between distributed entities, such as ION processes. 2 processes that instantiate 2 matching endpoints and connect them via correct names can communicate with defined interaction semantics. See details below.
Endpoint for an ION process that can invoke an RPC call to a public, named RPC endpoint, such a service, an agent or a process interface. RPC requires the definition of callable operations in a (service) interface. The caller calls an operation, specifies call arguments, optional headers and an optional timeout. The endpoint sends the request and waits for a private response. The endpoint blocks until the response is provided and relays it to the caller. Exceptions are reraised on the caller's side and timeouts are respected. The security context of the caller is used to make the call.
Endpoint for an ION process that acts as a public RPC server. Many processes can share the same public service name, thereby providing a scalable, reliable service in a worker pattern. The endpoint accepts available requests, checks and analyzes them, puts them through a defined set of interceptors and forwards them for execution to the endpoint callbacks, e.g. individual service operations. The result or exception is relayed back to the caller.
All callbacks are executing in the ION process control greenlet and are synchronized among all ION process aware listening endpoints. This means that there is no possibility for concurrency conflict while messages are processed, but this may limit the throughput of a process.
Endpoint for an ION process to publish an object as a message with descriptive headers to a name in the network. The endpoint does not know who listens to this name. This could be a broadcast or a private communication. See ProcessSubscriber.
Endpoint for an ION process to receive messages matching a given name and a given delivery queue. This endpoint is very flexible and can be specialized by subclasses or by configuration.
It is possible to subscribe for a name and have it delivered to a private queue, that is exclusive to the ION process that consumes. Alternatively a shared queue can be specified, so that multiple processes can round-robin consume from this queue.
The endpoint delivers received messages to a callback function. All callbacks are executing in the ION process control greenlet and are synchronized among all ION process aware listening endpoints.
Message consumption and delivery can be disabled and reenabled as needed. The size of the message queue and the number of connected consumers can be inspected.
Special class of ProcessSubscriber that provides functions to subscribe to event messages only, with sophisticated matching patterns, e.g. by type, super type, origin code, sub type, origin type or combinations thereof. Multiple processes can share one event queue if a common queue name is provided.
The endpoint delivers events to a callback. All callbacks are executing in the ION process control greenlet and are synchronized among all ION process aware listening endpoints.
Endpoint that publishes messages on a stream topic given a stream route expression.
Endpoint that subscribes to messages on a stream topic given a stream route expression.
Non ION process basic variant of a ProcessRPCClient for special cases such as in test cases. Some process specific message headers will be missing, limiting the use of the requests somewhat.
Non ION process basic variant of a ProcessRPCServer. The callbacks execute in their own concurrent greenlet, not synchronized with any other listening endpoints.
Non ION process basic variant of a ProcessPublisher.
Non ION process basic variant of a ProcessSubscriber.
Non ION process basic variant of a ProcessEventPublisher.
Non ION process basic variant of a ProcessEventSubscriber.
These include BaseEndpoint, ListeningBaseEndpoint, SendingBaseEndpoint and others that are common base classes for more advanced endpoints. These are typically not used directly, but there existence is important, in particular when trying to understand the code.
Each endpoint supports interaction instances. Each such conversation is managed by an EndpointUnit of a type matching the endpoint type.
Channels are the pathways of communication with the message broker. Multiple channels are multiplexed over the same connection. Each standing message consumer has its own channel and other channels exist for specific purposes.
Channel classes encapsulate the details of interacting with the message broker, which, given the asynchronous and distributed nature of the system, can be very complex. A channel implementation typically wraps a finite state machine, but provides a more application oriented view to the endpoint code. Errors are raised if the endpoint code does not use the channel according to the current state.
Different channel implementations exist, e.g. for sending and receiving channels. The channel layer also performs some optimizations, such as pooling channels, such that they can be reused.
Mostly in form of mapping channel action primitives to a specific set of AMQP client library asynchronous functions. But not restricted to AMQP - could be ZeroMQ or other.
This layer encapsulates a connection to a message broker or something similar.
These are higher level conceptual entities that the endpoint layers can work with instead of directly providing names. They represent either consumers, or producers, or intermediate entities in the messaging process. E.g. the QueueExchangeName object can represent a queue with a given name and a set of bindings that can be consumed from.
Besides participating as names in the endpoint layer, Exchange Objects can also be manipulated directly for greater control, e.g. adding or removing a queue binding, activating or deactivating message consumption etc.