Why X.400 Is Good for High Reliability Messaging

Summary

X.400 was specified in the 1980s, with the expectation that it would be the universal standard for email. While this did not happen, X.400 is still used for many applications, particularly where high reliability is required. This paper summarizes the key features of X.400 that make it good for applications needing high reliability, with particular focus on capabilities not available with Internet email.

The History of X.400 and Internet Email

X.400 was initially standardized by the ITU (International Telecommunications Union) in 1984, with a number of updates and extensions since then. X.400 was standardized jointly with ISO (International Organization for Standards) since 1988.

Internet email evolved in the 1970s as a practical solution by engineers working on the Arpanet (pre-Internet). It was standardized as a text messaging system in the 1980s, and extended with attachments and other capabilities in the late 80s and early 90s. It has become the worldwide system for email.

Why X.400 is rarely used for Inter-Personal Messaging

Inter-personal messaging is messaging primarily between humans, typically for general purpose business and personal use.

It is interesting to speculate why X.400 did not become the standard of choice for sending Inter-Personal messages. Possible factors include the complexity of the technology, and that it was primarily driven by PTTs (National Telecommunications Organizations) who were seeking to charge for email. However, there are now three overwhelming reasons why Internet mail is used for Inter Personal messaging.

  1. Connectivity. Everyone uses it, and so it makes sense to join them.
  2. There are a wide range of client and server products to support many applications.
  3. Internet mail has good functionality for Inter Personal messaging, in environments that do not have high reliability requirements.

Because general purpose Inter Personal messaging is such a dominant application, for which Internet mail is well suited and has been well tuned, it is in other areas of messaging that X.400 remains important.

Where X.400 is Used

X.400 is used in a number of environments where its additional benefits are needed, such as Military, Financial Data, Intelligence, Aviation and Government.

There are a number of types of application where it is used:

  1. For transferring data between systems. For example in financial applications to transfer EDI and other data.
  2. For inter-personal messaging in secure environments. It is quite common for users to make use of Internet Messaging for “low grade” communication and X.400 for “high grade”
  3. For structured communication, which can be considered to be messaging between roles rather than individuals. The military refer to this as “formal messaging”, contrasting with “informal messaging” (Inter Personal Messaging), and the STANAG 4406 specifications extend X.400 to provide this critical military service. Government organizations use X.400 in this way. The aviation use of X.400 to provide the ATS message service (AMHS) is also this type of application.

An interesting note is that in a vast majority of cases X.400 is used over the Internet (or over private TCP Networks), so it can be considered as an Internet application.

X.400 Architectural Advantages

X.400 was designed by people with substantial prior messaging experience, and a number of the advantages arise from the architecture of X.400. Although architecture is not a direct advantage, this section gives some insight into why X.400 has some of the benefits described later on.

Clean Protocol Layering

The key architectural concept in X.400 is the “Message Transfer Service”, (MTS) and the protocols and procedures for operation associated with this. The MTS carries generic content end to end, which can be an Inter Personal Message (IPM) or other data. There is a clear separation between MTS Envelope Information (e.g., Message Priority) or IPM Headers (e.g., Subject).

Internet email has retrospectively adopted a very similar architecture to X.400, but the separation between MTS and Content is not clean, due to requirements for backwards compatibility. Header fields are generic mechanism, used by both layers.

Extensibility

X.400 provides very generic message protocol extensibility, which is not available with Internet mail. It provides it in a number of places, and it provides it using mechanisms where X.400 users can define unique community extensions. This uniqueness is guaranteed by use of Object Identifiers.

Extensions in the MTS have the concept of “criticality” for transfer and delivery. This enables correct behavior in light of an unknown extension, and is very useful for introducing new capabilities. X.400 provides extensibility in a number of places, including:

  • Attachments (Internet email is also extensible here).
  • IPM Headings (Internet email is extensible, but does not have guaranteed uniqueness).
  • IPM Per Recipient Extensions.
  • Inter Personal Notification Extensions.
  • Delivery Report Extensions.
  • MTS Per-Message Extensions.
  • MTS Per-Recipient Extensions.

This extensibility is used to support both private extensions, and is also the mechanism used to update the X.400 standards with new capabilities, helping to ensure easy backwards compatibility.

Military messaging, defined in STANAG 4406 has made extensive use of X.400 extensibility to define military messaging extensions within the X.400 framework.

Standardized Internet messaging extensions exist. The extent and consistency of implementing these is very variable. For example, latest delivery time (RFC 2852) is not commonly implemented.

High Functionality

X.400 is a feature-rich messaging protocol, with a wide range of capabilities. This is helpful when defining new messaging applications. For example, the AMHS standards defined in the aviation industry have been able to work based entirely on standard X.400 features (no extensions), making use of capabilities such as “authorization time” and “precedence policy”.

Per Recipient Information

X.400 is structured so that parameters may be associated with individual recipients (as well as with message envelope and with an IPM). Internet messaging does not have this general purpose extensibility and it would be very hard to add in a backwards compatible manner. This extensibility is used to provide a number of X.400 services not available in Internet mail.

X.400 Implementation Advantages

There are a number of X.400 advantages that are not inherent to the technical specifications, but are in practice important reasons to choose X.400.

Vendor Approach

There are many vendors of Internet messaging solutions, with a wide variety of goals. X.400 vendors have a much tighter focus, and are generally concerned with management capabilities and high reliability. Because the X.400 markets demand high reliability, an X.400 vendor is likely to understand these requirements and have suitable products.

ISPs, PICS and Implementation Capabilities

The X.400 standards were designed with procurement in mind. The base specifications are structured into functional groups, which are referenced by International Standard Profiles (ISPs) that define collections of functional groups appropriate for different applications. The standards also include a number of PICS (Protocol Implementation Conformance Statement) proformas, which are essentially a table of X.400 features. Many X.400 vendors provide completed PICSs for their implementations that enable a customer to verify the functional details of what is being offered.

Internet mail is specified by a large number of related documents called RFCs (Request For Comments). Multiple specifications are used to achieve modularity of functionality. Because of the wide scope of Internet email, some implementations of some RFCs are rather deficient. Where a complex system with specific technical requirements is needed, this can place a quite high burden onto the system procurement.

Delivery Reports

X.400 provides delivery reports that offer positive acknowledgment of message delivery. This can be requested by user or by the message transfer system. This enables straightforward management of reliable delivery that may be transparent to the user. X.400 delivery reports are designed for both user and system processing X.400 specifies how Deljvery Reports interact with Distribution Lists.

Similar functionality is provided in Internet Messaging by Delivery Status Notifications. These are less suitable for automatic processing. They can only be requested by the user, and automatic correlation is only practical by use of a DSN feature that is not widely used (env-id). Also many popular Internet MTAs do not support the DSN format, but instead generate text messages for user consumption. Interaction between DSNs and distribution lists is inconsistent and problematic. All of this makes it impractical to do general system management of reliable delivery in support of "fire and forget" service.

Message Receipt

X.400 provides the Inter-Personal Notification service, which enables the recipient to send back a Receipt Notification that a message has been read (or not read). This service is important is support of high reliability services to ensure that messages have been read. Support of this service is generally well integrated with X.400 clients.

Internet Mail defines a specification for a similar service, know as Message Disposition Notification (MDN). Only a few clients support MDNs, and the popular Microsoft Outlook implements an equivalent service using a proprietary Microsoft protocol. Automated services using MDNs are in general problematic.

Unchanged Content

With X.400, content is transferred without change unless explicit content conversion occurs. The originator is able to prohibit content conversion, or to restrict content conversion to types of conversion that do not lead to loss of information.

Although there is no reason for Internet mail systems to change content, in practice content is often modified in transit.

Peer Authentication

Both X.400 and Internet mail provide mechanisms for authentication between components, and in particular for authentication between pairs of MTAs (Message Transfer Agents). Peer MTA authentication is generally desirable in a highly reliable Message Transfer System, to ensure that both source and destination MTA are correctly validated. Both X.400 and Internet mail support peer authentication. Peer authentication is used by default in X.400 networks, and is seen as a core feature. It is not generally used for Internet mail, and is often awkward or impossible to set up for Internet mail products.

Message Transport Features

This section looks at significant features of the X.400 Message Transport Service that are not available with Internet mail.

Priority

X.400 has a three level message priority system, extended in STANAG 4406 for military messaging to have six levels. The X.400 Message Transport System processes messages according to the priority. Many high reliability systems identify traffic with higher priority. In many high reliability systems, it is important to ensure high priority of certain critical messages. This is an important requirement for Military and Aviation systems.

Alternate Recipient

When a message is transferred close to the intended recipient, and there is a problem in delivering the message to the intended recipient, X.400 has the concept of “Alternate Recipient”. This has two variants:

1. "Recipient Domain Assigned Alternate Recipient". This

2. "Originator Specified Alternate Recipient". This is specified by the message sender, and in effect says "Use this address, if ".

Alternate recipient is often a highly desirable approach for high reliability messaging, when the goal is to ensure that a message is handled (rather than pushing a failure problem back to the originator).

Security

The X.400 Message Transport Service offers extensive security capabilities. Message security can be achieved in two fundamental ways:

  1. Message Transport Security, where security services are part of the message transport.
  2. Document security, where security (typically signature and encryption) are applied to the message content, which is then sent over a message transport without any security features.

For basic signature and encryption services, the overall result of these two approaches is similar.

Three popular open standards for email security are really document security. These are:

  • PGP. Pretty Good Privacy is a popular security standard for use with Internet Mail.
  • S/MIME uses the CMS (Cryptographic Message Standard) to sign and encrypt MIME documents. MIME is the document format used by Internet mail, and by other Internet standards such as the Web.
  • STANAG 4406 Military messaging uses CMS to sign and encrypt Military X.400 Messages. Here, the document is a message.

Message Transport Security can be used in conjunction with document security. There are three advantages to using Message Transport Security.

  1. It can be introduced incrementally, without impacting the format of messages (documents) transferred. This is an important benefit of the approach taken with AMHS (aviation).
  2. It secures all of the content. For example in S/MIME, the From/To/Subject our outside the secure part and can be tampered with.
  3. It introduces a range of security features that add value, and are not available with a document security approach (but could be used in conjunction with document security). These include:
    • Proof of submission.
    • Proof of delivery.
    • Non-repudiation of submission.
    • Message sequence integrity.
    • (Delivery) Report origin authentication.

Delivery Time Control

X.400 provides two controls on message delivery timing (complementary to the Message Expiry Time, which is an IPM feature provided by both X.400 and Internet mail).

  1. Deferred Delivery. A mechanism to specify the earliest time at which a message may be delivered
  2. Latest Delivery Time. A mechanism to specify the latest time at which a message may be delivered. In the event of failing to deliver in time, the MTS will non-deliver the message. Internet mail specifies an equivalent function in RFC 2852, but this is not widely implemented.

Latest Delivery Time can be important for managing time critical information in a high reliability system.

Other

Other features of the X.400 MTS that can be useful to high reliability message include:

  • Various operational characteristics of the X.400 MTS can be explicitly controlled by the originator. These include:
    • Prevention of content conversion
    • Prevention of content conversion where there is loss of information
    • Prevention of using alternate recipients
    • Prevention of use of distribution lists
  • X.400 Distribution Lists are a part of the X.400 Message Transport Service, and are managed with it.

Performance & Low Bandwidth

Messaging deployments are often in environments where bandwidth is high, and the considerations of this section are not relevant. Some high reliability messaging systems, such as military deployments, operate in bandwidth constrained environments where the points discussed in this section are important.

Seven vs Eight Bit

Internet messaging was originally a seven bit protocol. That is, only seven of the eight data bits are used, even when carrying binary data. The protocols have been extended to support binary transfers, but these extensions are not generally used because it is impractical to deploy them in a mixed environment with seven bit. This is compounded for binary data, as the preferred encoding of binary data (Base 64) leads to a 33% increase in data size. In practice, this means that Internet mail fails to make use of 25% of available bandwidth when transferring binary data.

Data Size

X.400 messages are more compact than Internet messages, due to use of the binary ASN.1 encoding as opposed to the text encoding of Internet mail. For example, Internet mail encodes “Subject:” using eleven bytes (8 for the text “Subject:” as space and CRLF). X.400 ASN.1 uses only two bytes. . HTML, frequently used in Internet mail, does not use a compact encoding, and can lead to size increase for small messages. Some measurements made by looking at transfer of minimal messages (single recipient, very short subject and text) were made, to give an informal analysis of the base overhead.

Internet:

  1. Internet message headers, generated by common email clients. 800-1200 bytes
  2. Typical minimal SMTP Transfer overhead: 200 bytes
  3. Typical SMTP Connection Establishment (no authentication, which is normal for SMTP) and tear down: 450 bytes

X.400:

  1. Typical minimum IPM Header: 250-300 bytes
  2. Typical minimum MTS Envelope. 350 bytes
  3. Typical P1 Data Transfer Overhead, down to TCP level. 90 bytes
  4. Typical P1 Connection establishment (with authentication) and teardown: 450 bytes

This data suggests:

  1. X.400 binary encoding is in general a win over text encoding for equivalent functionality.
  2. Normal internet message headers (generated by typical clients) carry a certain amount of extraneous data

Transfer Recovery

X.400 transfers messages using a Reliable Transfer Service, which enables partially transferred messages to resume transfer from the last checkpoint. This is useful to transfer larger messages over low reliability networks.

ACP 142 & STANAG 4406 Annexe E

For very low bandwidth deployments, the Military have defined X.400 extensions in two standards for use together:

  • STANAG 4406 Annexe E defines protocol optimizations and use of data compression.
  • ACP 142 defines an approach to multicast distribution that allows efficient use of broadcast networks such as satellite or radio, to send to multiple destinations at once.

These protocols are suitable for use with non-military deployments of X.400.

Directory Integration

Directory integration is an important advantage of X.400. The obvious use of directory, for “white pages” type lookup of recipients does not need protocol support. However, integration is important for high reliability messaging, which is described below.

Capability Verification

When sending a message, it is desirable to verify the recipient using directory lookup before the message is sent. This has the inherent advantage of ensuring that the message is being sent to a correct destination, and prevents the inefficiency and potential risks of an end to end delivery report.

The lookup will also verify capabilities of the recipient. The goal of this checking is to ensure that the recipient is appropriate and can process the message. Examples of checks:

  • Maximum message size, to ensure that the message is not too large for the recipient.
  • Service level supported by the recipient (e.g., AMHS check of extended versus basic service).
  • Gateway capabilities (e.g., AMHS checks for conversion to legacy systems).
  • Ability of client to handle specific data formats.
  • Security clearance, by verifying the recipient’s security clearance against the security label of the message.

High reliability environments will operate highly replicated directories in order to achieve this efficiently and robustly. Military messaging deployments take an extreme approach, and fully replicate the directory so that the necessary information is always available locally.

Protocol Support

The protocol support needed for this is that X.400 addressing can carry around a directory name associated with each address. This means that it is straightforward to integrate with directory by always making use of directory names in the context of each address. This is not possible for Internet mail, as directories suitable for capability verification are not keyed on Internet mail address.

Summary

This paper sets out detailed reasons as to why X.400 is the best choice for high reliability messaging. This makes clear why it is the choice for Aviation AMHS, and for Military STANAG 4406.