BikINI/ specification

The initial version of this specification is not complete until any issues are done.

The shell script I use to convert this document from markdown to html to internet draft format is detailed at markdown to html to internet-draft.

Abstract

BikINI is designed to allow multiple client computers to access an Internet message mail store on a central server. It operates over any reliable datastream protocol, but is commonly used over TCP. BikINI is designed to require minimal message parsing by the server, to be 8-bit clean for mail message content, and to be light-weight and easy-to-implement in comparison to other remote message access protocols.

Table of Contents

   1. Introduction
   2. Requirement Words
   3. Protocol Overview
      3.1. Link Level
      3.2. Commands and Responses
      3.3. Mail Store Architecture
      3.4. Message Attributes
      3.5. Capability Tokens
   4. States and Flow
      4.1. Opening State
      4.2. Index State
      4.3. Message-Store State
      4.4. Message-Retrieve State
      4.5. Closed State
   5. Data Format and Line Endings
   6. Commands
      6.1. Opening State
      6.2. Index State
   7. Formal Syntax
      7.1. Definitions
      7.2. Commands
   8. Examples
   9. Security Considerations
   10. IANA Considerations
   11. References
   12. Acknowledgments
   Author's Address

1. Introduction

The objective of BikINI is to provide a remote mail storage and access protocol which is easier to implement than [IMAP] and more suitable for remote storage and access of messages than [POP3]. To that end, multiple design decisions were made to keep the protocol as simple as possible.

1.1. Design Considerations

BikINI should implement only those features necessary to fulfill the minimum requirements to store and access messages online.

Server-side message parsing is resource-intensive and complex, and thus should be avoided. This precludes functionality like retrieval of single parts of MIME multipart messages, but makes implementation significantly simpler and reduces the resource-intensiveness of the server.

Case insensitivity is not necessary, therefore commands and responses in BikINI are case-sensitive. Clients and servers MUST send commands in upper case; upper and lower case characters MAY appear in identifiers, paths, and flags. The server MAY include upper and lower case characters in message identifiers, response modifiers, and descriptive text responses.

Arbitrary runs of whitespace around commands and arguments is not necessary. Clients and servers MUST NOT add padding of any type (including extra whitespace) to commands, arguments, or responses unless explicitly stated in this protocol specification.

High numbers of active clients must be supported. In particular, the immediate disconnect response modifier removes the need for large numbers of idle connections.

1.2. Terminology

The following definitions hold throughout this document.

In examples, "C:" and "S:" indicate lines sent by the client and server respectively.

Lines are defined as a series of characters excluding <LF>, and terminated by <LF>.

Characters are 7-bit US-ASCII unless otherwise specified.

User refers to a human user.

Client refers to the software being run by the user.

Session refers to the entire sequence of client commands and server responses from the initial establishment of the network connection until its termination.

The mail store is the top-level container accessed by a session.

Internet Message Format ([[IMF][]]) refers to the formats defined by [[RFC822][]], [RFC2822], and [RFC5322]. Where a specific version of the format is required, it will be specified by RFC number.

Message content may be an entire message or any portion of a message, and consists of arbitrary 8-bit data (servers and clients SHOULD store and send [[IMF][]] headers and messages).

2. Requirement Words

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

3. Protocol Overview

3.1. Link Level

The BikINI protocol requires a reliable data stream as an underlying link level protocol. This may be TCP/IP or another protocol. When operating over TCP/IP, BikINI is frequently used on TCP port [TO BE ASSIGNED BY IANA - 40 REQUESTED].

3.2. Commands and Responses

A BikINI connection consists of the establishment of a network connection between a client and server, and interactions between the client and server. The interactions consist of client commands, server data, and server completion result responses.

All interactions transmitted by client and server are in the form of lines or message content.

When message content is being transferred in either direction the interaction is 8-bit data. When the message content is [[IMF][]] (expected to be all of the time currently), lines in an individual message MUST be terminated with <LF> only.

The protocol receiver of a BikINI client or server is either reading a line, or is reading a sequence of octets with a known count followed by a line.

3.2.1. Client Protocol Sender and Server Protocol Receiver

The client command begins an operation.

There are three cases in which a line from the client does not represent a complete command.

  • When the client is authenticating via an AUTH method requiring an exchange of authentication tokens.
  • When the client requires preliminary approval before completing the PUT operation.
  • When the server requires envelope data before completing the SEND operation.

If the client may proceed the server sends a K single-line response indicating the client may send its message, envelope data, or authentication token(s). If the client may not proceed the server sends an A, E, U, or X single-line response indicating the command cannot be processed.

3.2.2. Server Protocol Sender and Client Protocol Receiver

The server MUST NOT send any data to the client except in response to a command.

Data transmitted by the server to the client takes one of three forms.

  • single-line response

    • A single-character status code (A, E, K, U, or X), optionally followed by one or more single-character response modifiers, followed by a single space, optionally followed by some text, and terminated with <LF>. Only the K code is a positive response.

  • multi-line response

    • A multi-line response has zero or more + lines, each terminated with <LF>, followed by a K single-line response to indicate completion of the multi-line response. All multi-line responses are positive responses. Servers MUST NOT add response modifiers to the continuation + lines of a multiline response. All response modifiers on a multiline response must be added to the trailing K response line. A multiline response with zero continuation lines is identical to a K single-line response.

  • message content response

    • A K single-line response, where the additional information section contains only the decimal number of octets of message content (SIZE) to follow. The server MUST NOT include additional padding, leading zeroes, or any information other than SIZE in the additional information section. The initial response line's terminating <LF> is followed by exactly SIZE octets of message content. The message content is followed by a K single-line response to indicate completion of the message content response.

With the exception of certain commands defined later, the client MUST NOT derive any information from the text portion of a A, E, K, U, or X single-line response, other than optional informational messages displayed to the user.

3.2.3. Server Response Codes

Server response lines consist of a single character response code, followed by zero or more response modifiers, followed by a single space, then followed by zero or more characters of additional information. The server MUST NOT send a space between the response code and response modifiers.

The additional information is required for some response codes, and optional for others (as specified for each command). If the additional information is required it has a strict format defined by the command in question. If the additional information section is optional it is free-form and continues until the end of the line.

If the additional information section is not supplied, then the server MUST send a space between the response code and the <LF> which terminates the response line.

  • K

    • Successful completion of a command.

  • X

    • Syntax error. The command was incorrect or not understood by the server. Servers SHOULD use the additional information section to describe the error in more detail.

  • A

    • Temporary failure. The command was syntactically correct, but failed due to a transient error. The command might succeed if tried again under similar conditions. This could be caused by software error, administrative prohibition or policy. Servers SHOULD use the additional information section to describe the reason for failure in more detail.

  • E

    • Permanent failure. The command was syntactically correct, but failed permanently. The command will not succeed if tried again under the same conditions. This could be due to software error, administrative prohibition, or policy (i.e. all TAG commands fail on a read-only mail store). Servers SHOULD use the additional information section to describe the reason for failure in more detail.

  • U

    • Not supported. The command was recognized, but is not supported by the server for technical or policy reasons. Servers MAY use the additional information section to describe the condition in more detail.

  • +

    • Successful completion of a command with a multiline response. Following the + response lines is one K line to indicate the completion of the multiline response.

3.2.4. Server Response Modifiers

Server reponses MAY include a non-repeating sequence of one or more single-character response modifiers immediately following the response code (i.e. not preceded by a space).

Servers MUST NOT send a response modifier not defined by this specification. Servers MUST NOT send the same response modifier more than once in a given response. Servers MAY send multiple response modifiers in any order. Clients MUST NOT expect response modifiers to be in any particular order.

The following response modifiers are defined.

  • C
    • The server is closing the connection after this response. The server will close the session immediately after sending the <LF> which terminates this response, exactly as if the client had sent a command. Servers MAY do this to manage load or for other reasons. Clients MUST NOT treat this as an error condition. If a client wishes to perform additional operations on this mail store after receiving a response including the C response modifier, they must establish a new connection, authorize with the command, and then proceed with their operation. Clients SHOULD NOT require user approval (i.e. clicking OK to a warning dialog) prior to doing this.

3.3. Mail Store Architecture

A mail store contains zero or more messages. Messages physically located in the system mail spool should be associated with the tag "INBOX".

3.4. Message Attributes

3.4.1. Message Identifier

An arbitrary length token assigned to each message. Message identifiers MUST NOT contain any whitespace characters.

Messages in BikINI are accessed strictly by means of their identifier. This identifier is guaranteed not to refer to any other message in the mail store.

3.4.2. Tags Message Attribute

A list of zero or more strings assigned to a message. Tags MUST NOT contain the comma (,) character.

Multiple clients have to cooperate when accessing the same mail store; they have to agree on the meanings of some tags (like New, Replied, Seen, etc.) for them to be useful. Clients SHOULD NOT set or clear tags in an arbitrary way. For example, if a given client just arbitrarily clears the Replied tag on all messages, then that tag is useless to other clients accessing the same mail store.

At a minimum, servers and clients MUST recognize the common tags and definitions below.

  • Draft -- the client considers this message a draft.

  • Flagged -- user-dependent, frequently used to mean "important".

  • New -- Message has not been retrieved (unless client manually sets).

  • Passed -- User passed this message to someone else (forwarded, bounced, etc).

  • Replied -- Client has sent a reply to this message.

  • Seen -- This message has been seen by the client.

The client may set or clear any tag.

The GET command clears the New tag of the message in question as a side-effect.

3.4.3. Internal Timestamp Message Attribute

The internal timestamp of the message on the server is the date and time when the message was received by the server.

If the message is received via the BikINI protocol, then the date and time of receipt is used as the internal timestamp.

If the message is not received via the BikINI protocol (i.e. delivered to the system underlying the mail store), then the internal timestamp WILL be based on when the new message was detected and (optionally) the type of message.

The format used is similar to that defined by [RFC3339] ("YYYYMMDDThhmmssTZD") without punctuation, and with a timezone offset indicator of "Z" to indicate Coordinated Time Universal (UTC).

Examples:

    19970716T192030Z
    20030301T120000Z

Servers MUST use UTC when presenting Internal Timestamp Message Attributes to a client.

3.4.4. Message Size Attribute

The number of octets in the message, expressed as a positive nonzero decimal with no leading zeroes.

If the sender (client or server) of a size attribute stores an [[IMF][]] message using an end-of-line convention other than Unix-style <LF>-only, the sender MUST convert the message to <LF>-only before calculating the message's size.

3.4.5. Envelope Recipient Attribute

The fully qualified address of the recipient of the message, at the time of delivery.

If delivered via the PUT command, the envelope recipient is set by the client.

If delivered directly to the underlying system, the envelope recipient is determined by parsing the message headers. If unable to be determined, the envelope recipient remains unset.

3.4.6. Envelope Sender Attribute

The fully qualified address of the sender of the message, at the time of delivery.

If delivered via the PUT command, the envelope sender is set by the client.

If delivered directly to the underlying system, the envelope sender is determined by parsing the message headers. If unable to be determined, the envelope sender remains unset.

3.5. Capability Tokens

BikINI is designed to be an extensible protocol. Clients should continue to work even if servers are upgraded to support new protocol features and vice versa. To accomplish this, clients need a way to determine what capabilities are supported by a given server.

The command provides information about the server's capabilities. The server's response is a multiline response, with one line per capability. The format of each line is a single capability token, optionally followed by additional information in a format specific to that token.

Servers MAY implement experimental or nonstandardized capabilities which are not listed in this specification using capability tokens beginning with "X-" (i.e. "X-FOO"). The BikINI protocol will never define a standard capability token beginning with "X-". I encourage client and server authors to register their "X-..." tokens with me to prevent collisions.

Authors should implement experimental commands with the "X-" prefix and use the command name as the capability token representing the command. For example, an experimental glob search command might use the command X-GLOBSEARCH and be advertised with the capability token X-GLOBSEARCH.

Any capability tokens not defined in this specification MUST begin with "X-". Clients MUST ignore tokens they do not understand.

Currently defined tokens are:

  • STARTTLS

    • The server accepts STARTTLS as a signal to initiate a Transport Layer Security (TLS) connection. This capability contains no additional information.
      See [TLS], [TLS-IMAP-POP3-ACAP], and [TLS-SMTP].

  • AUTH=PLAIN

    • The server accepts PLAIN as an AUTH method. This capability contains no additional information.
      See [AUTH-PLAIN].

  • AUTH=ANONYMOUS

    • The server accepts ANONYMOUS as an AUTH method. This capability's additional information is an optional list of tags available for anonymous reading separated by commas.
      See [AUTH-ANONYMOUS].

  • SEND

    • The server allows messages to be sent to recipients. This capability's additional information is an optional list of the protocols which could be used to SEND messages.

  • MESSAGE-SIZE

    • Servers MAY advertise their maximum incoming message size using this capability. This capability's additional information is the maximum allowed size in octets of incoming messages (octets) in decimal format, without leading zeroes or other padding. A single "0" (zero) character means the server does not impose a static limit on message size. The absence of this capability in the resonse to the CAPS command does not necessarily mean the server imposes no limit on message size, but simply that any possible limit is not advertised. Clients MUST be prepared for a server to refuse the SEND command based on the message size.

4. States and Flow

A BikINI server is in one of five states.

  • Opening state
  • Index state
  • Message-Store state
  • Message-Retrieve state
  • Closed state

All commands except CAPS are valid in only one state. It is a protocol error for the client to attempt a command while the connection is in an inappropriate state, and the server will respond with a syntax error response as described above.

4.1. Opening State

This state is entered when the connection is established. The server silently waits for the client to send either AUTH or STARTTLS command to login.

A failed AUTH command leaves the session in the opening state. Servers MAY limit the number of client login attempts by dropping the connection after a number of failed AUTH commands.

A successful AUTH command puts the server into the index state.

A successful STARTTLS command begins TLS negotiation and returns the the server to the opening state. Servers may require a successful STARTTLS command prior to allowing the client to use the AUTH command.

A session timeout puts the server into the closed state.

To close the session without logging in, the client MAY drop the connection without sending a command.

4.2. Index State

This state is entered upon successful completion of the AUTH command. The client MUST issue the AUTH command before the server will grant access to the mail store.

Preliminary success of the PUT command puts the server into the message-store state.

Preliminary success of the GET or GETHDR command puts the server into the message-retrieve state.

A QUIT command or session timeout puts the server into the closed state.

4.3. Message-Store State

This state is entered upon preliminary success of the PUT command, indicated by a positive single-line response from the server.

Transfer of <content-size> octets of 8-bit data, the string "finished", and <LF> by the client, followed by a positive single-line response from the server, puts the server into index state.

A session timeout puts the server into the closed state.

4.4. Message-Retrieve State

This state is entered upon the preliminary success of the GET or GETHDR command, indicated by a positive single-line response from the server.

Transfer of <content-size> octets of 8-bit data followed by a positive single-line response puts the server into the index state.

A session timeout puts the server into the closed state.

4.5. Closed State

This state is entered upon a QUIT command, a session timeout, or a server arbitrarily closing the connection. In this state, the server will close the socket without sending a warning.

5. Data Format and Line Endings

Many legacy protocols such as [SMTP] use <CRLF> to end every line in the message transmission. However, most servers store messages natively using <LF>-only line endings. This means that messages must be encoded when transmitting a locally-stored message, and decoded when storing a received message.

This leads to corruption of messages, or creation of messages which cannot be safely transmitted over such a protocol. If a protocol receiver converts incoming <CRLF> line endings to <LF>-only to store the messages, and then converts the <LF>-only lines back to <CRLF> line endings when transmitting the message, a message containing a bare <LF> character will be corrupted.

The conversion is avoided by specifying that message content is sent as a length-prefixed 8-bit message. A message might not end with a newline. If a message is "lines of text", such as an [[IMF][]] message, it is transmitted using <LF>-only line endings. If a BikINI protocol receiver wishes to store a message locally using <CRLF> line endings, it must perform the conversion after receiving the message, and must convert the message back to <LF>-only line endings before calculating the message's size and retransmitting the message over BikINI.

6. Commands

Experimental commands must start with "X-". The Bikini protocol will never define a standard command starting with this prefix.

Additional whitespace is included in examples when the example would otherwise be difficult to read. In cases where examples do not exactly match the formal syntax, follow the formal syntax.

Unless otherwise specified the possible response to any command will be any single-line response.

Unless otherwise specified the text portion of a single-line response is optional, free form, and continues until the end of the line.

6.1. Opening State

6.1.1. STARTTLS

Function: Start Transport Layer Security (TLS).

Syntax: STARTTLS

Notes:

  • Upon initial connection, a client MAY immediately send the STARTTLS command to secure further communication.

  • The positive single-line response will not be given until a TLS connection is successfully established.

  • The negative single-line response will be given in any case where the TLS connection cannot be successfully established.

Examples:

C:  STARTTLS
C & S:  <negotiate a TLS session, and check result of negotiation>
S:  K TLS connection established

C:  STARTTLS
S:  U STARTTLS not supported

C:  STARTTLS
S:  E STARTTLS not allowed from your address

C:  STARTTLS
S:  AC server busy, closing connection

6.1.2. AUTH

Function: Log in.

Syntax: AUTH auth-method [auth-token] [...]

Possible Responses:

  • Positive single-line. The text portion of the response is required and contains a request for more data ("token?"), a server-supplied token, or confirmation the user has authenticated ("ok").

  • Negative single-line.

Notes:

  • The parameter is required. The and other parameters vary depending on the method.

  • If a server does not support a particular for any reason, it MUST return a U response. This includes the case where an authentication method is implemented but forbidden by policy (such as not allowing insecure auth methods in an unencrypted session).

  • Upon initial connection, a client MAY immediately send the AUTH command using an auth-method of its choosing. Only if the server returns U does the client need to either try another well-known auth-method or retrieve the list of supported auth methods with the CAPS command. Clients MAY choose to remember what auth methods are supported by a particular host so as to avoid choosing an unsupported auth-method in their next session.

  • Servers MUST NOT close the session using the C (connection close) response modifier immediately after a positive-response to the AUTH command. If the login would normally be successful but the server wishes to close the session, it must use the A response code.

Examples:

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  K ok

C:  AUTH PLAIN
S:  K token?
C:  dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  K ok

C:  AUTH ANONYMOUS
S:  K ok

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  U plain not allowed in unencrypted session

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  E authorization failed

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  AC sorry, server too busy, closing connection

6.1.3. CAPS

Function: List the capabilities of the server.

Syntax: CAPS

Possible Responses:

  • Positive multi-line.

    • The text portion of the continuation lines is required and strictly formatted as follows:
      "capability-token [ SP capability-info ... ]"

    • The text portion of the final K-line is required and strictly formatted as follows:
      "server-name SP server-version"

  • Negative single-line.

Notes:

  • A server with no additional capabilities MUST NOT send a negative response simply because it has no capabilities to list. It MUST respond with an empty capability list (i.e. a simple K success response).

Examples:

C:  CAPS
S:  + AUTH=ANONYMOUS
    + AUTH=PLAIN
    + SEND
    K ExServer 0.9.0

C:  CAPS
S:  K ExServer 0.9.0

C:  CAPS
S:  EC service not available, closing connection

6.2. Index State

6.2.1. CAPS

Function: List the capabilities of the server. See CAPS command in Opening State above.

6.2.2. QUIT

Function: End session and close connection.

Syntax: QUIT

Possible Responses:

  • None. The server will close the connection immediately. The client SHOULD close the connection as soon as it has finished sending the QUIT command.

Notes:

  • This command cannot fail. The server MUST close the connection when the QUIT command is received.

  • The QUIT command is only valid in the index state. If a client wishes to close a connection while in the opening state (i.e. before successfully logging in), it just closes the connection without sending a command.

Examples:

C:  QUIT
[connection closes]

6.2.3. LISTTAGS

Function: Retrieve list of tags in mail store.

Syntax: LISTTAGS

Possible Responses:

  • Positive multi-line.

    • The text portion of each continuation line is required and consists of a single tag.

  • Negative single-line.

Notes:

  • As every BikINI mail store (as presented to the client) must contain the "inbox" tag, a positive response to the LISTTAGS command will never have zero continuation + lines. Servers MUST include the "inbox" tag in LISTTAGS output.

  • The order of tags in the listings is undefined and not guaranteed to be stable. Servers MAY output the continuation lines in any order they choose. Servers MAY change the order of tags on each LISTTAGS command. Clients MUST NOT assume the order of tags will be the same between two LISTTAGS commands.

  • Servers MAY support multiple clients accessing the same mail store simultaneously. Tags can therefore be created, deleted, or renamed other than in response to commands by the client. Clients MUST NOT assume the tag list is stable. Clients MUST be prepared for a tag to be renamed or deleted in between the last LISTTAGS command and subsequent attempts to utilize that tag.

Examples:

C:  LISTTAGS
S:  + inbox
    + proj
    + a
    + b
    + to-read
    K ok

C:  LISTTAGS
S:  A error reading mail store, try again later

C:  LISTTAGS
S:  EC service not available, connection closing

6.2.5. RMTAG

Function: Remove a TAG from the mail store.

Syntax: RMTAG tag

Notes:

  • The tag is disassociated from all messages, thus removing it from the mail store.

Examples:

C:  RMTAG proj
S:  K ok, proj removed from all messages

C:  RMTAG proj
S:  E can't remove proj; no such tag

C:  RMTAG proj
S:  E can't remove proj; mail store is read-only

6.2.8. RENTAG

Function: Rename an existing tag.

Syntax: RENTAG old-tag new-tag

Notes:

  • If old-tag does not exist, then RENTAG fails.

  • If new-tag already exists, then RENTAG fails.

Examples:

C:  RENTAG a old-a
S:  K ok, a renamed to old-a

C:  RENTAG a old-a
S:  E cannot rename a to old-a; tag does not exist

C:  RENTAG a old-a
S:  E cannot rename a to old-a; tag exists

6.2.9. LISTMSGS

Function: List the contents of a mail folder.

Syntax: LISTMSGS folder-path

Possible Responses:

  • Positive multi-line. The text portion of the continuation lines is required and strictly formatted.

  • Negative single-line.

Notes:

  • The format of the text in each multi-line response line is 'message-identifier SP [flags] ":" content-size ":" message-timestamp'.

  • Clients MUST be prepared for messages to be created, deleted, moved, or to have their metadata changed by events other than a command from the client. Clients MUST NOT assume a message's existence, timestamp, flags, size, or other information will not change between the LISTMSGS command and a subsequent command.

  • The successful response to the LISTMSGS command on an empty mail folder will have zero continuation + lines and one success K line.

Examples:

C:  LISTMSGS proj/a
S:  + 1046273407.M181707P22689Q2.localhost :20030302T121535Z
    + 1045853047.5854.localhost RS:3191:19961225T150824Z
    K ok

C:  LISTMSGS proj/a
S:  E no such folder

C:  LISTMSGS inbox
S:  K ok

6.2.10. GET

Function: Retrieve a message.

Syntax: GET message-path

Examples:

C:  GET qmail/1046119237.13497.localhost
S:  K 4534
    [message content]
    K okay, message complete

6.2.11. GETHDR

Function: Retrieve a message's header portion.

Syntax: GETHDR message-path

Examples:

C:  GETHDR qmail/1046119237.13497.localhost
S:  K 812
    [message header content]
    K okay, message header complete

6.2.12. PUT

Function: Store a message in an existing mail folder.

Syntax: PUT folder-path content-size

Possible Responses:

  • Positive single-line. The text portion of the response is required and contains the identifier for the message's new location.

  • Negative single-line.

Notes:

  • Once the server has responded to the PUT command with a positive response, the client sends exactly <content-size> octets of 8-bit message data to the server, followed by "finished", then <LF>.

  • The new message identifier is returned in the successful response to the message content.

  • The negative response is returned if <content-block> is more or less than <content-size> octets of 8-bit data.

Examples:

C:  PUT qmail 5431
S:  K go ahead
C:  [message content]
    finished
S:  K 1046119237.13497.localhost

6.2.13. RMMSG

Function: Delete a message from a mail folder.

Syntax: RMMSG message-path

Examples:

C:  RMMSG proj/a/1046119237.localhost
S:  K ok, message removed

C:  RMMSG proj/a/1046119237.localhost
S:  E can't remove proj/a/1046119237.localhost, does not exist

C:  RMMSG proj/a/1046119237.localhost
S:  E can't remove proj/a/1046119237.localhost, mail store read-only

C:  RMMSG proj/a
S:  E can't remove proj/a, not a message

6.2.14. MVMSG

Function: Move an existing message from one mail folder to another.

Syntax: MVMSG message-path folder-path

Possible Responses:

  • Positive single-line. The text portion of the response is required and contains the message-identifier for the message's new location.

  • Negative single-line.

Notes:

  • The new message identifier is returned in the successful response.

Examples:

C:  MVMSG inbox/1045853047.5854.example.net proj/a
S:  K 1045853047.5854.localhost

6.2.15. SETFLAG

Function: Set a flag on a message in an existing mail folder.

Syntax: SETFLAG flag message-path

Examples:

C:  SETFLAG F inbox/1045853047.5854.example.net
S:  K flag F set on inbox/1045853047.5854.example.net

C:  SETFLAG P inbox/1045853047.5854.example.net
S:  E inbox/1045853047.5854.example.net does not exist

6.2.16. CLEARFLAG

Function: Clear a flag on a message in an existing mail folder.

Syntax: CLEARFLAG flag message-path

Examples:

C:  CLEARFLAG F inbox/1045853047.5854.example.net
S:  K flag F cleared from inbox/1045853047.5854.example.net

C:  CLEARFLAG P inbox/1045853047.5854.example.net
S:  E inbox/1045853047.5854.example.net does not exist

6.2.17. EXPUNGE

Function: Expunge all messages with the Trashed flag.

Syntax: EXPUNGE [folder-path]

Notes:

  • If the client includes <folder-path>, then only the Trashed messages in <folder-path> are removed. If the client does not include <folder-path>, then all Trashed messages are removed.

  • If the client sends EXPUNGE when no Trashed messages exist, then the server MUST respond with the successful completion response code.

Examples:

C:  EXPUNGE inbox
S:  K inbox expunged

C:  EXPUNGE
S:  K all trashed messages expunged

C:  EXPUNGE proj/a
S:  E proj/a does not exist

6.2.18. SEND

Function: Ask the BikINI server to send an existing message to one or more recipients.

Syntax: SEND message-path content-size

Notes:

  • Once the server has responded to the SEND command with a positive response, the client sends exactly <content-size> octets of 8-bit envelope data to the server, followed by "finished", followed by <LF>. The format of the envelope is an envelope sender address, followed by an ASCII NUL, followed by an envelope recipient address, followed by an ASCII NUL, optionally followed by additional envelope recipient addresses, each followed by an ASCII NUL.

  • All addresses in the envelope data MUST be in fully-qualified form.

  • The server MAY use any method it chooses to deliver the message to the recipients. This may be by handing the message to the local Mail Transfer Agent (MTA) for delivery via [SMTP] or another protocol, by delivering directly into another BikINI mail store on the server, other methods, or a combination of these methods.

  • Some delivery methods may impose stricter requirements on the format of a message than BikINI does. For example, messages with partial final lines (no trailing <LF>) cannot be transmitted as-is over [SMTP]. The server MAY choose to modify the message to enable it to be delivered (i.e. add a trailing newline to the message when it is delivered via [SMTP]), or it MAY choose to refuse to deliver the message. Should the server modify the message to enable it to be delivered, then the server MUST notify the client of the modification.

  • When delivering a message via [SMTP], a BikINI server MUST ensure that the message to be delivered meets at least the basic requirements of messages according to [RFC2822]. In particular, the server MUST ensure that the message header contains all required fields, and that all fields are syntactically correct. The server MAY choose to enforce stricter requirements (such as ensuring that the envelope sender or "From:" header addresses belong to the user).

Examples:

C:  SEND proj/a/1046273407.M181707P22689Q2.localhost 65
S:  K ok, send envelope of 65 octets
C:  sender@host.domain\0recip1@example.org\0recip2@example.net\0
    finished
S:  K ok, message sent to 2 recipient(s)

C:  SEND proj/a/1046273407.M181707P22689Q2.localhost 65
S:  E SEND not permitted

C:  SEND proj/a/1046273407.M181707P22689Q2.localhost 65
S:  K ok, send envelope of 65 octets
C:  sender@host.domain\0recip1@example.org\0recip2@example.net\0
    finished
S:  A error queuing message, try again later

C:  SEND proj/a/1046273407.M181707P22689Q2.localhost 65
S:  K ok, send envelope of 65 octets
C:  sender@host.domain\0recip1@example.org\0recip2@example.net\0
    finished
S:  E you're not "sender@host.domain"

7. Formal Syntax

The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [RFC5234] including the core rules in Appendix B.1, [RFC3339] section 5.6, and [RFC3696].

Except as noted otherwise, all ABNF literal text strings are case-sensitive. Implementations MUST accept these strings in a case-sensitive fashion.

7.1. Definitions

NONZERODIGIT        = %x31-39
    ; Digits one through nine.

NORMCHAR            = VCHAR / SP

NUL                 = %x00

FIN                 = %x66 %x69 %x6E %x69 %x73 %x68 %x65 %x64
    ; Lowercase "finished".

AUTH                = %x41 %x55 %x54 %x48

CAPS                = %x43 %x41 %x50 %x53

CLEARFLAG           = %x43 %x4C %x45 %x41 %x52 %x46 %x4C %x41 %x47

EXPUNGE             = %x45 %x58 %x50 %x55 %x4E %x47 %x45

GET                 = %x47 %x45 %x54

GETHDR              = %x47 %x45 %x54 %x48 %x44 %x52

LISTDIRS            = %x4C %x49 %x53 %x54 %x44 %x49 %x52 %x53

LISTMSGS            = %x4C %x49 %x53 %x54 %x4D %x53 %x47 %x53

MVMSG               = %x4D %x56 %x4D %x53 %x47

PUT                 = %x50 %x55 %x54

QUIT                = %x51 %x55 %x49 %x54

RENTAG              = %x52 %x45 %x4E %x54 %x41 %x47

RMTAG               = %x52 %x4D %x54 %x41 %x47

RMMSG               = %x52 %x4D %x4D %x53 %x47

SEND                = %x53 %x45 %x4E %x44

SETFLAG             = %x53 %x45 %x54 %x46 %x4C %x41 %x47

STARTTLS            = %x53 %x54 %x41 %x52 %x54 %x54 %x4C %x53

ANONYMOUS           = %x41 %x4E %x4F %x4E %x59 %x4D %x4F %x55 %x53

PLAIN               = %x50 %x4C %x41 %x49 %x4E

local-part-char     = %x21 / %x23-2B / %x2D-3F / %x41-5A / %x5E-7E
    ; See [[RFC3696][]].

domain-char         = %x2D / %x41-5A / %x61-7A
    ; See [[RFC3696][]].

envelope-spec       = 1*64( local-part-char ) %x40
                      1*255( domain-char )
    ; See [[RFC3696][]].

path-separator      = "/"
    ; May not occur in path components.

path-component-char = %x21-2E / %x30-7E
    ; All visible (printing) US-ASCII characters except
    ; <path-separator>.

path-component      = 1*path-component-char

directory-path      = path-component
                      *(path-separator path-component)

mail-folder         = path-component

folder-path         = *(path-component path-separator) mail-folder
    ; Complete path from mail store root to mail folder name.

auth-token          = 0*NORMCHAR
    ; No <CR> or <LF>, probably, but what about other nonprinting
    ; characters?

message-identifier  = path-component
    ; Unique within a mail folder, identifies a message.  Persistent
    ; as long as message remains in a given mail folder.

message-path        = folder-path path-separator message-identifier

directory-listing   = directory-path path-separator
    ; In LISTDIRS output, the trailing "/" indicates a directory
    ; instead of a mail folder.

flag                = %x44 / %x46 / %x4E / %x50 / %x52 / %x53 / %x54
    ; Current flags D, F, N, P, R, S, and T are uppercase only
    ; because of case-sensitivity, new flags can be added later
    ; using "r", etc.

flags               = [%x44] [%x46] [%x4E] [%x50] [%x52] [%x53]
                      [%x54]
    ; Flags are in any order but cannot be repeated.

content-size        =  NONZERODIGIT *DIGIT
    ; Positive nonzero decimal number with no leading zeroes.

content-block       = 1*OCTET
    ; Content-size octets of 8-bit data.

message-timestamp   = date-fullyear date-month date-mday %x54
                      time-hour time-minute time-second %x5A
    ; Example 20020304T134508Z.  The terms <date-fullyear>,
    ; <date-month>, <date-mday>, <time-hour>, <time-minute>,
    ; <time-second> are defined in [[RFC3339][]].

message-list-line   = message-identifier SP flags ":"
                      content-size ":" message-timestamp LF
    ; If a flag is present, that means it is set.  Note flags are
    ; optional and may not be present if all flags are cleared.

negative-code       = %x41 / %x45 / %x55 / %x58
    ; Single character negative response code A, E, U, or X.

response-code       = %x2B / %x4B / negative-code
    ; Single character response code +, K, A, E, U, or X.

response-modifier   = %x43
    ; Single character response modifier "C".  New modifiers may be
    ; added later.

response-modifiers  = [%x43]
    ; Modifiers in any order but cannot be repeated.

response-info       = *NORMCHAR
    ; Zero or more visible characters and spaces.

positive-response   = %x4B SP response-info LF

negative-response   = negative-code response-modifiers SP
                      response-info LF

one-line-response   = negative-response / positive-response

response-line       = response-code response-modifiers SP
                      response-info LF

continuation-line   = %x2B SP 1*NORMCHAR LF

multiline-response  = *continuation-line positive-response
    ; Zero or more continuation lines followed by a positive
    ; response.

command             = AUTH / CAPS / CLEARFLAG / EXPUNGE / GET /
                      GETHDR / LISTDIRS / LISTMSGS /
                      RENTAG / MVMSG / PUT / QUIT /
                      RMTAG / RMMSG / SEND / SETFLAG /
                      STARTTLS

auth-method         = ANONYMOUS / PLAIN

7.2. Commands

This section details the formal grammar for each command and possible responses to the command.

7.2.1. STARTTLS

starttls-command    = STARTTLS LF

starttls-response   = one-line-response

7.2.2. AUTH

auth-command        = AUTH SP auth-method *( SP auth-token ) LF

token-query         = %x74 %x6F %x6B %x65 %x6E %x3F

auth-response       = negative-response
auth-response       =/ %x4B SP ( ( token-query ) /
                       ( %x6F %x6B ) / ( auth-token ) ) LF

auth-client-token   = auth-token LF

7.2.3. CAPS

caps-command        = CAPS LF

caps-response       = negative-response
caps-response       =/ *continuation-line
                       %x2B SP 1*VCHAR SP 1*VCHAR LF
    ; "K Server-type Version"

7.2.4. QUIT

quit-command        = QUIT LF

7.2.5. LISTDIRS

listdirs-command    = LISTDIRS LF

inbox               = %x69 %x6E %x62 %x6F %x78

listdirs-response   = negative-response
listdirs-response   =/ %x2B SP inbox LF *( %x2B SP
                       ( folder-path / directory-listing )
                       LF ) positive-response

7.2.7. RMTAG

rmdir-command       = RMTAG SP tag LF

rmdir-response      = one-line-response

7.2.10. RENTAG

rentag-command      = RENTAG SP tag-name SP tag-name LF

rentag-response     = one-line-response

7.2.11. LISTMSGS

listmsgs-command    = LISTMSGS SP folder-path LF

listmsgs-response   = negative-response
listmsgs-response   =/ *(%x2B SP message-list-line)
                       positive-response

7.2.12. GET

get-command         = GET SP message-path LF

get-response        = negative-response
get-response        =/ %x4B SP content-size LF
                       content-block positive-response
    ; <content-block> is <content-size> octets of 8-bit data

7.2.13. GETHDR

gethdr-command      = GETHDR SP message-path LF

gethdr-response     = negative-response
gethdr-response     =/ %x4B SP content-size LF
                       content-block positive-response
    ; <content-block> is <content-size> octets of 8-bit data
    ; The header ends with <LF> <LF>.

7.2.14. PUT

put-command         = PUT SP folder-path SP content-size LF

put-cmd-response    = one-line-response

put-message         = content-block FIN LF
    ; <content-block> is <content-size> octets of 8-bit data
    ; Only used if server response is positive.

put-msg-response    = negative-response
put-msg-response    =/ %x4B SP message-identifier LF

7.2.15. RMMSG

rmmsg-command       = RMMSG SP message-path LF

rmmsg-response      = one-line-response

7.2.16. MVMSG

mvmsg-command       = MVMSG SP message-path SP folder-path LF

mvmsg-response      = negative-response
mvmsg-response      =/ %x4B SP message-identifier

7.2.17. SETFLAG

setflag-command     = SETFLAG SP flag SP message-path LF

setflag-response    = one-line-response

7.2.18. CLEARFLAG

clearflag-command   = CLEARFLAG SP flag SP message-path LF

clearflag-response  = one-line-response

7.2.19. EXPUNGE

expunge-command     = EXPUNGE [ SP folder-path ] LF

expunge-response    = one-line-response

7.2.20. SEND

send-command        = SEND SP message-path SP content-size LF

send-envelope       = 2*( envelope-spec NUL ) FIN LF
    ; Only used if server response to <send-command> is positive.

send-response       = one-line-response
    ; Response to either SEND or the envelope data.

8. Examples

Here are examples of various BikINI commands and responses.

8.1. Log in and retrieve new messages from default incoming mail folder

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  K ok
C:  LISTMSGS inbox
S:  + 1044547809.6813_14.example.net S:13626:20010327T115332Z
    + 1045443985.21669.example.com S:3602:19980812T165402Z
    + 1044557867.9263_7.example.net NRS:15820:20030227T193206Z
    + 1044561911.10174_19.example.net S:5476:20021108T081428Z
    + 1044566229.11296_22.example.net NS:6644:20030227T195712Z
    K ok
C:  GET inbox/1044557867.9263_7.example.net
S:  K 15820
    Return-Path: <bob@familias.example.net>
    Delivered-To: jeff@familias.example.net
    [...]
    Hi, Jeff.  How's the wife and kids?
    [...]
    K ok
C:  GET inbox/1044566229.11296_22.example.net
S:  K 6644
    Return-Path: <tech@support.example.com>
    Delivered-To: web-test@discworld.dyndns.org
    [...]
    This is a message.
    [...]
    K ok
C:  LISTMSGS inbox
S:  + 1044547809.6813_14.example.net S:13626:20010327T115332Z
    + 1045443985.21669.example.com S:3602:19980812T165402Z
    + 1044557867.9263_7.example.net RS:15820:20030227T193206Z
    + 1044561911.10174_19.example.net S:5476:20021108T081428Z
    + 1044566229.11296_22.example.net S:6644:20030227T195712Z
    K ok
C:  QUIT
S:  K goodbye

Note that the message's N (new) flag is cleared by the successful GET operation.

8.2. Log in, retrieve the mail store structure, and store a message

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  K ok
C:  LISTDIRS
S:  + foobar/
    + foobar/pymsgauth
    + foobar/foobaz/
    + foobar/foobaz/mail
    + im2000
    + inbox
    + list-getmail
    + newfolder
    + nullmailer
    + python
    + qmail
    + spam
    K ok
C:  LISTMSGS foobar/pymsgauth
S:  K ok
C:  PUT foobar/pymsgauth 3534
S:  K go ahead
C:  Return-Path: <bikini@discworld.dyndns.org>
    To: "bikini mailing list" <bikini-devel@discworld.dnsalias.org>
    Subject: Development of mutt client backend
    [...]
    finished
S:  K 1045443985.Q49P3838.example.com
C:  LISTMSGS foobar/pymsgauth
S:  + 1045443985.Q49P3838.example.com :3534:20030304T113829Z
    K ok
C:  SETFLAG F foobar/pymsgauth/1045443985.Q49P3838.example.com
S:  K flag set
C:  LISTMSGS foobar/pymsgauth
S:  + 1045443985.Q49P3838.example.com F:3534:20030304T113829Z
    K ok
C:  QUIT
S:  K goodbye

The client retrieves the directory structure, lists the contents of mail folder "foobar/pymsgauth" (which is empty), stores a message (and is given the message identifier for the stored message by the server), lists the contents of the mail folder again (showing the message), sets the F flag on the message, lists the contents of the mail folder again (showing the flag is now set), and quits.

Note that the message's N (new) flag is initially clear when stored with the PUT command.

8.3. Log in procedure, including failed login due to unsupported method

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  U AUTH PLAIN unsupported
C:  CAPS
S:  + STARTTLS
    + MESSAGE-SIZE 2000000
    K ExServer 0.9.0
C:  STARTTLS
C & S:  <negotiate a TLS session, and check result of negotiation>
S:  K TLS connection established
C:  CAPS
S:  + AUTH=PLAIN
    + MESSAGE-SIZE 2000000
    K ExServer 0.9.0
C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
[...]

The client connects and tries to log in using "AUTH PLAIN". The server forbids this in unencrypted sessions (no STARTTLS), so it replies with U "unsupported". The client issues the STARTTLS command to encrypt this session, then issues the CAPS command to retrieve a list of supported AUTH methods and logs in using one of them.

Note that the server's CAPS response might be different after the STARTTLS command, as it might then include options such as "AUTH=PLAIN", and will not include STARTTLS.

9. Security Considerations

Security considerations about the email system and messages are covered in [RFC5322] and [SMTP], and will not be covered here.

9.1. Authentication

Authentication of the user is accomplished using a variety of AUTH methods including PLAIN. Implementations of this protocol SHOULD NOT allow PLAIN authentication unless the connection has been secured via a successfull STARTTLS command.

9.2. STARTTLS

Implementors MUST implement the TLS_RSA_WITH_RC4_128_SHA CipherSuite, and SHOULD implement the TLS_DHE_DSS_WITH_AES_256_CBC_SHA CipherSuite. This will ensure all compliant clients and servers can interoperate.

9.3. Announcements

While some small amount of security may be gained through hiding the server type and version, use of that data when debugging problems is beyond dispute. BikINI servers MUST make their type and version information available in the final K-line of the CAPS command response.

10. IANA Considerations

As a port will be used to access any servers using this protocol, the author requests assignment of TCP port 40 to BikINI.

11. References

11.1. Normative

[AUTH-ANONYMOUS] Zeilenga, K., "Anonymous Simple Authentication and Security Layer (SASL) Mechanism", RFC 4505, OpenLDAP Foundation, June 2006.
[AUTH-PLAIN] Zeilenga, K., "The PLAIN Simple Authentication and Security Layer (SASL) Mechanism", RFC 4616, OpenLDAP Foundation, August 2006.
[RFC2119] Bradner, "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, Harvard University, March 1997.
[RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: Timestamps", RFC 3339, July 2002.
[RFC3696] "Application Techniques for Checking and Transformation of Names" J. Klensin, RFC 3696, February 2004.
[RFC5234] Crocker, D., Ed., and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.
[RFC5322] Resnick, P., "Internet Message Format", STD 11, RFC 5322, QUALCOMM Incorporated, October 2008.
[TLS] Dierks, T. and E. Rescorla, "The Transport Layer Security (TLS) Protocol Version 1.2", RFC 5246, August 2008.
[TLS-IMAP-POP3-ACAP] Newman, C., "Using TLS with IMAP, POP3 and ACAP", RFC 2595, June 1999.
[TLS-SMTP] Hoffman, P., "SMTP Service Extension for Secure SMTP over Transport Layer Security", RFC 3207, February 2002.

11.2. Informative

[IMAP] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.
[POP3] Myers, J. and M. Rose, "Post Office Protocol - Version 3", STD 53, RFC 1939, May 1996.
[RFC2822] Resnick, P., Ed., "Internet Message Format", RFC 2822, April 2001.
[SASL] Melnikov, A., and Zeilenga, K., "Simple Authentication and Security Layer (SASL)", RFC 4422, Isode Limited and OpenLDAP Foundation, June 2006.
[SMTP] Klensin, J., "Simple Mail Transfer Protocol", RFC 5321, October 2008.

12. Acknowledgments

The author would like to thank Charles Cazabon for creating the original specification, server, and test client.

The author would also like to thank Bruce Guenter (contributor of the SASL code), Cory Wright, Jake Baillie, Russell Nelson, Mark Delany, and many others for contributions of code, documentation, document templates, valuable discussion, and other items to this project.

Finally, Chris Garrigues provided the quote which became the original motto for the project.

"IMAP sucks, but there really isn't anything else that even tries to do what IMAP does so we hate it and we live with it."

Author's Address

David J. Weller-Fahy
10009 Forest Lane
Midwest City, OK 73130
USA

EMail: dave@weller-fahy.com