BikINI/ specification

The initial version of this specification is not complete until any issues are done.

The shell script I use to convert this document from markdown to html to internet draft format is detailed at markdown to html to internet-draft.

Abstract

BikINI is designed to allow multiple client computers to access a mail message store on a central server. It operates over any reliable datastream protocol, but is commonly used over TCP. BikINI is designed to require minimal message parsing by the server, to be 8-bit clean for mail message content, and to be a lighter-weight and easier-to-implement than other remote message access protocols.

Table of Contents

   1. Introduction
   2. Requirement Words
   3. Protocol Overview
      3.1. Link Level
      3.2. Commands and Responses
      3.3. Mail Store Architecture
      3.4. Message Attributes
      3.5. Capability Tokens
   4. States and Flow
      4.1. Opening State
      4.2. Index State
      4.3. Message-Store State
      4.4. Message-Retrieve State
      4.5. Closed State
   5. Data Format and Line Endings
   6. Commands
      6.1. Opening State
      6.2. Index State
   7. Formal Syntax
      7.1. Definitions
      7.2. Commands
   8. Examples
   9. Security Considerations
   10. IANA Considerations
   11. References
   12. Acknowledgments
   Author's Address

1. Introduction

The objective of BikINI is to provide a remote mail storage and access protocol which is easier to implement than [IMAP] and more suitable for remote storage of messages than [POP3]. To that end, multiple design decisions were made to keep the protocol as simple as possible.

1.1. Design Considerations

Server-side message parsing is resource-intensive and complex, and thus should be avoided. This precludes functionality like retrieval of single parts of MIME multipart messages, but makes implementation significantly simpler and reduces the resource-intensiveness of the server.

Case insensitivity is no longer a necessary feature, and adds code for little reason otherwise. BikINI is a new protocol and doesn't require it, therefore commands and responses in BikINI are case-sensitive. Clients MUST send commands in upper case; upper and lower case characters MAY appear in identifiers, paths, and flags. The server MAY include upper and lower case characters in message identifiers, response modifiers, and descriptive text responses.

Allowing arbitrary runs of whitespace around commands and arguments is not necessary, and adds code to clients and servers for little purpose. BikINI clients and servers MUST NOT add padding of any type (including extra whitespace) to commands, arguments, or responses other than that explicitly stated in this protocol specification.

High numbers of active clients must be supported. In particular, the immediate disconnect response modifier removes the need for large numbers of idle connections on such systems.

1.2. Terminology

To minimize confusion, the following definitions hold throughout this document.

In examples, "C:" and "S:" indicate lines sent by the client and server respectively.

Lines are defined as a series of characters excluding <LF>, and terminated by <LF>.

Characters are 7-bit US-ASCII unless otherwise specified.

User refers to a human user.

Client refers to the software being run by the user.

Session refers to the entire sequence of client commands and server responses from the initial establishment of the network connection until its termination.

The mail store is the top-level container accessed by a session.

A directory is a container that contains zero or more directories, or zero or more mail folders, or both.

A folder (or mail folder) is a container that contains zero or more messages.

A message is arbitrary 8-bit data (servers and clients SHOULD store and send [RFC2822] and [RFC5322] email messages).

2. Requirement Words

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

3. Protocol Overview

3.1. Link Level

The BikINI protocol requires a reliable data stream as an underlying link level protocol. This may be TCP/IP or another protocol. When operating over TCP/IP, BikINI is frequently used on TCP port ???.

3.2. Commands and Responses

A BikINI connection consists of the establishment of a client/server network connection, an initial greeting from the server, and client/server interactions. These client/server interactions consist of a client command, server data, and a server completion result response.

All interactions transmitted by client and server are in the form of strings that end with <LF> and may not contain <LF>.

When message content (either a complete message, or only the header portion of a message) is being transferred in either direction the interaction is 8-bit data. When the message is an [RFC2822] message (expected to be all of the time at time of deployment of this protocol), lines in an individual message are terminated with Unix-style <LF> only.

The protocol receiver of a BikINI client or server is either reading a line, or is reading a sequence of octets with a known count followed by a line.

3.2.1. Client Protocol Sender and Server Protocol Receiver

The client command begins an operation.

There are two cases in which a line from the client does not represent a complete command. Those are when the client requires preliminary approval before completing the PUT operation, and when the client is authenticating via an AUTH method requiring an exchange of authentication tokens.

In this case, the server sends a K reply indicating the client may proceed to send its message data or authentication token(s), or an A, E, U, or X reply indicating the command cannot be processed.

3.2.2. Server Protocol Sender and Client Protocol Receiver

Data transmitted by the server to the client takes one of three forms.

The server MUST NOT send any data to the client except in response to a command.

With the exception of certain commands defined later, the client MUST NOT derive any information from the text portion of a A, E, K, U, or X single-line response, other than an optional informational message to display to the user.

3.2.3. Server Response Codes

Response codes are a single character, followed immediately by zero or more response modifiers, followed by a single space and zero or more characters of additional information. The server MUST NOT send a space between the response code and response modifiers.

The additional information is optional for some response codes (defined below). If the additional information is required (non-optional), it has a strict format defined by the command in question.

When the additional information section is optional, it is free-form and continues until the end of the line.

The server MUST send a space between the response code and the <LF> which terminates the response line if the additional information section is not supplied.

3.2.4. Server Response Modifiers

Server reponses MAY include a non-repeating sequence of one or more single-character response modifiers immediately following the response code (i.e. not preceded by a space).

Servers MUST NOT send a response modifier which is not defined in this specification. Servers MUST NOT send the same response modifier more than once in a given response. Servers MAY send multiple response modifiers in any order. Clients MUST NOT expect response modifiers to be in any particular order.

The following response modifiers are defined.

3.3. Mail Store Architecture

Mail folders can only contain messages, not directories or other mail folders. The root of a BikINI mail store is a directory with the path '' (i.e. the empty path). Mail folders are identified by path relative to the root of the mail store (i.e. "directory/folder"). Clients MUST NOT include a trailing slash when identifying a mail folder to a server. Servers MUST NOT include a trailing slash on mail folder paths in the response to the LISTDIRS command. Note that the term folder is chosen to be neutral to storage formats, as servers are free to implement mail folders using any method they choose.

Directories can only contain mail folders and other directories. Directories are identified by path relative to the root of the mail store (i.e. "directory" or "dir1/dir2"). Clients MUST NOT include a trailing slash when identifying a directory to a server. Servers MUST include a trailing slash on directory paths in the response to the LISTDIRS command to differentiate them from mail folders. The directory structure of a mail store MAY correspond to a filesystem structure on the server.

BikINI directories look like filesystem directories to the client. Mail folders look like opaque objects which contain messages. The whole BikINI mail store maps cleanly to a filesystem tree, making obvious implementations easy.

A mail store MAY contain directories, and it MUST contain at least one mail folder (the "inbox"). The "inbox" represents the default incoming mail directory of the user. Servers MAY logically map this to a mail folder physically outside of the rest of the mail store, such as the user's mail spool. The root of the mail store has the path '' (i.e., the empty path).

3.4. Message Attributes

In the case of messages delivered via the MVMSG command only the Message Identifier MAY be changed, all other attributes MUST remain the same.

3.4.1. Message Identifier

An arbitrary length token assigned to each message. Message identifiers MUST NOT contain any whitespace characters.

Messages in BikINI are accessed strictly by means of their folder and identifier. This identifier is guaranteed not to refer to any other message in the mail folder.

Unique identifiers persist as long as a given message resides in a particular folder (i.e. the server MAY change the message identifier if it is moved to another folder).

3.4.2. Flags Message Attribute

A list of zero or more boolean variables assigned to a message.

The following flags are defined.

The client may set or clear any flag.

The GET command clears the N flag of the message in question as a side-effect.

Multiple clients have to cooperate when accessing the same mail store; they have to agree on the meanings of some flags (like N, R, S, etc) for them to be useful. Clients SHOULD NOT set or clear flags in an arbitrary way; if a given client just arbitrarily clears the R flag on all messages, then that flag useless to other clients accessing the same mail store.

3.4.3. Internal Timestamp Message Attribute

The internal timestamp of the message on the server. This is a date and time which reflects when the message was received by the BikINI server.

If the message is received by the BikINI server, then it uses the timestamp the message was received as the internal timestamp.

If the message is not received by the BikINI server (i.e. the message is delivered to the filesystem mailbox underlying the mail folder), then the BikINI server will base the internal timestamp on when the new message was detected and what type of mailbox underlies the mail folder (i.e. maildir, mbox, mix, etc.).

The format used is similar to that of [RFC3339] without punctuation, "YYYYMMDDThhmmssTZD", with a timezone offset indicator of "Z" to indicate UTC (-0000).

Examples:

    19970716T192030Z
    20030301T120000Z

Servers MUST NOT use any other timezone offset when presenting the timestamp messages attribute to a client.

3.4.4. Message Size Attribute

The number of octets in the message (for GET, PUT, and LISTMSGS commands, or the number of octets in the message header for the GETHDR command), expressed as a positive nonzero decimal with no leading zeroes.

If the sender (client or server) of a size attribute stores an [RFC2822] message using an end-of-line convention other than Unix-style <LF>-only, the sender MUST convert the message to <LF>-only before calculating the message's size.

3.4.5. Envelope Recipient Attribute

The fully qualified address of the recipient of the message, at the time of delivery.

If delivered via the PUT command, the envelope recipient is the address in the top-most "Delivered-To:" field of the message headers. If there is no "Delivered-To:" field, then the envelope recipient attribute will remain unset.

If delivered directly to the underlying mailbox, the envelope recipient MUST be stored in the top-most "Delivered-To:" field of the message headers.

3.4.6. Envelope Sender Attribute

The fully qualified address of the sender of the message, at the time of delivery.

If delivered via the PUT command, the envelope recipient is the address in the "Return-Path:" trace field of the message headers. If there is no "Return-Path:" header, then the envelope recipient attribute may remain unset.

If delivered directly to the underlying mailbox, the envelope sender MUST be stored in the "Return-Path:" trace field of the message headers.

3.5. Capability Tokens

BikINI is designed to be an extensible protocol; clients should continue to work even if servers are upgraded to support new protocol features and vice versa. To accomplish this, clients need a way to determine what capabilities are supported by a given server.

The CAPS command provides this information. The server's response is a multiline response, with one line per capability. The format of each line is a single capability token, optionally followed by additional information in a format specific to that token.

Servers MAY implement experimental or nonstandardized capabilities which are not listed in this specification using capability tokens beginning with "X-" (i.e. "X-FOO"). The BikINI protocol will never define a standard capability token beginning with "X-". I encourage client and server authors to register their "X-..." tokens with me to prevent collisions. Authors should implement experimental commands with the "X-" prefix and use the command name as the capability token representing the command. For example, an experimental glob search command might use the command X-GLOBSEARCH and be advertised with the capability token X-GLOBSEARCH.

Servers MUST NOT send any capability tokens not defined in this specification with the exception of tokens beginning with "X-" as stated above. Clients MUST ignore tokens they do not understand.

Currently defined tokens are:

4. States and Flow

A BikINI server is in one of five states.

All commands except CAPS are valid in only one state. It is a protocol error for the client to attempt a command while the connection is in an inappropriate state, and the server will respond with a syntax error response as described above.

4.1. Opening State

This state is entered when the connection is established. The server does not issue a banner or any other greeting, but silently waits for the client to send the AUTH command to login.

A failed AUTH command leaves the session in the opening state. Servers MAY limit the number of login attempts a client may attempt by dropping the connection after a number of unsuccessful AUTH` commands.

A successful AUTH command puts the server into the index state.

A successful STARTTLS command begins TLS negotiation and returns the the server to the opening state.

A session timeout puts the server into the closed state.

To close the session without logging in, the client MAY drop the connection without sending a command.

4.2. Index State

This state is entered upon successful completion of the AUTH command. The client MUST issue the AUTH command before the server will grant access to the mail store.

Preliminary success of the PUT command puts the server into the message-store state.

Preliminary success of the GET or GETHDR command puts the server into the message-retrieve state.

A QUIT command or session timeout puts the server into the closed state.

4.3. Message-Store State

This state is entered upon preliminary success of the PUT command, indicated by a positive response by the server.

Transfer of <content-size> octets of 8-bit data, the string "finished", and <LF> by the client, followed by a response from the server, puts the server into index state.

A session timeout puts the server into the closed state.

4.4. Message-Retrieve State

This state is entered upon the preliminary success of the GET or GETHDR command, indicated by a "K <SP> <content-size> <LF>" response from the server.

Transfer of <content-size> octets of 8-bit data followed by a positive response puts the server into the index state.

A session timeout puts the server into the closed state.

4.5. Closed State

This state is entered upon a QUIT command, a session timeout, or a server arbitrarily closing the connection. In this state, the server will close the socket without sending a warning.

5. Data Format and Line Endings

Many legacy protocols such as [SMTP] use <CRLF> to end every line in the message transmission. However, most servers store messages natively using <LF>-only line endings. This means that messages must be encoded when transmitting a locally-stored message, and decoded when storing a received message.

This leads to corruption of messages, or creation of messages which cannot be safely transmitted over such a protocol. If a protocol receiver converts incoming <CRLF> line endings to <LF>-only to store the messages, and then converts the <LF>-only lines back to <CRLF> line endings when transmitting the message, a message containing a bare <LF> character will be corrupted.

The conversion is avoided by specifying that message content is sent as a length-prefixed 8-bit message. A message might not end with a newline. If a message is "lines of text", such as an [RFC2822] message, it is transmitted using <LF>-only line endings. If a BikINI protocol receiver wishes to store a message locally using <CRLF> line endings, it must perform the conversion after receiving the message, and must convert the message back to <LF>-only line endings before calculating the message's size and retransmitting the message over BikINI.

6. Commands

Experimental commands must start with "X-". The Bikini protocol will never define a standard command starting with this prefix.

Additional whitespace is included in examples when the example would otherwise be difficult to read. In cases where examples do not exactly match the formal syntax, follow the formal syntax.

6.1. Opening State

6.1.1. STARTTLS

Function: Start Transport Layer Security (TLS).

Syntax: STARTTLS

Possible Responses:

Notes:

Examples:

C:  STARTTLS
C & S:  <negotiate a TLS session, and check result of negotiation>
S:  K TLS connection established

C:  STARTTLS
S:  U STARTTLS not supported

C:  STARTTLS
S:  E STARTTLS not allowed from your address

C:  STARTTLS
S:  AC server busy, closing connection

6.1.2. AUTH

Function: Log in.

Syntax: AUTH auth-method [auth-token] [...]

Possible Responses:

Notes:

Examples:

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  K ok

C:  AUTH PLAIN
S:  K token?
C:  dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  K ok

C:  AUTH ANONYMOUS
S:  K ok

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  U plain not allowed in unencrypted session

C:  AUTH LOGIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  U login auth-method not available

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  E authorization failed

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  AC sorry, server too busy, closing connection

6.1.3. CAPS

Function: List the capabilities of the server.

Syntax: CAPS

Possible Responses:

Notes:

Examples:

C:  CAPS
S:  + AUTH=ANONYMOUS
    + AUTH=PLAIN
    + AUTH=KERBEROS_V5
    + SEND
    K Example 0.9.0

C:  CAPS
S:  K Example 0.9.0

C:  CAPS
S:  EC service not available, closing connection

6.2. Index State

6.2.1. CAPS

Function: List the capabilities of the server. See CAPS command in Opening State above.

6.2.2. QUIT

Function: End session and close connection.

Syntax: QUIT

Possible Responses:

Notes:

Examples:

C:  QUIT
[connection closes]

6.2.3. LISTDIRS

Function: Retrieve list of directories and mail folders in mail store.

Syntax: LISTDIRS

Possible Responses:

Notes:

Examples:

C:  LISTDIRS
S:  + inbox
    + proj/
    + proj/a
    + proj/b
    + empty-directory/
    K ok

C:  LISTDIRS
S:  A error reading mail store, try again later

C:  LISTDIRS
S:  EC service not available, connection closing

6.2.4. MKDIR

Function: Create a new directory in the mail store.

Syntax: MKDIR directory-path

Possible Responses:

Notes:

Examples:

C:  MKDIR proj/a
S:  K ok, directory created

C:  MKDIR proj/a
S:  E directory projects does not exist

C:  MKDIR shared/business-plan
S:  A error creating directory, try again later

6.2.5. RMDIR

Function: Delete a directory in the mail store.

Syntax: RMDIR directory-path

Possible Responses:

Notes:

Examples:

C:  RMDIR proj/a
S:  K ok, directory removed

C:  RMDIR proj/a
S:  E can't remove proj/a; directory not empty

C:  RMDIR proj/b
S:  E can't remove proj/b; not a directory

C:  RMDIR proj/a
S:  E can't remove proj/a; mail-store is read-only

6.2.6. MKFOLDER

Function: Create a new mail folder in the mail store.

Syntax: MKFOLDER folder-path

Possible Responses:

Notes:

Examples:

C:  MKFOLDER proj/a/archive-2002-10
S:  K ok, folder created

C:  MKFOLDER proj/a/archive-2002-10
S:  E cannot create proj/a/archive-2002-10, proj/a doesn't exist

C:  MKFOLDER sent-mail/daveh@dnh.example.net
S:  E mail-store is read-only

6.2.7. RMFOLDER

Function: Delete a mail folder from the mail store.

Syntax: RMFOLDER folder-path

Possible Responses:

Notes:

Examples:

C:  RMFOLDER proj/a/archive-2002-10
S:  K ok, folder deleted

C:  RMFOLDER proj/a/archive-2002-10
S:  E folder doesn't exist

C:  RMFOLDER shared-folders/daveh@dnh.example.net
S:  E shared directory is read-only

6.2.8. MVFOLDER

Function: Move a mail folder from one directory to another, or rename a mail folder.

Syntax: MVFOLDER old-folder-path new-folder-path

Possible Responses:

Examples:

C:  MVFOLDER proj/a archive/old-proj/a
S:  K ok, folder moved

C:  MVFOLDER proj/dogfood-super proj/dogfood-supper
S:  K ok, folder renamed

C:  MVFOLDER proj/a archive/old-proj/alpha
S:  E cannot move proj/a ; archive/old-proj/ doesn't exist

C:  MVFOLDER shared/daveh@example.net shared/daveh@corp.example.net
S:  E shared directory is read-only

6.2.9. LISTMSGS

Function: List the contents of a mail folder.

Syntax: LISTMSGS folder-path

Possible Responses:

Notes:

Examples:

C:  LISTMSGS proj/a
S:  + 1046273407.M181707P22689Q2.localhost :20030302T121535Z
    + 1045853047.5854.localhost RS:3191:19961225T150824Z
    K ok

C:  LISTMSGS proj/a
S:  E no such folder

C:  LISTMSGS inbox
S:  K ok

6.2.10. GET

Function: Retrieve a message.

Syntax: GET message-path

Possible Responses:

Examples:

C:  GET qmail/1046119237.13497.localhost
S:  K 4534
    [message content]
    K okay, message complete

6.2.11. GETHDR

Function: Retrieve a message's header portion.

Syntax: GETHDR message-path

Possible Responses:

Examples:

C:  GETHDR qmail/1046119237.13497.localhost
S:  K 812
    [message header content]
    K okay, message header complete

6.2.12. PUT

Function: Store a message in an existing mail folder.

Syntax: PUT folder-path content-size

Possible Responses:

Notes:

Examples:

C:  PUT qmail 5431
S:  K go ahead
C:  [message content]
    finished
S:  K 1046119237.13497.localhost

6.2.13. RMMSG

Function: Delete a message from a mail folder.

Syntax: RMMSG message-path

Possible Responses:

Examples:

C:  RMMSG proj/a/1046119237.localhost
S:  K ok, message removed

C:  RMMSG proj/a/1046119237.localhost
S:  E can't remove proj/a/1046119237.localhost, does not exist

C:  RMMSG proj/a/1046119237.localhost
S:  E can't remove proj/a/1046119237.localhost, mail store read-only

C:  RMMSG proj/a
S:  E can't remove proj/a, not a message

6.2.14. MVMSG

Function: Move an existing message from one mail folder to another.

Syntax: MVMSG message-path folder-path

Possible Responses:

Notes:

Examples:

C:  MVMSG inbox/1045853047.5854.example.net proj/a
S:  K 1045853047.5854.localhost

6.2.15. SETFLAG

Function: Set a flag on a message in an existing mail folder.

Syntax: SETFLAG flag message-path

Possible Responses:

Examples:

C:  SETFLAG F inbox/1045853047.5854.example.net
S:  K flag F set on inbox/1045853047.5854.example.net

C:  SETFLAG P inbox/1045853047.5854.example.net
S:  E inbox/1045853047.5854.example.net does not exist

6.2.16. CLEARFLAG

Function: Clear a flag on a message in an existing mail folder.

Syntax: CLEARFLAG flag message-path

Possible Responses:

Examples:

C:  CLEARFLAG F inbox/1045853047.5854.example.net
S:  K flag F cleared from inbox/1045853047.5854.example.net

C:  CLEARFLAG P inbox/1045853047.5854.example.net
S:  E inbox/1045853047.5854.example.net does not exist

6.2.17. EXPUNGE

Function: Expunge all messages with the Trashed flag.

Syntax: EXPUNGE [folder-path]

Possible Responses:

Notes:

Examples:

C:  EXPUNGE inbox
S:  K inbox expunged

C:  EXPUNGE
S:  K all trashed messages expunged

C:  EXPUNGE proj/a
S:  E proj/a does not exist

6.2.18. SEND

Function: Ask the BikINI server to send an existing message to one or more recipients.

Syntax: SEND message-path content-size

Possible Responses:

Notes:

Examples:

C:  SEND proj/a/1046273407.M181707P22689Q2.localhost 65
S:  K ok, send envelope of 65 octets
C:  sender@host.domain\0recip1@example.org\0recip2@example.net\0
    finished
S:  K ok, message sent to 2 recipient(s)

C:  SEND proj/a/1046273407.M181707P22689Q2.localhost 65
S:  E SEND not permitted

C:  SEND proj/a/1046273407.M181707P22689Q2.localhost 65
S:  K ok, send envelope of 65 octets
C:  sender@host.domain\0recip1@example.org\0recip2@example.net\0
    finished
S:  A error queuing message, try again later

C:  SEND proj/a/1046273407.M181707P22689Q2.localhost 65
S:  K ok, send envelope of 65 octets
C:  sender@host.domain\0recip1@example.org\0recip2@example.net\0
    finished
S:  E you're not "sender@host.domain"

7. Formal Syntax

The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [RFC5234] including the core rules in Appendix B.1, [RFC3339] section 5.6, and [RFC3696].

Except as noted otherwise, all ABNF literal text strings are case-sensitive. Implementations MUST accept these strings in a case-sensitive fashion.

7.1. Definitions

NONZERODIGIT        = %x31-39
    ; Digits one through nine.

NORMCHAR            = VCHAR / SP

NUL                 = %x00

FIN                 = %x66 %x69 %x6E %x69 %x73 %x68 %x65 %x64
    ; Lowercase "finished".

AUTH                = %x41 %x55 %x54 %x48

CAPS                = %x43 %x41 %x50 %x53

CLEARFLAG           = %x43 %x4C %x45 %x41 %x52 %x46 %x4C %x41 %x47

EXPUNGE             = %x45 %x58 %x50 %x55 %x4E %x47 %x45

GET                 = %x47 %x45 %x54

GETHDR              = %x47 %x45 %x54 %x48 %x44 %x52

LISTDIRS            = %x4C %x49 %x53 %x54 %x44 %x49 %x52 %x53

LISTMSGS            = %x4C %x49 %x53 %x54 %x4D %x53 %x47 %x53

MKDIR               = %x4D %x4B %x44 %x49 %x52

MKFOLDER            = %x4D %x4B %x46 %x4F %x4C %x44 %x45 %x52

MVFOLDER            = %x4D %x56 %x46 %x4F %x4C %x44 %x45 %x52

MVMSG               = %x4D %x56 %x4D %x53 %x47

PUT                 = %x50 %x55 %x54

QUIT                = %x51 %x55 %x49 %x54

RMDIR               = %x52 %x4D %x44 %x49 %x52

RMFOLDER            = %x52 %x4D %x46 %x4F %x4C %x44 %x45 %x52

RMMSG               = %x52 %x4D %x4D %x53 %x47

SEND                = %x53 %x45 %x4E %x44

SETFLAG             = %x53 %x45 %x54 %x46 %x4C %x41 %x47

STARTTLS            = %x53 %x54 %x41 %x52 %x54 %x54 %x4C %x53

ANONYMOUS           = %x41 %x4E %x4F %x4E %x59 %x4D %x4F %x55 %x53

DIGEST-MD5          = %x44 %x49 %x47 %x45 %x53
                      %x54 %x2D %x4D %x44 %x35

GSSAPI              = %x47 %x53 %x53 %x41 %x50 %x49

KERBEROS_V5         = %x4B %x45 %x52 %x42 %x45 %x52
                      %x4F %x53 %x5F %x56 %x35

PLAIN               = %x50 %x4C %x41 %x49 %x4E

SCRAM-SHA1          = %x53 %x43 %x52 %x41 %x4D
                      %x2D %x53 %x48 %x41 %x31

local-part-char     = %x21 / %x23-2B / %x2D-3F / %x41-5A / %x5E-7E
    ; See [[RFC3696][]].

domain-char         = %x2D / %x41-5A / %x61-7A
    ; See [[RFC3696][]].

envelope-spec       = 1*64( local-part-char ) %x40
                      1*255( domain-char )
    ; See [[RFC3696][]].

path-separator      = "/"
    ; May not occur in path components.

path-component-char = %x21-2E / %x30-7E
    ; All visible (printing) US-ASCII characters except
    ; <path-separator>.

path-component      = 1*path-component-char

directory-path      = path-component
                      *(path-separator path-component)

mail-folder         = path-component

folder-path         = *(path-component path-separator) mail-folder
    ; Complete path from mail store root to mail folder name.

auth-token          = 0*NORMCHAR
    ; No <CR> or <LF>, probably, but what about other nonprinting
    ; characters?

message-identifier  = path-component
    ; Unique within a mail folder, identifies a message.  Persistent
    ; as long as message remains in a given mail folder.

message-path        = folder-path path-separator message-identifier

directory-listing   = directory-path path-separator
    ; In LISTDIRS output, the trailing "/" indicates a directory
    ; instead of a mail folder.

flag                = %x44 / %x46 / %x4E / %x50 / %x52 / %x53 / %x54
    ; Current flags D, F, N, P, R, S, and T are uppercase only
    ; because of case-sensitivity, new flags can be added later
    ; using "r", etc.

flags               = [%x44] [%x46] [%x4E] [%x50] [%x52] [%x53]
                      [%x54]
    ; Flags are in any order but cannot be repeated.

content-size        =  NONZERODIGIT *DIGIT
    ; Positive nonzero decimal number with no leading zeroes.

content-block       = 1*OCTET
    ; Content-size octets of 8-bit data.

message-timestamp   = date-fullyear date-month date-mday %x54
                      time-hour time-minute time-second %x5A
    ; Example 20020304T134508Z.  The terms <date-fullyear>,
    ; <date-month>, <date-mday>, <time-hour>, <time-minute>,
    ; <time-second> are defined in [[RFC3339][]].

message-list-line   = message-identifier SP flags ":"
                      content-size ":" message-timestamp LF
    ; If a flag is present, that means it is set.  Note flags are
    ; optional and may not be present if all flags are cleared.

negative-code       = %x41 / %x45 / %x55 / %x58
    ; Single character negative response code A, E, U, or X.

response-code       = %x2B / %x4B / negative-code
    ; Single character response code +, K, A, E, U, or X.

response-modifier   = %x43
    ; Single character response modifier "C".  New modifiers may be
    ; added later.

response-modifiers  = [%x43]
    ; Modifiers in any order but cannot be repeated.

response-info       = *NORMCHAR
    ; Zero or more visible characters and spaces.

positive-response   = %x4B SP response-info LF

negative-response   = negative-code response-modifiers SP
                      response-info LF

one-line-response   = negative-response / positive-response

response-line       = response-code response-modifiers SP
                      response-info LF

continuation-line   = %x2B SP 1*NORMCHAR LF

multiline-response  = *continuation-line positive-response
    ; Zero or more continuation lines followed by a positive
    ; response.

command             = AUTH / CAPS / CLEARFLAG / EXPUNGE / GET /
                      GETHDR / LISTDIRS / LISTMSGS / MKDIR /
                      MKFOLDER / MVFOLDER / MVMSG / PUT / QUIT /
                      RMDIR / RMFOLDER / RMMSG / SEND / SETFLAG /
                      STARTTLS

auth-method         = ANONYMOUS / DIGEST-MD5 / GSSAPI /
                      KERBEROS_V5 / PLAIN / SCRAM-SHA1

7.2. Commands

This section details the formal grammar for each command and possible responses to the command.

7.2.1. STARTTLS

starttls-command    = STARTTLS LF

starttls-response   = one-line-response

7.2.2. AUTH

auth-command        = AUTH SP auth-method *( SP auth-token ) LF

token-query         = %x74 %x6F %x6B %x65 %x6E %x3F

auth-response       = negative-response
auth-response       =/ %x4B SP ( ( token-query ) /
                       ( %x6F %x6B ) / ( auth-token ) ) LF

auth-client-token   = auth-token LF

7.2.3. CAPS

caps-command        = CAPS LF

caps-response       = negative-response
caps-response       =/ *continuation-line
                       %x2B SP 1*VCHAR SP 1*VCHAR LF
    ; "K Server-type Version"

7.2.4. QUIT

quit-command        = QUIT LF

7.2.5. LISTDIRS

listdirs-command    = LISTDIRS LF

inbox               = %x69 %x6E %x62 %x6F %x78

listdirs-response   = negative-response
listdirs-response   =/ %x2B SP inbox LF *( %x2B SP
                       ( folder-path / directory-listing )
                       LF ) positive-response

7.2.6. MKDIR

mkdir-command       = MKDIR SP directory-path LF

mkdir-response      = one-line-response

7.2.7. RMDIR

rmdir-command       = RMDIR SP directory-path LF

rmdir-response      = one-line-response

7.2.8. MKFOLDER

mkfolder-command    = MKFOLDER SP folder-path LF

mkfolder-response   = one-line-response

7.2.9. RMFOLDER

rmfolder-command    = RMFOLDER SP folder-path LF

rmfolder-response   = one-line-response

7.2.10. MVFOLDER

mvfolder-command    = MVFOLDER SP folder-path SP folder-path LF

mvfolder-response   = one-line-response

7.2.11. LISTMSGS

listmsgs-command    = LISTMSGS SP folder-path LF

listmsgs-response   = negative-response
listmsgs-response   =/ *(%x2B SP message-list-line)
                       positive-response

7.2.12. GET

get-command         = GET SP message-path LF

get-response        = negative-response
get-response        =/ %x4B SP content-size LF
                       content-block positive-response
    ; <content-block> is <content-size> octets of 8-bit data

7.2.13. GETHDR

gethdr-command      = GETHDR SP message-path LF

gethdr-response     = negative-response
gethdr-response     =/ %x4B SP content-size LF
                       content-block positive-response
    ; <content-block> is <content-size> octets of 8-bit data
    ; The header ends with <LF> <LF>.

7.2.14. PUT

put-command         = PUT SP folder-path SP content-size LF

put-cmd-response    = one-line-response

put-message         = content-block FIN LF
    ; <content-block> is <content-size> octets of 8-bit data
    ; Only used if server response is positive.

put-msg-response    = negative-response
put-msg-response    =/ %x4B SP message-identifier LF

7.2.15. RMMSG

rmmsg-command       = RMMSG SP message-path LF

rmmsg-response      = one-line-response

7.2.16. MVMSG

mvmsg-command       = MVMSG SP message-path SP folder-path LF

mvmsg-response      = negative-response
mvmsg-response      =/ %x4B SP message-identifier

7.2.17. SETFLAG

setflag-command     = SETFLAG SP flag SP message-path LF

setflag-response    = one-line-response

7.2.18. CLEARFLAG

clearflag-command   = CLEARFLAG SP flag SP message-path LF

clearflag-response  = one-line-response

7.2.19. EXPUNGE

expunge-command     = EXPUNGE [ SP folder-path ] LF

expunge-response    = one-line-response

7.2.20. SEND

send-command        = SEND SP message-path SP content-size LF

send-envelope       = 2*( envelope-spec NUL ) FIN LF
    ; Only used if server response to <send-command> is positive.

send-response       = one-line-response
    ; Response to either SEND or the envelope data.

8. Examples

Here are examples of various BikINI commands and responses.

8.1. Log in and retrieve new messages from default incoming mail folder

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  K ok
C:  LISTMSGS inbox
S:  + 1044547809.6813_14.example.net S:13626:20010327T115332Z
    + 1045443985.21669.example.com S:3602:19980812T165402Z
    + 1044557867.9263_7.example.net NRS:15820:20030227T193206Z
    + 1044561911.10174_19.example.net S:5476:20021108T081428Z
    + 1044566229.11296_22.example.net NS:6644:20030227T195712Z
    K ok
C:  GET inbox/1044557867.9263_7.example.net
S:  K 15820
    Return-Path: <bob@familias.example.net>
    Delivered-To: jeff@familias.example.net
    [...]
    Hi, Jeff.  How's the wife and kids?
    [...]
    K ok
C:  GET inbox/1044566229.11296_22.example.net
S:  K 6644
    Return-Path: <tech@support.example.com>
    Delivered-To: web-test@discworld.dyndns.org
    [...]
    This is a message.
    [...]
    K ok
C:  LISTMSGS inbox
S:  + 1044547809.6813_14.example.net S:13626:20010327T115332Z
    + 1045443985.21669.example.com S:3602:19980812T165402Z
    + 1044557867.9263_7.example.net RS:15820:20030227T193206Z
    + 1044561911.10174_19.example.net S:5476:20021108T081428Z
    + 1044566229.11296_22.example.net S:6644:20030227T195712Z
    K ok
C:  QUIT
S:  K goodbye

Note that the message's N (new) flag is cleared by the successful GET operation.

8.2. Log in, retrieve the mail store structure, and store a message

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  K ok
C:  LISTDIRS
S:  + foobar/
    + foobar/pymsgauth
    + foobar/foobaz/
    + foobar/foobaz/mail
    + im2000
    + inbox
    + list-getmail
    + newfolder
    + nullmailer
    + python
    + qmail
    + spam
    K ok
C:  LISTMSGS foobar/pymsgauth
S:  K ok
C:  PUT foobar/pymsgauth 3534
S:  K go ahead
C:  Return-Path: <bikini@discworld.dyndns.org>
    To: "bikini mailing list" <bikini-devel@discworld.dnsalias.org>
    Subject: Development of mutt client backend
    [...]
    finished
S:  K 1045443985.Q49P3838.example.com
C:  LISTMSGS foobar/pymsgauth
S:  + 1045443985.Q49P3838.example.com :3534:20030304T113829Z
    K ok
C:  SETFLAG F foobar/pymsgauth/1045443985.Q49P3838.example.com
S:  K flag set
C:  LISTMSGS foobar/pymsgauth
S:  + 1045443985.Q49P3838.example.com F:3534:20030304T113829Z
    K ok
C:  QUIT
S:  K goodbye

The client retrieves the directory structure, lists the contents of mail folder "foobar/pymsgauth" (which is empty), stores a message (and is given the message identifier for the stored message by the server), lists the contents of the mail folder again (showing the message), sets the F flag on the message, lists the contents of the mail folder again (showing the flag is now set), and quits.

Note that the message's N (new) flag is initially clear when stored with the PUT command.

8.3. Log in procedure, including failed login due to unsupported method

C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
S:  U AUTH PLAIN unsupported
C:  CAPS
S:  + STARTTLS
    + AUTH=GSSAPI
    + AUTH=KERBEROS_V5
    + MESSAGE-SIZE 2000000
    K Example 0.9.0
C:  STARTTLS
C & S:  <negotiate a TLS session, and check result of negotiation>
S:  K TLS connection established
C:  CAPS
S:  + AUTH=PLAIN
    + AUTH=GSSAPI
    + AUTH=KERBEROS_V5
    + MESSAGE-SIZE 2000000
    K Example 0.9.0
C:  AUTH PLAIN dXNlckBob3N0LmRvbWFpbgBteSBwYXNzd29yZAA=
[...]

The client connects and tries to log in using "AUTH PLAIN". The server forbids this in unencrypted sessions (no STARTTLS), so it replies with U "unsupported". The client issues the STARTTLS command to encrypt this session, then issues the CAPS command to retrieve a list of supported AUTH methods and logs in using one of them.

Note that the server's CAPS response might be different after the STARTTLS command, as it might then include options such as "AUTH=PLAIN", and will not include STARTTLS.

9. Security Considerations

Security considerations about the email system and messages are covered in [RFC5322] and [SMTP], and will not be covered here.

9.1. Authentication

Authentication of the user is accomplished using a variety of AUTH methods including PLAIN. Implementations of this protocol SHOULD NOT allow PLAIN authentication unless the connection has been secured via a successfull STARTTLS command.

9.2. STARTTLS

Implementors MUST implement the TLS_RSA_WITH_RC4_128_SHA CipherSuite, and SHOULD implement the TLS_DHE_DSS_WITH_AES_256_CBC_SHA CipherSuite. This will ensure all compliant clients and servers can interoperate.

9.3. Announcements

While some small amount of security may be gained through hiding the server type and version, use of that data when debugging problems is beyond dispute. BikINI servers MUST make their type and version information available in the final K-line of the CAPS command response.

10. IANA Considerations

As a port will be used to access any servers using this protocol, the author requests assignment of TCP port 40 to BikINI.

11. References

11.1. Normative

[AUTH-ANONYMOUS] Zeilenga, K., "Anonymous Simple Authentication and Security Layer (SASL) Mechanism", RFC 4505, OpenLDAP Foundation, June 2006.
[AUTH-PLAIN] Zeilenga, K., "The PLAIN Simple Authentication and Security Layer (SASL) Mechanism", RFC 4616, OpenLDAP Foundation, August 2006.
[AUTH-DIGEST-MD5] Leach, P., and Newman, C., "Using Digest Authentication as a SASL Mechanism", RFC 2831, Microsoft and Innosoft, May 2000.
[AUTH-GSSAPI] Melnikov, A., Ed., "The Kerberos V5 ("GSSAPI") Simple Authentication and Security Layer (SASL) Mechanism", RFC 4752, November 2006.
[AUTH-KERBEROSV5] Neuman, C., Yu, T., Hartman, S., and K. Raeburn, "The Kerberos Network Authentication Service (V5)", RFC 4120, July 2005.
[AUTH-SCRAM-SHA1] Menon-Sen, A., Melnikov, A., Newman, C., and Williams, N., "Salted Challenge Response (SCRAM) SASL and GSS-API Mechanism", Internet Draft, May 2009.
[RFC2119] Bradner, "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, Harvard University, March 1997.
[RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: Timestamps", RFC 3339, July 2002.
[RFC3696] "Application Techniques for Checking and Transformation of Names" J. Klensin, RFC 3696, February 2004.
[RFC5234] Crocker, D., Ed., and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.
[RFC5322] Resnick, P., "Internet Message Format", STD 11, RFC 5322, QUALCOMM Incorporated, October 2008.
[TLS] Dierks, T. and E. Rescorla, "The Transport Layer Security (TLS) Protocol Version 1.2", RFC 5246, August 2008.
[TLS-IMAP-POP3-ACAP] Newman, C., "Using TLS with IMAP, POP3 and ACAP", RFC 2595, June 1999.
[TLS-SMTP] Hoffman, P., "SMTP Service Extension for Secure SMTP over Transport Layer Security", RFC 3207, February 2002.

11.2. Informative

[IMAP] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.
[POP3] Myers, J. and M. Rose, "Post Office Protocol - Version 3", STD 53, RFC 1939, May 1996.
[RFC2822] Resnick, P., Ed., "Internet Message Format", RFC 2822, April 2001.
[SASL] Melnikov, A., and Zeilenga, K., "Simple Authentication and Security Layer (SASL)", RFC 4422, Isode Limited and OpenLDAP Foundation, June 2006.
[SMTP] Klensin, J., "Simple Mail Transfer Protocol", RFC 5321, October 2008.

12. Acknowledgments

The author would like to thank Charles Cazabon for creating the original specification, server, and test client.

The author would also like to thank Bruce Guenter (contributor of the SASL code), Cory Wright, Jake Baillie, Russell Nelson, Mark Delany, and many others for contributions of code, documentation, document templates, valuable discussion, and other items to this project.

Finally, Chris Garrigues provided the quote which became the original motto for the project.

"IMAP sucks, but there really isn't anything else that even tries to do what IMAP does so we hate it and we live with it."

Author's Address

David J. Weller-Fahy
10009 Forest Lane
Midwest City, OK 73130
USA

EMail: dave@weller-fahy.com