Request Buffers

All requests must have a valid crp_buf initialized by one of the following functions:

crypto_use_buf()

Uses an array of len bytes pointed to by buf as the data buffer.

crypto_use_mbuf()

Uses the network memory buffer m as the data buffer.

crypto_use_uio()

Uses the scatter/gather list uio as the data buffer.

crypto_use_vmpage()

Uses the array of vm_page_t structures as the data buffer.

One of the following functions should be used to initialize crp_obuf for requests that use separate input and output buffers:

crypto_use_output_buf()

Uses an array of len bytes pointed to by buf as the output buffer.

crypto_use_output_mbuf()

Uses the network memory buffer m as the output buffer.

crypto_use_output_uio()

Uses the scatter/gather list uio as the output buffer.

crypto_use_output_vmpage()

Uses the array of vm_page_t structures as the output buffer.

Request Regions

For requests with separate input and output data buffers, the AAD, IV, and payload regions are always defined as regions in the input buffer, and a separate payload output region is defined to hold the output of encryption or decryption in the output buffer. The digest region describes a region in the input data buffer for requests that verify an existing digest. For requests that compute a digest, the digest region describes a region in the output data buffer. Note that the only data written to the output buffer is the encryption or decryption result and any computed digest. AAD and IV regions are not copied from the input buffer into the output buffer but are only used as inputs.

The following regions are defined:

Requests are permitted to operate on only a subset of the data buffer. For example, requests from IPsec operate on network packets that include headers not used as either additional authentication data (AAD) or payload data.

Request Operations

Compression requests support the following operations:

CRYPTO_OP_COMPRESS

Compress the data in the payload region of the data buffer.

CRYPTO_OP_DECOMPRESS

Decompress the data in the payload region of the data buffer.

Cipher requests support the following operations:

CRYPTO_OP_ENCRYPT

Encrypt the data in the payload region of the data buffer.

CRYPTO_OP_DECRYPT

Decrypt the data in the payload region of the data buffer.

Digest requests support the following operations:

CRYPTO_OP_COMPUTE_DIGEST

Calculate a digest over the payload region of the data buffer and store the result in the digest region.

CRYPTO_OP_VERIFY_DIGEST

Calculate a digest over the payload region of the data buffer. Compare the calculated digest to the existing digest from the digest region. If the digests match, complete the request successfully. If the digests do not match, fail the request with EBADMSG.

AEAD and Encrypt-then-Authenticate requests support the following operations:

CRYPTO_OP_ENCRYPT | CRYPTO_OP_COMPUTE_DIGEST

Encrypt the data in the payload region of the data buffer. Calculate a digest over the AAD and payload regions and store the result in the data buffer.

CRYPTO_OP_DECRYPT | CRYPTO_OP_VERIFY_DIGEST

Calculate a digest over the AAD and payload regions of the data buffer. Compare the calculated digest to the existing digest from the digest region. If the digests match, decrypt the payload region. If the digests do not match, fail the request with EBADMSG.

Request AAD

Request ESN

If the ESN is stored in crp_esn, CSP_F_ESN should be set in csp_flags. This use case is dedicated for encrypt and authenticate mode, since the high-order 32 bits of the sequence number are appended after the Next Header (RFC 4303).

AEAD modes supply the ESN in a separate AAD buffer (see e.g. RFC 4106, Chapter 5 AAD Construction).

Request IV and/or Nonce

Requests that store part, but not all, of the IV in the data buffer should store the partial IV in the data buffer and pass the full IV separately in crp_iv.

Request and Callback Scheduling

crypto_dispatch() can pass the request to the session's driver via three different methods:

1. The request is queued to a taskqueue backed by a pool of worker threads. By default the pool is sized to provide one thread for each CPU. Worker threads dequeue requests and pass them to the driver asynchronously.
2. The request is passed to the driver synchronously in the context of the thread invoking crypto_dispatch().
3. The request is queued to a queue of pending requests. A single worker thread dequeues requests and passes them to the driver asynchronously.

To select the first method (taskqueue backed by multiple threads), requests should set CRYPTO_F_ASYNC. To always use the third method (queue to single worker thread), requests should set CRYPTO_F_BATCH. If both flags are set, CRYPTO_F_ASYNC takes precedence. If neither flag is set, crypto_dispatch() will first attempt the second method (invoke driver synchronously). If the driver is blocked, the request will be queued using the third method. One caveat is that the first method is only used for requests using software drivers which use host CPUs to process requests. Requests whose session is associated with a hardware driver will ignore CRYPTO_F_ASYNC and only use CRYPTO_F_BATCH to determine how requests should be scheduled.

In addition to bypassing synchronous dispatch in crypto_dispatch(), CRYPTO_F_BATCH requests additional changes aimed at optimizing batches of requests to the same driver. When the worker thread processes a request with CRYPTO_F_BATCH, it will search the pending request queue for any other requests for the same driver, including requests from different sessions. If any other requests are present, CRYPTO_HINT_MORE is passed to the driver's process method. Drivers may use this to batch completion interrupts.

Callback function scheduling is simpler than request scheduling. Callbacks can either be invoked synchronously from crypto_done(), or they can be queued to a pool of worker threads. This pool of worker threads is also sized to provide one worker thread for each CPU by default. Note that a callback function invoked synchronously from crypto_done() must follow the same restrictions placed on threaded interrupt handlers.

By default, callbacks are invoked asynchronously by a worker thread. If CRYPTO_F_CBIMM is set, the callback is always invoked synchronously from crypto_done(). If CRYPTO_F_CBIFSYNC is set, the callback is invoked synchronously if the request was processed by a software driver or asynchronously if the request was processed by a hardware driver.

If a request was scheduled to the taskqueue via CRYPTO_F_ASYNC, callbacks are always invoked asynchronously ignoring CRYPTO_F_CBIMM and CRYPTO_F_CBIFSYNC. In this case, CRYPTO_F_ASYNC_KEEPORDER may be set to ensure that callbacks for requests on a given session are invoked in the same order that requests were queued to the session via crypto_dispatch(). This flag is used by IPsec to ensure that decrypted network packets are passed up the network stack in roughly the same order they were received.

Other Request Fields

crp_session

A reference to the active session. This is set when the request is created by crypto_getreq() and should not be modified. Drivers can use this to fetch driver-specific session state or session parameters.

crp_etype

Error status. Either zero on success, or an error if a request fails. Set by drivers prior to completing a request via crypto_done().

crp_flags

A bitmask of flags. The following flags are available in addition to flags discussed previously:

CRYPTO_F_DONE

Set by crypto_done before calling crp_callback. This flag is not very useful and will likely be removed in the future. It can only be safely checked from the callback routine at which point it is always set.

crp_cipher_key

Pointer to a request-specific encryption key. If this value is not set, the request uses the session encryption key.

crp_auth_key

Pointer to a request-specific authentication key. If this value is not set, the request uses the session authentication key.

crp_opaque

An opaque pointer. This pointer permits users of the cryptographic framework to store information about a request to be used in the callback.

crp_callback

Callback function. This must point to a callback function of type void (*)(struct cryptop *). The callback function should inspect crp_etype to determine the status of the completed operation. It should also arrange for the request to be freed via crypto_freereq().

crp_olen

Used with compression and decompression requests to describe the updated length of the payload region in the data buffer.

If a compression request increases the size of the payload, then the data buffer is unmodified, the request completes successfully, and crp_olen is set to the size the compressed data would have used. Callers can compare this to the payload region length to determine if the compressed data was discarded.