Lifting, Lowering and Serialization

UniFFI is able to transfer rich data types back-and-forth between the Rust code and the foreign-language code via a process we refer to as "lowering" and "lifting".

Recall that UniFFI interoperates between different languages by defining a C-style FFI layer which operates in terms of primitive data types and plain functions. To transfer data from one side of this layer to the other, the sending side "lowers" the data from a language-specific data type into one of the primitive types supported by the FFI-layer functions, and the receiving side "lifts" that primitive type into its own language-specific data type.

Lifting and lowering simple types such as integers is done by directly casting the value to and from an appropriate type. Non-trivial types such as Strings, Optionals and Records, etc. are lowered to a byte buffer called a RustBuffer internally.

For example, a Python str is passed to Rust by lowering it to a RustBuffer, which is then lifted to a Rust String

Lifting and Lowering when calling functions

As a concrete example, consider this interface for accumulating a list of integers:

namespace example {
  sequence<i32> add_to_list(i32 item);
}

Calling this function from foreign language code involves lowering the arguments, calling an FFI function, lifting the arguments, then calling the original function.

Details:

1. UniFFI generates an add_to_list function in the foreign language (Python in the example diagram). In this function the item argument and the return type are language-native types.
2. The generated function lowers each argument. Since the item argument is a plain integer, it is lowered by casting to an int32_t.
3. The generated Python function passes the lowered arguments to the Rust scaffolding function. This is a repr(C) FFI function in Rust library and named uniffi_fn_add_to_list in this example.
4. The Rust scaffolding function lifts each argument received over the FFI into a native Rust type. Since item is a plain integer no conversion is needed.
5. The Rust scaffolding passes the lifted arguments to the user-defined Rust add_to_list function, which then executes normally.
6. The Rust scaffolding function receives the return value and now needs to lower the it to pass it back across the FFI. Since this type's ::FfiType is a RustBuffer, it's lowered by serializing the values into a byte buffer (RustBuffer), which is then returned.
7. The generated Python function receives the return value, and then lifts it to a native data type. Since this type's ::FfiType is a RustBuffer, it's lifted by deserializing a language-native list of integers from the RustBuffer.

Lowered Types

UDL Type	Representation in the C FFI
i8/i16/i32/i64	int8_t/int16_t/int32_t/int64_t
u8/u16/u32/u64	uint8_t/uint16_t/uint32_t/uint64_t
f32/float	float
f64/double	double
boolean	int8_t, either 0 or 1
string	RustBuffer struct pointing to utf8 bytes
bytes	Same as sequence<u8>
timestamp	RustBuffer struct pointing to a i64 representing seconds and a u32 representing nanoseconds
duration	RustBuffer struct pointing to a u64 representing seconds and a u32 representing nanoseconds
T?	RustBuffer struct pointing to serialized bytes
sequence<T>	RustBuffer struct pointing to serialized bytes
record<string, T>	RustBuffer struct pointing to serialized bytes
enum and [Enum] interface	RustBuffer struct pointing to serialized bytes
dictionary	RustBuffer struct pointing to serialized bytes
interface	void* opaque pointer to object on the heap

Serialization Format

When serializing complex data types into a byte buffer, UniFFI uses an ad-hoc fixed-width format which is designed mainly for simplicity. The details of this format are internal only and may change between versions of UniFFI.

UDL Type	Representation in serialized bytes
i8/i16/i32/i64	Fixed-width 1/2/4/8-byte signed integer, big-endian
u8/u16/u32/u64	Fixed-width 1/2/4/8-byte unsigned integer, big-endian
f32/float	Fixed-width 4-byte float, big-endian
f64/double	Fixed-width 8-byte double, big-endian
boolean	Fixed-width 1-byte signed integer, either 0 or 1
string	Serialized i32 length followed by utf-8 string bytes; no trailing null
T?	If null, serialized boolean false; if non-null, serialized boolean true followed by serialized T
sequence<T>	Serialized i32 item count followed by serialized items; each item is a serialized T
record<string, T>	Serialized i32 item count followed by serialized items; each item is a serialized string followed by a serialized T
enum and [Enum] interface	Serialized i32 indicating variant, numbered in declaration order starting from 1, followed by the serialized values of the variant's fields in declaration order
dictionary	The serialized value of each field, in declaration order
interface	Fixed-width 8-byte unsigned integer encoding a pointer to the object on the heap

Note that length fields in this format are serialized as signed integers despite the fact that they will always be non-negative. This is to help ease compatibility with JVM-based languages since the JVM uses signed 32-bit integers for its size fields internally.