Rendering Foreign Bindings

This document details the general system that UniFFI uses to render the foreign bindings code.

The Askama template engine

Our foreign bindings generation uses the Askama template rendering engine. Askama uses a compile-time macro system that allows the template code to use Rust types directly, calling their methods passing them to normal Rust functions.

The task of the templates is to render the ComponentInterface, which is the Rust representation of the UDL file, into a bindings source file. This mainly consists of rendering source code for each Type from the UDL.

Type matching

One of the main sources of complexity when generating the bindings is handling types. UniFFI supports a large number of types, each of which corresponds to a variant of the Type enum. At one point there was a fairly large number of "mega-match" functions, each one matching against all Type variants. This made the code difficult to understand, because the functionality for one kind of type was split up.

Our current system for handling this is to have exactly 2 matches against Type:

  • One match lives in the template code. We map each Type variant to a template file that renders definitions and helper code, including:
    • Class definitions for records, enums, and objects.
    • Base classes and helper classes, for example ObjectRuntime.kt contains shared functionality for all the Type::Object types.
    • The FfiConverter class definition. This handles lifting and lowering types across the FFI for the type.
    • Initialization functions
    • Importing dependencies
    • See Types.kt for an example.
  • The other match lives in the Rust code. We map each Type variant to a implementation of the CodeType trait that renders identifiers and names related to the type, including:

Why is the code organized like this? For a few reasons:

  • Defining Askama templates in Rust required a lot of boilerplate. When the Rust code was responsible for rendering the class definitions, helper classes, etc., it needed to define a lot of Askama template structs which lead to a lot of extra lines of code (see PR #1189)
  • It's easier to access global state from the template code. Since the Rust code only handles names and identifiers, it only needs access to the Type instance itself, not the ComponentInterface or the Config. This simplifies the Rust side of things (see PR #1191). Accessing the ComponentInterface and Config from the template code is easy, we simply define these as fields on the top-level template Struct then they are accessible from all child templates.
  • Putting logic in the template code makes it easier to implement external types. For example, at one point the logic to lift/lower a type lived in the Rust code as a function that generated the expression in the foreign language. However, it was not clear at all how to make this work for external types, it would probably require parsing multiple UDL files and managing multiple ComponentInterfaces. Putting the logic to lift/lower the type in the FfiConverter class simplifies this, because we can import the external FfiConverter class and use that. We only need to know the name of the FfiConverter class which is a simpler task.

Askama extensions

A couple parts of this system require us to "extend" the functionality of Askama (i.e. adding hacks to workaround its limitations).

Adding imports

We want our type template files to specify what needs to be imported, but we don't want it to render the import statements directly. The imports should be rendered at the top of the file and de-duped in case multiple types require the same import. We handle this by:

  • Defining a separate Askama template struct that loops over all types and renders the definition/helper code for them.
  • That struct also stores a BTreeSet that contains the needed import statements and has an add_import() method that the template code calls. Using a BTreeSet ensures the imports stay de-duped and sorted.
  • Rendering this template as a separate pass. The rendered string and the list of imports get passed to the main template which arranges for them to be placed in the correct location.

Including templates once

We want our type template files to render runtime code, but only once. For example, we only want to render ObjectRuntime.kt once, even if there are multiple Object types defined in the UDL file. To handle this the type template defines an include_once_check() method, which tests if we've included a file before. The template code then uses that to guard the Askama {% include %} statement. See Object.kt for an example