data-model.md

The Experience Builder Data Model

In the rest of this document, Experience Builder will be written as XB.

This builds on top of the XB Components doc. Please read that first.

Some of the examples here refer to details that component types that use XB Shape Matching into Field Types doc. It should be possible to first read this without having read that, to understand the big picture. It is recommended to first read this, then that one, followed by a second pass of this document.

It also builds on top of the XB Config Management doc, which itself refers back to this one for a few things. The data model is built on top of the configuration architecture.

Also see the diagram.

Finding issues 🐛, code 🤖 & people 👯‍♀️

Related XB issue queue components:

1. Data model

Those issue queue components also have corresponding entries in CODEOWNERS.

If anything is unclear or missing in this document, create an issue in one of those issue queue components and assign it to one of us! 😊 🙏

1. Terminology

1.1 Existing Drupal Terminology that is crucial for XB

• content entity: an entity that can be created by a Content Creator, containing various fields, potentially including the XB field type, of a particular entity type (e.g. "node")
• data type: Drupal's smallest unit of representing data, defines semantics and typically comes with validation logic and convenience methods for interacting with the data it represents ⚠️ Not all data types in Drupal core do what they say, see \Drupal\experience_builder\Plugin\DataTypeOverride\UriOverride for example. ⚠️
• field: synonym of field item list
• field prop: a property defined by a field type, with a value for that property on such a field item, represented by a data type. Often a single prop exists (typically: value), but not always (for example: the image field type: target_id, entity, alt, title, width, height — with entity a computed field prop)
• field instance: a definition for instantiating a field type into a field item list containing >=1 field item
• field item: the instantiation of a field type
• field item list: to support multiple-cardinality values, Drupal core has opted to wrap every field item in a list — even if a particular field instance is single-cardinality
• field type: metadata plus a class defining the field props that exist on this field type, requires a field instance to be used
• SDC: see XB Components doc
• theme region: see XB Config Management doc
• view mode: view modes lets a content entity be displayed in multiple ways

1.2 XB terminology

• component: see XB Components doc
• Component config entity: see XB Config Management doc
• component instance: a UUID uniquely identifying this instance + component version + values for each required component input (if any) + optionally values for its component slots (if any)
• component node: one of the node types in the UI data model, representing a component instance in the component tree
• component input: see XB Components doc
• component slot: see XB Components doc
• Component Source Plugin: see XB Components doc
• component tree: a tree of component instances, by placing >=1 component instances in a particular order in another component instance's slot
• component tree field type: XB's field type that allows storing a component tree ⚠️ This is currently limited to the "default" view mode, and hence one component tree per content entity. ⚠️
• component tree root: the root of the component tree is the special case: it does not exist in another component, but it behaves the same as any other component slot
• component type: see XB Components doc
• component version: a version (a deterministic hash) identifying the version of a Component config entity either because the underlying component itself changed, or because the default static prop sources changed due to modified shape matching
• content type template: see XB Config Management doc.
• layout: synonym of component tree
• prop expression: see XB Shape Matching into Field Types doc
• prop source: see XB Shape Matching into Field Types doc
• static prop source: see XB Shape Matching into Field Types doc
• dynamic prop source: see XB Shape Matching into Field Types doc
• region node: one of the node types in the UI data model, representing a theme region's component tree
• slot node: one of the node types in the UI data model, representing a component instance's component slot
• XB field: an instance of the component tree field type
• XB field type: see component tree field type

2. Product requirements

This uses the terms defined above.

This adds to the product requirements listed in XB Components doc and XB Config Management doc.

(There are more, but these in particular affect XB's data model.)

• MUST have validation logic that generates consistent validation error messages for either content (a component tree created by the Content Creator and stored in a content entity) or config (a component tree created by the Site Builder and stored in a content type template)
• MUST support both symmetric and asymmetric translations (same vs different layout per translation, respectively)
• SHOULD facilitate real-time collaborative editing

3. Implementation

This uses the terms defined above.

Given a component developed by a Front-End Developer: how does XB allow a Content Creator to place a component instance in the component tree, specify values for the component inputs and component slots?

3.1 Data Model: from Front-End Developer to an XB data model that empowers the Content Creator

Moved to the XB Shape Matching into Field Types doc.

3.2 Data Model: storing a component tree

The component tree is represented by a \Drupal\experience_builder\Plugin\Field\FieldType\ComponentTreeItemList, which contains one field value for each component instance in the tree. Each component instance is represented by a \Drupal\experience_builder\Plugin\Field\FieldType\ComponentTreeItem, which each allow accessing the Component config entity and Component Source Plugin that represents the component.

See \Drupal\experience_builder\Plugin\Field\FieldType\ComponentTreeItem + its validation constraint.

XB defines a new XB field type with the following field props:

• uuid — A unique ID for this component instance
• component_id — This is the ID of the Component config entity this component instance references
• parent_uuid — If this component instance is placed inside another component instance in the tree, the UUID of the parent component instance
• slot — If this component instance is placed inside another component instance in the tree, the machine name of the component slot in which it is placed. This slot must exist in the parent component instance.
• inputs — see 3.2.2

When parent_uuid and slot are empty, the component instance is at the root of the component tree.

Additionally there are two computed field props:

• component - this is an entity reference to the Component config entity the component instance uses, meaning also the appropriate version will be loaded. Any methods on the Component config entity can be chained. E.g. $item->get('component')?->getComponentSource().
• parent_item - this is a data reference to the sibling \Drupal\experience_builder\Plugin\Field\FieldType\ComponentTreeItem in the tree that represents the component instance's parent component instance in the component tree. If the component instance has no parent, this will be NULL. Any methods on the parent component instance can be chained, e.g. $item->get('parent_item')->getComponent()?->getComponentSource()?->getSlotDefinitions()

Additionally, convenience methods for accessing/setting values on the ComponentTreeItem exist including:

• getParentUuid(): ?string - gets the value of parent_uuid if it exists
• getParentComponentTreeItem(): ?ComponentTreeItem - gets the parent component instance if it exists
• getSlot(): ?string - gets the component slot machine name if it exists
• getComponent(): ComponentInterface - gets the Component config entity at the specified version
• getComponentId(): string - gets the ID of the Component config entity
• getComponentVersion(): string - gets the version of the Component config entity used for this instance
• getUuid(): string - gets the UUID of the component instance
• getInputs(): ?array - gets the explicit inputs of the component instance as an array (JSON decoded)
• getInput(): ?string - gets the explicit inputs of the component instance as a string (JSON encoded)
• setInput(array|string $input): static - sets the inputs, can be passed as either a string (JSON encoded) or an array
• getLabel(): ?string - gets the (optional) label for the component instance to provide context for content authors
• setLabel(?string $label): self - sets the (optional) label for the component instance to provide context for content authors

Storing these as separate field props simplifies supporting both symmetric and asymmetric translations:

• the inputs column group (just the inputs column) group SHOULD always be translatable
• the tree column group (comprising uuid, component_id, component_version, parent_uuid and slot) can be either:
  1. marked translatable for asymmetric translations (a different component tree per content entity translation)
  2. marked untranslatable for symmetric translations (same component tree for all content entity translations)

(Drupal's Content Translation module natively supports configuring this.)

3.2.1 The columns (field props) storing the tree structure

The uuid, component_id, component_version, parent_uuid and slot columns model the tree structure.

See \Drupal\experience_builder\Plugin\DataType\ComponentTreeStructure + its validation constraint.

These columns always meet the following requirements

1. every component instance is represented by a "uuid, component_id, component_version" triple, with:

• the value for "component_id" being the ID of a Component config entity (NOT that of the underlying component)
• the value for "component_version" being a version on the (versioned!) Component config entity (see \Drupal\experience_builder\Entity\VersionedConfigEntityInterface::getVersions())
• the "uuid" being a randomly generated UUID

2. Any top-level items have NULL for both the parent_uuid and slot.
3. Nested components must have a value for both the parent_uuid and slot.
   1. The parent_uuid must exist in a sibling field item in the ComponentTreeItemList.
   2. The slot must be present in the parent component's slot definitions
   3. The parent_uuid must not be the same as the uuid - you cannot reference yourself as a parent
4. Each uuid must be unique in the list of items
5. The delta of each field item represents the order that components in the same level of the tree appear in.

3.2.2 The column (field prop) storing the component input values

See

• \Drupal\experience_builder\Plugin\DataType\ComponentInputs
• \Drupal\experience_builder\ComponentSource\ComponentSourceInterface::getExplicitInput()
• \Drupal\experience_builder\ComponentSource\ComponentSourceInterface::validateComponentInput()

This uses 3.1.

The component tree's inputs field prop has a trivial representation that could easily change. It is stored as a JSON blob, and meets the following requirements:

1. it contains opaque arrays that are validated by that source's ::validateComponentInput() and are decodable using that source's ::getExplicitInput()
2. the inputs for a given component live in the same field-item as its corresponding uuid, component_id, parent_uuid and slot

Note: this simplifies different (symmetric) translation strategies: it's trivial to either reuse another translation's inputs field prop (to show what to translate from) or not reuse anything at all — that needs only array intersection.

Note: a welcome bonus is that when real-time collaborative editing is eventually added, one user can move a component instance while another edits the inputs of that same component instance, without causing a conflict. This is because editing will be specific to a component instance, which is modeled as a single delta.

No validation is necessary for this field prop, because it is more easily validated at the field item level of the XB field type, not at the field prop level — there, the aforementioned ::validateComponentInput() method is called for every component instance encountered in the stored component tree. If the Component Source Plugin complains, a validation error occurs.

Example: A simple tree showing a root item (41595148-e5c1-4873-b373-be3ae6e21340) with a child (3b305d86-86a7-4684-8664-7ef1fc2be070) in the body slot, plus another root item (41595148-e5c1-4873-b373-be3ae6e21340).

[
  'uuid' => '41595148-e5c1-4873-b373-be3ae6e21340',
  'component_id' => 'sdc.xb_test_sdc.props-slots',
  'component_version' => 'ab4d3ddce315cf64',
  'inputs' => [
    'heading' => [
      'sourceType' => 'static:field_item:string',
      'value' => "Hello, world!",
      'expression' => 'ℹ︎string␟value',
    ],
  ],
],
[
  'uuid' => '3b305d86-86a7-4684-8664-7ef1fc2be070',
  'component_id' => 'sdc.xb_test_sdc.props-no-slots',
  'component_version' => '95f4f1d5ee47663b',
  'parent_uuid' => '41595148-e5c1-4873-b373-be3ae6e21340',
  'slot' => 'the_body',
  'inputs' => [
    'heading' => [
      'sourceType' => 'static:field_item:string',
      'value' => "It's me!",
      'expression' => 'ℹ︎string␟value',
    ],
  ],
  [
    'uuid' => '41595148-e5c1-4873-b373-be3ae6e21340',
    'component_id' => 'block.system_branding_block',
    'component_version' => '247a23298360adb2',
    // Example, that populates a Block component instance.
    // Note how much simpler the stored information is, because it uses the Block system's native input UX:
    'inputs' => [
      'label' => '',
      'label_display' => FALSE,
      'use_site_logo' => TRUE,
      'use_site_name' => TRUE,
      'use_site_slogan' => TRUE,
    ],
  ],
],

3.2.3 Validation

Assuming the tree column groups (uuid, component_id, parent_uuid and slot) has already been validated, a component tree described in an XB field then is valid when: for each component instance in the tree field prop:

1. getting the explicit input using ComponentSourceInterface::getExplicitInput() (which for Block requires no extra work but for SDC involves resolving the stored prop sources, resulting in values to be passed to the corresponding component inputs)
2. calling ComponentSourceInterface::validateComponentInput() (which for Block uses config schema validation and for SDC checking if \Drupal\Core\Theme\Component\ComponentValidator::validateProps() does not throw an exception)

3.2.4 Facilitating component inputs changes

When a component evolves, some component inputs cannot happen without also updating the stored component tree. In other words: an upgrade path is necessary if a Front-End Developer makes certain drastic changes:

• renaming a component input
• changing the schema of a component input
• adding a new required component input

Here too, storing the inputs as separate field props is helpful. An upgrade path for a component would require logic somewhat like this:

1. SQL query to search the component_id column for uses of this component, capture the UUIDs. If 0 matches: break.
2. If >0 matches, PHP logic computes the necessary changes.
3. Insert the updated inputs JSON blob into that specific delta.

The above sequence assumes doing this per-entity. But this can actually be done per entity-type, or more precisely: per XB field. So if the XB field type is only used for one entity type but is used in many bundles (i.e. many different content entity types), then a single query can find all component instances of the evolving component. After that point, the typical Drupal update path best practices apply. The key observation here: it is possible to efficiently find all uses of a component.

3.3 Data Model: rendering a stored component tree

See \Drupal\experience_builder\Plugin\Field\FieldType\ComponentTreeItemList.

This uses 3.2.1, 3.2.2 and 3.2.3.

Thanks to the validation in 3.2.3, it is guaranteed that each individual component instance can be rendered. But the goal is of course to render a component tree (not component instances), by starting at the root and rendering each component instance in the specified component slot.

To hydrate the stored component tree:

1. get (flat) list of component instances from the tree field prop (3.2.1): a list of uuids
2. load the corresponding Component config entity for each component instance given its UUID, which in turn enables loading the corresponding Component Source Plugin
3. get the explicit input from the inputs field prop (3.2.2) for each component instance, by using the Component Source Plugin's ::getExplicitInput() method
   • for Component Source Plugins with their own input UX (such as BlockComponent), that's just forwarding the stored values
   • for those without their own input UX (such as SingleDirectoryComponent), that may require additional resolving or evaluating (such as resolving the stored prop source — see 3.1)
4. pass those explicit inputs to each component instance, resulting in a list of hydrated component instances
5. transform that list to a tree by respecting the tree field prop (3.2.1), by placing nested component instances in the specified component slot of the specified parent component instance (special case: the root)

To render the stored component tree, it must first be hydrated it (see above), after which it can be converted to a render array.

3.4 UI Data Model: communicating a component tree to the front end

All prior sections refer to the data model that is stored (on the back end). But what makes sense on the back end does not necessarily make sense on the front end:

• the back end must integrate with many (server-side) Drupal subsystems, and it should as much as possible avoid burdening the front end with those implementation details
• the front end has different data structure needs, specifically the need for highly frequent changes, including concurrent ones during collaborative real-time editing

The front end needs the component tree to generate a preview that the end user can modify and interact with. For this we split the tree into layout and model parts. The layout represents the tree's overall structure and the model represents data for each component within that tree. The model is stored as a flat structure so it can more easily be queried by the front end.

The front end layout is a set of nodes, where each node can be one of the following types, represented by the nodeType key:

• 'component' which represents a component instance
• 'slot' which represents a component slot in a component instance.
• 'region' which represents a separate theme region in the user interface.

Each node in the component tree is described with a nodeType key which is one of the above 3 strings.

The top level of the layout structure is an array of zero or more region nodes.

3.4.1 component nodes

A component node represents a single component instance in the component tree and will contain zero or more component slots.

component nodes have the following keys

• uuid: a unique identifier for the component instance.
• type: an opaque string containing a Component config entity ID + version that this instantiates
• name: a name assigned by a Content Creator, to for example distinguish this particular component instance of some component among the 20 such in the current component tree
• slots: an object of slot nodes representing each component slot of this component instance (including empty slots)

An example simple component instance of a component with no component slots, and with a name for the component instance specified by the Content Creator:

{
  "nodeType": "component",
  "id": "380aaa26-5678-4c86-9b32-12161ea34196",
  "name": "Most Important Heading",
  "type": "sdc.xb_test_sdc.heading@1b4f8df7c94d7e3c",
  "slots": []
}

An example simple component instance of a component with a single component slot that is empty:

{
  "nodeType": "component",
  "id": "177122af-1679-4ee4-b700-dcf5ab376c4a",
  "type": "sdc.xb_test_sdc.one_column@f6a3a392e98e8342",
  "slots": [
    {
      "id": "177122af-1679-4ee4-b700-dcf5ab376c4a/content",
      "name": "content",
      "nodeType": "slot",
      "components": []
    }
  ]
}

3.4.2 slot nodes

A slot node must be the child of a component node.

slot nodes have the following keys

• name: a human-readable name that may be displayed to the user.
• components: an array of component nodes that represent the top-level component instances for this component slot
• id: a unique ID made up of the uuid of the parent component followed by the component slot name, separated by a slash.

{
  "nodeType": "slot",
  "id": "380aaa26-5678-4c86-9b32-12161ea34196/column_one",
  "name": "Column one",
  "components": []
}

3.4.3 region nodes

A region node can only exist at the top level in the layout tree and can be thought of as a special case of a slot that applies to the page rather than a component. Just like a slot node, it can contain zero or more component nodes.

region nodes have the following keys

• id is the identifier of the theme region.
• name: a human-readable name that may be displayed to the user.
• components: an array of component nodes that represent the top-level component instances for this theme region

The theme region with the ID of content is treated specially by the server, and assumed to contain the content entity. The front end should not need to do anything special here except perhaps default to editing the content region (but perhaps the server should express this default via a flag somewhere?).

{
  "nodeType": "region",
  "id": "content",
  "name": "Content",
  "components": []
}

3.4.4 The complete API response

The API response contains two top level keys:

• layout: the component tree described above, using the 3 layout tree node types
• model: an array of model data for each component node in the tree, keyed by the UUID of the component instance.

(What if the model and layout get out of sync? We could theoretically have UUIDs that don't have model values, or model values that are orphaned and don't have corresponding components in the layout. The server side's validation logic forbids saving in this case.)

A complete example, with three region nodes:

• A 'header' region with a single component instance.
• A 'content' region with multiple, nested component instances a tree.
• An empty 'footer' region.

{
  "layout": [
    {
      "nodeType": "region",
      "id": "header",
      "name": "Header",
      "components": [
        {
          "nodeType": "component",
          "id": "a164fa84-0460-40b0-a428-bf332b4a792a",
          "type": "block.system_branding_block@247a23298360adb2",
          "slots": []
        }
      ]
    },
    {
      "nodeType": "region",
      "id": "content",
      "name": "Content",
      "components": [
        {
          "nodeType": "component",
          "id": "97fb7bb9-4c8e-4fdc-87a8-c39ac9e8e618",
          "type": "sdc.xb_test_sdc.two_column@e5ef92acda2ee2d1",
          "slots": [
            {
              "nodeType": "slot",
              "id": "97fb7bb9-4c8e-4fdc-87a8-c39ac9e8e618/column_one",
              "components": [
                {
                  "nodeType": "component",
                  "id": "e8ecc571-0221-40d8-9ab2-262389fabd58",
                  "type": "sdc.xb_test_sdc.heading@1b4f8df7c94d7e3c",
                  "slots": []
                },
                {
                  "nodeType": "component",
                  "id": "baf231e8-b214-4e3e-93d3-5d3f03a1eae9",
                  "type": "sdc.xb_test_sdc.druplicon@some-version-string",
                  "slots": []
                }
              ]
            },
            {
              "nodeType": "slot",
              "id": "97fb7bb9-4c8e-4fdc-87a8-c39ac9e8e618/column_two",
              "components": [
                {
                  "nodeType": "component",
                  "id": "39648574-b937-4a5a-b1b2-9db0f30ae315",
                  "type": "sdc.xb_test_sdc.one_column@f6a3a392e98e8342",
                  "slots": [
                    {
                      "nodeType": "slot",
                      "id": "39648574-b937-4a5a-b1b2-9db0f30ae315/content",
                      "components": [
                        {
                          "nodeType": "component",
                          "id": "a1cfa9f1-0088-45d9-b837-39571485b75e",
                          "type": "sdc.xb_test_sdc.my-hero",
                          "slots": []
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "nodeType": "region",
      "id": "footer",
      "name": "Footer",
      "components": []
    }
  ],
  "model": {
    "a164fa84-0460-40b0-a428-bf332b4a792a": {},
    "97fb7bb9-4c8e-4fdc-87a8-c39ac9e8e618": {},
    "e8ecc571-0221-40d8-9ab2-262389fabd58": {
      "text": "Heading",
      "style": "primary",
      "element": "h1"
    },
    "baf231e8-b214-4e3e-93d3-5d3f03a1eae9": {},
    "39648574-b937-4a5a-b1b2-9db0f30ae315": {},
    "a1cfa9f1-0088-45d9-b837-39571485b75e": {
      "heading": "Hero",
      "subheading": "My subheading"
    }
  }
}