Your First Modality

This guide walks through building a complete modality from scratch. By the end, you will have a working Flashcard modality with front/back text, fully wired into a Session for dispatch and snapshot emission.

A single struct implements all three core traits: Reducer, Component, and Modality. This guide builds them up one at a time.

Define the State

Every modality splits its state into two parts:

Synced — persisted via LoroDoc, shared across peers via CRDT sync.
Ephemeral — runtime-only state (selection, hover, transient UI flags). Defaults on creation, never persisted.

use modality_core::prelude::*;
use std::collections::HashMap;

/// Persisted state -- synced via Loro CRDT.
#[derive(Clone, PartialEq, Synced, HasResourceMeta)]
pub struct FlashcardSynced {
    pub placements: Vec<ComponentPlacement<FlashcardPosition>>,
    pub resource_meta: ResourceMeta,
}

/// Runtime-only state -- never persisted.
#[derive(Default)]
pub struct FlashcardEphemeral {
    pub is_flipped: bool,
    pub selected_placement_id: Option<String>,
    pub output: Vec<FlashcardElement>,
}

/// The combined state container.
pub type FlashcardState = State<FlashcardSynced, FlashcardEphemeral>;

The State<S, E> wrapper provides .synced and .ephemeral field access. Session requires Ephemeral: Default so it can construct the initial ephemeral state on open.

Position type

Each modality defines how components are spatially arranged. For a flashcard, position is simple — just front or back:

#[derive(Clone, PartialEq, Hash)]
pub enum FlashcardPosition {
    Front,
    Back,
}

Output type

Define what compilation produces. This is what Dart renders:

#[derive(Clone)]
pub struct FlashcardElement {
    pub side: FlashcardPosition,
    pub content: String,
}

Define the Intent

Intents are the external mutations your modality accepts. Every user action becomes an intent dispatched through Session::dispatch().

#[derive(CommandTool)]
pub enum FlashcardIntent {
    /// Flip the card to show the other side.
    Flip,

    /// Set the text content for a placement.
    SetText {
        placement_id: String,
        text: String,
    },

    /// Add a new placement to the card.
    AddPlacement {
        id: String,
        position: FlashcardPosition,
        component_key: String,
    },

    /// Remove a placement from the card.
    #[tool(hidden)]
    RemovePlacement {
        placement_id: String,
    },
}

#[derive(CommandTool)] makes each variant available as an AI tool. Mark variants with #[tool(hidden)] to exclude them from AI while keeping them dispatchable from Dart.

Implement Reducer

Reducer handles state management. The key insight: reduce() is static (no &self). It receives &mut State and an Effects handle for side effects.

pub struct Flashcard {
    state: FlashcardState,
    services: Arc<ModalityServices>,
    cmd_tx: CommandSender<Command<FlashcardIntent, ()>>,
    cmd_rx: Mutex<Receiver<Command<FlashcardIntent, ()>>>,
}

impl Reducer for Flashcard {
    type State = FlashcardState;
    type Intent = FlashcardIntent;
    type Feedback = ();  // no async feedback for this simple modality
    type Services = ModalityServices;

    fn state(&self) -> &Self::State { &self.state }
    fn state_mut(&mut self) -> &mut Self::State { &mut self.state }
    fn services(&self) -> &Arc<Self::Services> { &self.services }
    fn sender(&self) -> &CommandSender<Command<Self::Intent, Self::Feedback>> { &self.cmd_tx }
    fn receiver(&self) -> &Mutex<Receiver<Command<Self::Intent, Self::Feedback>>> { &self.cmd_rx }

    fn reduce(
        cmd: Command<FlashcardIntent, ()>,
        state: &mut FlashcardState,
        fx: &mut Effects<ModalityServices, Command<FlashcardIntent, ()>>,
    ) {
        match cmd {
            Command::Intent(intent) => match intent {
                FlashcardIntent::Flip => {
                    state.ephemeral.is_flipped = !state.ephemeral.is_flipped;
                }
                FlashcardIntent::SetText { placement_id, text } => {
                    // Find the placement and update its bindings
                    if let Some(p) = state.synced.placements.iter_mut()
                        .find(|p| p.id == placement_id)
                    {
                        p.bindings.insert("text".to_string(), PropertyValue::Text(text));
                    }
                    // Report success to the AI bridge
                    fx.services().agent.tool_output(serde_json::json!({
                        "status": "ok",
                        "placement_id": placement_id,
                    }));
                }
                FlashcardIntent::AddPlacement { id, position, component_key } => {
                    state.synced.placements.push(ComponentPlacement {
                        id,
                        component_key,
                        component_id: None,
                        position,
                        bindings: HashMap::new(),
                    });
                }
                FlashcardIntent::RemovePlacement { placement_id } => {
                    state.synced.placements.retain(|p| p.id != placement_id);
                }
            },
            Command::Feedback(()) => {}
        }
    }
}

A few things to notice:

reduce is a static function — it takes state: &mut FlashcardState, not &mut self.
Side effects go through fx. Use fx.services() for service access, fx.send() for follow-up commands, fx.spawn() for async work.
Flip only touches ephemeral state (not synced), so it will not trigger a Loro commit.
SetText mutates synced state (placements), which triggers commit and CRDT sync.

Implement Component

Component defines the modality’s property schema and compilation logic. Compilation turns state + property values into renderable output.

impl Component for Flashcard {
    type Metadata = FlashcardMetadata;
    type Output = Vec<FlashcardElement>;

    fn properties(&self) -> Vec<PropertySchema> {
        vec![
            PropertySchema {
                key: "title".to_string(),
                name: "Card Title".to_string(),
                property_type: PropertyType::Text { max_length: Some(100) },
                required: false,
                default_value: None,
            },
        ]
    }

    fn compile(
        &self,
        metadata: &FlashcardMetadata,
        values: &HashMap<String, PropertyValue>,
    ) -> Result<Vec<FlashcardElement>, CompileError> {
        // Delegate to layout + assemble (the default pattern)
        let pairs = self.layout(
            |placement, meta, resolved_values| {
                self.compile_child(placement, meta, resolved_values)
            },
            values,
        )?;
        Ok(self.assemble(pairs))
    }
}

Metadata

Metadata describes the modality-level context that child components need. It must implement Hash for the incremental compilation cache.

#[derive(Clone, Hash, Serialize)]
pub struct FlashcardMetadata {
    pub card_width: u32,
    pub card_height: u32,
}

Implement Modality

Modality ties everything together. It extends both Reducer (state management) and Component (compilation), adding layout orchestration and snapshot emission.

impl Modality for Flashcard {
    type Synced = FlashcardSynced;
    type Ephemeral = FlashcardEphemeral;
    type Position = FlashcardPosition;
    type ChildComponent = dyn FlashcardComponent;
    type Snapshot = FlashcardSnapshot;

    fn new(state: FlashcardState, services: Arc<ModalityServices>) -> Self {
        let (cmd_tx, cmd_rx) = mpsc::channel();
        Flashcard {
            state,
            services,
            cmd_tx: CommandSender::new(cmd_tx),
            cmd_rx: Mutex::new(cmd_rx),
        }
    }

    fn placements(&self) -> &[ComponentPlacement<FlashcardPosition>] {
        &self.state.synced.placements
    }

    fn metadata(&self) -> FlashcardMetadata {
        FlashcardMetadata {
            card_width: 600,
            card_height: 400,
        }
    }

    fn child_metadata(
        &self,
        placement: &ComponentPlacement<FlashcardPosition>,
    ) -> <Self::ChildComponent as Component>::Metadata {
        // Child components receive the card dimensions as their bounds
        Rect {
            x: 0.0,
            y: 0.0,
            width: 600.0,
            height: 400.0,
        }
    }

    fn compile_child(
        &self,
        placement: &ComponentPlacement<FlashcardPosition>,
        meta: &<Self::ChildComponent as Component>::Metadata,
        values: &HashMap<String, PropertyValue>,
    ) -> Result<<Self::ChildComponent as Component>::Output, CompileError> {
        let component = resolve_component(&placement.component_key)
            .ok_or(CompileError::UnknownComponent(placement.component_key.clone()))?;
        component.compile(meta, values)
    }

    fn assemble(
        &self,
        pairs: Vec<(
            ComponentPlacement<FlashcardPosition>,
            <Self::ChildComponent as Component>::Output,
        )>,
    ) -> Vec<FlashcardElement> {
        pairs.into_iter()
            .flat_map(|(placement, elements)| {
                elements.into_iter().map(move |e| FlashcardElement {
                    side: placement.position.clone(),
                    content: format!("{:?}", e),
                })
            })
            .collect()
    }

    fn snapshot(
        &self,
        ai: &AiState,
        export: &ExportState,
        version: u64,
    ) -> FlashcardSnapshot {
        FlashcardSnapshot {
            elements: self.state.ephemeral.output.clone(),
            is_flipped: self.state.ephemeral.is_flipped,
            is_streaming: ai.is_streaming,
            version,
        }
    }

    // layout() uses the default implementation, which iterates placements,
    // resolves bindings, and calls compile_child for each.
}

Snapshot

The snapshot is the DTO pushed to Dart via sinks. Each modality defines its own concrete type:

#[derive(Clone)]
pub struct FlashcardSnapshot {
    pub elements: Vec<FlashcardElement>,
    pub is_flipped: bool,
    pub is_streaming: bool,
    pub version: u64,
}

Wire into a Session

Session<M, I> is the orchestrator that owns your modality, the Loro document, sinks, and cache. To use your modality, create a session with the appropriate intent type:

use modality_session::{Session, SessionIntent, Doc};

// The session type for Flashcard
type FlashcardSession = Session<Flashcard, SessionIntent<FlashcardIntent>>;

Opening a session

pub fn open_flashcard_session(
    id: &str,
    backend: Arc<dyn StoreBackend>,
    sink: Box<dyn Sink<FlashcardSnapshot>>,
) -> FlashcardSession {
    let session_services = Arc::new(SessionServices { backend: backend.clone() });

    // Load or create synced state
    let synced = match backend.load(id) {
        Ok(bytes) => {
            let doc = Doc::<FlashcardSynced>::from_bytes(&bytes);
            doc.synced()
        }
        Err(_) => FlashcardSynced::default(),
    };

    let state = State {
        synced,
        ephemeral: FlashcardEphemeral::default(),
    };

    let (agent_service, agent_rx) = AgentService::new();
    let services = Arc::new(ModalityServices::new(agent_service));

    Session::new(state, services, session_services, vec![sink], agent_rx)
}

Dispatching intents

From Dart (via FRB), dispatch intents through the session:

// Flip the card
session.dispatch(Command::Intent(
    SessionIntent::Modality(FlashcardIntent::Flip)
));

// Set front text
session.dispatch(Command::Intent(
    SessionIntent::Modality(FlashcardIntent::SetText {
        placement_id: "front-text".to_string(),
        text: "What is photosynthesis?".to_string(),
    })
));

Each dispatch() call runs the full lifecycle:

flowchart LR
    A[dispatch] --> B[drain feedback]
    B --> C[reduce]
    C --> D[send loop]
    D --> E[recompile]
    E --> F[commit if changed]
    F --> G[version++]
    G --> H[emit snapshot]
    H --> I[spawn async]

The snapshot emitted at step H arrives at your Dart sink, updating the UI.

Recap

Building a modality requires:

State structs — Synced (persisted) and Ephemeral (runtime), plus Position and output types.
Intent enum — all external mutations, with #[derive(CommandTool)] for AI.
Reducer impl — static reduce() that matches on intents and mutates state.
Component impl — properties() schema and compile() output.
Modality impl — new(), placements(), metadata(), child_metadata(), compile_child(), assemble(), snapshot().
Session wiring — Session<Flashcard, SessionIntent<FlashcardIntent>> owns everything.

The same Flashcard struct implements all three traits. State flows downward through compilation. Mutations flow upward through intents and the dispatch lifecycle.

Next Steps

Adding a Component — build a child component for your modality
Adding an Intent — add new intent variants with effects
Wiring AI — make your modality AI-generatable