radiance · Radiance self-hosting compiler

compiler/ lib/ examples/ std/ arch/ collections/ lang/ alloc/ ast/ gen/ il/ lower/ module/ parser/ resolver/ scanner/ alloc.rad 4.2 KiB ast.rad 22.6 KiB gen.rad 265 B il.rad 15.1 KiB lower.rad 258.3 KiB module.rad 14.1 KiB package.rad 1.4 KiB parser.rad 78.7 KiB resolver.rad 227.9 KiB scanner.rad 23.4 KiB sexpr.rad 7.0 KiB strings.rad 2.2 KiB sys/ arch.rad 65 B collections.rad 36 B fmt.rad 3.8 KiB intrinsics.rad 206 B io.rad 1.2 KiB lang.rad 222 B mem.rad 2.2 KiB sys.rad 167 B testing.rad 2.4 KiB tests.rad 11.6 KiB vec.rad 3.1 KiB std.rad 231 B scripts/ seed/ test/ vim/ .gitignore 353 B .gitsigners 112 B LICENSE 1.1 KiB Makefile 3.1 KiB README 2.5 KiB std.lib 987 B std.lib.test 252 B
lib/std/lang/lower.rad 258.3 KiB raw
//! AST to IL lowering pass.
//!
//! This module converts the typed AST produced by the resolver into a linear
//! SSA-based intermediate language (IL). The IL is suitable for further
//! optimization and code generation.
//!
//! # Design Overview
//!
//! The lowering process works in two main phases:
//!
//! 1. Module-level pass: Iterates over top-level declarations, lowering
//!    each function independently while accumulating global data items (strings,
//!    constants, static arrays) into a shared data section.
//!
//! 2. Function-level pass: For each function, constructs an SSA-form control
//!    flow graph (CFG) with basic blocks connected by jumps and branches. Uses
//!    a simplified SSA construction algorithm where variables are tracked per-block
//!    and block parameters are inserted lazily when a variable is used before
//!    being defined in a block.
//!
//! # Memory Model
//!
//! All allocations use an arena allocator passed through the `Lowerer` context.
//! The IL is entirely stack-based at runtime -- there's no heap allocation during
//! program execution. Aggregate values (records, slices, optionals) are passed
//! by pointer on the stack.
//!
//! # SSA Construction
//!
//! SSA form is built incrementally as the AST is walked. When a variable is
//! defined (via `defVar`), its current value is recorded in the current block.
//! When a variable is used (via `useVar`), the algorithm either:
//! - Returns the value if defined in this block
//! - Recurses to predecessors if the block is sealed (all predecessors known)
//! - Inserts a block parameter if the variable value must come from multiple paths
//!
//! Block "sealing" indicates that all predecessor edges are known, enabling
//! the SSA construction to resolve cross-block variable references.
//!
//! # SSA Variable API
//!
//! 1. `newVar` creates a logical variable that can be defined differently in each block.
//! 2. `defVar` defines the variable's value in the current block.
//! 3. `useVar` reads the variable; if called in a block with multiple
//!             predecessors that defined it differently, the SSA algorithm
//!             automatically creates a block parameter.
//!
//! # Expression Lowering
//!
//! Expressions produce IL values which can be:
//!
//! - Imm(i64): immediate/constant values
//! - Reg(u32): SSA register references
//! - Sym(name): symbol references (for function pointers, data addresses)
//! - Undef: for unused values
//!
//! For aggregate types (records, arrays, slices, optionals), the "value" is
//! actually a pointer to stack-allocated memory containing the aggregate.
//!
//! # Block ordering invariant
//!
//! The register allocator processes blocks in forward index order and uses a
//! single global assignment array. This means a value's definition block must
//! have a lower index than any block that uses the value (except through
//! back-edges, where block parameters handle the merge). The lowerer maintains
//! this by creating blocks in control-flow order. For `for` loops, the step
//! block is created lazily (after the loop body) so that its index is higher
//! than all body blocks -- see [`lowerForLoop`] and [`getOrCreateContinueBlock`].
//!
//! A more robust alternative would be per-block register maps with edge-copy
//! resolution (as in QBE's `rega.c`), which is block-order-independent. That
//! would eliminate this invariant at the cost of a more complex allocator.
//!
//! # Notes on constant data lowering
//!
//! The lowerer flattens constant AST expressions directly into IL data values.
//! Primitive constants use [`resolver::ConstValue`] as an intermediate form.
//! Record padding is explicit: `undef * N;` for N padding bytes.
//!
//!   AST Node (ArrayLit, RecordLit, literals, etc.)
//!       ↓ [`lowerConstData`]
//!   resolver::ConstValue (Bool, Char, String, Int)
//!       ↓ [`constValueToDataItem`]
//!   il::DataItem (Val, Sym, Str, Undef)
//!       ↓
//!   il::DataValue
//!       ↓
//!   il::Data
//!
use std::fmt;
use std::io;
use std::lang::alloc;
use std::mem;
use std::lang::ast;
use std::lang::il;
use std::lang::module;
use std::lang::resolver;

// TODO: Search for all `_ as i32` to ensure that casts from u32 to i32 don't
// happen, since they are potentially truncating values.

// TODO: Support constant union lowering.
// TODO: Void unions should be passed by value.

////////////////////
// Error Handling //
////////////////////

/// Lowering errors are typically unrecoverable since they indicate bugs in
/// the resolver or malformed AST that should have been caught earlier.
pub union LowerError {
    /// A node's symbol was not set before lowering.
    MissingSymbol(*ast::Node),
    /// A node's type was not set before lowering.
    MissingType(*ast::Node),
    /// A node's constant value was not set before lowering.
    MissingConst(*ast::Node),
    /// An optional type was expected.
    ExpectedOptional,
    /// A record type was expected.
    ExpectedRecord,
    /// An array was expected.
    ExpectedArray,
    /// A block was expected.
    ExpectedBlock(*ast::Node),
    /// An identifier was expected.
    ExpectedIdentifier,
    /// A function parameter was expected.
    ExpectedFunctionParam,
    /// A function type was expected.
    ExpectedFunction,
    /// Trying to lower loop construct outside of loop.
    OutsideOfLoop,
    /// Invalid variable use.
    InvalidUse,
    /// Unexpected node value.
    UnexpectedNodeValue(*ast::Node),
    /// Unexpected type.
    UnexpectedType(*resolver::Type),

    /// Missing control flow target.
    MissingTarget,
    /// Missing metadata that should have been set by resolver.
    MissingMetadata,
    /// Expected a slice or array type for operation.
    ExpectedSliceOrArray,
    /// Expected a variant symbol.
    ExpectedVariant,
    /// Expected a call expression.
    ExpectedCall,
    /// Field not found in record or invalid field access.
    FieldNotFound,
    /// Assignment to an immutable variable.
    ImmutableAssignment,
    /// Nil used in non-optional context.
    NilInNonOptional,
    /// Invalid argument count for builtin.
    InvalidArgCount,
    /// Feature or pattern not supported by the lowerer.
    Unsupported,
    /// Unknown intrinsic function.
    UnknownIntrinsic,
    /// Allocation failure.
    AllocationFailed,
}

/// Print a LowerError for debugging.
pub fn printError(err: LowerError) {
    match err {
        case LowerError::MissingSymbol(_) => io::print("MissingSymbol"),
        case LowerError::MissingType(_) => io::print("MissingType"),
        case LowerError::MissingConst(_) => io::print("MissingConst"),
        case LowerError::ExpectedOptional => io::print("ExpectedOptional"),
        case LowerError::ExpectedRecord => io::print("ExpectedRecord"),
        case LowerError::ExpectedArray => io::print("ExpectedArray"),
        case LowerError::ExpectedBlock(_) => io::print("ExpectedBlock"),
        case LowerError::ExpectedIdentifier => io::print("ExpectedIdentifier"),
        case LowerError::ExpectedFunctionParam => io::print("ExpectedFunctionParam"),
        case LowerError::ExpectedFunction => io::print("ExpectedFunction"),
        case LowerError::OutsideOfLoop => io::print("OutsideOfLoop"),
        case LowerError::InvalidUse => io::print("InvalidUse"),
        case LowerError::UnexpectedNodeValue(_) => io::print("UnexpectedNodeValue"),
        case LowerError::UnexpectedType(_) => io::print("UnexpectedType"),
        case LowerError::MissingTarget => io::print("MissingTarget"),
        case LowerError::MissingMetadata => io::print("MissingMetadata"),
        case LowerError::ExpectedSliceOrArray => io::print("ExpectedSliceOrArray"),
        case LowerError::ExpectedVariant => io::print("ExpectedVariant"),
        case LowerError::ExpectedCall => io::print("ExpectedCall"),
        case LowerError::FieldNotFound => io::print("FieldNotFound"),
        case LowerError::ImmutableAssignment => io::print("ImmutableAssignment"),
        case LowerError::NilInNonOptional => io::print("NilInNonOptional"),
        case LowerError::InvalidArgCount => io::print("InvalidArgCount"),
        case LowerError::Unsupported => io::print("Unsupported"),
        case LowerError::UnknownIntrinsic => io::print("UnknownIntrinsic"),
        case LowerError::AllocationFailed => io::print("AllocationFailed"),
    }
}

///////////////
// Constants //
///////////////

/// Maximum nesting depth of loops.
const MAX_LOOP_DEPTH: u32 = 16;
/// Maximum number of `catch` clauses per `try`.
const MAX_CATCH_CLAUSES: u32 = 32;

// Slice Layout
//
// A slice is a fat pointer consisting of a data pointer and length.
// `{ ptr: u32, len: u32 }`.

/// Slice data pointer offset.
const SLICE_PTR_OFFSET: i32 = 0;
/// Offset of slice length in slice data structure.
const SLICE_LEN_OFFSET: i32 = 8;
/// Offset of slice capacity in slice data structure.
const SLICE_CAP_OFFSET: i32 = 12;

// Trait Object Layout
//
// A trait object is a fat pointer consisting of a data pointer and a
// v-table pointer. `{ data: *T, vtable: *VTable }`.

/// Trait object data pointer offset.
const TRAIT_OBJ_DATA_OFFSET: i32 = 0;
/// Trait object v-table pointer offset.
const TRAIT_OBJ_VTABLE_OFFSET: i32 = 8;

// Tagged Value Layout (optionals, tagged unions)
//
// Optionals and unions use 1-byte tags. Results use 8-byte tags.
//
// `{ tag: u8, [padding], payload: T }`
//
// Optionals use `tag: 0` for `nil` and `tag: 1` otherwise.
// When `T` is a pointer, the entire optional is stored as a single pointer.
//
// Tagged unions have a payload the size of the maximum variant size.

/// Offset of tag in tagged value data structure.
const TVAL_TAG_OFFSET: i32 = 0;
/// Offset of value in result data structure (8-byte tag).
const RESULT_VAL_OFFSET: i32 = 8;

//////////////////////////
// Core Data Structures //
//////////////////////////

/// Options controlling the lowering pass.
pub record LowerOptions {
    /// Whether to emit source location info.
    debug: bool,
    /// Whether to lower `@test` functions.
    buildTest: bool,
}

/// Module-level lowering context. Shared across all function lowerings.
/// Holds global state like the data section (strings, constants) and provides
/// access to the resolver for type queries.
pub record Lowerer {
    /// Arena for IL allocations. All IL nodes are allocated here.
    arena: *mut alloc::Arena,
    /// Allocator backed by the arena.
    allocator: alloc::Allocator,
    /// Resolver for type information. Used to query types, symbols, and
    /// compile-time constant values during lowering.
    resolver: *resolver::Resolver,
    /// Module graph for cross-module symbol resolution.
    moduleGraph: ?*module::ModuleGraph,
    /// Package name for qualified symbol names.
    pkgName: *[u8],
    /// Current module being lowered.
    currentMod: ?u16,
    /// Global data items (string literals, constants, static arrays).
    /// These become the data sections in the final binary.
    data: *mut [il::Data],
    /// Lowered functions.
    fns: *mut [*il::Fn],
    /// Map of function symbols to qualified names.
    fnSyms: *mut [FnSymEntry],
    /// Global error type tag table. Maps nominal types to unique tags.
    errTags: *mut [ErrTagEntry],
    /// Next error tag to assign (starts at 1; 0 = success).
    errTagCounter: u32,
    /// Lowering options.
    options: LowerOptions,
}

/// Entry mapping a function symbol to its qualified name.
record FnSymEntry {
    sym: *resolver::Symbol,
    qualName: *[u8],
}

/// Entry in the global error tag table.
record ErrTagEntry {
    /// The type of this error, identified by its interned pointer.
    ty: resolver::Type,
    /// The globally unique tag assigned to this error type (non-zero).
    tag: u32,
}

/// Compute the maximum size of any error type in a throw list.
fn maxErrSize(throwList: *[*resolver::Type]) -> u32 {
    let mut maxSize: u32 = 0;
    for ty in throwList {
        let size = resolver::getTypeLayout(*ty).size;
        if size > maxSize {
            maxSize = size;
        }
    }
    return maxSize;
}

/// Get or assign a globally unique error tag for the given error type.
/// Tag `0` is reserved for success; error tags start at `1`.
fn getOrAssignErrorTag(self: *mut Lowerer, errType: resolver::Type) -> u32 {
    for entry in self.errTags {
        if entry.ty == errType {
            return entry.tag;
        }
    }
    let tag = self.errTagCounter;

    self.errTagCounter += 1;
    self.errTags.append(ErrTagEntry { ty: errType, tag }, self.allocator);

    return tag;
}

/// Builder for accumulating data values during constant lowering.
record DataValueBuilder {
    allocator: alloc::Allocator,
    values: *mut [il::DataValue],
    /// Whether all values pushed are undefined.
    allUndef: bool,
}

/// Result of lowering constant data.
record ConstDataResult {
    values: *[il::DataValue],
    isUndefined: bool,
}

/// Create a new builder.
fn dataBuilder(allocator: alloc::Allocator) -> DataValueBuilder {
    return DataValueBuilder { allocator, values: &mut [], allUndef: true };
}

/// Append a data value to the builder.
fn dataBuilderPush(b: *mut DataValueBuilder, value: il::DataValue) {
    b.values.append(value, b.allocator);

    if value.item != il::DataItem::Undef {
        b.allUndef = false;
    }
}

/// Return the accumulated values.
fn dataBuilderFinish(b: *DataValueBuilder) -> ConstDataResult {
    return ConstDataResult {
        values: &b.values[..],
        isUndefined: b.allUndef,
    };
}

///////////////////////////
// SSA Variable Tracking //
///////////////////////////

// The SSA construction algorithm tracks variable definitions per-block.
// Each source-level variable gets a [`Var`] handle, and each block maintains
// a mapping from [`Var`] to current SSA value. When control flow merges,
// block parameters are inserted to merge values from different control flow paths.

/// A variable handle. Represents a source-level variable during lowering.
/// The same [`Var`] can have different SSA values in different blocks.
// TODO: Use `Var(u32)`.
pub record Var {
    /// Index into the function's variables array.
    id: u32,
}

/// Metadata for a source-level variable, stored once per function.
///
/// Each variable declaration in the source creates one [`VarData`] entry in the
/// function's `variables` array, indexed by `id`. This contains static
/// properties that don't change across basic blocks.
///
/// Per-block SSA values are tracked separately in [`BlockData::vars`] as [`?il::Val`],
/// where `nil` means "not yet assigned in this block". Together they implement
/// SSA construction.
///
record VarData {
    /// Variable name, used by [`lookupVarByName`] to resolve identifiers.
    /// Nil for anonymous variables (e.g., internal loop counters).
    name: ?*[u8],
    /// IL type of this variable. Set at declaration time and used when
    /// generating loads, stores, and type-checking assignments.
    type: il::Type,
    /// Whether this variable was declared with `mut`. Controls whether [`defVar`]
    /// is allowed after the initial definition.
    mutable: bool,
    /// Whether this variable's address has been taken (e.g. via `&mut x`).
    /// When true, the SSA value is a pointer to a stack slot and reads/writes
    /// must go through memory instead of using the cached SSA value directly.
    addressTaken: bool,
}

/// Links a function parameter to its corresponding variable for the entry block.
/// After creating the entry block, we iterate through these to define initial values.
record FnParamBinding {
    /// The variable that receives this parameter's value.
    var: Var,
    /// SSA register containing the parameter value from the caller.
    reg: il::Reg,
}

////////////////////////////////
// Basic Block Representation //
////////////////////////////////

// During lowering, we build a CFG of basic blocks. Each block accumulates
// instructions until terminated by a jump, branch, return, or unreachable.
// Blocks can be created before they're filled (forward references for jumps).

/// A handle to a basic block within the current function.
/// Block handles are stable, they don't change as more blocks are added.
pub record BlockId {
    /// Index into the function's block array.
    n: u32,
}

/// Internal block state during construction.
///
/// The key invariants:
///
/// - A block is "open" if it has no terminator; instructions can be added.
/// - A block is "sealed" when all predecessor edges are known.
/// - Sealing is required before SSA construction can insert block parameters.
///
/// This differs from the final [`il::Block`] which is immutable and fully formed.
record BlockData {
    /// Block label for debugging and IL printing.
    label: *[u8],
    /// Block parameters for merging values at control flow joins. These
    /// receive values from predecessor edges when control flow merges.
    params: *mut [il::Param],
    /// Variable ids corresponding to each parameter. Used to map block params
    /// back to source variables when building argument lists for jumps.
    paramVars: *mut [u32],
    /// Instructions accumulated so far. The last instruction should eventually
    /// be a terminator.
    instrs: *mut [il::Instr],
    /// Debug source locations, one per instruction. Only populated when
    /// debug info is enabled.
    locs: *mut [il::SrcLoc],
    /// Predecessor block ids. Used for SSA construction to propagate values
    /// from predecessors when a variable is used before being defined locally.
    preds: *mut [u32],
    /// The current SSA value of each variable in this block. Indexed by variable
    /// id. A `nil` means the variable wasn't assigned in this block. Updated by
    /// [`defVar`], queried by [`useVarInBlock`].
    vars: *mut [?il::Val],
    /// Sealing state. Once sealed, all predecessors are known and we can resolve
    /// variable uses that need to pull values from predecessors.
    sealState: Sealed,
    /// Loop nesting depth when this block was created.
    loopDepth: u32,
}

/// Block sealing state for SSA construction.
///
/// A block is "unsealed" while its predecessors are still being discovered.
/// During this time, variables used before being defined locally are tracked.
/// Once all predecessors are known, the block is sealed and those variables
/// are resolved via [`resolveBlockArgs`].
union Sealed {
    /// Block is unsealed; predecessors may still be added.
    No { incompleteVars: *mut [u32] },
    /// Block is sealed; all predecessors are known.
    Yes,
}

///////////////////////////////////
// Loop and Control Flow Context //
///////////////////////////////////

/// Context for break/continue statements within a loop.
/// Each nested loop pushes a new context onto the loop stack.
pub record LoopCtx {
    /// Where `break` should transfer control (the loop's exit block).
    breakTarget: BlockId,
    /// Where `continue` should transfer control.
    continueTarget: ?BlockId,
}

/// Logical operator.
union LogicalOp { And, Or }

/// Iterator state for for-loop lowering.
union ForIter {
    /// Range iterator: `for i in 0..n`.
    Range {
        valVar: Var,
        indexVar: ?Var,
        endVal: il::Val,
        valType: il::Type,
        unsigned: bool,
    },
    /// Collection iterator: `for elem in slice`.
    Collection {
        valVar: ?Var,
        idxVar: Var,
        dataReg: il::Reg,
        lengthVal: il::Val,
        elemType: *resolver::Type,
    },
}


//////////////////////////////
// Pattern Matching Support //
//////////////////////////////

// Match expressions are lowered by evaluating the subject once, then emitting
// a chain of comparison-and-branch sequences for each arm. The algorithm
// handles several subject types specially:
//
// - Optional pointers: compared against `null`.
// - Optional aggregates: tag checked then payload extracted.
// - Unions: tag compared against variant indices.

/// Cached information about a match subject. Computed once and reused across
/// all arms to avoid redundant lowering and type queries.
record MatchSubject {
    /// The lowered subject value.
    val: il::Val,
    /// Source-level type from the resolver.
    type: resolver::Type,
    /// IL-level type for code generation.
    ilType: il::Type,
    /// The type that binding arms should use. For optionals, this is the
    /// inner type; for regular values, it's the same as `type`.
    bindType: resolver::Type,
    /// Classification of how the subject should be compared and destructured.
    kind: MatchSubjectKind,
    /// How bindings are created: by value, or by reference.
    by: resolver::MatchBy,
}

/// Classifies a match subject by how it should be compared and destructured.
union MatchSubjectKind {
    /// Regular value: direct equality comparison.
    Regular,
    /// Optional with null pointer optimization: `?*T`, `?*[T]`.
    OptionalPtr,
    /// Optional aggregate `?T`: tagged union with payload.
    OptionalAggregate,
    /// Union type: tag compared against variant indices.
    Union(resolver::UnionType),
}

/// Determine the kind of a match subject from its type.
fn matchSubjectKind(type: resolver::Type) -> MatchSubjectKind {
    if resolver::isOptionalPointer(type) {
        return MatchSubjectKind::OptionalPtr;
    }
    if resolver::isOptionalAggregate(type) {
        return MatchSubjectKind::OptionalAggregate;
    }
    if let info = unionInfoFromType(type) {
        return MatchSubjectKind::Union(info);
    }
    return MatchSubjectKind::Regular;
}

//////////////////////////
// Field Access Support //
//////////////////////////

/// Result of resolving a field access expression to a memory location.
record FieldRef {
    /// Base pointer register (points to the container).
    base: il::Reg,
    /// Byte offset of the field within the aggregate.
    offset: i32,
    /// Type of the field value.
    fieldType: resolver::Type,
}

/// Result of computing an element pointer for array/slice subscript operations.
/// Used by [`lowerElemPtr`] to return both the element address register and
/// the element type for subsequent load or address-of operations.
record ElemPtrResult {
    /// Register holding the computed element address.
    elemReg: il::Reg,
    /// Source-level type of the element.
    elemType: resolver::Type,
}

/////////////////////////////
// Function Lowering State //
/////////////////////////////

/// Per-function lowering state. Created fresh for each function and contains
/// all the mutable state needed during function body lowering.
record FnLowerer {
    /// Reference to the module-level lowerer.
    low: *mut Lowerer,
    /// Allocator for IL allocations.
    allocator: alloc::Allocator,
    /// Type signature of the function being lowered.
    fnType: *resolver::FnType,
    /// Function name, used as prefix for generated data symbols.
    fnName: *[u8],

    // ~ SSA variable tracking ~ //

    /// Metadata (name, type, mutability) for each variable. Indexed by variable
    /// id. Doesn't change after declaration. For the SSA value of a variable in
    /// a specific block, see [`BlockData::vars`].
    vars: *mut [VarData],
    /// Parameter-to-variable bindings, initialized in the entry block.
    params: *mut [FnParamBinding],

    // ~ Basic block management ~ //

    /// Block storage array, indexed by block id.
    blockData: *mut [BlockData],
    /// The entry block for this function.
    entryBlock: ?BlockId,
    /// The block currently receiving new instructions.
    currentBlock: ?BlockId,

    // ~ Loop management ~ //

    /// Stack of loop contexts for break/continue resolution.
    loopStack: *mut [LoopCtx],
    /// Current nesting depth (index into loopStack).
    loopDepth: u32,

    // ~ Counters ~ //

    /// Counter for generating unique block labels like `then#0`, `loop#1`, etc.
    labelCounter: u32,
    /// Counter for generating unique data names within this function.
    /// Each literal gets a name like `fnName#N`.
    dataCounter: u32,
    /// Counter for generating SSA register numbers.
    regCounter: u32,
    /// When the function returns an aggregate type, the caller passes a hidden
    /// pointer as the first parameter. The callee writes the return value into
    /// this buffer and returns the pointer.
    returnReg: ?il::Reg,

    // ~ Debug info ~ //

    /// Current debug source location, set when processing AST nodes.
    srcLoc: il::SrcLoc,
}

/////////////////////////////////
// Module Lowering Entry Point //
/////////////////////////////////

/// Lower a complete module AST to an IL program.
///
/// This is the main entry point for the lowering pass. It:
/// 1. Counts functions to preallocate the output array.
/// 2. Iterates over top-level declarations, lowering each.
/// 3. Returns the complete IL program with functions and data section.
///
/// The resolver must have already processed the AST -- we rely on its type
/// annotations, symbol table, and constant evaluations.
pub fn lower(
    res: *resolver::Resolver,
    root: *ast::Node,
    pkgName: *[u8],
    arena: *mut alloc::Arena
) -> il::Program throws (LowerError) {
    let mut low = Lowerer {
        arena: arena,
        allocator: alloc::arenaAllocator(arena),
        resolver: res,
        moduleGraph: nil,
        pkgName,
        currentMod: nil,
        data: &mut [],
        fns: &mut [],
        fnSyms: &mut [],
        errTags: &mut [],
        errTagCounter: 1,
        options: LowerOptions { debug: false, buildTest: false },
    };
    let defaultFnIdx = try lowerDecls(&mut low, root, true);

    return il::Program {
        data: &low.data[..],
        fns: &low.fns[..],
        defaultFnIdx,
    };
}

/////////////////////////////////
// Multi-Module Lowering API   //
/////////////////////////////////

/// Create a lowerer for multi-module compilation.
pub fn lowerer(
    res: *resolver::Resolver,
    graph: *module::ModuleGraph,
    pkgName: *[u8],
    arena: *mut alloc::Arena,
    options: LowerOptions
) -> Lowerer {
    return Lowerer {
        arena,
        allocator: alloc::arenaAllocator(arena),
        resolver: res,
        moduleGraph: graph,
        pkgName,
        currentMod: nil,
        data: &mut [],
        fns: &mut [],
        fnSyms: &mut [],
        errTags: &mut [],
        errTagCounter: 1,
        options,
    };
}

/// Lower a module's AST into the lowerer accumulator.
/// Call this for each module in the package, then use `finalize` to get the program.
pub fn lowerModule(
    low: *mut Lowerer,
    moduleId: u16,
    root: *ast::Node,
    isRoot: bool
) -> ?u32 throws (LowerError) {
    low.currentMod = moduleId;
    return try lowerDecls(low, root, isRoot);
}

/// Lower all top-level declarations in a block.
fn lowerDecls(low: *mut Lowerer, root: *ast::Node, isRoot: bool) -> ?u32 throws (LowerError) {
    let case ast::NodeValue::Block(block) = root.value else {
        throw LowerError::ExpectedBlock(root);
    };
    let stmtsList = block.statements;
    let mut defaultFnIdx: ?u32 = nil;

    for node in stmtsList {
        match node.value {
            case ast::NodeValue::FnDecl(decl) => {
                if let f = try lowerFnDecl(low, node, decl) {
                    if isRoot and checkAttr(decl.attrs, ast::Attribute::Default) {
                        defaultFnIdx = low.fns.len;
                    }
                    low.fns.append(f, low.allocator);
                }
            }
            case ast::NodeValue::ConstDecl(decl) => {
                try lowerDataDecl(low, node, decl.value, true);
            }
            case ast::NodeValue::StaticDecl(decl) => {
                try lowerDataDecl(low, node, decl.value, false);
            }
            case ast::NodeValue::InstanceDecl { traitName, targetType, methods } => {
                try lowerInstanceDecl(low, node, traitName, targetType, methods);
            }
            else => {},
        }
    }
    return defaultFnIdx;
}

/// Finalize lowering and return the unified IL program.
pub fn finalize(low: *Lowerer, defaultFnIdx: ?u32) -> il::Program {
    return il::Program {
        data: &low.data[..],
        fns: &low.fns[..],
        defaultFnIdx,
    };
}

/////////////////////////////////
// Qualified Name Construction //
/////////////////////////////////

/// Get module path segments for the current or specified module.
/// Returns empty slice if no module graph or module not found.
fn getModulePath(self: *mut Lowerer, modId: ?u16) -> *[*[u8]] {
    let graph = self.moduleGraph else {
        return &[];
    };
    let mut id = modId;
    if id == nil {
        id = self.currentMod;
    }
    let actualId = id else {
        return &[];
    };
    let entry = module::get(graph, actualId) else {
        return &[];
    };
    return module::moduleQualifiedPath(entry);
}

/// Build a qualified name string for a symbol.
/// If `modId` is nil, uses current module.
fn qualifyName(self: *mut Lowerer, modId: ?u16, name: *[u8]) -> *[u8] {
    let path = getModulePath(self, modId);
    if path.len == 0 {
        return name;
    }
    return il::formatQualifiedName(self.arena, path, name);
}

/// Register a function symbol with its qualified name.
/// Called when lowering function declarations, so cross-package calls can find
/// the function by name.
fn registerFnSym(self: *mut Lowerer, sym: *resolver::Symbol, qualName: *[u8]) {
    self.fnSyms.append(FnSymEntry { sym, qualName }, self.allocator);
}

/// Look up a function's qualified name by its symbol.
/// Returns `nil` if the symbol wasn't registered (e.g. callee's module is not yet lowered).
// TODO: This is kind of dubious as an optimization, if it depends on the order
// in which modules are lowered.
// TODO: Use a hash table here?
fn lookupFnSym(self: *Lowerer, sym: *resolver::Symbol) -> ?*[u8] {
    for entry in self.fnSyms {
        if entry.sym == sym {
            return entry.qualName;
        }
    }
    return nil;
}

/// Set the package context for lowering.
/// Called before lowering each package.
pub fn setPackage(self: *mut Lowerer, graph: *module::ModuleGraph, pkgName: *[u8]) {
    self.moduleGraph = graph;
    self.pkgName = pkgName;
    self.currentMod = nil;
}

/// Create a new function lowerer for a given function type and name.
fn fnLowerer(
    self: *mut Lowerer,
    node: *ast::Node,
    fnType: *resolver::FnType,
    qualName: *[u8]
) -> FnLowerer {
    let loopStack = try! alloc::allocSlice(self.arena, @sizeOf(LoopCtx), @alignOf(LoopCtx), MAX_LOOP_DEPTH) as *mut [LoopCtx];

    let mut fnLow = FnLowerer {
        low: self,
        allocator: alloc::arenaAllocator(self.arena),
        fnType: fnType,
        fnName: qualName,
        vars: &mut [],
        params: &mut [],
        blockData: &mut [],
        entryBlock: nil,
        currentBlock: nil,
        loopStack,
        loopDepth: 0,
        labelCounter: 0,
        dataCounter: 0,
        regCounter: 0,
        returnReg: nil,
        srcLoc: undefined,
    };
    if self.options.debug {
        let modId = self.currentMod else {
            panic "fnLowerer: debug enabled but no current module";
        };
        fnLow.srcLoc = il::SrcLoc {
            moduleId: modId,
            offset: node.span.offset,
        };
    }
    return fnLow;
}

/// Lower a function declaration.
///
/// This sets up the per-function lowering state, processes parameters,
/// then lowers the function body into a CFG of basic blocks.
///
/// For throwing functions, the return type is a result aggregate
/// rather than the declared return type.
fn lowerFnDecl(self: *mut Lowerer, node: *ast::Node, decl: ast::FnDecl) -> ?*il::Fn throws (LowerError) {
    if not shouldLowerFn(&decl, self.options.buildTest) {
        return nil;
    }
    let case ast::NodeValue::Ident(name) = decl.name.value else {
        throw LowerError::ExpectedIdentifier;
    };
    let data = resolver::nodeData(self.resolver, node);
    let case resolver::Type::Fn(fnType) = data.ty else {
        throw LowerError::ExpectedFunction;
    };
    let isExtern = checkAttr(decl.attrs, ast::Attribute::Extern);

    // Build qualified function name for multi-module compilation.
    let qualName = qualifyName(self, nil, name);

    // Register function symbol for cross-package call resolution.
    if let sym = data.sym {
        registerFnSym(self, sym, qualName);
    }
    let mut fnLow = fnLowerer(self, node, fnType, qualName);

    // If the function returns an aggregate or is throwing, prepend a hidden
    // return parameter. The caller allocates the buffer and passes it
    // as the first argument; the callee writes the return value into it.
    if requiresReturnParam(fnType) and not isExtern {
        fnLow.returnReg = nextReg(&mut fnLow);
    }
    let lowParams = try lowerParams(&mut fnLow, *fnType, decl.sig.params, nil);
    let func = try! alloc::alloc(self.arena, @sizeOf(il::Fn), @alignOf(il::Fn)) as *mut il::Fn;

    *func = il::Fn {
        name: qualName,
        params: lowParams,
        returnType: undefined,
        isExtern,
        blocks: &[],
    };
    // Throwing functions return a result aggregate (word-sized pointer).
    // TODO: The resolver should set an appropriate type that takes into account
    //       the throws list. It shouldn't set the return type to the "success"
    //       value only.
    if fnType.throwList.len > 0 {
        func.returnType = il::Type::W64;
    } else {
        func.returnType = ilType(self, *fnType.returnType);
    }
    let body = decl.body else {
        // Extern functions have no body.
        assert isExtern;
        return func;
    };
    func.blocks = try lowerFnBody(&mut fnLow, body);

    return func;
}

/// Build a qualified name of the form "Type::method".
fn instanceMethodName(self: *mut Lowerer, modId: ?u16, typeName: *[u8], methodName: *[u8]) -> *[u8] {
    let sepLen: u32 = 2; // "::"
    let totalLen = typeName.len + sepLen + methodName.len;
    let buf = try! alloc::allocSlice(self.arena, 1, 1, totalLen) as *mut [u8];
    let mut pos: u32 = 0;

    pos += try! mem::copy(&mut buf[pos..], typeName);
    pos += try! mem::copy(&mut buf[pos..], "::");
    pos += try! mem::copy(&mut buf[pos..], methodName);

    // TODO: Assert that `pos` equals `totalLen`.

    return qualifyName(self, modId, &buf[..totalLen]);
}

/// Build a v-table data name of the form "vtable::Type::Trait".
fn vtableName(self: *mut Lowerer, modId: ?u16, typeName: *[u8], traitName: *[u8]) -> *[u8] {
    let prefix = "vtable::";
    let sepLen: u32 = 2; // "::"
    let totalLen = prefix.len + typeName.len + sepLen + traitName.len;
    let buf = try! alloc::allocSlice(self.arena, 1, 1, totalLen) as *mut [u8];
    let mut pos: u32 = 0;

    pos += try! mem::copy(&mut buf[pos..], prefix);
    pos += try! mem::copy(&mut buf[pos..], typeName);
    pos += try! mem::copy(&mut buf[pos..], "::");
    pos += try! mem::copy(&mut buf[pos..], traitName);

    // TODO: Assert that `pos` equals `totalLen`.

    return qualifyName(self, modId, &buf[..totalLen]);
}

/// Lower an instance declaration (`instance Trait for Type { ... }`).
///
/// Each method in the instance block is lowered as a standalone function
/// with a qualified name of the form `Type::method`. A read-only v-table
/// data record is emitted containing pointers to these functions, ordered
/// by the trait's method indices. The v-table is later referenced when
/// constructing trait objects for dynamic dispatch.
fn lowerInstanceDecl(
    self: *mut Lowerer,
    node: *ast::Node,
    traitNameNode: *ast::Node,
    targetTypeNode: *ast::Node,
    methods: *mut [*ast::Node]
) throws (LowerError) {
    // Look up the trait and type from the resolver.
    let traitSym = resolver::nodeData(self.resolver, traitNameNode).sym
        else throw LowerError::MissingSymbol(traitNameNode);
    let case resolver::SymbolData::Trait(traitInfo) = traitSym.data
        else throw LowerError::MissingMetadata;
    let typeSym = resolver::nodeData(self.resolver, targetTypeNode).sym
        else throw LowerError::MissingSymbol(targetTypeNode);

    let tName = traitSym.name;
    let typeName = typeSym.name;

    // Lower each instance method as a regular function.
    // Collect qualified names for the v-table. Empty entries are filled
    // later from inherited supertrait methods.
    let mut methodNames: [*[u8]; ast::MAX_TRAIT_METHODS] = undefined;
    let mut methodNameSet: [bool; ast::MAX_TRAIT_METHODS] = [false; ast::MAX_TRAIT_METHODS];

    for methodNode in methods {
        let case ast::NodeValue::InstanceMethodDecl {
            name, receiverName, receiverType, sig, body
        } = methodNode.value else continue;

        let case ast::NodeValue::Ident(mName) = name.value else {
            throw LowerError::ExpectedIdentifier;
        };
        let qualName = instanceMethodName(self, nil, typeName, mName);

        // Get the function type from the resolver.
        let data = resolver::nodeData(self.resolver, methodNode);
        let case resolver::Type::Fn(fnType) = data.ty else {
            throw LowerError::ExpectedFunction;
        };

        // Register the method symbol.
        if let sym = data.sym {
            registerFnSym(self, sym, qualName);
        }

        // Lower the method as a normal function.
        let mut fnLow = fnLowerer(self, methodNode, fnType, qualName);
        if requiresReturnParam(fnType) {
            fnLow.returnReg = nextReg(&mut fnLow);
        }
        let lowParams = try lowerParams(&mut fnLow, *fnType, sig.params, receiverName);
        let func = try! alloc::alloc(self.arena, @sizeOf(il::Fn), @alignOf(il::Fn)) as *mut il::Fn;

        *func = il::Fn {
            name: qualName,
            params: lowParams,
            returnType: ilType(self, *fnType.returnType),
            isExtern: false,
            blocks: &[],
        };
        if fnType.throwList.len > 0 {
            func.returnType = il::Type::W64;
        }
        func.blocks = try lowerFnBody(&mut fnLow, body);
        self.fns.append(func, self.allocator);

        let method = resolver::findTraitMethod(traitInfo, mName)
            else panic "lowerInstanceDecl: method not found in trait";

        methodNames[method.index] = qualName;
        methodNameSet[method.index] = true;
    }

    // Fill inherited method slots from supertraits.
    // These methods were already lowered as part of the supertrait instance
    // declarations and use the same `Type::method` qualified name.
    for method, i in traitInfo.methods {
        if not methodNameSet[i] {
            methodNames[i] = instanceMethodName(self, nil, typeName, method.name);
        }
    }

    // Create v-table in data section, used for dynamic dispatch.
    let vName = vtableName(self, nil, typeName, tName);
    let values = try! alloc::allocSlice(
        self.arena, @sizeOf(il::DataValue), @alignOf(il::DataValue), traitInfo.methods.len as u32
    ) as *mut [il::DataValue];

    for i in 0..traitInfo.methods.len {
        values[i] = il::DataValue {
            item: il::DataItem::Fn(methodNames[i]),
            count: 1,
        };
    }
    self.data.append(il::Data {
        name: vName,
        size: traitInfo.methods.len as u32 * resolver::PTR_SIZE,
        alignment: resolver::PTR_SIZE,
        readOnly: true,
        isUndefined: false,
        values: &values[..traitInfo.methods.len as u32],
    }, self.allocator);
}

/// Check if a function should be lowered.
fn shouldLowerFn(decl: *ast::FnDecl, buildTest: bool) -> bool {
    if checkAttr(decl.attrs, ast::Attribute::Test) {
        return buildTest;
    }
    return true;
}

/// Check if a specific attribute is present in the attribute set.
fn checkAttr(attrs: ?ast::Attributes, attr: ast::Attribute) -> bool {
    if let a = attrs {
        return ast::attributesContains(&a, attr);
    }
    return false;
}

/// Create a label with a numeric suffix, eg. `@base0`.
/// This ensures unique labels like `@then0`, `@then1`, etc.
fn labelWithSuffix(self: *mut FnLowerer, base: *[u8], suffix: u32) -> *[u8] throws (LowerError) {
    let mut digits: [u8; fmt::U32_STR_LEN] = undefined;
    let suffixText = fmt::formatU32(suffix, &mut digits[..]);
    let totalLen = base.len + suffixText.len;
    let buf = try! alloc::allocSlice(self.low.arena, 1, 1, totalLen) as *mut [u8];

    try! mem::copy(&mut buf[..base.len], base);
    try! mem::copy(&mut buf[base.len..totalLen], suffixText);

    return &buf[..totalLen];
}

/// Generate a unique label by appending the global counter to the base.
fn nextLabel(self: *mut FnLowerer, base: *[u8]) -> *[u8] throws (LowerError) {
    let idx = self.labelCounter;
    self.labelCounter += 1;

    return try labelWithSuffix(self, base, idx);
}

///////////////////////////////
// Data Section Construction //
///////////////////////////////

// Functions for building the data section from const/static
// declarations and inline literals.

/// Convert a constant integer payload to a signed 64-bit value.
fn constIntToI64(intVal: resolver::ConstInt) -> i64 {
    return (0 - intVal.magnitude as i64) if intVal.negative else intVal.magnitude as i64;
}

/// Convert a scalar constant value to an i64.
/// String constants are not handled here and should be checked before calling.
/// Panics if a non-scalar value is passed.
fn constToScalar(val: resolver::ConstValue) -> i64 {
    match val {
        case resolver::ConstValue::Bool(b) => return 1 if b else 0,
        case resolver::ConstValue::Char(c) => return c as i64,
        case resolver::ConstValue::Int(i) => return constIntToI64(i),
        else => panic,
    }
}

/// Convert a constant value to an IL value.
fn constValueToVal(self: *mut FnLowerer, val: resolver::ConstValue, node: *ast::Node) -> il::Val throws (LowerError) {
    if let case resolver::ConstValue::String(s) = val {
        return try lowerStringLit(self, node, s);
    }
    return il::Val::Imm(constToScalar(val));
}

/// Convert a resolver constant value to an IL data initializer item.
fn constValueToDataItem(self: *mut Lowerer, val: resolver::ConstValue, typ: resolver::Type) -> il::DataItem {
    if let case resolver::ConstValue::String(s) = val {
        return il::DataItem::Str(s);
    }
    // Bool and char are byte-sized; integer uses the declared type.
    let mut irTyp = il::Type::W8;
    if let case resolver::ConstValue::Int(_) = val {
        irTyp = ilType(self, typ);
    }
    return il::DataItem::Val { typ: irTyp, val: constToScalar(val) };
}

/// Lower a const or static declaration to the data section.
fn lowerDataDecl(
    self: *mut Lowerer,
    node: *ast::Node,
    value: *ast::Node,
    readOnly: bool
) throws (LowerError) {
    let data = resolver::nodeData(self.resolver, node);
    let sym = data.sym else {
        throw LowerError::MissingSymbol(node);
    };
    if data.ty == resolver::Type::Unknown {
        throw LowerError::MissingType(node);
    }
    let layout = resolver::getTypeLayout(data.ty);
    let qualName = qualifyName(self, nil, sym.name);
    let mut b = dataBuilder(self.allocator);
    try lowerConstDataInto(self, value, data.ty, layout.size, qualName, &mut b);
    let result = dataBuilderFinish(&b);

    self.data.append(il::Data {
        name: qualName,
        size: layout.size,
        alignment: layout.alignment,
        readOnly,
        isUndefined: result.isUndefined,
        values: result.values,
    }, self.allocator);
}

/// Emit the in-memory representation of a slice header: `{ ptr, len, cap }`.
fn dataSliceHeader(b: *mut DataValueBuilder, dataSym: *[u8], len: u32) {
    dataBuilderPush(b, il::DataValue {
        item: il::DataItem::Sym(dataSym),
        count: 1
    });
    dataBuilderPush(b, il::DataValue {
        item: il::DataItem::Val {
            typ: il::Type::W32,
            val: len as i64
        },
        count: 1
    });
    dataBuilderPush(b, il::DataValue {
        item: il::DataItem::Val {
            typ: il::Type::W32,
            val: len as i64
        },
        count: 1
    });
}

/// Lower a compile-time `&[...]` expression to a concrete slice header.
fn lowerConstAddressSliceInto(
    self: *mut Lowerer,
    addr: ast::AddressOf,
    ty: resolver::Type,
    dataPrefix: *[u8],
    b: *mut DataValueBuilder
) throws (LowerError) {
    let case resolver::Type::Slice { mutable, .. } = ty
        else throw LowerError::ExpectedSliceOrArray;
    let targetTy = resolver::typeFor(self.resolver, addr.target)
        else throw LowerError::MissingType(addr.target);
    let case resolver::Type::Array(arrInfo) = targetTy
        else throw LowerError::ExpectedArray;

    let mut nested = dataBuilder(self.allocator);
    let layout = resolver::getTypeLayout(targetTy);
    try lowerConstDataInto(self, addr.target, targetTy, layout.size, dataPrefix, &mut nested);

    let backing = dataBuilderFinish(&nested);
    let readOnly = not mutable;
    let mut dataName: *[u8] = undefined;
    if readOnly {
        if let found = findConstData(self, backing.values, layout.alignment) {
            dataName = found;
        } else {
            dataName = try pushDeclData(self, layout.size, layout.alignment, readOnly, backing.values, dataPrefix);
        }
    } else {
        dataName = try pushDeclData(self, layout.size, layout.alignment, readOnly, backing.values, dataPrefix);
    }
    dataSliceHeader(b, dataName, arrInfo.length);
}

/// Lower a constant expression into a builder, padding to slotSize.
fn lowerConstDataInto(
    self: *mut Lowerer,
    node: *ast::Node,
    ty: resolver::Type,
    slotSize: u32,
    dataPrefix: *[u8],
    b: *mut DataValueBuilder
) throws (LowerError) {
    let layout = resolver::getTypeLayout(ty);

    // Function pointer references in constant data.
    if let case resolver::Type::Fn(_) = ty {
        let sym = resolver::nodeData(self.resolver, node).sym
            else throw LowerError::MissingSymbol(node);
        let modId = resolver::moduleIdForSymbol(self.resolver, sym);
        let qualName = qualifyName(self, modId, sym.name);
        dataBuilderPush(b, il::DataValue {
            item: il::DataItem::Fn(qualName), count: 1,
        });
        let pad = slotSize - layout.size;
        if pad > 0 {
            dataBuilderPush(b, il::DataValue {
                item: il::DataItem::Undef, count: pad,
            });
        }
        return;
    }

    match node.value {
        case ast::NodeValue::Undef => {
            dataBuilderPush(b, il::DataValue {
                item: il::DataItem::Undef,
                count: layout.size
            });
        }
        case ast::NodeValue::ArrayLit(elems) =>
            try lowerConstArrayLitInto(self, elems, ty, dataPrefix, b),
        case ast::NodeValue::ArrayRepeatLit(repeat) =>
            try lowerConstArrayRepeatInto(self, repeat, ty, dataPrefix, b),
        case ast::NodeValue::RecordLit(recLit) =>
            try lowerConstRecordLitInto(self, node, recLit, ty, dataPrefix, b),
        case ast::NodeValue::Call(call) => {
            let calleeSym = resolver::nodeData(self.resolver, call.callee).sym
                else throw LowerError::MissingSymbol(call.callee);
            match calleeSym.data {
                case resolver::SymbolData::Variant { .. } =>
                    try lowerConstUnionVariantInto(self, node, calleeSym, ty, call.args, dataPrefix, b),
                case resolver::SymbolData::Type(resolver::NominalType::Record(recInfo)) => {
                    try lowerConstRecordCtorInto(self, call.args, recInfo, dataPrefix, b);
                }
                else => throw LowerError::MissingConst(node),
            }
        }
        case ast::NodeValue::AddressOf(addr) => {
            try lowerConstAddressSliceInto(self, addr, ty, dataPrefix, b);
        }
        case ast::NodeValue::Ident(_) => {
            // Identifier referencing a constant.
            let sym = resolver::nodeData(self.resolver, node).sym
                else throw LowerError::MissingSymbol(node);
            let case ast::NodeValue::ConstDecl(decl) = sym.node.value
                else throw LowerError::MissingConst(node);

            try lowerConstDataInto(self, decl.value, ty, slotSize, dataPrefix, b);
        },
        else => {
            // Scalar values: integers, bools, strings, void union variants, etc.
            let val = resolver::constValueEntry(self.resolver, node)
                else throw LowerError::MissingConst(node);

            if let case resolver::ConstValue::String(s) = val {
                if let case resolver::Type::Slice { .. } = ty {
                    let strSym = try getOrCreateStringData(self, s, dataPrefix);
                    dataSliceHeader(b, strSym, s.len);
                } else {
                    dataBuilderPush(b, il::DataValue {
                        item: constValueToDataItem(self, val, ty),
                        count: 1
                    });
                }
            } else {
                dataBuilderPush(b, il::DataValue {
                    item: constValueToDataItem(self, val, ty),
                    count: 1
                });
            }
        }
    }
    // Pad to fill the slot.
    let padding = slotSize - layout.size;
    if padding > 0 {
        dataBuilderPush(b, il::DataValue { item: il::DataItem::Undef, count: padding });
    }
}

/// Flatten a constant array literal `[a, b, c]` into a builder.
fn lowerConstArrayLitInto(
    self: *mut Lowerer,
    elems: *mut [*ast::Node],
    ty: resolver::Type,
    dataPrefix: *[u8],
    b: *mut DataValueBuilder
) throws (LowerError) {
    let case resolver::Type::Array(arrInfo) = ty
        else throw LowerError::ExpectedArray;
    let elemTy = *arrInfo.item;
    let elemLayout = resolver::getTypeLayout(elemTy);

    for elem in elems {
        try lowerConstDataInto(self, elem, elemTy, elemLayout.size, dataPrefix, b);
    }
}

/// Build data values for a constant array repeat literal `[item; count]`.
fn lowerConstArrayRepeatInto(
    self: *mut Lowerer,
    repeat: ast::ArrayRepeatLit,
    ty: resolver::Type,
    dataPrefix: *[u8],
    b: *mut DataValueBuilder
) throws (LowerError) {
    let case resolver::Type::Array(arrInfo) = ty
        else throw LowerError::ExpectedArray;
    let length = arrInfo.length;
    let elemTy = *arrInfo.item;
    let elemLayout = resolver::getTypeLayout(elemTy);

    if let case ast::NodeValue::Undef = repeat.item.value {
        dataBuilderPush(b, il::DataValue {
            item: il::DataItem::Undef,
            count: elemLayout.size * length
        });
    } else if let val = resolver::constValueEntry(self.resolver, repeat.item) {
        dataBuilderPush(b, il::DataValue {
            item: constValueToDataItem(self, val, elemTy),
            count: length
        });
    } else {
        for _ in 0..length {
            try lowerConstDataInto(self, repeat.item, elemTy, elemLayout.size, dataPrefix, b);
        }
    }
}

/// Build data values for a constant record literal.
/// Each field is lowered with a slot size that includes trailing padding.
fn lowerConstRecordLitInto(
    self: *mut Lowerer,
    node: *ast::Node,
    recLit: ast::RecordLit,
    ty: resolver::Type,
    dataPrefix: *[u8],
    b: *mut DataValueBuilder
) throws (LowerError) {
    match ty {
        case resolver::Type::Nominal(resolver::NominalType::Record(recInfo)) => {
            try lowerConstRecordCtorInto(self, recLit.fields, recInfo, dataPrefix, b);
        }
        case resolver::Type::Nominal(resolver::NominalType::Union(_)) => {
            let typeName = recLit.typeName else {
                throw LowerError::ExpectedVariant;
            };
            let sym = resolver::nodeData(self.resolver, typeName).sym else {
                throw LowerError::MissingSymbol(typeName);
            };
            try lowerConstUnionVariantInto(self, node, sym, ty, recLit.fields, dataPrefix, b);
        }
        else => throw LowerError::ExpectedRecord,
    }
}

/// Build data values for record constants.
fn lowerConstRecordCtorInto(
    self: *mut Lowerer,
    args: *mut [*ast::Node],
    recInfo: resolver::RecordType,
    dataPrefix: *[u8],
    b: *mut DataValueBuilder
) throws (LowerError) {
    let layout = recInfo.layout;
    for argNode, i in args {
        let mut valueNode = argNode;
        if let case ast::NodeValue::RecordLitField(fieldLit) = argNode.value {
            valueNode = fieldLit.value;
        }
        let fieldInfo = recInfo.fields[i];
        let fieldOffset = fieldInfo.offset as u32;

        // Slot extends to the next field's offset,
        // or record size for the last field.
        let slotEnd = recInfo.fields[i + 1].offset as u32 if i + 1 < recInfo.fields.len else layout.size;
        let slotSize = slotEnd - fieldOffset;

        try lowerConstDataInto(self, valueNode, fieldInfo.fieldType, slotSize, dataPrefix, b);
    }
}

/// Build data values for a constant union variant value from payload fields/args.
fn lowerConstUnionVariantInto(
    self: *mut Lowerer,
    node: *ast::Node,
    variantSym: *mut resolver::Symbol,
    ty: resolver::Type,
    payloadArgs: *mut [*ast::Node],
    dataPrefix: *[u8],
    b: *mut DataValueBuilder
) throws (LowerError) {
    let case resolver::SymbolData::Variant { type: payloadType, index, .. } = variantSym.data
        else throw LowerError::UnexpectedNodeValue(node);

    let unionInfo = unionInfoFromType(ty) else {
        throw LowerError::MissingMetadata;
    };
    let unionLayout = resolver::getTypeLayout(ty);
    let payloadSlotSize = unionLayout.size - unionInfo.valOffset;

    // Tag byte.
    dataBuilderPush(b, il::DataValue {
        item: il::DataItem::Val {
            typ: il::Type::W8,
            val: index as i64
        },
        count: 1
    });
    // Padding between tag and payload.
    if unionInfo.valOffset > 1 {
        dataBuilderPush(b, il::DataValue {
            item: il::DataItem::Undef,
            count: unionInfo.valOffset - 1
        });
    }
    if payloadType == resolver::Type::Void {
        if payloadSlotSize > 0 {
            dataBuilderPush(b, il::DataValue {
                item: il::DataItem::Undef,
                count: payloadSlotSize
            });
        }
        return;
    }

    let case resolver::Type::Nominal(resolver::NominalType::Record(payloadRec)) = payloadType else {
        throw LowerError::ExpectedRecord;
    };
    let payloadLayout = payloadRec.layout;
    try lowerConstRecordCtorInto(self, payloadArgs, payloadRec, dataPrefix, b);

    // Unused bytes in the union payload slot for smaller variants.
    if payloadSlotSize > payloadLayout.size {
        dataBuilderPush(b, il::DataValue {
            item: il::DataItem::Undef,
            count: payloadSlotSize - payloadLayout.size
        });
    }
}

/// Find an existing string data entry with matching content.
// TODO: Optimize with hash table or remove?
fn findStringData(self: *Lowerer, s: *[u8]) -> ?*[u8] {
    for d in self.data {
        if d.values.len == 1 {
            if let case il::DataItem::Str(existing) = d.values[0].item {
                if mem::eq(existing, s) {
                    return d.name;
                }
            }
        }
    }
    return nil;
}

/// Generate a unique name for declaration-local backing data entries.
fn nextDeclDataName(self: *mut Lowerer, prefix: *[u8], count: u32) -> *[u8] {
    let mut digits: [u8; fmt::U32_STR_LEN] = undefined;
    let suffix = fmt::formatU32(count, &mut digits[..]);
    let suffixStart = prefix.len + 1;
    let totalLen = suffixStart + suffix.len;
    let buf = try! alloc::allocSlice(self.arena, 1, 1, totalLen) as *mut [u8];

    try! mem::copy(&mut buf[..prefix.len], prefix);
    buf[prefix.len] = '$';
    try! mem::copy(&mut buf[suffixStart..], suffix);

    return &buf[..totalLen];
}

/// Append a data entry using a declaration-scoped name (`prefix$N`).
fn pushDeclData(
    self: *mut Lowerer,
    size: u32,
    alignment: u32,
    readOnly: bool,
    values: *[il::DataValue],
    dataPrefix: *[u8]
) -> *[u8] throws (LowerError) {
    let name = nextDeclDataName(self, dataPrefix, self.data.len);
    self.data.append(il::Data { name, size, alignment, readOnly, isUndefined: false, values }, self.allocator);

    return name;
}

/// Find or create read-only string data and return its symbol name.
fn getOrCreateStringData(
    self: *mut Lowerer,
    s: *[u8],
    dataPrefix: *[u8]
) -> *[u8] throws (LowerError) {
    if let existing = findStringData(self, s) {
        return existing;
    }
    let values = try! alloc::allocSlice(
        self.arena, @sizeOf(il::DataValue), @alignOf(il::DataValue), 1
    ) as *mut [il::DataValue];

    values[0] = il::DataValue {
        item: il::DataItem::Str(s),
        count: 1
    };
    return try pushDeclData(self, s.len, 1, true, &values[..1], dataPrefix);
}

/// Compare two data items for structural equality.
/// Unlike raw byte comparison, this correctly ignores padding bytes in unions.
fn dataItemEq(a: il::DataItem, b: il::DataItem) -> bool {
    match a {
        case il::DataItem::Val { typ: aTyp, val: aVal } =>
            if let case il::DataItem::Val { typ: bTyp, val: bVal } = b {
                return aTyp == bTyp and aVal == bVal;
            } else {
                return false;
            },
        case il::DataItem::Sym(aPtr) =>
            if let case il::DataItem::Sym(bPtr) = b {
                return mem::eq(aPtr, bPtr);
            } else {
                return false;
            },
        case il::DataItem::Fn(aName) =>
            if let case il::DataItem::Fn(bName) = b {
                return mem::eq(aName, bName);
            } else {
                return false;
            },
        case il::DataItem::Str(aStr) =>
            if let case il::DataItem::Str(bStr) = b {
                return mem::eq(aStr, bStr);
            } else {
                return false;
            },
        case il::DataItem::Undef =>
            if let case il::DataItem::Undef = b {
                return true;
            } else {
                return false;
            },
    }
}

/// Compare two data value slices for structural equality.
fn dataValuesEq(a: *[il::DataValue], b: *[il::DataValue]) -> bool {
    if a.len != b.len {
        return false;
    }
    for i in 0..a.len {
        if a[i].count != b[i].count {
            return false;
        }
        if not dataItemEq(a[i].item, b[i].item) {
            return false;
        }
    }
    return true;
}

/// Find an existing read-only slice data entry with matching values.
// TODO: Optimize with hash table or remove?
fn findSliceData(self: *Lowerer, values: *[il::DataValue], alignment: u32) -> ?*[u8] {
    for d in self.data {
        if d.alignment == alignment and d.readOnly and dataValuesEq(d.values, values) {
            return d.name;
        }
    }
    return nil;
}

/// Find existing const data entry with matching content.
/// Handles both string data and slice data.
fn findConstData(self: *Lowerer, values: *[il::DataValue], alignment: u32) -> ?*[u8] {
    // Fast path for strings.
    if values.len == 1 and alignment == 1 {
        if let case il::DataItem::Str(s) = values[0].item {
            return findStringData(self, s);
        }
    }
    // General case: byte comparison of data values.
    return findSliceData(self, values, alignment);
}

/// Lower constant data to a slice value.
/// Creates or reuses a data section entry, then builds a slice header on the stack.
fn lowerConstDataAsSlice(
    self: *mut FnLowerer,
    values: *[il::DataValue],
    alignment: u32,
    readOnly: bool,
    elemTy: *resolver::Type,
    mutable: bool,
    length: u32
) -> il::Val throws (LowerError) {
    let elemLayout = resolver::getTypeLayout(*elemTy);
    let size = elemLayout.size * length;
    let mut dataName: *[u8] = undefined;
    if readOnly {
        if let found = findConstData(self.low, values, alignment) {
            dataName = found;
        } else {
            dataName = try nextDataName(self);
            self.low.data.append(il::Data { name: dataName, size, alignment, readOnly, isUndefined: false, values }, self.low.allocator);
        }
    } else {
        dataName = try nextDataName(self);
        self.low.data.append(il::Data { name: dataName, size, alignment, readOnly, isUndefined: false, values }, self.low.allocator);
    }

    // Get data address.
    let ptrReg = nextReg(self);
    emit(self, il::Instr::Copy { dst: ptrReg, val: il::Val::DataSym(dataName) });

    return try buildSliceValue(
        self, elemTy, mutable, il::Val::Reg(ptrReg), il::Val::Imm(length as i64), il::Val::Imm(length as i64)
    );
}

/// Generate a unique data name for inline literals, eg. `fnName/N`
fn nextDataName(self: *mut FnLowerer) -> *[u8] throws (LowerError) {
    let counter = self.dataCounter;
    self.dataCounter += 1;
    return try labelWithSuffix(self, self.fnName, counter);
}

/// Get the next available SSA register.
fn nextReg(self: *mut FnLowerer) -> il::Reg {
    let reg = il::Reg { n: self.regCounter };
    self.regCounter += 1;
    return reg;
}

/// Remove the last block parameter and its associated variable.
/// Used when detecting a trivial phi that can be eliminated.
fn removeLastBlockParam(self: *mut FnLowerer, block: BlockId) {
    let blk = getBlockMut(self, block);
    if blk.params.len > 0 {
        // TODO: Use `pop`?
        blk.params = @sliceOf(blk.params.ptr, blk.params.len - 1, blk.params.cap);
    }
    if blk.paramVars.len > 0 {
        // TODO: Use `pop`?
        blk.paramVars = @sliceOf(blk.paramVars.ptr, blk.paramVars.len - 1, blk.paramVars.cap);
    }
}

/// Rewrite cached SSA values for a variable across all blocks, and also
/// rewrite any terminator arguments that reference the provisional register.
/// The latter is necessary because recursive SSA resolution may have already
/// patched terminator arguments with the provisional value before it was
/// found to be trivial.
fn rewriteCachedVarValue(self: *mut FnLowerer, v: Var, from: il::Val, to: il::Val) {
    for i in 0..self.blockData.len {
        let blk = getBlockMut(self, BlockId { n: i });
        if blk.vars[v.id] == from {
            blk.vars[v.id] = to;
        }
        if blk.instrs.len > 0 {
            let ix = blk.instrs.len - 1;
            match &mut blk.instrs[ix] {
                case il::Instr::Jmp { args, .. } =>
                    rewriteValInSlice(*args, from, to),
                case il::Instr::Br { thenArgs, elseArgs, .. } => {
                    rewriteValInSlice(*thenArgs, from, to);
                    rewriteValInSlice(*elseArgs, from, to);
                }
                case il::Instr::Switch { defaultArgs, cases, .. } => {
                    rewriteValInSlice(*defaultArgs, from, to);
                    for j in 0..cases.len {
                        rewriteValInSlice(cases[j].args, from, to);
                    }
                }
                else => {}
            }
        }
    }
}

/// Replace all occurrences of `from` with `to` in an args slice.
fn rewriteValInSlice(args: *mut [il::Val], from: il::Val, to: il::Val) {
    for i in 0..args.len {
        if args[i] == from {
            args[i] = to;
        }
    }
}

////////////////////////////
// Basic Block Management //
////////////////////////////

// Basic blocks are the fundamental unit of the CFG. Each block contains a
// sequence of instructions ending in a terminator (jump, branch, return).
//
// The block management API supports forward references -- you can create a
// block before switching to it and emitting instructions.
// This is essential for control flow where we need to reference target blocks
// before we've built them (e.g., the "else" block when building "then").
//
// Sealing is a key concept for SSA construction: a block is sealed once all
// its predecessor edges are known. Before sealing, we can't resolve variable
// uses that require looking up values from predecessors.

/// Create a new basic block with the given label base.
///
/// The block is initially unsealed (predecessors may be added later) and empty.
/// Returns a [`BlockId`] that can be used for jumps and branches. The block must
/// be switched to via [`switchToBlock`] before instructions can be emitted.
fn createBlock(self: *mut FnLowerer, labelBase: *[u8]) -> BlockId throws (LowerError) {
    let label = try nextLabel(self, labelBase);
    let id = BlockId { n: self.blockData.len };
    let varCount = self.fnType.localCount;
    let vars = try! alloc::allocSlice(self.low.arena, @sizeOf(?il::Val), @alignOf(?il::Val), varCount) as *mut [?il::Val];

    for i in 0..varCount {
        vars[i] = nil;
    }
    self.blockData.append(BlockData {
        label,
        params: &mut [],
        paramVars: &mut [],
        instrs: &mut [],
        locs: &mut [],
        preds: &mut [],
        vars,
        sealState: Sealed::No { incompleteVars: &mut [] },
        loopDepth: self.loopDepth,
    }, self.allocator);

    return id;
}

/// Create a new block with a single parameter.
fn createBlockWithParam(
    self: *mut FnLowerer,
    labelBase: *[u8],
    param: il::Param
) -> BlockId throws (LowerError) {
    let block = try createBlock(self, labelBase);
    let blk = getBlockMut(self, block);
    blk.params.append(param, self.allocator);

    return block;
}

/// Switch to building a different block.
/// All subsequent `emit` calls will add instructions to this block.
fn switchToBlock(self: *mut FnLowerer, block: BlockId) {
    self.currentBlock = block;
}

/// Seal a block, indicating all predecessor edges are now known.
///
/// Sealing enables SSA construction to resolve variable uses by looking up
/// values from predecessors and inserting block parameters as needed. It
/// does not prevent instructions from being added to the block.
fn sealBlock(self: *mut FnLowerer, block: BlockId) throws (LowerError) {
    let blk = getBlockMut(self, block);
    let case Sealed::No { incompleteVars } = blk.sealState else {
        return; // Already sealed.
    };
    blk.sealState = Sealed::Yes;

    // Complete all incomplete block parameters.
    for varId in incompleteVars {
        try resolveBlockArgs(self, block, Var { id: varId });
    }
}

/// Seal a block and switch to it.
fn switchToAndSeal(self: *mut FnLowerer, block: BlockId) throws (LowerError) {
    try sealBlock(self, block);
    switchToBlock(self, block);
}

/// Get block data by block id.
fn getBlock(self: *FnLowerer, block: BlockId) -> *BlockData {
    return &self.blockData[block.n];
}

/// Get mutable block data by block id.
fn getBlockMut(self: *mut FnLowerer, block: BlockId) -> *mut BlockData {
    return &mut self.blockData[block.n];
}

/// Get the current block being built.
fn currentBlock(self: *FnLowerer) -> BlockId {
    let block = self.currentBlock else {
        panic "currentBlock: no current block";
    };
    return block;
}

//////////////////////////
// Instruction Emission //
//////////////////////////

/// Emit an instruction to the current block.
fn emit(self: *mut FnLowerer, instr: il::Instr) {
    let blk = self.currentBlock else panic;
    let mut block = getBlockMut(self, blk);

    // Record source location alongside instruction when enabled.
    if self.low.options.debug {
        block.locs.append(self.srcLoc, self.allocator);
    }
    block.instrs.append(instr, self.allocator);
}

/// Emit an unconditional jump to `target`.
fn emitJmp(self: *mut FnLowerer, target: BlockId) throws (LowerError) {
    emit(self, il::Instr::Jmp { target: target.n, args: &mut [] });
    addPredecessor(self, target, currentBlock(self));
}

/// Emit an unconditional jump to `target` with a single argument.
fn emitJmpWithArg(self: *mut FnLowerer, target: BlockId, arg: il::Val) throws (LowerError) {
    let args = try allocVal(self, arg);
    emit(self, il::Instr::Jmp { target: target.n, args });
    addPredecessor(self, target, currentBlock(self));
}

/// Emit an unconditional jump to `target` and switch to it.
fn switchAndJumpTo(self: *mut FnLowerer, target: BlockId) throws (LowerError) {
    try emitJmp(self, target);
    switchToBlock(self, target);
}

/// Emit a conditional branch based on `cond`.
fn emitBr(self: *mut FnLowerer, cond: il::Reg, thenBlock: BlockId, elseBlock: BlockId) throws (LowerError) {
    assert thenBlock != elseBlock;
    emit(self, il::Instr::Br {
        op: il::CmpOp::Ne,
        typ: il::Type::W32,
        a: il::Val::Reg(cond),
        b: il::Val::Imm(0),
        thenTarget: thenBlock.n,
        thenArgs: &mut [],
        elseTarget: elseBlock.n,
        elseArgs: &mut [],
    });
    addPredecessor(self, thenBlock, currentBlock(self));
    addPredecessor(self, elseBlock, currentBlock(self));
}

/// Emit a compare-and-branch instruction with the given comparison op.
fn emitBrCmp(
    self: *mut FnLowerer,
    op: il::CmpOp,
    typ: il::Type,
    a: il::Val,
    b: il::Val,
    thenBlock: BlockId,
    elseBlock: BlockId
) throws (LowerError) {
    assert thenBlock != elseBlock;
    emit(self, il::Instr::Br {
        op, typ, a, b,
        thenTarget: thenBlock.n, thenArgs: &mut [],
        elseTarget: elseBlock.n, elseArgs: &mut [],
    });
    addPredecessor(self, thenBlock, currentBlock(self));
    addPredecessor(self, elseBlock, currentBlock(self));
}

/// Emit a guard that traps with `ebreak` when a comparison is false.
fn emitTrapUnlessCmp(
    self: *mut FnLowerer,
    op: il::CmpOp,
    typ: il::Type,
    a: il::Val,
    b: il::Val
) throws (LowerError) {
    let passBlock = try createBlock(self, "guard#pass");
    let trapBlock = try createBlock(self, "guard#trap");

    try emitBrCmp(self, op, typ, a, b, passBlock, trapBlock);
    try switchToAndSeal(self, trapBlock);

    emit(self, il::Instr::Ebreak);
    emit(self, il::Instr::Unreachable);

    try switchToAndSeal(self, passBlock);
}

/// Emit a conditional branch. Uses fused compare-and-branch for simple scalar
/// comparisons, falls back to separate comparison plus branch otherwise.
fn emitCondBranch(
    self: *mut FnLowerer,
    cond: *ast::Node,
    thenBlock: BlockId,
    elseBlock: BlockId
) throws (LowerError) {
    // Try fused compare-and-branch for simple scalar comparisons.
    if let case ast::NodeValue::BinOp(binop) = cond.value {
        let leftTy = resolver::typeFor(self.low.resolver, binop.left) else {
            throw LowerError::MissingType(binop.left);
        };
        let rightTy = resolver::typeFor(self.low.resolver, binop.right) else {
            throw LowerError::MissingType(binop.right);
        };
        if not isAggregateType(leftTy) and not isAggregateType(rightTy) {
            let unsigned = isUnsignedType(leftTy);
            if let op = cmpOpFrom(binop.op, unsigned) {
                let a = try lowerExpr(self, binop.left);
                let b = try lowerExpr(self, binop.right);
                let typ = ilType(self.low, leftTy);

                // Swap operands if needed.
                match binop.op {
                    case ast::BinaryOp::Gt => // `a > b` = `b < a`
                        try emitBrCmp(self, op, typ, b, a, thenBlock, elseBlock),
                    case ast::BinaryOp::Lte => // `a <= b` = `!(b < a)`
                        try emitBrCmp(self, op, typ, b, a, elseBlock, thenBlock),
                    case ast::BinaryOp::Gte => // `a >= b` = `!(a < b)`
                        try emitBrCmp(self, op, typ, a, b, elseBlock, thenBlock),
                    else =>
                        try emitBrCmp(self, op, typ, a, b, thenBlock, elseBlock),
                }
                return;
            }
        }
    }
    // Fallback: evaluate condition and emit boolean branch.
    let condVal = try lowerExpr(self, cond);
    let condReg = emitValToReg(self, condVal);

    try emitBr(self, condReg, thenBlock, elseBlock);
}

/// Emit a 32-bit store instruction at the given offset.
fn emitStoreW32At(self: *mut FnLowerer, src: il::Val, dst: il::Reg, offset: i32) {
    emit(self, il::Instr::Store { typ: il::Type::W32, src, dst, offset });
}

/// Emit a 32-bit store instruction with default alignment.
fn emitStoreW32(self: *mut FnLowerer, src: il::Val, dst: il::Reg) {
    emitStoreW32At(self, src, dst, 0);
}

/// Emit a 32-bit load instruction at the given offset.
fn emitLoadW32At(self: *mut FnLowerer, dst: il::Reg, src: il::Reg, offset: i32) {
    emit(self, il::Instr::Load { typ: il::Type::W32, dst, src, offset });
}

/// Emit a 32-bit load instruction with default alignment.
fn emitLoadW32(self: *mut FnLowerer, dst: il::Reg, src: il::Reg) {
    emitLoadW32At(self, dst, src, 0);
}

/// Emit an 8-bit store instruction.
fn emitStoreW8(self: *mut FnLowerer, src: il::Val, dst: il::Reg) {
    emit(self, il::Instr::Store { typ: il::Type::W8, src, dst, offset: 0 });
}

/// Emit an 8-bit store instruction at the given offset.
fn emitStoreW8At(self: *mut FnLowerer, src: il::Val, dst: il::Reg, offset: i32) {
    emit(self, il::Instr::Store { typ: il::Type::W8, src, dst, offset });
}

/// Emit an 8-bit load instruction.
fn emitLoadW8(self: *mut FnLowerer, dst: il::Reg, src: il::Reg) {
    emit(self, il::Instr::Load { typ: il::Type::W8, dst, src, offset: 0 });
}

/// Emit an 8-bit load instruction at the given offset.
fn emitLoadW8At(self: *mut FnLowerer, dst: il::Reg, src: il::Reg, offset: i32) {
    emit(self, il::Instr::Load { typ: il::Type::W8, dst, src, offset });
}

/// Emit a 64-bit store instruction at the given offset.
fn emitStoreW64At(self: *mut FnLowerer, src: il::Val, dst: il::Reg, offset: i32) {
    emit(self, il::Instr::Store { typ: il::Type::W64, src, dst, offset });
}

/// Emit a 64-bit store instruction with default alignment.
fn emitStoreW64(self: *mut FnLowerer, src: il::Val, dst: il::Reg) {
    emitStoreW64At(self, src, dst, 0);
}

/// Emit a 64-bit load instruction at the given offset.
fn emitLoadW64At(self: *mut FnLowerer, dst: il::Reg, src: il::Reg, offset: i32) {
    emit(self, il::Instr::Load { typ: il::Type::W64, dst, src, offset });
}

/// Emit a 64-bit load instruction with default alignment.
fn emitLoadW64(self: *mut FnLowerer, dst: il::Reg, src: il::Reg) {
    emitLoadW64At(self, dst, src, 0);
}

/// Load a tag from memory at `src` plus `offset` with the given IL type.
fn loadTag(self: *mut FnLowerer, src: il::Reg, offset: i32, tagType: il::Type) -> il::Val {
    let dst = nextReg(self);
    emit(self, il::Instr::Load { typ: tagType, dst, src, offset });
    return il::Val::Reg(dst);
}

/// Load the data pointer from a slice value.
fn loadSlicePtr(self: *mut FnLowerer, sliceReg: il::Reg) -> il::Reg {
    let ptrReg = nextReg(self);
    emitLoadW64At(self, ptrReg, sliceReg, SLICE_PTR_OFFSET);
    return ptrReg;
}

/// Load the length from a slice value.
fn loadSliceLen(self: *mut FnLowerer, sliceReg: il::Reg) -> il::Val {
    let lenReg = nextReg(self);
    emitLoadW32At(self, lenReg, sliceReg, SLICE_LEN_OFFSET);
    return il::Val::Reg(lenReg);
}

/// Load the capacity from a slice value.
fn loadSliceCap(self: *mut FnLowerer, sliceReg: il::Reg) -> il::Val {
    let capReg = nextReg(self);
    emitLoadW32At(self, capReg, sliceReg, SLICE_CAP_OFFSET);
    return il::Val::Reg(capReg);
}

/// Emit a load instruction for a scalar value at `src` plus `offset`.
/// For reading values that may be aggregates, use `emitRead` instead.
fn emitLoad(self: *mut FnLowerer, src: il::Reg, offset: i32, typ: resolver::Type) -> il::Val {
    let dst = nextReg(self);
    let ilTyp = ilType(self.low, typ);

    if isSignedType(typ) {
        emit(self, il::Instr::Sload { typ: ilTyp, dst, src, offset });
    } else {
        emit(self, il::Instr::Load { typ: ilTyp, dst, src, offset });
    }
    return il::Val::Reg(dst);
}

/// Read a value from memory at `src` plus `offset`. Aggregates are represented
/// as pointers, so we return the address directly. Scalars are loaded via [`emitLoad`].
fn emitRead(self: *mut FnLowerer, src: il::Reg, offset: i32, typ: resolver::Type) -> il::Val {
    if isAggregateType(typ) {
        let ptr = emitPtrOffset(self, src, offset);
        return il::Val::Reg(ptr);
    }
    return emitLoad(self, src, offset, typ);
}

/// Emit a copy instruction that loads a data symbol's address into a register.
fn emitDataAddr(self: *mut FnLowerer, sym: *resolver::Symbol) -> il::Reg {
    let dst = nextReg(self);
    let modId = resolver::moduleIdForSymbol(self.low.resolver, sym);
    let qualName = qualifyName(self.low, modId, sym.name);

    emit(self, il::Instr::Copy { dst, val: il::Val::DataSym(qualName) });

    return dst;
}

/// Emit a copy instruction that loads a function's address into a register.
fn emitFnAddr(self: *mut FnLowerer, sym: *resolver::Symbol) -> il::Reg {
    let dst = nextReg(self);
    let modId = resolver::moduleIdForSymbol(self.low.resolver, sym);
    let qualName = qualifyName(self.low, modId, sym.name);

    emit(self, il::Instr::Copy { dst, val: il::Val::FnAddr(qualName) });

    return dst;
}

/// Emit pattern tests for a single pattern.
fn emitPatternMatch(
    self: *mut FnLowerer,
    subject: *MatchSubject,
    pattern: *ast::Node,
    matchBlock: BlockId,
    fallthrough: BlockId
) throws (LowerError) {
    // Wildcards always match; array patterns are tested element-by-element
    // during binding, so they also unconditionally enter the match block.
    if isWildcardPattern(pattern) {
        try emitJmp(self, matchBlock);
        return;
    }
    if let case ast::NodeValue::ArrayLit(_) = pattern.value {
        try emitJmp(self, matchBlock);
        return;
    }
    let isNil = pattern.value == ast::NodeValue::Nil;

    match subject.kind {
        case MatchSubjectKind::OptionalPtr if isNil => {
            // Null pointer optimization: branch on the data pointer being null.
            let nilReg = try optionalNilReg(self, subject.val, subject.type);
            try emitBrCmp(self, il::CmpOp::Eq, il::Type::W64, il::Val::Reg(nilReg), il::Val::Imm(0), matchBlock, fallthrough);
        }
        case MatchSubjectKind::OptionalAggregate => {
            let base = emitValToReg(self, subject.val);

            if isNil { // Optional aggregate: `nil` means tag is zero.
                let tagReg = tvalTagReg(self, base);
                try emitBr(self, tagReg, fallthrough, matchBlock);
            } else {
                if isAggregateType(subject.bindType) {
                    // TODO: Why?
                    throw LowerError::Unsupported;
                }
                let pattVal = try lowerExpr(self, pattern);
                let pattReg = emitValToReg(self, pattVal);
                let eq = try lowerOptionalEq(self, subject.bindType, base, pattReg, 0);
                let eqReg = emitValToReg(self, eq);

                try emitBr(self, eqReg, matchBlock, fallthrough);
            }
        }
        case MatchSubjectKind::Union(unionInfo) => {
            assert not isNil;

            let case resolver::NodeExtra::UnionVariant { tag: variantTag, .. } =
                resolver::nodeData(self.low.resolver, pattern).extra
            else {
                throw LowerError::ExpectedVariant;
            };
            // Void unions are passed by value (the tag itself).
            // Non-void unions are passed by reference (need to load tag).
            // When matching by reference, always load from the pointer.
            if unionInfo.isAllVoid {
                let mut tagVal = subject.val;
                match subject.by {
                    case resolver::MatchBy::Ref, resolver::MatchBy::MutRef => {
                        let base = emitValToReg(self, subject.val);
                        tagVal = loadTag(self, base, 0, il::Type::W8);
                    }
                    case resolver::MatchBy::Value => {}
                }
                try emitBrCmp(self, il::CmpOp::Eq, il::Type::W8, tagVal, il::Val::Imm(variantTag as i64), matchBlock, fallthrough);
            } else {
                let base = emitValToReg(self, subject.val);
                let tagReg = tvalTagReg(self, base);

                try emitBrCmp(self, il::CmpOp::Eq, il::Type::W8, il::Val::Reg(tagReg), il::Val::Imm(variantTag as i64), matchBlock, fallthrough);
            }
        }
        else => { // Value comparison.
            assert not isNil;
            let pattVal = try lowerExpr(self, pattern);
            if isAggregateType(subject.type) {
                // Aggregate types (slices, records, etc.) need structural
                // comparison via lowerAggregateEq rather than scalar compare.
                let subjectReg = emitValToReg(self, subject.val);
                let pattReg = emitValToReg(self, pattVal);
                let eq = try lowerAggregateEq(self, subject.type, subjectReg, pattReg, 0);
                let eqReg = emitValToReg(self, eq);
                try emitBr(self, eqReg, matchBlock, fallthrough);
            } else {
                try emitBrCmp(self, il::CmpOp::Eq, subject.ilType, subject.val, pattVal, matchBlock, fallthrough);
            }
        }
    }
}

/// Emit branches for multiple patterns. The first pattern that matches
/// causes a jump to the match block. If no patterns match, we jump to the
/// fallthrough block.
fn emitPatternMatches(
    self: *mut FnLowerer,
    subject: *MatchSubject,
    patterns: *mut [*ast::Node],
    matchBlock: BlockId,
    fallthrough: BlockId
) throws (LowerError) {
    assert patterns.len > 0;

    for i in 0..(patterns.len - 1) {
        let pattern = patterns[i];
        let nextArm = try createBlock(self, "arm");
        try emitPatternMatch(self, subject, pattern, matchBlock, nextArm);

        // Seal the intermediate arm block: all predecessor edges are known
        // This ensures SSA construction can resolve variable uses through
        // single-predecessor optimization instead of creating unresolved block
        // parameters.
        try switchToAndSeal(self, nextArm);
    }
    // Handle last pattern: go to fallthrough block on failure.
    let last = patterns[patterns.len - 1];
    try emitPatternMatch(self, subject, last, matchBlock, fallthrough);
}

/// Emit a match binding pattern.
/// Binding patterns always match for regular values, but for optionals they
/// check for the presence of a value. Jumps to `valuePresent` on success,
/// `valueAbsent` on failure.
fn emitBindingTest(
    self: *mut FnLowerer,
    subject: *MatchSubject,
    valuePresent: BlockId,
    valueAbsent: BlockId
) throws (LowerError) {
    match subject.kind {
        case MatchSubjectKind::OptionalPtr, MatchSubjectKind::OptionalAggregate => {
            let nilReg = try optionalNilReg(self, subject.val, subject.type);
            try emitBr(self, nilReg, valuePresent, valueAbsent);
        }
        else => {
            // Regular values always match binding patterns unconditionally.
            try emitJmp(self, valuePresent);
        }
    }
}

/// Emit a jump to target if the current block hasn't terminated, then seal the target block.
fn emitJmpAndSeal(self: *mut FnLowerer, target: BlockId) throws (LowerError) {
    if not blockHasTerminator(self) {
        try emitJmp(self, target);
    }
    try sealBlock(self, target);
}

/// Check if the current block already has a terminator instruction.
fn blockHasTerminator(self: *FnLowerer) -> bool {
    let blk = getBlock(self, currentBlock(self));
    if blk.instrs.len == 0 {
        return false;
    }
    match blk.instrs[blk.instrs.len - 1] {
        case il::Instr::Ret { .. },
             il::Instr::Jmp { .. },
             il::Instr::Br { .. },
             il::Instr::Switch { .. },
             il::Instr::Unreachable =>
            return true,
        else =>
            return false,
    }
}

/// Emit a jump to merge block if the current block hasn't terminated.
///
/// This is used after lowering branches of an if-else, for example. If the
/// branch diverges, the block already has a terminator and no jump is needed.
///
/// This handles cases like:
///
///     if cond {
///         return 1;   // @then diverges, no jump to merge.
///     } else {
///         return 0;   // @else diverges, no jump to merge.
///     }
///
/// In the above example, the merge block stays `nil`, and no code is generated
/// after the `if`. The merge block is created on first use.
fn emitMergeIfUnterminated(self: *mut FnLowerer, mergeBlock: *mut ?BlockId) throws (LowerError) {
    if not blockHasTerminator(self) {
        if *mergeBlock == nil {
            *mergeBlock = try createBlock(self, "merge");
        }
        let target = *mergeBlock else { throw LowerError::MissingTarget; };
        try emitJmp(self, target);
    }
}

//////////////////////////////////
// Control Flow Edge Management //
//////////////////////////////////

/// Add a predecessor edge from `pred` to `target`.
/// Must be called before the target block is sealed. Duplicates are ignored.
fn addPredecessor(self: *mut FnLowerer, target: BlockId, pred: BlockId) {
    let blk = getBlockMut(self, target);
    if blk.sealState == Sealed::Yes {
        panic "addPredecessor: adding predecessor to sealed block";
    }
    let preds = &mut blk.preds;
    for i in 0..preds.len {
        if preds[i] == pred.n { // Avoid duplicate predecessor entries.
            return;
        }
    }
    preds.append(pred.n, self.allocator);
}

/// Finalize all blocks and return the block array.
fn finalizeBlocks(self: *mut FnLowerer) -> *[il::Block] throws (LowerError) {
    let blocks = try! alloc::allocSlice(
        self.low.arena, @sizeOf(il::Block), @alignOf(il::Block), self.blockData.len
    ) as *mut [il::Block];

    for i in 0..self.blockData.len {
        let data = &self.blockData[i];

        blocks[i] = il::Block {
            label: data.label,
            params: &data.params[..],
            instrs: data.instrs,
            locs: &data.locs[..],
            preds: &data.preds[..],
            loopDepth: data.loopDepth,
        };
    }
    return &blocks[..self.blockData.len];
}

/////////////////////
// Loop Management //
/////////////////////

/// Enter a loop context for break/continue handling.
/// `continueBlock` is `nil` when the continue target is created lazily.
fn enterLoop(self: *mut FnLowerer, breakBlock: BlockId, continueBlock: ?BlockId) {
    if self.loopDepth >= self.loopStack.len {
        panic "enterLoop: loop depth overflow";
    }
    let slot = &mut self.loopStack[self.loopDepth];

    slot.breakTarget = breakBlock;
    slot.continueTarget = continueBlock;
    self.loopDepth += 1;
}

/// Exit the current loop context.
fn exitLoop(self: *mut FnLowerer) {
    if self.loopDepth == 0 {
        panic "exitLoop: loopDepth is zero";
    }
    self.loopDepth -= 1;
}

/// Get the current loop context.
fn currentLoop(self: *mut FnLowerer) -> ?*mut LoopCtx {
    if self.loopDepth == 0 {
        return nil;
    }
    return &mut self.loopStack[self.loopDepth - 1];
}

/// Get or lazily create the continue target block for the current loop.
fn getOrCreateContinueBlock(self: *mut FnLowerer) -> BlockId throws (LowerError) {
    let ctx = currentLoop(self) else {
        throw LowerError::OutsideOfLoop;
    };
    if let block = ctx.continueTarget {
        return block;
    }
    let block = try createBlock(self, "step");
    ctx.continueTarget = block;
    return block;
}

/// Allocate a slice of values in the lowering arena.
fn allocVals(self: *mut FnLowerer, len: u32) -> *mut [il::Val] throws (LowerError) {
    return try! alloc::allocSlice(self.low.arena, @sizeOf(il::Val), @alignOf(il::Val), len) as *mut [il::Val];
}

/// Allocate a single-value slice in the lowering arena.
fn allocVal(self: *mut FnLowerer, val: il::Val) -> *mut [il::Val] throws (LowerError) {
    let args = try allocVals(self, 1);
    args[0] = val;
    return args;
}

////////////////////////
// SSA Var Management //
////////////////////////

// This section implements SSA construction following "Simple and Efficient
// Construction of Static Single Assignment Form" (Braun et al., 2013). The IL
// uses block parameters (equivalent to phi nodes) that receive values via
// terminator arguments. Block args are resolved eagerly at seal time via
// [`resolveBlockArgs`], matching Braun's on-the-fly approach.
//
// In SSA form, each variable definition creates a unique value. When control
// flow diverges and merges (like after an if-else), a variable might have
// different definitions from different paths.
//
// # What "Value" means
//
// An [`il::Val`] is a compile-time representation of *where* a runtime value
// lives, not the runtime value itself. Values can live in registers, as symbol
// references, or as immediate values, ie. static constants.
//
// When we "find the value" of a variable, we're answering: "Which SSA register
// (or constant) represents this variable at this program point?"
//
// Each source-level variable gets a `Var` handle on declaration. When a
// variable is defined via [`defVar`], its SSA value is recorded in the current
// block's variable mapping. When a variable is used with [`useVar`] or
// [`useVarInBlock`], we either return the local definition or recursively look
// up the value from predecessor blocks. When multiple predecessors define
// different values for a given variable, we insert a block parameter
// (equivalent to a phi node).
//
// The algorithm used by [`useVarInBlock`] handles three cases:
//
// 1. **Local definition exists**: If the variable was assigned in this block,
//    return that value immediately (fast path).
//
// 2. **Sealed block with single predecessor**: If all incoming edges are known
//    and there's exactly one predecessor, recurse to that predecessor. No merge
//    is needed since there's only one path. The result is cached.
//
// 3. **Multiple predecessors**: Create a block parameter to receive the merged
//    value. If the block is sealed, immediately look up each predecessor's value
//    and patch their terminators via [`resolveBlockArgs`]. If unsealed, defer by
//    recording the variable in `incompleteVars`; when [`sealBlock`] is called,
//    all incomplete block params are resolved at that point.
//
// Consider this code:
//
//     let mut x = 1;
//     if cond {
//         x = 2;
//     } else {
//         x = f();   // Result in register %r.
//     }
//     print(x);      // Which value?
//
// The Control Flow Graph (CFG) looks like:
//
//         [entry]
//           _|_
//          /   \
//     [then]   [else]
//     x = 2    x = %r
//         \    /
//        [merge]
//         use(x)
//
// At the `print(x)` point, the compiler doesn't know which branch ran (that's
// a runtime decision), but it needs to emit code that works for either case.
// This is where [`useVarInBlock`] is called.
//
// The generated IL looks like this:
//
//     @then
//       jmp @merge(2);           // pass immediate `2`
//     @else
//       jmp @merge(%r);          // pass register `%r`
//     @merge(w32 %m)             // `%m` receives whichever value arrives at runtime
//       call w32 $print(%m);     // `print` is called with runtime value in `%m`
//
// A block is "sealed" when all predecessor edges are known. This is crucial
// because until sealed, we can't know how many paths merge into the block,
// and thus can't create the right number of block parameter arguments.
//
// If a block isn't sealed but we need a variable's value, we still create
// a block parameter, but defer filling in the terminator arguments. When the
// block is later sealed via [`sealBlock`], all incomplete block params are resolved.

/// Declare a new source-level variable and define its initial value.
/// If called before any block exists (e.g., for parameters), the definition is skipped.
fn newVar(
    self: *mut FnLowerer,
    name: ?*[u8],
    type: il::Type,
    mutable: bool,
    val: il::Val
) -> Var {
    let id = self.vars.len;
    self.vars.append(VarData { name, type, mutable, addressTaken: false }, self.allocator);

    let v = Var { id };
    if self.currentBlock != nil {
        defVar(self, v, val);
    }
    return v;
}

/// Define (write) a variable. Record the SSA value of a variable in the
/// current block. Called when a variable is assigned or initialized (`let`
/// bindings, assignments, loop updates). When [`useVar`] is later called,
/// it will retrieve this value.
fn defVar(self: *mut FnLowerer, v: Var, val: il::Val) {
    assert v.id < self.vars.len;
    getBlockMut(self, currentBlock(self)).vars[v.id] = val;
}

/// Use (read) the current value of a variable in the current block.
/// May insert block parameters if the value must come from predecessors.
fn useVar(self: *mut FnLowerer, v: Var) -> il::Val throws (LowerError) {
    return try useVarInBlock(self, currentBlock(self), v);
}

/// Resolve which SSA definition of a variable reaches a use point in a given block.
///
/// Given a variable and a block where it's used, this function finds the
/// correct [`il::Val`] that holds the variable's value at that program point.
/// When control flow merges from multiple predecessors with different
/// definitions, it creates a block parameter to unify them.
fn useVarInBlock(self: *mut FnLowerer, block: BlockId, v: Var) -> il::Val throws (LowerError) {
    assert v.id < self.vars.len;

    let blk = getBlockMut(self, block);
    if let val = blk.vars[v.id] {
        return val;
    }
    // Entry block cannot have block parameters. If variable isn't defined
    // in entry, we return undefined.
    if block == self.entryBlock {
        return il::Val::Undef;
    }
    if blk.sealState == Sealed::Yes {
        if blk.preds.len == 0 {
            // Variable used in sealed block with no predecessors.
            throw LowerError::InvalidUse;
        }
        // Single predecessor means no merge needed, variable is implicitly
        // available without a block parameter.
        if blk.preds.len == 1 {
            let pred = BlockId { n: blk.preds[0] };
            if pred.n != block.n {
                let val = try useVarInBlock(self, pred, v);
                blk.vars[v.id] = val; // Cache.
                return val;
            }
        }
    }
    // Multiple predecessors or unsealed block: need a block parameter to merge
    // the control flow paths.
    return try createBlockParam(self, block, v);
}

/// Look up a variable by name in the current scope.
/// Searches from most recently declared to first, enabling shadowing.
fn lookupVarByName(self: *FnLowerer, name: *[u8]) -> ?Var {
    let mut id = self.vars.len;
    while id > 0 {
        id -= 1;
        if let varName = self.vars[id].name {
            if mem::eq(varName, name) {
                return Var { id };
            }
        }
    }
    return nil;
}

/// Look up a local variable bound to an identifier node.
fn lookupLocalVar(self: *FnLowerer, node: *ast::Node) -> ?Var {
    let case ast::NodeValue::Ident(name) = node.value else {
        return nil;
    };
    return lookupVarByName(self, name);
}

/// Save current lexical variable scope depth.
fn enterVarScope(self: *FnLowerer) -> u32 {
    return self.vars.len;
}

/// Restore lexical variable scope depth.
fn exitVarScope(self: *mut FnLowerer, savedVarsLen: u32) {
    self.vars = @sliceOf(self.vars.ptr, savedVarsLen, self.vars.cap);
}

/// Get the metadata for a variable.
fn getVar(self: *FnLowerer, v: Var) -> *VarData {
    assert v.id < self.vars.len;
    return &self.vars[v.id];
}

/// Create a block parameter to merge a variable's value from multiple
/// control flow paths.
///
/// Called when [`useVarInBlock`] can't find a local definition and the block has
/// multiple predecessors. For example, when `x` is used in `@end` but defined
/// differently in `@then` and `@else`:
///
///     @then
///       jmp @end(1);            // x = 1
///     @else
///       jmp @end(2);            // x = 2
///     @end(w32 %1)              // x = %1, merged from predecessors
///       ret %1;
///
/// This function creates a fresh register `%1` as a block parameter, then patches
/// each predecessor's jump to pass its value of `x` as an argument.
fn createBlockParam(self: *mut FnLowerer, block: BlockId, v: Var) -> il::Val throws (LowerError) {
    // Entry block must not have block parameters.
    if block == self.entryBlock {
        panic "createBlockParam: entry block must not have block parameters";
    }
    // Allocate a register to hold the merged value.
    let reg = nextReg(self);
    let type = getVar(self, v).type;

    // Create block parameter and add it to the block.
    let param = il::Param { value: reg, type };
    let blk = getBlockMut(self, block);
    blk.params.append(param, self.allocator);
    blk.paramVars.append(v.id, self.allocator); // Associate variable with parameter.

    // Record that this variable's value in this block is now the parameter register.
    // This must happen before the predecessor loop to handle self-referential loops.
    blk.vars[v.id] = il::Val::Reg(reg);

    match &mut blk.sealState {
        case Sealed::No { incompleteVars } => {
            // Block unsealed: defer until sealing.
            incompleteVars.append(v.id, self.allocator);
        },
        case Sealed::Yes => {
            // Block sealed: check for trivial phi before committing. If all
            // predecessors provide the same value, we can remove the param we
            // just created and use that value directly.
            if let trivial = try getTrivialPhiVal(self, block, v) {
                let provisional = il::Val::Reg(reg);
                removeLastBlockParam(self, block);
                rewriteCachedVarValue(self, v, provisional, trivial);
                getBlockMut(self, block).vars[v.id] = trivial;
                return trivial;
            }
            // Non-trivial phi: patch predecessors to pass their values.
            try resolveBlockArgs(self, block, v);
        },
    }
    return il::Val::Reg(reg);
}

/// Complete a block parameter by looking up the variable's value in all
/// predecessors and patching their terminator instructions with edge arguments.
///
/// This is the block-parameter equivalent of adding operands to a phi-function in
/// traditional SSA. Where a phi-function merges values at the join point:
///
///     x3 = phi(x1, x2)
///
/// This representation avoids the need for phi nodes to reference their
/// predecessor blocks explicitly, since the control flow edges already encode
/// that information.
fn resolveBlockArgs(self: *mut FnLowerer, block: BlockId, v: Var) throws (LowerError) {
    let blk = getBlock(self, block);

    // Find the parameter index corresponding to this variable.
    // Each variable that needs merging gets its own block parameter slot.
    let mut paramIdx: u32 = 0;
    for i in 0..blk.paramVars.len {
        if blk.paramVars[i] == v.id {
            paramIdx = i;
            break;
        }
    }

    // For each predecessor, recursively look up the variable's reaching definition
    // in that block, then patch the predecessor's terminator to pass that value
    // as an argument to this block's parameter.
    for predId in blk.preds {
        let pred = BlockId { n: predId };
        // This may recursively trigger more block arg resolution if the
        // predecessor also needs to look up the variable from its predecessors.
        let val = try useVarInBlock(self, pred, v);
        if val == il::Val::Undef {
            panic "createBlockParam: predecessor provides undef value for block parameter";
        }
        patchTerminatorArg(self, pred, block.n, paramIdx, val);
    }
}

/// Check if a block parameter is trivial, i.e. all predecessors provide
/// the same value. Returns the trivial value if so.
fn getTrivialPhiVal(self: *mut FnLowerer, block: BlockId, v: Var) -> ?il::Val throws (LowerError) {
    let blk = getBlock(self, block);
    // Get the block parameter register.
    let paramReg = blk.vars[v.id];
    // Check if all predecessors provide the same value.
    let mut sameVal: ?il::Val = nil;

    for predId in blk.preds {
        let pred = BlockId { n: predId };
        let val = try useVarInBlock(self, pred, v);

        // Check if this is a self-reference.
        // This happens in cycles where the loop back-edge passes the phi to
        // itself. We skip self-references when checking for trivial phis.
        if let reg = paramReg {
            if val == reg {
                // Self-reference, skip this predecessor.
            } else if let sv = sameVal {
                if val != sv {
                    // Multiple different values, not trivial.
                    return nil;
                }
            } else {
                sameVal = val;
            }
        } else { // No param reg set yet, can't be trivial.
            return nil;
        }
    }
    return sameVal;
}

/// Patch switch case arguments for a specific target.
// TODO: Inline this when we don't have register pressure issues.
fn patchSwitchCases(self: *mut FnLowerer, cases: *mut [il::SwitchCase], target: u32, paramIdx: u32, val: il::Val) {
    let idx = paramIdx + 1;
    for i in 0..cases.len {
        if cases[i].target == target {
            cases[i].args = growArgs(self, cases[i].args, idx);
            cases[i].args[paramIdx] = val;
        }
    }
}

/// Patch a single terminator argument for a specific edge. This is used during
/// SSA construction to pass variable values along control flow edges.
fn patchTerminatorArg(
    self: *mut FnLowerer,
    from: BlockId,         // The predecessor block containing the terminator to patch.
    target: u32,           // The index of the target block we're passing the value to.
    paramIdx: u32,         // The index of the block parameter to set.
    val: il::Val           // The value to pass as the argument.
) {
    let data = getBlockMut(self, from);
    let ix = data.instrs.len - 1; // The terminator is always the last instruction.

    // TODO: We shouldn't need to use a mutable subscript here, given that the
    // fields are already mutable.
    match &mut data.instrs[ix] {
        case il::Instr::Jmp { args, .. } => {
            *args = growArgs(self, *args, paramIdx + 1);
            args[paramIdx] = val;
        }
        case il::Instr::Br { thenTarget, thenArgs, elseTarget, elseArgs, .. } => {
            // Nb. both branches could target the same block (e.g. `if cond { x } else { x }`).
            if *thenTarget == target {
                *thenArgs = growArgs(self, *thenArgs, paramIdx + 1);
                thenArgs[paramIdx] = val;
            }
            if *elseTarget == target {
                *elseArgs = growArgs(self, *elseArgs, paramIdx + 1);
                elseArgs[paramIdx] = val;
            }
        }
        case il::Instr::Switch { defaultTarget, defaultArgs, cases, .. } => {
            if *defaultTarget == target {
                *defaultArgs = growArgs(self, *defaultArgs, paramIdx + 1);
                defaultArgs[paramIdx] = val;
            }
            patchSwitchCases(self, *cases, target, paramIdx, val);
        }
        else => {
            // Other terminators (e.g. `Ret`, `Unreachable`) don't have successor blocks.
        }
    }
}

/// Grow an args array to hold at least the given capacity.
fn growArgs(self: *mut FnLowerer, args: *mut [il::Val], capacity: u32) -> *mut [il::Val] {
    if args.len >= capacity {
        return args;
    }
    let newArgs = try! alloc::allocSlice(
        self.low.arena, @sizeOf(il::Val), @alignOf(il::Val), capacity
    ) as *mut [il::Val];

    for arg, i in args {
        newArgs[i] = arg;
    }
    for i in args.len..capacity {
        newArgs[i] = il::Val::Undef;
    }
    return newArgs;
}

/// Extract the parameter name from an [`FnParam`] AST node value.
fn paramName(value: *ast::NodeValue) -> *[u8] throws (LowerError) {
    let case ast::NodeValue::FnParam(param) = *value else {
        throw LowerError::ExpectedFunctionParam;
    };
    let case ast::NodeValue::Ident(name) = param.name.value else {
        throw LowerError::ExpectedIdentifier;
    };
    return name;
}

/// Lower function parameters. Declares variables for each parameter.
/// When a receiver name is passed, we're handling a trait method.
fn lowerParams(
    self: *mut FnLowerer,
    fnType: resolver::FnType,
    astParams: *mut [*ast::Node],
    receiverName: ?*ast::Node
) -> *[il::Param] throws (LowerError) {
    let offset: u32 = 1 if self.returnReg != nil else 0;
    let totalLen = fnType.paramTypes.len as u32 + offset;
    if totalLen == 0 {
        return &[];
    }
    assert fnType.paramTypes.len as u32 <= resolver::MAX_FN_PARAMS;

    let params = try! alloc::allocSlice(
        self.low.arena, @sizeOf(il::Param), @alignOf(il::Param), totalLen
    ) as *mut [il::Param];

    if let reg = self.returnReg {
        params[0] = il::Param { value: reg, type: il::Type::W64 };
    }
    for i in 0..fnType.paramTypes.len as u32 {
        let type = ilType(self.low, *fnType.paramTypes[i]);
        let reg = nextReg(self);

        params[i + offset] = il::Param { value: reg, type };

        // Declare the parameter variable. For the receiver, the name comes
        // from the receiver node.
        // For all other parameters, the name comes from the AST params.
        let mut name: *[u8] = undefined;
        if let recNode = receiverName {
            if i == 0 {
                let case ast::NodeValue::Ident(recName) = recNode.value else {
                    throw LowerError::ExpectedIdentifier;
                };
                name = recName;
            } else {
                name = try paramName(&astParams[i - 1].value);
            }
        } else {
            name = try paramName(&astParams[i].value);
        }
        let v = newVar(self, name, type, false, il::Val::Undef);

        self.params.append(FnParamBinding { var: v, reg }, self.allocator);
    }
    return params;
}

/// Resolve match subject.
fn lowerMatchSubject(self: *mut FnLowerer, subject: *ast::Node) -> MatchSubject throws (LowerError) {
    let mut val = try lowerExpr(self, subject);
    let subjectType = resolver::typeFor(self.low.resolver, subject) else {
        throw LowerError::MissingType(subject);
    };
    let unwrapped = resolver::unwrapMatchSubject(subjectType);

    // When matching an aggregate by value, copy it to a fresh stack slot so
    // that bindings are independent of the original memory.  Without this,
    // the lowerer returns a pointer into the source and mutations to the
    // source silently corrupt the bound variables.
    if unwrapped.by == resolver::MatchBy::Value and isAggregateType(unwrapped.effectiveTy) {
        val = try emitStackVal(self, unwrapped.effectiveTy, val);
    }

    let mut bindType = unwrapped.effectiveTy;
    if let case resolver::Type::Optional(inner) = unwrapped.effectiveTy {
        bindType = *inner;
    }
    let ilType = ilType(self.low, unwrapped.effectiveTy);
    let kind = matchSubjectKind(unwrapped.effectiveTy);

    return MatchSubject { val, type: unwrapped.effectiveTy, ilType, bindType, kind, by: unwrapped.by };
}

/// Check whether a case pattern is an unconditional wildcard.
fn isWildcardPattern(pattern: *ast::Node) -> bool {
    match pattern.value {
        case ast::NodeValue::Placeholder => return true,
        else => return false,
    }
}

/// Check whether a pattern node is a destructuring pattern that looks
/// through structure (union variant, record literal, scope access).
/// Identifiers, placeholders, and plain literals are not destructuring.
fn isDestructuringPattern(pattern: *ast::Node) -> bool {
    match pattern.value {
        case ast::NodeValue::Call(_),
             ast::NodeValue::RecordLit(_),
             ast::NodeValue::ScopeAccess(_) => return true,
        else => return false,
    }
}

/// Check whether an AST node is the `undefined` literal.
fn isUndef(node: *ast::Node) -> bool {
    match node.value {
        case ast::NodeValue::Undef => return true,
        else => return false,
    }
}

/// Load the tag byte from a tagged value aggregate (optionals and unions).
fn tvalTagReg(self: *mut FnLowerer, base: il::Reg) -> il::Reg {
    let tagReg = nextReg(self);
    emitLoadW8At(self, tagReg, base, TVAL_TAG_OFFSET);
    return tagReg;
}

/// Load the tag word from a result aggregate.
fn resultTagReg(self: *mut FnLowerer, base: il::Reg) -> il::Reg {
    let tagReg = nextReg(self);
    emitLoadW64At(self, tagReg, base, TVAL_TAG_OFFSET);
    return tagReg;
}

/// Get the register to compare against `0` for optional `nil` checking.
/// For null-ptr-optimized types, loads the data pointer, or returns it
/// directly for scalar pointers. For aggregates, returns the tag register.
fn optionalNilReg(self: *mut FnLowerer, val: il::Val, typ: resolver::Type) -> il::Reg throws (LowerError) {
    let reg = emitValToReg(self, val);

    match typ {
        case resolver::Type::Optional(resolver::Type::Slice { .. }) => {
            let ptrReg = nextReg(self);
            emitLoadW64At(self, ptrReg, reg, SLICE_PTR_OFFSET);
            return ptrReg;
        }
        case resolver::Type::Optional(resolver::Type::Pointer { .. }) => return reg,
        case resolver::Type::Optional(_) => return tvalTagReg(self, reg),
        else => return reg,
    }
}

/// Lower an optional nil check (`opt == nil` or `opt != nil`).
fn lowerNilCheck(self: *mut FnLowerer, opt: *ast::Node, isEq: bool) -> il::Val throws (LowerError) {
    let optTy = resolver::typeFor(self.low.resolver, opt) else {
        throw LowerError::MissingType(opt);
    };
    // Handle `nil == nil` or `nil != nil`.
    if optTy == resolver::Type::Nil {
        return il::Val::Imm(1) if isEq else il::Val::Imm(0);
    }
    let val = try lowerExpr(self, opt);
    let cmpReg = try optionalNilReg(self, val, optTy);

    // Null-pointer-optimized types compare a 64-bit pointer against zero.
    // Aggregate optionals compare an 8-bit tag byte against zero.
    let cmpType = il::Type::W64 if resolver::isOptionalPointer(optTy) else il::Type::W8;

    let op = il::BinOp::Eq if isEq else il::BinOp::Ne;
    return emitTypedBinOp(self, op, cmpType, il::Val::Reg(cmpReg), il::Val::Imm(0));
}

/// Load the payload value from a tagged value aggregate at the given offset.
fn tvalPayloadVal(self: *mut FnLowerer, base: il::Reg, payload: resolver::Type, valOffset: i32) -> il::Val {
    if payload == resolver::Type::Void {
        return il::Val::Undef;
    }
    return emitRead(self, base, valOffset, payload);
}

/// Compute the address of the payload in a tagged value aggregate.
fn tvalPayloadAddr(self: *mut FnLowerer, base: il::Reg, valOffset: i32) -> il::Val {
    return il::Val::Reg(emitPtrOffset(self, base, valOffset));
}

/// Bind a variable to a tagged value's payload.
fn bindPayloadVariable(
    self: *mut FnLowerer,
    name: *[u8],
    subjectVal: il::Val,
    bindType: resolver::Type,
    matchBy: resolver::MatchBy,
    valOffset: i32,
    mutable: bool
) -> Var throws (LowerError) {
    let base = emitValToReg(self, subjectVal);
    let mut payload: il::Val = undefined;

    match matchBy {
        case resolver::MatchBy::Value =>
            payload = tvalPayloadVal(self, base, bindType, valOffset),
        case resolver::MatchBy::Ref, resolver::MatchBy::MutRef =>
            payload = tvalPayloadAddr(self, base, valOffset),
    };
    return newVar(self, name, ilType(self.low, bindType), mutable, payload);
}

fn bindMatchVariable(
    self: *mut FnLowerer,
    subject: *MatchSubject,
    binding: *ast::Node,
    mutable: bool
) -> ?Var throws (LowerError) {
    // Only bind if the pattern is an identifier.
    let case ast::NodeValue::Ident(name) = binding.value else {
        return nil;
    };
    // For optional aggregates, extract the payload from the tagged value.
    // The tag check already passed, so we know the payload is valid.
    if let case MatchSubjectKind::OptionalAggregate = subject.kind {
        let valOffset = resolver::getOptionalValOffset(subject.bindType) as i32;
        return try bindPayloadVariable(self, name, subject.val, subject.bindType, subject.by, valOffset, mutable);
    }
    // Declare the variable in the current block's scope.
    return newVar(self, name, ilType(self.low, subject.bindType), mutable, subject.val);
}

/// Bind variables from inside case patterns (union variants, records, slices).
/// `failBlock` is passed when nested patterns may require additional tests
/// that branch on mismatch (e.g. nested union variant tests).
fn bindPatternVariables(self: *mut FnLowerer, subject: *MatchSubject, patterns: *mut [*ast::Node], failBlock: BlockId) throws (LowerError) {
    for pattern in patterns {

        // Handle simple variant patterns like `Variant(x)`.
        if let arg = resolver::variantPatternBinding(self.low.resolver, pattern) {
            let case MatchSubjectKind::Union(unionInfo) = subject.kind
                else panic "bindPatternVariables: expected union subject";
            let valOffset = unionInfo.valOffset as i32;

            // Get the actual field type from the variant's record info.
            // This preserves the original data layout type (e.g. `*T`) even when
            // the resolver resolved the pattern against a dereferenced type (`T`).
            let variantExtra = resolver::nodeData(self.low.resolver, pattern).extra;
            let case resolver::NodeExtra::UnionVariant { ordinal, .. } = variantExtra
                else panic "bindPatternVariables: expected variant extra";
            let payloadType = unionInfo.variants[ordinal].valueType;
            let payloadRec = resolver::getRecord(payloadType)
                else panic "bindPatternVariables: expected record payload";
            let fieldType = payloadRec.fields[0].fieldType;

            match arg.value {
                case ast::NodeValue::Ident(name) => {
                    try bindPayloadVariable(self, name, subject.val, fieldType, subject.by, valOffset, false);
                }
                case ast::NodeValue::Placeholder => {}
                else => {
                    // Nested pattern inside a variant call, e.g. `Variant(Inner { x, y })`.
                    let base = emitValToReg(self, subject.val);
                    let payloadBase = emitPtrOffset(self, base, valOffset);
                    let fieldInfo = resolver::RecordField {
                        name: nil,
                        fieldType,
                        offset: 0,
                    };
                    try bindFieldVariable(self, arg, payloadBase, fieldInfo, subject.by, failBlock);
                }
            }
        }
        match pattern.value {
            // Compound variant patterns like `Variant { a, b }`.
            case ast::NodeValue::RecordLit(lit) =>
                try bindRecordPatternFields(self, subject, pattern, lit, failBlock),
            // Array patterns like `[a, b, c]`.
            case ast::NodeValue::ArrayLit(items) =>
                try bindArrayPatternElements(self, subject, items, failBlock),
            // Literals, wildcards, identifiers: no bindings needed.
            else => {},
        }
    }
}

/// Bind variables from an array literal pattern (e.g., `[a, 1, c]`).
/// Each element is either bound as a variable, skipped (placeholder), or
/// tested against the subject element, branching to `failBlock` on mismatch.
fn bindArrayPatternElements(
    self: *mut FnLowerer,
    subject: *MatchSubject,
    items: *mut [*ast::Node],
    failBlock: BlockId
) throws (LowerError) {
    let case resolver::Type::Array(arrInfo) = subject.type
        else throw LowerError::ExpectedSliceOrArray;

    let elemTy = *arrInfo.item;
    let elemLayout = resolver::getTypeLayout(elemTy);
    let stride = elemLayout.size as i32;
    let base = emitValToReg(self, subject.val);

    for elem, i in items {
        let fieldInfo = resolver::RecordField {
            name: nil,
            fieldType: elemTy,
            offset: (i as i32) * stride,
        };
        try bindFieldVariable(self, elem, base, fieldInfo, subject.by, failBlock);
    }
}

fn bindRecordPatternFields(self: *mut FnLowerer, subject: *MatchSubject, pattern: *ast::Node, lit: ast::RecordLit, failBlock: BlockId) throws (LowerError) {
    // No fields to bind (e.g., `{ .. }`).
    if lit.fields.len == 0 {
        return;
    }
    // Get the union type info from the subject.
    let case MatchSubjectKind::Union(unionInfo) = subject.kind
        else panic "bindRecordPatternFields: expected union subject";

    // Get the variant index from the pattern node.
    let case resolver::NodeExtra::UnionVariant { ordinal: variantOrdinal, .. } =
        resolver::nodeData(self.low.resolver, pattern).extra
    else throw LowerError::MissingMetadata;

    // Get the record type from the variant's payload type.
    let payloadType = unionInfo.variants[variantOrdinal].valueType;
    let recInfo = resolver::getRecord(payloadType)
        else throw LowerError::ExpectedRecord;

    // Get the payload base pointer which points to the record within the tagged union.
    let base = emitValToReg(self, subject.val);
    let valOffset = unionInfo.valOffset as i32;
    let payloadBase = emitPtrOffset(self, base, valOffset);

    try bindNestedRecordFields(self, payloadBase, lit, recInfo, subject.by, failBlock);
}

/// Emit a field read from a base pointer.
fn emitFieldRead(self: *mut FnLowerer, base: il::Reg, fieldInfo: resolver::RecordField, matchBy: resolver::MatchBy) -> il::Val {
    match matchBy {
        case resolver::MatchBy::Value => {
            return emitRead(self, base, fieldInfo.offset, fieldInfo.fieldType);
        }
        case resolver::MatchBy::Ref, resolver::MatchBy::MutRef => {
            return il::Val::Reg(emitPtrOffset(self, base, fieldInfo.offset));
        }
    }
}

/// Bind a single record field to a pattern variable, with support for nested
/// pattern tests that branch to `failBlock` on mismatch.
fn bindFieldVariable(
    self: *mut FnLowerer,
    binding: *ast::Node,
    base: il::Reg,
    fieldInfo: resolver::RecordField,
    matchBy: resolver::MatchBy,
    failBlock: BlockId
) throws (LowerError) {
    match binding.value {
        case ast::NodeValue::Ident(name) => {
            let val = emitFieldRead(self, base, fieldInfo, matchBy);
            let _ = newVar(self, name, ilType(self.low, fieldInfo.fieldType), false, val);
        }
        case ast::NodeValue::Placeholder => {}
        case ast::NodeValue::RecordLit(lit) => {
            // Check if this record literal is a union variant pattern.
            if let keyNode = resolver::patternVariantKeyNode(binding) {
                if let case resolver::NodeExtra::UnionVariant { .. } = resolver::nodeData(self.low.resolver, keyNode).extra {
                    try emitNestedFieldTest(self, binding, base, fieldInfo, matchBy, failBlock);
                    return;
                }
            }
            // Plain nested record destructuring pattern.
            // Auto-deref: if the field is a pointer, load it first.
            let mut derefType = fieldInfo.fieldType;
            let mut nestedBase = emitPtrOffset(self, base, fieldInfo.offset);
            if let case resolver::Type::Pointer { target, .. } = fieldInfo.fieldType {
                let ptrReg = nextReg(self);
                emitLoadW64(self, ptrReg, nestedBase);
                nestedBase = ptrReg;
                derefType = *target;
            }
            let recInfo = resolver::getRecord(derefType)
                else throw LowerError::ExpectedRecord;
            try bindNestedRecordFields(self, nestedBase, lit, recInfo, matchBy, failBlock);
        }
        else => {
            // Nested pattern requiring a test (union variant scope access, literal, etc).
            try emitNestedFieldTest(self, binding, base, fieldInfo, matchBy, failBlock);
        }
    }
}

/// Emit a nested pattern test for a record field value, branching to
/// `failBlock` if the pattern does not match. On success, continues in
/// a fresh block and binds any nested variables.
fn emitNestedFieldTest(
    self: *mut FnLowerer,
    pattern: *ast::Node,
    base: il::Reg,
    fieldInfo: resolver::RecordField,
    matchBy: resolver::MatchBy,
    failBlock: BlockId
) throws (LowerError) {
    let mut fieldType = fieldInfo.fieldType;
    let fieldPtr = emitPtrOffset(self, base, fieldInfo.offset);

    // Auto-deref: when the field is a pointer and the pattern destructures
    // the pointed-to value, load the pointer and use the target type.
    // The loaded pointer becomes the base address for the nested subject.
    let mut derefBase: ?il::Reg = nil;
    if let case resolver::Type::Pointer { target, .. } = fieldType {
        if isDestructuringPattern(pattern) {
            let ptrReg = nextReg(self);
            emitLoadW64(self, ptrReg, fieldPtr);
            derefBase = ptrReg;
            fieldType = *target;
        }
    }

    // Build a MatchSubject for the nested field.
    let ilTy = ilType(self.low, fieldType);
    let kind = matchSubjectKind(fieldType);

    // Determine the subject value.
    let mut val = il::Val::Reg(fieldPtr);
    if let reg = derefBase {
        // Auto-deref: the loaded pointer is the address of the target value.
        val = il::Val::Reg(reg);
    } else if not isAggregateType(fieldType) {
        // Scalar: load the value.
        val = emitRead(self, base, fieldInfo.offset, fieldType);
    }

    let nestedSubject = MatchSubject {
        val,
        type: fieldType,
        ilType: ilTy,
        bindType: fieldType,
        kind,
        by: matchBy,
    };

    // Emit the pattern test: on success jump to continueBlock, on fail to failBlock.
    let continueBlock = try createBlock(self, "nest");
    try emitPatternMatch(self, &nestedSubject, pattern, continueBlock, failBlock);
    try switchToAndSeal(self, continueBlock);

    // After the test succeeds, bind any nested variables.
    let patterns: *mut [*ast::Node] = &mut [pattern];
    try bindPatternVariables(self, &nestedSubject, patterns, failBlock);
}

/// Bind variables from a nested record literal pattern.
fn bindNestedRecordFields(
    self: *mut FnLowerer,
    base: il::Reg,
    lit: ast::RecordLit,
    recInfo: resolver::RecordType,
    matchBy: resolver::MatchBy,
    failBlock: BlockId
) throws (LowerError) {
    for fieldNode in lit.fields {
        let case ast::NodeValue::RecordLitField(field) = fieldNode.value else {
            throw LowerError::UnexpectedNodeValue(fieldNode);
        };
        let fieldIdx = resolver::recordFieldIndexFor(self.low.resolver, fieldNode)
            else throw LowerError::MissingMetadata;
        if fieldIdx >= recInfo.fields.len {
            throw LowerError::MissingMetadata;
        }
        let fieldInfo = recInfo.fields[fieldIdx];

        try bindFieldVariable(self, field.value, base, fieldInfo, matchBy, failBlock);
    }
}

/// Lower function body to a list of basic blocks.
fn lowerFnBody(self: *mut FnLowerer, body: *ast::Node) -> *[il::Block] throws (LowerError) {
    // Create and switch to entry block.
    let entry = try createBlock(self, "entry");
    self.entryBlock = entry;
    switchToBlock(self, entry);

    /// Bind parameter registers to variables in the entry block.
    for def in self.params {
        defVar(self, def.var, il::Val::Reg(def.reg));
    }
    /// Lower function body.
    try lowerBlock(self, body);

    // Add implicit return if body doesn't diverge.
    if not blockHasTerminator(self) {
        if self.fnType.throwList.len > 0 {
            if *self.fnType.returnType == resolver::Type::Void {
                // Implicit `void` return in throwing function: wrap in result success.
                let val = try buildResult(self, 0, nil, resolver::Type::Void);
                try emitRetVal(self, val);
            } else {
                // Non-void throwing function without explicit return should
                // not happen.
                panic "lowerFnBody: missing return in non-void function";
            }
        } else {
            emit(self, il::Instr::Ret { val: nil });
        }
    }
    return try finalizeBlocks(self);
}

/// Lower a scalar match as a switch instruction.
fn lowerMatchSwitch(self: *mut FnLowerer, prongs: *mut [*ast::Node], subject: *MatchSubject, mergeBlock: *mut ?BlockId) throws (LowerError) {
    let mut blocks: [BlockId; 32] = undefined;
    let mut cases: *mut [il::SwitchCase] = &mut [];
    let mut defaultIdx: u32 = 0;
    let entry = currentBlock(self);

    for p, i in prongs {
        let case ast::NodeValue::MatchProng(prong) = p.value
            else throw LowerError::UnexpectedNodeValue(p);

        match prong.arm {
            case ast::ProngArm::Binding(_), ast::ProngArm::Else => {
                blocks[i] = try createBlock(self, "default");
                defaultIdx = i;
            }
            case ast::ProngArm::Case(pats) => {
                blocks[i] = try createBlock(self, "case");
                for pat in pats {
                    let cv = resolver::constValueFor(self.low.resolver, pat)
                        else throw LowerError::MissingConst(pat);

                    cases.append(il::SwitchCase {
                        value: constToScalar(cv),
                        target: blocks[i].n,
                        args: &mut []
                    }, self.allocator);
                }
            }
        }
        addPredecessor(self, blocks[i], entry);
    }
    emit(self, il::Instr::Switch {
        val: subject.val,
        defaultTarget: blocks[defaultIdx].n,
        defaultArgs: &mut [],
        cases: &mut cases[..]
    });

    for p, i in prongs {
        let case ast::NodeValue::MatchProng(prong) = p.value
            else throw LowerError::UnexpectedNodeValue(p);

        try switchToAndSeal(self, blocks[i]);
        try lowerNode(self, prong.body);
        try emitMergeIfUnterminated(self, mergeBlock);
    }
    if let blk = *mergeBlock {
        try switchToAndSeal(self, blk);
    }
}

/// Lower a match statement.
///
/// Processes prongs sequentially, generating comparison code and branches for each.
/// Prongs are processed in source order, so earlier prongs take precedence.
///
/// Guards are handled by emitting an additional branch after pattern matching
/// but before the body.
///
/// Example:
///
///   match x {
///       case 0 => return 0,
///       case 1 => return 1,
///       else => return 2,
///   }
///
/// Generates:
///
///   arm#0:
///       br.eq w32 %x 0 case#0 arm#1;    // if `x == 0`, jump to case#0, else arm#1
///   case#0:
///       ret 0;                          // `case 0` body
///   arm#1:
///       br.eq w32 %x 1 case#1 arm#2;    // if `x == 1`, jump to case#1, else arm#2
///   case#1:
///       ret 1;                          // `case 1` body
///   arm#2:
///       jmp else#0;                     // fallthrough to `else`
///   else#0:
///       ret 2;                          // `else` body
///
/// Example: Binding with guard
///
///   match x {
///       y if y > 0 => return y,
///       else => return 0,
///   }
///
/// Generates:
///
///   arm#0:
///       jmp guard#0;                    // catch-all binding, jump to guard
///   case#0(w32 %y):
///       ret %y;                         // guarded case body, receives bound var
///   guard#0:
///       sgt w32 %cmp %x 0;              // evaluate guard `y > 0`
///       br.ne w32 %cmp 0 case#0 arm#1;  // if `true`, jump to case body
///   arm#1:
///       jmp else#0;                     // guard failed, fallthrough to `else`
///   else#0:
///       ret 0;                          // `else` body
///
fn lowerMatch(self: *mut FnLowerer, node: *ast::Node, m: ast::Match) throws (LowerError) {
    assert m.prongs.len > 0;

    let prongs = m.prongs;
    // Lower the subject expression once; reused across all arms.
    let subject = try lowerMatchSubject(self, m.subject);
    // Merge block created lazily if any arm needs it (i.e., doesn't diverge).
    let mut mergeBlock: ?BlockId = nil;

    // Use `switch` instruction for matches with constant patterns.
    if resolver::isMatchConst(self.low.resolver, node) {
        try lowerMatchSwitch(self, prongs, &subject, &mut mergeBlock);
        return;
    }
    // Fallback: chained branches.
    let firstArm = try createBlock(self, "arm");
    try emitJmp(self, firstArm);
    try switchToAndSeal(self, firstArm);

    for prongNode, i in prongs {
        let prongScope = enterVarScope(self);
        let case ast::NodeValue::MatchProng(prong) = prongNode.value
            else panic "lowerMatch: expected match prong";

        let isLastArm = i + 1 == prongs.len;
        let hasGuard = prong.guard != nil;
        let catchAll = resolver::isProngCatchAll(self.low.resolver, prongNode);

        // Entry block: guard block if present, otherwise the body block.
        // The guard block must be created before the body block so that
        // block indices are in reverse post-order (RPO), which the register
        // allocator requires.
        let mut entryBlock: BlockId = undefined;
        if hasGuard {
            entryBlock = try createBlock(self, "guard");
        }
        // Body block: where the case body lives.
        let mut bodyLabel = "case";
        if prong.arm == ast::ProngArm::Else {
            bodyLabel = "else";
        }
        let mut bodyBlock = try createBlock(self, bodyLabel);
        if not hasGuard {
            entryBlock = bodyBlock;
        }
        // Fallthrough block: jumped to when pattern or guard fails.
        let nextArm = try createBlock(self, "arm");

        // Emit pattern test: branch to entry block on match, next arm on fail.
        match prong.arm {
            case ast::ProngArm::Binding(_) if not catchAll =>
                try emitBindingTest(self, &subject, entryBlock, nextArm),
            case ast::ProngArm::Case(patterns) if not catchAll =>
                try emitPatternMatches(self, &subject, patterns, entryBlock, nextArm),
            else =>
                try emitJmp(self, entryBlock),
        }
        // Switch to entry block, where any variable bindings need to be created.
        try switchToAndSeal(self, entryBlock);

        // Bind pattern variables after successful match. Note that the guard
        // has not been evaluated yet. Nested patterns may emit additional
        // tests that branch to `nextArm` on failure, switching the current
        // block.
        match prong.arm {
            case ast::ProngArm::Binding(pat) =>
                try bindMatchVariable(self, &subject, pat, false),
            case ast::ProngArm::Case(patterns) =>
                try bindPatternVariables(self, &subject, patterns, nextArm),
            else => {},
        }

        // Evaluate guard if present; can still fail to next arm.
        if let g = prong.guard {
            try emitCondBranch(self, g, bodyBlock, nextArm);
        } else if currentBlock(self).n != bodyBlock.n {
            // Nested tests changed the current block. Create a new body block
            // after the nest blocks to maintain RPO ordering, and jump to it.
            bodyBlock = try createBlock(self, bodyLabel);
            try emitJmp(self, bodyBlock);
        }
        // Lower prong body and jump to merge if unterminated.
        try switchToAndSeal(self, bodyBlock);
        try lowerNode(self, prong.body);
        try emitMergeIfUnterminated(self, &mut mergeBlock);
        exitVarScope(self, prongScope);

        // Switch to next arm, unless last arm without guard.
        if not isLastArm or hasGuard {
            try switchToAndSeal(self, nextArm);
            if isLastArm {
                // Last arm with guard: guard failure jumps to merge.
                try emitMergeIfUnterminated(self, &mut mergeBlock);
            }
        }
    }
    // Continue in merge block if we have one, ie. if at least one arm doesn't
    // diverge.
    if let blk = mergeBlock {
        try switchToAndSeal(self, blk);
    }
}

/// Lower an `if let` statement.
fn lowerIfLet(self: *mut FnLowerer, cond: ast::IfLet) throws (LowerError) {
    let savedVarsLen = enterVarScope(self);
    let subject = try lowerMatchSubject(self, cond.pattern.scrutinee);
    let mut thenBlock: BlockId = undefined;
    if cond.pattern.guard == nil {
        thenBlock = try createBlock(self, "then");
    }
    let elseBlock = try createBlock(self, "else");
    let mut mergeBlock: ?BlockId = nil;

    // Pattern match: jump to @then on success, @else on failure.
    try lowerPatternMatch(self, &subject, &cond.pattern, &mut thenBlock, "then", elseBlock);

    // Lower then branch.
    try lowerNode(self, cond.thenBranch);
    try emitMergeIfUnterminated(self, &mut mergeBlock);

    // Lower else branch.
    try switchToAndSeal(self, elseBlock);
    if let elseBranch = cond.elseBranch {
        try lowerNode(self, elseBranch);
    }
    try emitMergeIfUnterminated(self, &mut mergeBlock);

    if let blk = mergeBlock {
        try switchToAndSeal(self, blk);
    }
    exitVarScope(self, savedVarsLen);
}

/// Emit pattern match branch with optional guard, and bind variables.
/// Used by `if-let`, `let-else`, and `while-let` lowering.
///
/// When a guard is present, the guard block is created before `successBlock`
/// to ensure block indices are in RPO.
fn lowerPatternMatch(
    self: *mut FnLowerer,
    subject: *MatchSubject,
    pat: *ast::PatternMatch,
    successBlock: *mut BlockId,
    successLabel: *[u8],
    failBlock: BlockId
) throws (LowerError) {
    // If guard present, pattern match jumps to @guard, then guard evaluation
    // jumps to `successBlock` or `failBlock`. Otherwise, jump directly to
    // `successBlock`.
    let mut targetBlock: BlockId = undefined;
    if pat.guard != nil {
        targetBlock = try createBlock(self, "guard");
        *successBlock = try createBlock(self, successLabel);
    } else {
        targetBlock = *successBlock;
    }
    match pat.kind {
        case ast::PatternKind::Case => {
            let patterns: *mut [*ast::Node] = &mut [pat.pattern];
            // Jump to `targetBlock` if the pattern matches, `failBlock` otherwise.
            try emitPatternMatches(self, subject, patterns, targetBlock, failBlock);
            try switchToAndSeal(self, targetBlock);
            // Bind any variables inside the pattern. Nested patterns may
            // emit additional tests that branch to `failBlock`, switching
            // the current block.
            try bindPatternVariables(self, subject, patterns, failBlock);
        }
        case ast::PatternKind::Binding => {
            // Jump to `targetBlock` if there is a value present, `failBlock` otherwise.
            try emitBindingTest(self, subject, targetBlock, failBlock);
            try switchToAndSeal(self, targetBlock);
            // Bind the matched value to the pattern variable.
            try bindMatchVariable(self, subject, pat.pattern, pat.mutable);
        }
    }
    // Handle guard: on success jump to `successBlock`, on failure jump to `failBlock`.
    if let g = pat.guard {
        try emitCondBranch(self, g, *successBlock, failBlock);
        try switchToAndSeal(self, *successBlock);
    } else if currentBlock(self).n != targetBlock.n {
        // Nested tests changed the current block. Create a new success block
        // after the nest blocks to maintain RPO ordering, and jump to it.
        *successBlock = try createBlock(self, successLabel);
        try emitJmp(self, *successBlock);
        try switchToAndSeal(self, *successBlock);
    }
}

/// Lower a `let-else` statement.
fn lowerLetElse(self: *mut FnLowerer, letElse: ast::LetElse) throws (LowerError) {
    let subject = try lowerMatchSubject(self, letElse.pattern.scrutinee);
    let mut mergeBlock: BlockId = undefined;
    if letElse.pattern.guard == nil {
        mergeBlock = try createBlock(self, "merge");
    }
    // Else branch executes when the pattern fails to match.
    let elseBlock = try createBlock(self, "else");

    // Evaluate pattern and jump to @end or @else.
    try lowerPatternMatch(self, &subject, &letElse.pattern, &mut mergeBlock, "merge", elseBlock);
    try switchToAndSeal(self, elseBlock);
    try lowerNode(self, letElse.elseBranch);

    // Continue in @merge. The else branch must diverge, so @merge has only
    // one predecessor.
    try switchToAndSeal(self, mergeBlock);
}

/// Lower a `while let` loop as a match-driven loop.
fn lowerWhileLet(self: *mut FnLowerer, w: ast::WhileLet) throws (LowerError) {
    let savedVarsLen = enterVarScope(self);
    // Create control flow blocks: loop header, body (created lazily when
    // there's a guard), and exit.
    let whileBlock = try createBlock(self, "while");
    let mut bodyBlock: BlockId = undefined;
    if w.pattern.guard == nil {
        bodyBlock = try createBlock(self, "body");
    }
    let endBlock = try createBlock(self, "merge");

    // Enter loop context and jump to loop header.
    enterLoop(self, endBlock, whileBlock);
    try switchAndJumpTo(self, whileBlock);
    let subject = try lowerMatchSubject(self, w.pattern.scrutinee);

    // Evaluate pattern and jump to loop body or loop end.
    try lowerPatternMatch(self, &subject, &w.pattern, &mut bodyBlock, "body", endBlock);

    // Lower loop body, jump back to loop header, and exit loop context.
    try lowerBlock(self, w.body);
    try emitJmpAndSeal(self, whileBlock);

    exitLoop(self);
    try switchToAndSeal(self, endBlock);
    exitVarScope(self, savedVarsLen);
}

///////////////////
// Node Lowering //
///////////////////

/// Lower an AST node.
fn lowerNode(self: *mut FnLowerer, node: *ast::Node) throws (LowerError) {
    if self.low.options.debug {
        self.srcLoc.offset = node.span.offset;
    }
    match node.value {
        case ast::NodeValue::Block(_) => {
            try lowerBlock(self, node);
        }
        case ast::NodeValue::Return { value } => {
            try lowerReturnStmt(self, node, value);
        }
        case ast::NodeValue::Throw { expr } => {
            try lowerThrowStmt(self, expr);
        }
        case ast::NodeValue::Let(l) => {
            try lowerLet(self, node, l);
        }
        case ast::NodeValue::ConstDecl(decl) => {
            // Local constants lower to data declarations and emit no runtime code.
            try lowerDataDecl(self.low, node, decl.value, true);
        }
        case ast::NodeValue::StaticDecl(decl) => {
            // Local statics lower to data declarations and emit no runtime code.
            try lowerDataDecl(self.low, node, decl.value, false);
        }
        case ast::NodeValue::If(i) => {
            try lowerIf(self, i);
        }
        case ast::NodeValue::IfLet(i) => {
            try lowerIfLet(self, i);
        }
        case ast::NodeValue::Assign(a) => {
            try lowerAssign(self, a);
        }
        case ast::NodeValue::Loop { body } => {
            try lowerLoop(self, body);
        }
        case ast::NodeValue::While(w) => {
            try lowerWhile(self, w);
        }
        case ast::NodeValue::WhileLet(w) => {
            try lowerWhileLet(self, w);
        }
        case ast::NodeValue::For(f) => {
            try lowerFor(self, node, f);
        }
        case ast::NodeValue::Break => {
            try lowerBreak(self);
        }
        case ast::NodeValue::Continue => {
            try lowerContinue(self);
        }
        case ast::NodeValue::Match(m) => {
            try lowerMatch(self, node, m);
        }
        case ast::NodeValue::LetElse(letElse) => {
            try lowerLetElse(self, letElse);
        }
        case ast::NodeValue::ExprStmt(expr) => {
            let _ = try lowerExpr(self, expr);
        }
        case ast::NodeValue::Panic { .. } => {
            emit(self, il::Instr::Unreachable);
        }
        case ast::NodeValue::Assert { condition, .. } => {
            // Lower `assert <cond>` as: `if not cond { unreachable; }`.
            let thenBlock = try createBlock(self, "assert.fail");
            let endBlock = try createBlock(self, "assert.ok");

            // Branch: if condition is `true`, go to `endBlock`; if `false`, go to `thenBlock`.
            try emitCondBranch(self, condition, endBlock, thenBlock);
            try sealBlock(self, thenBlock);

            // Emit `unreachable` in the failure block.
            switchToBlock(self, thenBlock);
            emit(self, il::Instr::Unreachable);

            // Continue after the assert.
            try switchToAndSeal(self, endBlock);
        }
        else => {
            // Treat as expression statement, discard result.
            let _ = try lowerExpr(self, node);
        }
    }
}

/// Lower a code block.
fn lowerBlock(self: *mut FnLowerer, node: *ast::Node) throws (LowerError) {
    let case ast::NodeValue::Block(blk) = node.value else {
        throw LowerError::ExpectedBlock(node);
    };
    let savedVarsLen = enterVarScope(self);
    for stmt in blk.statements {
        try lowerNode(self, stmt);

        // If the statement diverges, further statements are unreachable.
        if blockHasTerminator(self) {
            exitVarScope(self, savedVarsLen);
            return;
        }
    }
    exitVarScope(self, savedVarsLen);
}

///////////////////////////////////////
// Record and Aggregate Type Helpers //
///////////////////////////////////////

/// Extract the nominal record info from a resolver type.
fn recordInfoFromType(typ: resolver::Type) -> ?resolver::RecordType {
    let case resolver::Type::Nominal(resolver::NominalType::Record(recInfo)) = typ
        else return nil;
    return recInfo;
}

/// Extract the nominal union info from a resolver type.
fn unionInfoFromType(typ: resolver::Type) -> ?resolver::UnionType {
    let case resolver::Type::Nominal(resolver::NominalType::Union(unionInfo)) = typ
        else return nil;
    return unionInfo;
}

/// Return the effective type of a node after any coercion applied by
/// the resolver. `lowerExpr` already materializes the coercion in the
/// IL value, so the lowerer must use the post-coercion type when
/// choosing how to compare or store that value.
fn effectiveType(self: *mut FnLowerer, node: *ast::Node) -> resolver::Type throws (LowerError) {
    let ty = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    if let coerce = resolver::coercionFor(self.low.resolver, node) {
        if let case resolver::Coercion::OptionalLift(optTy) = coerce {
            return optTy;
        }
    }
    return ty;
}

/// Check if a resolver type lowers to an aggregate in memory.
fn isAggregateType(typ: resolver::Type) -> bool {
    match typ {
        case resolver::Type::Nominal(_) => {
            // Void unions are small enough to pass by value.
            return not resolver::isVoidUnion(typ);
        }
        case resolver::Type::Slice { .. } => return true,
        case resolver::Type::TraitObject { .. } => return true,
        case resolver::Type::Array(_) => return true,
        case resolver::Type::Nil => return true,
        case resolver::Type::Optional(resolver::Type::Pointer { .. }) => {
            // Optional pointers are scalar (single word) due to NPO.
            return false;
        }
        case resolver::Type::Optional(_) => {
            // All other optionals, including optional slices, are aggregates.
            return true;
        }
        else => return false,
    }
}

/// Check if a resolver type is a small aggregate that can be
/// passed or returned by value in a register.
fn isSmallAggregate(typ: resolver::Type) -> bool {
    match typ {
        case resolver::Type::Nominal(_) => {
            if resolver::isVoidUnion(typ) {
                return false;
            }
            let layout = resolver::getTypeLayout(typ);
            return layout.size <= resolver::PTR_SIZE;
        }
        else => return false,
    }
}

/// Whether a function needs a hidden return parameter.
///
/// This is the case for throwing functions, which return a result aggregate,
/// and for functions returning large aggregates that cannot be passed in
/// registers.
fn requiresReturnParam(fnType: *resolver::FnType) -> bool {
    return fnType.throwList.len > 0
        or (isAggregateType(*fnType.returnType)
        and not isSmallAggregate(*fnType.returnType));
}

/// Check if a node is a void union variant literal (e.g. `Color::Red`).
/// If so, returns the variant's tag index. This enables optimized comparisons
/// that only check the tag instead of doing full aggregate comparison.
fn voidVariantIndex(res: *resolver::Resolver, node: *ast::Node) -> ?i64 {
    let data = resolver::nodeData(res, node);
    let sym = data.sym else {
        return nil;
    };
    let case resolver::SymbolData::Variant { type: payloadType, index, .. } = sym.data else {
        return nil;
    };
    // Only void variants can use tag-only comparison.
    if payloadType != resolver::Type::Void {
        return nil;
    }
    return index as i64;
}

/// Check if an expression has persistent storage, ie. is an "lvalue".
/// Such expressions need to be copied when used to initialize a variable,
/// since their storage continues to exist independently. Temporaries (literals,
/// call results) can be adopted directly without copying.
fn hasStorage(node: *ast::Node) -> bool {
    match node.value {
        case ast::NodeValue::Ident(_),
             ast::NodeValue::FieldAccess(_),
             ast::NodeValue::Subscript { .. },
             ast::NodeValue::Deref(_) => return true,
        else => return false,
    }
}

/// Reserve stack storage for a value of the given type.
fn emitReserve(self: *mut FnLowerer, typ: resolver::Type) -> il::Reg throws (LowerError) {
    let layout = resolver::getTypeLayout(typ);
    return emitReserveLayout(self, layout);
}

/// Reserve stack storage with an explicit layout.
fn emitReserveLayout(self: *mut FnLowerer, layout: resolver::Layout) -> il::Reg {
    let dst = nextReg(self);

    emit(self, il::Instr::Reserve {
        dst,
        size: il::Val::Imm(layout.size as i64),
        alignment: layout.alignment,
    });
    return dst;
}

/// Store a value into an address.
fn emitStore(self: *mut FnLowerer, base: il::Reg, offset: i32, typ: resolver::Type, src: il::Val) throws (LowerError) {
    // `undefined` values need no store.
    if let case il::Val::Undef = src {
        return;
    }
    if isAggregateType(typ) {
        let dst = emitPtrOffset(self, base, offset);
        let src = emitValToReg(self, src);
        let layout = resolver::getTypeLayout(typ);

        emit(self, il::Instr::Blit { dst, src, size: il::Val::Imm(layout.size as i64) });
    } else {
        emit(self, il::Instr::Store {
            typ: ilType(self.low, typ),
            src,
            dst: base,
            offset,
        });
    }
}

/// Allocate stack space for a value and store it. Returns a pointer to the value.
fn emitStackVal(self: *mut FnLowerer, typ: resolver::Type, val: il::Val) -> il::Val throws (LowerError) {
    let ptr = try emitReserve(self, typ);
    try emitStore(self, ptr, 0, typ, val);
    return il::Val::Reg(ptr);
}

/// Generic helper to build any tagged aggregate.
/// Reserves space based on the provided layout, stores the tag, and optionally
/// stores the payload value at `valOffset`.
fn buildTagged(
    self: *mut FnLowerer,
    layout: resolver::Layout,
    tag: i64,
    payload: ?il::Val,
    payloadType: resolver::Type,
    tagSize: u32,
    valOffset: i32
) -> il::Val throws (LowerError) {
    let dst = nextReg(self);
    emit(self, il::Instr::Reserve {
        dst,
        size: il::Val::Imm(layout.size as i64),
        alignment: layout.alignment,
    });
    if tagSize == 1 {
        emitStoreW8At(self, il::Val::Imm(tag), dst, TVAL_TAG_OFFSET);
    } else {
        emitStoreW64At(self, il::Val::Imm(tag), dst, TVAL_TAG_OFFSET);
    }

    if let val = payload {
        if payloadType != resolver::Type::Void {
            try emitStore(self, dst, valOffset, payloadType, val);
        }
    }
    return il::Val::Reg(dst);
}

/// Wrap a value in an optional type.
///
/// For optional pointers (`?*T`), the value is returned as-is since pointers
/// use zero to represent `nil`. For other optionals, builds a tagged aggregate.
/// with the tag set to `1`, and the value as payload.
fn wrapInOptional(self: *mut FnLowerer, val: il::Val, optType: resolver::Type) -> il::Val throws (LowerError) {
    let case resolver::Type::Optional(inner) = optType else {
        throw LowerError::ExpectedOptional;
    };
    // Null-pointer-optimized (NPO) types are used as-is -- valid values are never null.
    if resolver::isNullableType(*inner) {
        return val;
    }
    let layout = resolver::getTypeLayout(optType);
    let valOffset = resolver::getOptionalValOffset(*inner) as i32;

    return try buildTagged(self, layout, 1, val, *inner, 1, valOffset);
}

/// Build a `nil` value for an optional type.
///
/// For optional pointers (`?*T`), returns an immediate `0` (null pointer).
/// For other optionals, builds a tagged aggregate with tag set to `0` (absent).
fn buildNilOptional(self: *mut FnLowerer, optType: resolver::Type) -> il::Val throws (LowerError) {
    match optType {
        case resolver::Type::Optional(resolver::Type::Pointer { .. }) => {
            return il::Val::Imm(0);
        }
        case resolver::Type::Optional(resolver::Type::Slice { item, mutable }) => {
            return try buildSliceValue(self, item, mutable, il::Val::Imm(0), il::Val::Imm(0), il::Val::Imm(0));
        }
        case resolver::Type::Optional(inner) => {
            let valOffset = resolver::getOptionalValOffset(*inner) as i32;
            return try buildTagged(self, resolver::getTypeLayout(optType), 0, nil, *inner, 1, valOffset);
        }
        else => throw LowerError::ExpectedOptional,
    }
}

/// Build a result value for throwing functions.
fn buildResult(
    self: *mut FnLowerer,
    tag: i64,
    payload: ?il::Val,
    payloadType: resolver::Type
) -> il::Val throws (LowerError) {
    let successType = *self.fnType.returnType;
    let layout = resolver::getResultLayout(
        successType, self.fnType.throwList
    );
    return try buildTagged(self, layout, tag, payload, payloadType, 8, RESULT_VAL_OFFSET);
}

/// Build a slice aggregate from a data pointer, length and capacity.
fn buildSliceValue(
    self: *mut FnLowerer,
    elemTy: *resolver::Type,
    mutable: bool,
    ptrVal: il::Val,
    lenVal: il::Val,
    capVal: il::Val
) -> il::Val throws (LowerError) {
    let sliceType = resolver::Type::Slice { item: elemTy, mutable };
    let dst = try emitReserve(self, sliceType);
    let ptrTy = resolver::Type::Pointer { target: elemTy, mutable };

    try emitStore(self, dst, SLICE_PTR_OFFSET, ptrTy, ptrVal);
    try emitStore(self, dst, SLICE_LEN_OFFSET, resolver::Type::U32, lenVal);
    try emitStore(self, dst, SLICE_CAP_OFFSET, resolver::Type::U32, capVal);

    return il::Val::Reg(dst);
}

/// Build a trait object fat pointer from a data pointer and a v-table.
fn buildTraitObject(
    self: *mut FnLowerer,
    dataVal: il::Val,
    traitInfo: *resolver::TraitType,
    inst: *resolver::InstanceEntry
) -> il::Val throws (LowerError) {
    let vName = vtableName(self.low, inst.moduleId, inst.concreteTypeName, traitInfo.name);

    // Reserve space for the trait object on the stack.
    let slot = emitReserveLayout(self, resolver::Layout {
        size: resolver::PTR_SIZE * 2,
        alignment: resolver::PTR_SIZE,
    });

    // Store data pointer.
    emit(self, il::Instr::Store {
        typ: il::Type::W64,
        src: dataVal,
        dst: slot,
        offset: TRAIT_OBJ_DATA_OFFSET,
    });

    // Store v-table address.
    emit(self, il::Instr::Store {
        typ: il::Type::W64,
        src: il::Val::DataSym(vName),
        dst: slot,
        offset: TRAIT_OBJ_VTABLE_OFFSET,
    });
    return il::Val::Reg(slot);
}

/// Compute a field pointer by adding a byte offset to a base address.
fn emitPtrOffset(self: *mut FnLowerer, base: il::Reg, offset: i32) -> il::Reg {
    if offset == 0 {
        return base;
    }
    let dst = nextReg(self);

    emit(self, il::Instr::BinOp {
        op: il::BinOp::Add,
        typ: il::Type::W64,
        dst,
        a: il::Val::Reg(base),
        b: il::Val::Imm(offset as i64),
    });
    return dst;
}

/// Emit an element address computation for array/slice indexing.
/// Computes: `base + idx * stride`.
fn emitElem(self: *mut FnLowerer, stride: u32, base: il::Reg, idx: il::Val) -> il::Reg {
    // Optimize: if index is zero.
    if idx == il::Val::Imm(0) {
        return base;
    }
    // Compute `offset = idx * stride`.
    let offset = nextReg(self);
    emit(self, il::Instr::BinOp {
        op: il::BinOp::Mul,
        typ: il::Type::W64,
        dst: offset,
        a: idx,
        b: il::Val::Imm(stride as i64)
    });
    // Compute `dst = base + offset`.
    let dst = nextReg(self);
    emit(self, il::Instr::BinOp {
        op: il::BinOp::Add,
        typ: il::Type::W64,
        dst,
        a: il::Val::Reg(base),
        b: il::Val::Reg(offset)
    });
    return dst;
}

/// Emit a typed binary operation, returning the result as a value.
fn emitTypedBinOp(self: *mut FnLowerer, op: il::BinOp, typ: il::Type, a: il::Val, b: il::Val) -> il::Val {
    let dst = nextReg(self);
    emit(self, il::Instr::BinOp { op, typ, dst, a, b });
    return il::Val::Reg(dst);
}

/// Emit a tag comparison for void variant equality/inequality.
fn emitTagCmp(self: *mut FnLowerer, op: ast::BinaryOp, val: il::Val, tagIdx: i64, valType: resolver::Type) -> il::Val
    throws (LowerError)
{
    let reg = emitValToReg(self, val);

    // For all-void unions, the value *is* the tag, not a pointer.
    let mut tag: il::Val = undefined;
    if resolver::isVoidUnion(valType) {
        tag = il::Val::Reg(reg);
    } else {
        tag = loadTag(self, reg, TVAL_TAG_OFFSET, il::Type::W8);
    }
    let binOp = il::BinOp::Eq if op == ast::BinaryOp::Eq else il::BinOp::Ne;
    return emitTypedBinOp(self, binOp, il::Type::W8, tag, il::Val::Imm(tagIdx));
}

/// Logical "and" between two values. Returns the result in a register.
fn emitLogicalAnd(self: *mut FnLowerer, left: ?il::Val, right: il::Val) -> il::Val {
    let prev = left else {
        return right;
    };
    return emitTypedBinOp(self, il::BinOp::And, il::Type::W32, prev, right);
}

//////////////////////////
// Aggregate Comparison //
//////////////////////////

/// Emit an equality test for values at an offset of the given base registers.
fn emitEqAtOffset(
    self: *mut FnLowerer,
    left: il::Reg,
    right: il::Reg,
    offset: i32,
    fieldType: resolver::Type
) -> il::Val throws (LowerError) {
    // For aggregate types, pass offset through and compare recursively.
    if isAggregateType(fieldType) {
        return try lowerAggregateEq(self, fieldType, left, right, offset);
    }
    // For scalar types, load and compare directly.
    let a = emitLoad(self, left, offset, fieldType);
    let b = emitLoad(self, right, offset, fieldType);
    let dst = nextReg(self);
    emit(self, il::Instr::BinOp { op: il::BinOp::Eq, typ: ilType(self.low, fieldType), dst, a, b });

    return il::Val::Reg(dst);
}

/// Compare two record values for equality.
fn lowerRecordEq(
    self: *mut FnLowerer,
    recInfo: resolver::RecordType,
    a: il::Reg,
    b: il::Reg,
    offset: i32
) -> il::Val throws (LowerError) {
    let mut result: ?il::Val = nil;

    for field in recInfo.fields {
        let cmp = try emitEqAtOffset(self, a, b, offset + field.offset, field.fieldType);

        result = emitLogicalAnd(self, result, cmp);
    }
    if let r = result {
        return r;
    }
    return il::Val::Imm(1);
}

/// Compare two slice values for equality.
fn lowerSliceEq(
    self: *mut FnLowerer,
    elemTy: *resolver::Type,
    mutable: bool,
    a: il::Reg,
    b: il::Reg,
    offset: i32
) -> il::Val throws (LowerError) {
    let ptrTy = resolver::Type::Pointer { target: elemTy, mutable };
    let ptrEq = try emitEqAtOffset(self, a, b, offset + SLICE_PTR_OFFSET, ptrTy);
    let lenEq = try emitEqAtOffset(self, a, b, offset + SLICE_LEN_OFFSET, resolver::Type::U32);

    return emitTypedBinOp(self, il::BinOp::And, il::Type::W32, ptrEq, lenEq);
}

/// Check if a type may contain uninitialized payload bytes when used as the
/// inner type of a `nil` optional. Unions with non-void variants and nested
/// optionals fall into this category; records and primitives do not.
fn needsGuardedPayloadCmp(inner: resolver::Type) -> bool {
    if let _ = unionInfoFromType(inner) {
        return true;
    }
    match inner {
        case resolver::Type::Optional(_) => return true,
        else => return false,
    }
}

/// Compare two optional aggregate values for equality.
///
/// Two optionals are equal when their tags match and either both are `nil` or
/// their payloads are equal.
///
/// For inner types that are safe to compare even when uninitialised, we use a
/// branchless formulation: `tagEq AND (tagNil OR payloadEq)`
///
/// For inner types that may contain uninitialized data when `nil` (unions,
/// nested optionals), the payload comparison is guarded behind a branch
/// so that `nil` payloads are never inspected.
fn lowerOptionalEq(
    self: *mut FnLowerer,
    inner: resolver::Type,
    a: il::Reg,
    b: il::Reg,
    offset: i32
) -> il::Val throws (LowerError) {
    let valOffset = resolver::getOptionalValOffset(inner) as i32;

    // Load tags.
    let tagA = loadTag(self, a, offset + TVAL_TAG_OFFSET, il::Type::W8);
    let tagB = loadTag(self, b, offset + TVAL_TAG_OFFSET, il::Type::W8);

    // For simple inner types (no unions/nested optionals), use branchless comparison.
    // TODO: Inline this function.
    if not needsGuardedPayloadCmp(inner) {
        let tagEq = emitTypedBinOp(self, il::BinOp::Eq, il::Type::W8, tagA, tagB);
        let tagNil = emitTypedBinOp(self, il::BinOp::Eq, il::Type::W8, tagA, il::Val::Imm(0));
        let payloadEq = try emitEqAtOffset(self, a, b, offset + valOffset, inner);

        return emitTypedBinOp(self, il::BinOp::And, il::Type::W32, tagEq,
            emitTypedBinOp(self, il::BinOp::Or, il::Type::W32, tagNil, payloadEq));
    }

    // For complex inner types, use branching comparison to avoid inspecting
    // uninitialized payload bytes.
    let resultReg = nextReg(self);
    let mergeBlock = try createBlockWithParam(self, "opteq#merge", il::Param {
        value: resultReg, type: il::Type::W8
    });
    let nilCheck = try createBlock(self, "opteq#nil");
    let payloadCmp = try createBlock(self, "opteq#payload");

    let falseArgs = try allocVal(self, il::Val::Imm(0));
    let trueArgs = try allocVal(self, il::Val::Imm(1));

    // Check if tags differ.
    emit(self, il::Instr::Br {
        op: il::CmpOp::Eq, typ: il::Type::W8, a: tagA, b: tagB,
        thenTarget: nilCheck.n, thenArgs: &mut [],
        elseTarget: mergeBlock.n, elseArgs: falseArgs,
    });
    addPredecessor(self, nilCheck, currentBlock(self));
    addPredecessor(self, mergeBlock, currentBlock(self));

    // Check if both are `nil`.
    try switchToAndSeal(self, nilCheck);
    emit(self, il::Instr::Br {
        op: il::CmpOp::Ne, typ: il::Type::W8, a: tagA, b: il::Val::Imm(0),
        thenTarget: payloadCmp.n, thenArgs: &mut [],
        elseTarget: mergeBlock.n, elseArgs: trueArgs,
    });
    addPredecessor(self, payloadCmp, currentBlock(self));
    addPredecessor(self, mergeBlock, currentBlock(self));

    // Both are non-`nil`, compare payloads.
    try switchToAndSeal(self, payloadCmp);
    let payloadEq = try emitEqAtOffset(self, a, b, offset + valOffset, inner);
    try emitJmpWithArg(self, mergeBlock, payloadEq);
    try switchToAndSeal(self, mergeBlock);

    return il::Val::Reg(resultReg);
}

/// Compare two union values for equality.
///
/// Two unions are equal iff their tags match and, for non-void variants,
/// their payloads are also equal. The comparison proceeds as follows.
///
/// First, compare the tags. If they differ, the unions are not equal, so
/// jump to the merge block with `false`. If they match, jump to the tag
/// block to determine which variant we're dealing with.
///
/// The tag block uses a switch on the tag value to dispatch to the appropriate
/// comparison block. Void variants jump directly to merge with `true`.
/// Non-void variants each have their own payload block that compares the
/// payload and jumps to the merge block with the result.
///
/// The merge block collects results from all paths via a block parameter
/// and returns the final equality result.
///
/// For all-void unions, we skip the control flow entirely and just compare
/// the tags directly.
///
/// TODO: Could be optimized to branchless when all non-void variants share
/// the same payload type: `tagEq AND (isVoidVariant OR payloadEq)`.
fn lowerUnionEq(
    self: *mut FnLowerer,
    unionInfo: resolver::UnionType,
    a: il::Reg,
    b: il::Reg,
    offset: i32
) -> il::Val throws (LowerError) {
    // Compare tags.
    let tagA = loadTag(self, a, offset + TVAL_TAG_OFFSET, il::Type::W8);
    let tagB = loadTag(self, b, offset + TVAL_TAG_OFFSET, il::Type::W8);

    // Fast path: all-void union just needs tag comparison.
    if unionInfo.isAllVoid {
        return emitTypedBinOp(self, il::BinOp::Eq, il::Type::W8, tagA, tagB);
    }
    // Holds the equality result.
    let resultReg = nextReg(self);

    // Where control flow continues after equality check is done. Receives
    // the result as a parameter.
    let mergeBlock = try createBlockWithParam(self, "eq#merge", il::Param {
        value: resultReg, type: il::Type::W8
    });
    // Where we switch on the tag to compare payloads.
    let tagBlock = try createBlock(self, "eq#tag");

    // Compare tags: if they differ, jump to merge with `false`; otherwise check payloads.
    let falseArgs = try allocVal(self, il::Val::Imm(0));

    assert tagBlock != mergeBlock;

    // TODO: Use the helper once the compiler supports more than eight function params.
    emit(self, il::Instr::Br {
        op: il::CmpOp::Eq, typ: il::Type::W8, a: tagA, b: tagB,
        thenTarget: tagBlock.n, thenArgs: &mut [],
        elseTarget: mergeBlock.n, elseArgs: falseArgs,
    });
    addPredecessor(self, tagBlock, currentBlock(self));
    addPredecessor(self, mergeBlock, currentBlock(self));

    // Create comparison blocks for each non-void variant and build switch cases.
    // Void variants jump directly to merge with `true`.
    let trueArgs = try allocVal(self, il::Val::Imm(1));
    let cases = try! alloc::allocSlice(
        self.low.arena, @sizeOf(il::SwitchCase), @alignOf(il::SwitchCase), unionInfo.variants.len as u32
    ) as *mut [il::SwitchCase];

    let mut caseBlocks: [?BlockId; resolver::MAX_UNION_VARIANTS] = undefined;
    for variant, i in unionInfo.variants {
        if variant.valueType == resolver::Type::Void {
            cases[i] = il::SwitchCase {
                value: i as i64,
                target: mergeBlock.n,
                args: trueArgs
            };
            caseBlocks[i] = nil;
        } else {
            let payloadBlock = try createBlock(self, "eq#payload");
            cases[i] = il::SwitchCase {
                value: i as i64,
                target: payloadBlock.n,
                args: &mut []
            };
            caseBlocks[i] = payloadBlock;
        }
    }

    // Emit switch in @tag block. Default arm is unreachable since we cover all variants.
    let unreachableBlock = try createBlock(self, "eq#unreachable");
    try switchToAndSeal(self, tagBlock);
    emit(self, il::Instr::Switch {
        val: tagA,
        defaultTarget: unreachableBlock.n,
        defaultArgs: &mut [],
        cases
    });

    // Add predecessor edges for switch targets.
    addPredecessor(self, unreachableBlock, tagBlock);
    for i in 0..unionInfo.variants.len {
        if let caseBlock = caseBlocks[i] {
            addPredecessor(self, caseBlock, tagBlock);
        } else {
            addPredecessor(self, mergeBlock, tagBlock);
        }
    }
    let valOffset = unionInfo.valOffset as i32;

    // Emit payload comparison blocks for non-void variants.
    for variant, i in unionInfo.variants {
        if let caseBlock = caseBlocks[i] {
            try switchToAndSeal(self, caseBlock);
            let payloadEq = try emitEqAtOffset(
                self, a, b, offset + valOffset, variant.valueType
            );
            try emitJmpWithArg(self, mergeBlock, payloadEq);
        }
    }
    // Emit unreachable block.
    try switchToAndSeal(self, unreachableBlock);
    emit(self, il::Instr::Unreachable);

    try switchToAndSeal(self, mergeBlock);
    return il::Val::Reg(resultReg);
}

/// Compare two array values for equality, element by element.
fn lowerArrayEq(
    self: *mut FnLowerer,
    arr: resolver::ArrayType,
    a: il::Reg,
    b: il::Reg,
    offset: i32
) -> il::Val throws (LowerError) {
    let elemLayout = resolver::getTypeLayout(*arr.item);
    let stride = elemLayout.size as i32;
    let mut result: ?il::Val = nil;

    for i in 0..arr.length {
        let elemOffset = offset + (i as i32) * stride;
        let cmp = try emitEqAtOffset(self, a, b, elemOffset, *arr.item);
        result = emitLogicalAnd(self, result, cmp);
    }
    if let r = result {
        return r;
    }
    // Empty arrays are always equal.
    return il::Val::Imm(1);
}

/// Compare two aggregate values for equality.
fn lowerAggregateEq(
    self: *mut FnLowerer,
    typ: resolver::Type,
    a: il::Reg,
    b: il::Reg,
    offset: i32
) -> il::Val throws (LowerError) {
    match typ {
        case resolver::Type::Optional(resolver::Type::Slice { item, mutable }) => {
            // Optional slices use null pointer optimization.
            return try lowerSliceEq(self, item, mutable, a, b, offset);
        }
        case resolver::Type::Optional(inner) => {
            return try lowerOptionalEq(self, *inner, a, b, offset);
        }
        case resolver::Type::Slice { item, mutable } =>
            return try lowerSliceEq(self, item, mutable, a, b, offset),
        case resolver::Type::Array(arr) =>
            return try lowerArrayEq(self, arr, a, b, offset),
        case resolver::Type::Nominal(resolver::NominalType::Record(recInfo)) =>
            return try lowerRecordEq(self, recInfo, a, b, offset),
        case resolver::Type::Nominal(resolver::NominalType::Union(unionInfo)) =>
            return try lowerUnionEq(self, unionInfo, a, b, offset),
        else => {
            let recInfo = recordInfoFromType(typ) else {
                throw LowerError::ExpectedRecord;
            };
            return try lowerRecordEq(self, recInfo, a, b, offset);
        }
    }
}

/// Lower a record literal expression. Handles both plain records and union variant
/// record literals like `Union::Variant { field: value }`.
fn lowerRecordLit(self: *mut FnLowerer, node: *ast::Node, lit: ast::RecordLit) -> il::Val throws (LowerError) {
    let typ = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    match typ {
        case resolver::Type::Nominal(resolver::NominalType::Record(recInfo)) => {
            let dst = try emitReserve(self, typ);
            try lowerRecordFields(self, dst, &recInfo, lit.fields, 0);

            return il::Val::Reg(dst);
        }
        case resolver::Type::Nominal(resolver::NominalType::Union(_)) => {
            // TODO: This can be inlined once we have a real register allocator.
            return try lowerUnionRecordLit(self, typ, lit);
        }
        else => throw LowerError::UnexpectedType(&typ),
    }
}

/// Lower a union variant record literal like `Union::Variant { field: value }`.
fn lowerUnionRecordLit(self: *mut FnLowerer, typ: resolver::Type, lit: ast::RecordLit) -> il::Val throws (LowerError) {
    let typeName = lit.typeName else {
        throw LowerError::ExpectedVariant;
    };
    let sym = resolver::nodeData(self.low.resolver, typeName).sym else {
        throw LowerError::MissingSymbol(typeName);
    };
    let case resolver::SymbolData::Variant { type: payloadType, index, .. } = sym.data else {
        throw LowerError::ExpectedVariant;
    };
    let recInfo = recordInfoFromType(payloadType) else {
        throw LowerError::ExpectedRecord;
    };
    let unionInfo = unionInfoFromType(typ) else {
        throw LowerError::MissingMetadata;
    };
    let valOffset = unionInfo.valOffset as i32;
    let dst = try emitReserve(self, typ);

    emitStoreW8At(self, il::Val::Imm(index as i64), dst, TVAL_TAG_OFFSET);
    try lowerRecordFields(self, dst, &recInfo, lit.fields, valOffset);

    return il::Val::Reg(dst);
}

/// Lower fields of a record literal into a destination register.
/// The `offset` is added to each field's offset when storing.
fn lowerRecordFields(
    self: *mut FnLowerer,
    dst: il::Reg,
    recInfo: *resolver::RecordType,
    fields: *mut [*ast::Node],
    offset: i32
) throws (LowerError) {
    for fieldNode, i in fields {
        let case ast::NodeValue::RecordLitField(field) = fieldNode.value else {
            throw LowerError::UnexpectedNodeValue(fieldNode);
        };
        let mut fieldIdx: u32 = i;
        if recInfo.labeled {
            let idx = resolver::recordFieldIndexFor(self.low.resolver, fieldNode) else {
                throw LowerError::MissingMetadata;
            };
            fieldIdx = idx;
        }
        // Skip `undefined` fields, they need no initialization.
        // Emitting a blit from an uninitialised reserve produces a
        // phantom SSA source value that the backend cannot handle.
        if not isUndef(field.value) {
            let fieldTy = recInfo.fields[fieldIdx].fieldType;
            let fieldVal = try lowerExpr(self, field.value);
            try emitStore(self, dst, offset + recInfo.fields[fieldIdx].offset, fieldTy, fieldVal);
        }
    }
}

/// Lower an unlabeled record constructor call.
fn lowerRecordCtor(self: *mut FnLowerer, nominal: *resolver::NominalType, args: *mut [*ast::Node]) -> il::Val throws (LowerError) {
    let case resolver::NominalType::Record(recInfo) = *nominal else {
        throw LowerError::ExpectedRecord;
    };
    let typ = resolver::Type::Nominal(nominal);
    let dst = try emitReserve(self, typ);

    for argNode, i in args {
        // Skip `undefined` arguments.
        if not isUndef(argNode) {
            let fieldTy = recInfo.fields[i].fieldType;
            let argVal = try lowerExpr(self, argNode);
            try emitStore(self, dst, recInfo.fields[i].offset, fieldTy, argVal);
        }
    }
    return il::Val::Reg(dst);
}

/// Lower an array literal expression like `[1, 2, 3]`.
fn lowerArrayLit(self: *mut FnLowerer, node: *ast::Node, elements: *mut [*ast::Node]) -> il::Val
    throws (LowerError)
{
    let typ = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let case resolver::Type::Array(arrInfo) = typ else {
        throw LowerError::ExpectedArray;
    };
    let elemTy = *arrInfo.item;
    let elemLayout = resolver::getTypeLayout(elemTy);
    let dst = try emitReserve(self, typ);

    for elemNode, i in elements {
        let elemVal = try lowerExpr(self, elemNode);
        let offset = i * elemLayout.size;

        try emitStore(self, dst, offset as i32, elemTy, elemVal);
    }
    return il::Val::Reg(dst);
}

/// Lower an array repeat literal expression like `[42; 3]`.
/// Unrolls the initialization at compile time.
// TODO: Beyond a certain length, lower this to a loop.
fn lowerArrayRepeatLit(self: *mut FnLowerer, node: *ast::Node, repeat: ast::ArrayRepeatLit) -> il::Val
    throws (LowerError)
{
    let typ = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let case resolver::Type::Array(arrInfo) = typ else {
        throw LowerError::ExpectedArray;
    };
    let elemTy = *arrInfo.item;
    let length = arrInfo.length;
    let elemLayout = resolver::getTypeLayout(elemTy);
    let dst = try emitReserve(self, typ);

    // Evaluate the repeated item once.
    let repeatVal = try lowerExpr(self, repeat.item);

    // Unroll: store at each offset.
    for i in 0..length {
        let offset = i * elemLayout.size;
        try emitStore(self, dst, offset as i32, elemTy, repeatVal);
    }
    return il::Val::Reg(dst);
}

/// Lower a union constructor call like `Union::Variant(...)`.
fn lowerUnionCtor(self: *mut FnLowerer, node: *ast::Node, sym: *mut resolver::Symbol, call: ast::Call) -> il::Val
    throws (LowerError)
{
    let unionTy = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let case resolver::SymbolData::Variant { type: payloadType, index, .. } = sym.data else {
        throw LowerError::ExpectedVariant;
    };
    let unionInfo = unionInfoFromType(unionTy) else {
        throw LowerError::MissingMetadata;
    };
    let valOffset = unionInfo.valOffset as i32;
    let mut payloadVal: ?il::Val = nil;
    if payloadType != resolver::Type::Void {
        let case resolver::Type::Nominal(payloadNominal) = payloadType else {
            throw LowerError::MissingMetadata;
        };
        payloadVal = try lowerRecordCtor(self, payloadNominal, call.args);
    }
    return try buildTagged(self, resolver::getTypeLayout(unionTy), index as i64, payloadVal, payloadType, 1, valOffset);
}

/// Lower a field access into a pointer to the field.
fn lowerFieldRef(self: *mut FnLowerer, access: ast::Access) -> FieldRef throws (LowerError) {
    let parentTy = resolver::typeFor(self.low.resolver, access.parent) else {
        throw LowerError::MissingType(access.parent);
    };
    let subjectTy = resolver::autoDeref(parentTy);
    let fieldIdx = resolver::recordFieldIndexFor(self.low.resolver, access.child) else {
        throw LowerError::MissingMetadata;
    };
    let fieldInfo = resolver::getRecordField(subjectTy, fieldIdx) else {
        throw LowerError::FieldNotFound;
    };
    let baseVal = try lowerExpr(self, access.parent);
    let baseReg = emitValToReg(self, baseVal);

    return FieldRef {
        base: baseReg,
        offset: fieldInfo.offset,
        fieldType: fieldInfo.fieldType,
    };
}

/// Lower a field access expression.
fn lowerFieldAccess(self: *mut FnLowerer, access: ast::Access) -> il::Val throws (LowerError) {
    let fieldRef = try lowerFieldRef(self, access);
    return emitRead(self, fieldRef.base, fieldRef.offset, fieldRef.fieldType);
}

/// Lower a slice range expression into a slice header value.
fn lowerSliceRange(
    self: *mut FnLowerer,
    container: *ast::Node,
    range: ast::Range,
    sliceNode: *ast::Node
) -> il::Val throws (LowerError) {
    let info = resolver::sliceRangeInfoFor(self.low.resolver, sliceNode) else {
        throw LowerError::MissingMetadata;
    };
    let baseVal = try lowerExpr(self, container);
    let baseReg = emitValToReg(self, baseVal);

    // Extract data pointer and container length.
    let mut dataReg = baseReg;
    let mut containerLen: il::Val = undefined;
    if let cap = info.capacity { // Slice from array.
        containerLen = il::Val::Imm(cap as i64);
    } else { // Slice from slice.
        dataReg = loadSlicePtr(self, baseReg);
        containerLen = loadSliceLen(self, baseReg);
    }

    // Compute range bounds.
    let mut startVal: il::Val = il::Val::Imm(0);
    if let start = range.start {
        startVal = try lowerExpr(self, start);
    }
    let mut endVal = containerLen;
    if let end = range.end {
        endVal = try lowerExpr(self, end);
    }
    // If the start value is known to be zero, the slice length is just the end
    // value. Otherwise, we have to compute it.
    let mut sliceLen = endVal;

    // Only compute range offset and length if the start value is not
    // statically known to be zero.
    if startVal != il::Val::Imm(0) {
        // Offset the data pointer by the start value.
        dataReg = emitElem(
            self, resolver::getTypeLayout(*info.itemType).size, dataReg, startVal
        );
        // Compute the length as `end - start`.
        let lenReg = nextReg(self);
        emit(self, il::Instr::BinOp {
            op: il::BinOp::Sub,
            typ: il::Type::W32,
            dst: lenReg,
            a: endVal,
            b: startVal,
        });
        sliceLen = il::Val::Reg(lenReg);
    }
    return try buildSliceValue(
        self, info.itemType, info.mutable, il::Val::Reg(dataReg), sliceLen, sliceLen
    );
}

/// Lower an address-of (`&x`) expression.
fn lowerAddressOf(self: *mut FnLowerer, node: *ast::Node, addr: ast::AddressOf) -> il::Val throws (LowerError) {
    // Handle subscript: `&ary[i]` or `&ary[start..end]`.
    if let case ast::NodeValue::Subscript { container, index } = addr.target.value {
        if let case ast::NodeValue::Range(range) = index.value {
            return try lowerSliceRange(self, container, range, node);
        }
        let result = try lowerElemPtr(self, container, index);

        return il::Val::Reg(result.elemReg);
    }
    // Handle field address: `&x.field`.
    if let case ast::NodeValue::FieldAccess(access) = addr.target.value {
        let fieldRef = try lowerFieldRef(self, access);
        let ptr = emitPtrOffset(self, fieldRef.base, fieldRef.offset);

        return il::Val::Reg(ptr);
    }
    // Handle variable address: `&x`
    if let case ast::NodeValue::Ident(_) = addr.target.value {
        if let v = lookupLocalVar(self, addr.target) {
            let val = try useVar(self, v);
            let typ = resolver::typeFor(self.low.resolver, addr.target) else {
                throw LowerError::MissingType(addr.target);
            };
            // For aggregates, the value is already a pointer.
            if isAggregateType(typ) {
                return val;
            }
            // For scalars, if we've already materialized a stack slot for this
            // variable, the SSA value is that slot pointer.
            if self.vars[v.id].addressTaken {
                // Already address-taken; return existing stack pointer.
                return val;
            }
            // Materialize a stack slot using the declaration's resolved
            // layout so `align(N)` on locals is honored.
            let layout = resolver::getLayout(self.low.resolver, addr.target, typ);
            let slot = emitReserveLayout(self, layout);
            try emitStore(self, slot, 0, typ, val);
            let stackVal = il::Val::Reg(slot);

            self.vars[v.id].addressTaken = true;
            defVar(self, v, stackVal);

            return stackVal;
        }
        // Fall back to symbol lookup for constants/statics.
        if let sym = resolver::nodeData(self.low.resolver, addr.target).sym {
            return il::Val::Reg(emitDataAddr(self, sym));
        } else {
            throw LowerError::MissingSymbol(node);
        }
    }
    // Handle dereference address: `&(*ptr) = ptr`.
    if let case ast::NodeValue::Deref(target) = addr.target.value {
        return try lowerExpr(self, target);
    }
    // Handle slice literal: `&[1, 2, 3]`.
    if let case ast::NodeValue::ArrayLit(elements) = addr.target.value {
        return try lowerSliceLiteral(self, node, addr.target, elements);
    }
    throw LowerError::UnexpectedNodeValue(addr.target);
}

/// Lower a slice literal like `&[1, 2, 3]`.
/// If all elements are constants, creates static data. Otherwise, allocates
/// stack space and stores elements at runtime.
fn lowerSliceLiteral(
    self: *mut FnLowerer,
    sliceNode: *ast::Node,
    arrayNode: *ast::Node,
    elements: *mut [*ast::Node]
) -> il::Val throws (LowerError) {
    // Get the slice type from the address-of expression.
    let sliceTy = resolver::typeFor(self.low.resolver, sliceNode) else {
        throw LowerError::MissingType(sliceNode);
    };
    let case resolver::Type::Slice { item, mutable } = sliceTy else {
        throw LowerError::UnexpectedType(&sliceTy);
    };
    if elements.len == 0 { // Empty slices don't need to be stored as data.
        return try buildSliceValue(self, item, mutable, il::Val::Imm(0), il::Val::Imm(0), il::Val::Imm(0));
    }
    if resolver::isConstExpr(self.low.resolver, arrayNode) {
        let elemLayout = resolver::getTypeLayout(*item);
        return try lowerConstSliceLiteral(self, item, mutable, elements, elemLayout);
    } else {
        return try lowerRuntimeSliceLiteral(self, item, mutable, elements);
    }
}

/// Lower a slice literal with all constant elements to static data.
fn lowerConstSliceLiteral(
    self: *mut FnLowerer,
    elemTy: *resolver::Type,
    mutable: bool,
    elements: *mut [*ast::Node],
    elemLayout: resolver::Layout
) -> il::Val throws (LowerError) {
    // Build data values for all elements using standard data lowering.
    let mut b = dataBuilder(self.low.allocator);
    for elem in elements {
        try lowerConstDataInto(self.low, elem, *elemTy, elemLayout.size, self.fnName, &mut b);
    }
    let result = dataBuilderFinish(&b);
    let readOnly = not mutable;

    return try lowerConstDataAsSlice(self, result.values, elemLayout.alignment, readOnly, elemTy, mutable, elements.len);
}

/// Lower a slice literal with non-constant elements.
fn lowerRuntimeSliceLiteral(
    self: *mut FnLowerer,
    elemTy: *resolver::Type,
    mutable: bool,
    elements: *mut [*ast::Node]
) -> il::Val throws (LowerError) {
    let elemLayout = resolver::getTypeLayout(*elemTy);
    let arraySize = elements.len * elemLayout.size;
    let arrayReg = nextReg(self);

    // Reserve stack space for slice elements.
    emit(self, il::Instr::Reserve {
        dst: arrayReg,
        size: il::Val::Imm(arraySize as i64),
        alignment: elemLayout.alignment
    });
    // Store each element.
    for elemNode, i in elements {
        let elemVal = try lowerExpr(self, elemNode);
        let offset = i * elemLayout.size;

        try emitStore(self, arrayReg, offset as i32, *elemTy, elemVal);
    }
    let lenVal = il::Val::Imm(elements.len as i64);

    return try buildSliceValue(self, elemTy, mutable, il::Val::Reg(arrayReg), lenVal, lenVal);
}

/// Lower the common element pointer computation for subscript operations.
/// Handles both arrays and slices by resolving the container type, extracting
/// the data pointer (for slices), and emitting an [`il::Instr::Elem`] to compute
/// the element address.
fn lowerElemPtr(
    self: *mut FnLowerer, container: *ast::Node, index: *ast::Node
) -> ElemPtrResult throws (LowerError) {
    let containerTy = resolver::typeFor(self.low.resolver, container) else {
        throw LowerError::MissingType(container);
    };
    let subjectTy = resolver::autoDeref(containerTy);
    let indexVal = try lowerExpr(self, index);
    let baseVal = try lowerExpr(self, container);
    let baseReg = emitValToReg(self, baseVal);

    let mut dataReg = baseReg;
    let mut elemType: resolver::Type = undefined;

    match subjectTy {
        case resolver::Type::Slice { item, .. } => {
            elemType = *item;
            let sliceLen = loadSliceLen(self, baseReg);
            // Runtime safety check: index must be strictly less than slice length.
            try emitTrapUnlessCmp(self, il::CmpOp::Ult, il::Type::W32, indexVal, sliceLen);

            dataReg = loadSlicePtr(self, baseReg);
        }
        case resolver::Type::Array(arrInfo) => {
            elemType = *arrInfo.item;
            // Runtime safety check: index must be strictly less than array length.
            // Skip when the index is a compile-time constant, since we check
            // that in the resolver.
            if not resolver::isConstExpr(self.low.resolver, index) {
                let arrLen = il::Val::Imm(arrInfo.length as i64);
                try emitTrapUnlessCmp(self, il::CmpOp::Ult, il::Type::W32, indexVal, arrLen);
            }
        }
        else => throw LowerError::ExpectedSliceOrArray,
    }
    let elemLayout = resolver::getTypeLayout(elemType);
    let elemReg = emitElem(self, elemLayout.size, dataReg, indexVal);

    return ElemPtrResult { elemReg, elemType };
}

/// Lower a dereference expression.
fn lowerDeref(self: *mut FnLowerer, node: *ast::Node, target: *ast::Node) -> il::Val throws (LowerError) {
    let type = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let ptrVal = try lowerExpr(self, target);
    let ptrReg = emitValToReg(self, ptrVal);

    return emitRead(self, ptrReg, 0, type);
}

/// Lower a subscript expression.
fn lowerSubscript(self: *mut FnLowerer, node: *ast::Node, container: *ast::Node, index: *ast::Node) -> il::Val
    throws (LowerError)
{
    if let case ast::NodeValue::Range(_) = index.value {
        panic "lowerSubscript: range subscript must use address-of (&)";
    }
    let result = try lowerElemPtr(self, container, index);

    return emitRead(self, result.elemReg, 0, result.elemType);
}

/// Lower a let binding.
fn lowerLet(self: *mut FnLowerer, node: *ast::Node, l: ast::Let) throws (LowerError) {
    // Evaluate value.
    let val = try lowerExpr(self, l.value);
    // Handle placeholder pattern: `let _ = expr;`
    if let case ast::NodeValue::Placeholder = l.ident.value {
        return;
    }
    let case ast::NodeValue::Ident(name) = l.ident.value else {
        throw LowerError::ExpectedIdentifier;
    };
    let typ = resolver::typeFor(self.low.resolver, l.value) else {
        throw LowerError::MissingType(l.value);
    };
    let ilType = ilType(self.low, typ);
    let mut varVal = val;

    // Aggregates with persistent storage need a local copy to avoid aliasing.
    // Temporaries such as literals or call results can be adopted directly.
    // This is because aggregates are represented as memory addresses
    // internally, even though they have value semantics, so without an explicit
    // copy, only the address is is written. Function calls on the other hand
    // reserve their own local stack space, so copying would be redundant.
    if isAggregateType(typ) and hasStorage(l.value) {
        varVal = try emitStackVal(self, typ, val);
    }

    // If the resolver determined that this variable's address is taken
    // anywhere in the function, allocate a stack slot immediately so the
    // SSA value is always a pointer. This avoids mixing integer and pointer
    // values in loop phis when `&var` or `&mut var` appears inside a loop.
    if not isAggregateType(typ) {
        if let sym = resolver::nodeData(self.low.resolver, node).sym {
            if let case resolver::SymbolData::Value { addressTaken, .. } = sym.data; addressTaken {
                let layout = resolver::getLayout(self.low.resolver, node, typ);
                let slot = emitReserveLayout(self, layout);
                try emitStore(self, slot, 0, typ, varVal);

                let v = newVar(self, name, ilType, l.mutable, il::Val::Reg(slot));
                self.vars[v.id].addressTaken = true;

                return;
            }
        }
    }
    let _ = newVar(self, name, ilType, l.mutable, varVal);
}

/// Lower an if statement: `if <cond> { <then> } else { <else> }`.
///
/// With else branch:
///
///     @entry -> (true)  @then ---> @merge <--.
///         |                                   )
///         `---> (false) @else ---------------'
///
/// Without else branch:
///
///     @entry -> (true)  @then ---> @end <--.
///         |                                 )
///         `---- (false) -------------------'
///
fn lowerIf(self: *mut FnLowerer, i: ast::If) throws (LowerError) {
    let thenBlock = try createBlock(self, "then");

    if let elseNode = i.elseBranch { // If-else case.
        let elseBlock = try createBlock(self, "else");
        try emitCondBranch(self, i.condition, thenBlock, elseBlock);

        // Both @then and @else have exactly one predecessor (@entry), so we can
        // seal them immediately.
        try sealBlock(self, thenBlock);
        try sealBlock(self, elseBlock);

        // The merge block is created lazily by [`emitMergeIfUnterminated`]. We
        // only need it if at least one branch doesn't diverge (i.e., needs to
        // continue execution after the `if`). If both branches diverge (eg.
        // both `return`), no merge block is created and control flow
        // doesn't continue past the `if` statement.
        let mut mergeBlock: ?BlockId = nil;

        // Lower the @then block: switch to it, emit its code, then jump to
        // merge if the block doesn't diverge.
        switchToBlock(self, thenBlock);
        try lowerBlock(self, i.thenBranch);
        try emitMergeIfUnterminated(self, &mut mergeBlock);

        // Lower the @else block similarly.
        switchToBlock(self, elseBlock);
        try lowerBlock(self, elseNode);
        try emitMergeIfUnterminated(self, &mut mergeBlock);

        // If a merge block was created (at least one branch flows into it),
        // switch to it for subsequent code. The merge block's predecessors
        // are the branches that jumped to it, so we seal it now.
        // If the merge block is `nil`, both branches diverged and there's no
        // continuation point.
        if let blk = mergeBlock {
            try switchToAndSeal(self, blk);
        }
    } else { // If without `else`.
        // The false branch goes directly to @end, which also serves as the
        // merge point after @then completes.
        let endBlock = try createBlock(self, "merge");

        try emitCondBranch(self, i.condition, thenBlock, endBlock);

        // @then has one predecessor (@entry), seal it immediately.
        // @end is not sealed yet because @then might also jump to it.
        try sealBlock(self, thenBlock);

        // Lower the @then block, then jump to @end and seal it.
        // Unlike the if-else case, @end is always created because there is no
        // else branch that can diverge.
        switchToBlock(self, thenBlock);
        try lowerBlock(self, i.thenBranch);
        try emitJmpAndSeal(self, endBlock);

        // Continue execution at @end.
        try switchToAndSeal(self, endBlock);
    }
}

/// Lower an assignment statement.
fn lowerAssign(self: *mut FnLowerer, a: ast::Assign) throws (LowerError) {
    // Evaluate assignment value.
    let rhs = try lowerExpr(self, a.right);

    match a.left.value {
        case ast::NodeValue::Ident(_) => {
            // First try local variable lookup.
            if let v = lookupLocalVar(self, a.left) {
                if not getVar(self, v).mutable {
                    throw LowerError::ImmutableAssignment;
                }
                let leftTy = resolver::typeFor(self.low.resolver, a.left) else {
                    throw LowerError::MissingType(a.left);
                };
                if isAggregateType(leftTy) or getVar(self, v).addressTaken {
                    // Aggregates and address-taken scalars are represented as
                    // pointers to stack memory. Store through the pointer.
                    let val = try useVar(self, v);
                    let dst = emitValToReg(self, val);

                    try emitStore(self, dst, 0, leftTy, rhs);
                } else {
                    // Scalars are tracked directly in SSA. Each assignment
                    // records a new SSA value.
                    defVar(self, v, rhs);
                }
            } else {
                // Fall back to static variable assignment.
                try lowerStaticAssign(self, a.left, rhs);
            }
        }
        case ast::NodeValue::FieldAccess(access) => {
            let fieldRef = try lowerFieldRef(self, access);
            try emitStore(self, fieldRef.base, fieldRef.offset, fieldRef.fieldType, rhs);
        }
        case ast::NodeValue::Deref(target) => {
            // Assignment through pointer dereference: `*ptr = value`.
            let ptrVal = try lowerExpr(self, target);
            let ptrReg = emitValToReg(self, ptrVal);
            let targetTy = resolver::typeFor(self.low.resolver, a.left) else {
                throw LowerError::MissingType(a.left);
            };
            try emitStore(self, ptrReg, 0, targetTy, rhs);
        }
        case ast::NodeValue::Subscript { container, index } => {
            // Assignment to array/slice element: `arr[i] = value`.
            let result = try lowerElemPtr(self, container, index);
            try emitStore(self, result.elemReg, 0, result.elemType, rhs);
        }
        else => throw LowerError::UnexpectedNodeValue(a.left),
    }
}

///////////////////
// Loop Lowering //
///////////////////

// All loop forms are lowered to a common structure:
//
// - Loop header block: evaluates condition if any, branches to body or exit.
// - Body block: executes loop body, may contain break/continue.
// - Step block (for `for` loops): increments counter, jumps back to header.
// - Exit block: target for break statements and normal loop exit.
//
// The loop stack tracks break/continue targets for nested loops.

/// Lower an infinite loop: `loop { <body> }`.
///
///   @entry -> @loop -> @loop
///               |
///               `----> @end
///
fn lowerLoop(self: *mut FnLowerer, body: *ast::Node) throws (LowerError) {
    let loopBlock = try createBlock(self, "loop");
    let endBlock = try createBlock(self, "merge");

    // Enter the loop with the given break and continue targets.
    // `break` jumps to `endBlock`,
    // `continue` jumps to `loopBlock`.
    enterLoop(self, endBlock, loopBlock);
    // Switch to and jump to the loop block, then lower all loop body statements into it.
    try switchAndJumpTo(self, loopBlock);
    try lowerBlock(self, body);

    // If the loop body doesn't diverge, jump back to the start.
    // This creates the infinite loop.
    // All predecessors are known, we can seal the loop block.
    try emitJmpAndSeal(self, loopBlock);
    // Exit the loop.
    exitLoop(self);

    // Only seal end block if it's actually reachable (ie. has predecessors).
    // If the loop has no breaks and only exits via return, the end block
    // remains unreachable and isn't added to the CFG.
    if getBlock(self, endBlock).preds.len > 0 {
        try switchToAndSeal(self, endBlock);
    }
}

/// Lower a while loop: `while <cond> { <body> }`.
///
///   @entry -> @loop -> (true)  @body -> @loop
///               |
///               `----> (false) @end
///
fn lowerWhile(self: *mut FnLowerer, w: ast::While) throws (LowerError) {
    let whileBlock = try createBlock(self, "while");
    let bodyBlock = try createBlock(self, "body");
    let endBlock = try createBlock(self, "merge");

    enterLoop(self, endBlock, whileBlock);

    // Loop condition.
    try switchAndJumpTo(self, whileBlock);

    // Based on the condition, either jump to the body,
    // or to the end of the loop.
    try emitCondBranch(self, w.condition, bodyBlock, endBlock);

    // Lower loop body and jump back to loop condition check.
    try switchToAndSeal(self, bodyBlock);
    try lowerBlock(self, w.body);
    try emitJmpAndSeal(self, whileBlock);

    try switchToAndSeal(self, endBlock);
    exitLoop(self);
}

/// Emit an increment of a variable by `1`.
fn emitIncrement(self: *mut FnLowerer, v: Var, typ: il::Type) throws (LowerError) {
    let cur = try useVar(self, v);
    let next = nextReg(self);

    emit(self, il::Instr::BinOp { op: il::BinOp::Add, typ, dst: next, a: cur, b: il::Val::Imm(1) });
    defVar(self, v, il::Val::Reg(next));
}

/// Common for-loop lowering for both range and collection iterators.
///
/// The step block is created lazily (after the loop body) so that it gets
/// a block index higher than all body blocks. This ensures the register
/// allocator processes definitions before uses in forward block order,
/// avoiding stale assignments when a value defined deep in the body flows
/// through the step block as a block argument.
fn lowerForLoop(self: *mut FnLowerer, iter: *ForIter, body: *ast::Node) throws (LowerError) {
    let loopBlock = try createBlock(self, "loop");
    let bodyBlock = try createBlock(self, "body");
    let endBlock = try createBlock(self, "merge");

    enterLoop(self, endBlock, nil);
    try switchAndJumpTo(self, loopBlock);

    // Emit condition check.
    match *iter {
        case ForIter::Range { valVar, endVal, valType, unsigned, .. } => {
            let curVal = try useVar(self, valVar);
            let cmp = il::CmpOp::Ult if unsigned else il::CmpOp::Slt;
            try emitBrCmp(self, cmp, valType, curVal, endVal, bodyBlock, endBlock);
        }
        case ForIter::Collection { idxVar, lengthVal, .. } => {
            let curIdx = try useVar(self, idxVar);
            try emitBrCmp(self, il::CmpOp::Slt, il::Type::W32, curIdx, lengthVal, bodyBlock, endBlock);
        }
    }
    // Switch to loop body.
    try switchToAndSeal(self, bodyBlock);

    // Emit element binding, only for collections.
    // Reads the element at the current index.
    if let case ForIter::Collection { valVar, idxVar, dataReg, elemType, .. } = *iter {
        if let v = valVar {
            let curIdx = try useVar(self, idxVar);
            let elemReg = emitElem(self, resolver::getTypeLayout(*elemType).size, dataReg, curIdx);
            let val = emitRead(self, elemReg, 0, *elemType);

            defVar(self, v, val);
        }
    }
    // Lower the loop body and jump to the @step block.
    try lowerBlock(self, body);
    let stepBlock = try getOrCreateContinueBlock(self);
    try emitJmpAndSeal(self, stepBlock);

    // Switch to step block and emit step/increment.
    switchToBlock(self, stepBlock);

    match *iter {
        case ForIter::Range { valVar, valType, indexVar, .. } => {
            try emitIncrement(self, valVar, valType);

            if let idxVar = indexVar {
                try emitIncrement(self, idxVar, il::Type::W32);
            }
        }
        case ForIter::Collection { idxVar, .. } => {
            try emitIncrement(self, idxVar, il::Type::W32);
        }
    }
    try emitJmp(self, loopBlock);

    exitLoop(self);
    try sealBlock(self, loopBlock);
    try sealBlock(self, endBlock);
    switchToBlock(self, endBlock);
}

/// Lower a `for` loop over a range, array, or slice.
fn lowerFor(self: *mut FnLowerer, node: *ast::Node, f: ast::For) throws (LowerError) {
    let savedVarsLen = enterVarScope(self);
    let info = resolver::forLoopInfoFor(self.low.resolver, node) else {
        throw LowerError::MissingMetadata;
    };
    match info {
        case resolver::ForLoopInfo::Range { valType, range, bindingName, indexName } => {
            let endExpr = range.end else {
                throw LowerError::MissingMetadata;
            };
            let mut startVal = il::Val::Imm(0);
            if let start = range.start {
                startVal = try lowerExpr(self, start);
            }
            let endVal = try lowerExpr(self, endExpr);
            let iterType = ilType(self.low, *valType);
            let valVar = newVar(self, bindingName, iterType, false, startVal);

            let mut indexVar: ?Var = nil;
            if indexName != nil { // Optional index always starts at zero.
                indexVar = newVar(self, indexName, il::Type::W32, false, il::Val::Imm(0));
            }
            let iter = ForIter::Range {
                valVar, indexVar, endVal, valType: iterType,
                unsigned: isUnsignedType(*valType),
            };

            try lowerForLoop(self, &iter, f.body);
        }
        case resolver::ForLoopInfo::Collection { elemType, length, bindingName, indexName } => {
            let containerVal = try lowerExpr(self, f.iterable);
            let containerReg = emitValToReg(self, containerVal);

            let mut dataReg = containerReg;
            let mut lengthVal: il::Val = undefined;
            if let len = length { // Array (length is known).
                lengthVal = il::Val::Imm(len as i64);
            } else { // Slice (length must be loaded).
                lengthVal = loadSliceLen(self, containerReg);
                dataReg = loadSlicePtr(self, containerReg);
            }
            // Declare index value binidng.
            let idxVar = newVar(self, indexName, il::Type::W32, false, il::Val::Imm(0));

            // Declare element value binding.
            let mut valVar: ?Var = nil;
            if bindingName != nil {
                valVar = newVar(
                    self,
                    bindingName,
                    ilType(self.low, *elemType),
                    false,
                    il::Val::Undef
                );
            }
            let iter = ForIter::Collection { valVar, idxVar, dataReg, lengthVal, elemType };

            try lowerForLoop(self, &iter, f.body);
        }
    }
    exitVarScope(self, savedVarsLen);
}

/// Lower a break statement.
fn lowerBreak(self: *mut FnLowerer) throws (LowerError) {
    let ctx = currentLoop(self) else {
        throw LowerError::OutsideOfLoop;
    };
    try emitJmp(self, ctx.breakTarget);
}

/// Lower a continue statement.
fn lowerContinue(self: *mut FnLowerer) throws (LowerError) {
    let block = try getOrCreateContinueBlock(self);
    try emitJmp(self, block);
}

/// Emit a return, blitting into the caller's return buffer if needed.
///
/// When the function has a return buffer parameter, the value is blitted
/// into the buffer and the buffer pointer is returned. Otherwise, the value is
/// returned directly.
fn emitRetVal(self: *mut FnLowerer, val: il::Val) throws (LowerError) {
    if let retReg = self.returnReg {
        let src = emitValToReg(self, val);
        let size = retBufSize(self);

        emit(self, il::Instr::Blit { dst: retReg, src, size: il::Val::Imm(size as i64) });
        emit(self, il::Instr::Ret { val: il::Val::Reg(retReg) });
    } else if isSmallAggregate(*self.fnType.returnType) {
        let src = emitValToReg(self, val);
        let dst = nextReg(self);

        emit(self, il::Instr::Load { typ: il::Type::W64, dst, src, offset: 0 });
        emit(self, il::Instr::Ret { val: il::Val::Reg(dst) });
    } else {
        emit(self, il::Instr::Ret { val });
    }
}

/// Compute the size of the return buffer for the current function.
fn retBufSize(self: *mut FnLowerer) -> u32 {
    if self.fnType.throwList.len > 0 {
        let successType = *self.fnType.returnType;

        return resolver::getResultLayout(successType, self.fnType.throwList).size;
    }
    return resolver::getTypeLayout(*self.fnType.returnType).size;
}

/// Lower a return statement.
fn lowerReturnStmt(self: *mut FnLowerer, node: *ast::Node, value: ?*ast::Node) throws (LowerError) {
    let mut val = il::Val::Undef;
    if let expr = value {
        val = try lowerExpr(self, expr);
    }
    val = try applyCoercion(self, node, val);
    try emitRetVal(self, val);
}

/// Lower a throw statement.
fn lowerThrowStmt(self: *mut FnLowerer, expr: *ast::Node) throws (LowerError) {
    assert self.fnType.throwList.len > 0;

    let errType = resolver::typeFor(self.low.resolver, expr) else {
        throw LowerError::MissingType(expr);
    };
    let tag = getOrAssignErrorTag(self.low, errType) as i64;
    let errVal = try lowerExpr(self, expr);
    let resultVal = try buildResult(self, tag, errVal, errType);

    try emitRetVal(self, resultVal);
}

/// Ensure a value is in a register (eg. for branch conditions).
fn emitValToReg(self: *mut FnLowerer, val: il::Val) -> il::Reg {
    match val {
        case il::Val::Reg(r) => return r,
        case il::Val::Imm(_), il::Val::DataSym(_), il::Val::FnAddr(_) => {
            let dst = nextReg(self);
            emit(self, il::Instr::Copy { dst, val });
            return dst;
        }
        case il::Val::Undef => {
            // TODO: We shouldn't hit this case, right? A register shouldn't be needed
            // if the value is undefined.
            return nextReg(self);
        }
    }
}

/// Lower a logical `and`/`or` with short-circuit evaluation.
///
/// Short-circuit evaluation skips evaluating the right operand when the left
/// operand already determines the result:
///
/// - In `a and b`, if `a` is false, result is false without evaluating `b`.
/// - In `a or b`, if `a` is true, result is true without evaluating `b`.
///
/// This matters when `b` has side effects, or is expensive to evaluate.
///
/// Example: `a and b`
///
///     @entry
///       br %a @then @else;
///     @then
///       // Evaluate b into %b
///       // ...
///       jmp @end(%b);
///     @else
///       jmp @end(0);                  // Skip evaluating b, result is false
///     @end(w8 %result)
///       ret %result;
///
/// Example: `a or b`
///
///     @entry
///       br %a @then @else;
///     @then
///       jmp @end(1);                  // Skip evaluating b, result is true
///     @else
///       // Evaluate b into %b
///       // ...
///       jmp @end(%b);
///     @end(w8 %result)
///       ret %result;
///
fn lowerLogicalOp(
    self: *mut FnLowerer,
    binop: ast::BinOp,
    thenLabel: *[u8],
    elseLabel: *[u8],
    mergeLabel: *[u8],
    op: LogicalOp
) -> il::Val throws (LowerError) {
    let thenBlock = try createBlock(self, thenLabel);
    let elseBlock = try createBlock(self, elseLabel);

    let resultReg = nextReg(self);
    let mergeBlock = try createBlockWithParam(
        self, mergeLabel, il::Param { value: resultReg, type: il::Type::W8 }
    );
    // Evaluate left operand and branch.
    try emitCondBranch(self, binop.left, thenBlock, elseBlock);

    // Block that skips evaluating `b`.
    let mut shortCircuitBlock: BlockId = undefined;
    // Block that evaluates `b`.
    let mut evalBlock: BlockId = undefined;
    // Result when short-circuiting (`0` or `1`).
    let mut shortCircuitVal: i64 = undefined;

    match op {
        case LogicalOp::And => {
            shortCircuitBlock = elseBlock;
            evalBlock = thenBlock;
            shortCircuitVal = 0;
        }
        case LogicalOp::Or => {
            shortCircuitBlock = thenBlock;
            evalBlock = elseBlock;
            shortCircuitVal = 1;
        }
    }
    // Emit short-circuit branch: jump to merge with constant result.
    try switchToAndSeal(self, shortCircuitBlock);
    try emitJmpWithArg(self, mergeBlock, il::Val::Imm(shortCircuitVal));

    // Emit evaluation branch: evaluate right operand and jump to merge.
    try switchToAndSeal(self, evalBlock);
    try emitJmpWithArg(self, mergeBlock, try lowerExpr(self, binop.right));

    try switchToAndSeal(self, mergeBlock);
    return il::Val::Reg(resultReg);
}

/// Lower a conditional expression (`thenExpr if condition else elseExpr`).
fn lowerCondExpr(self: *mut FnLowerer, node: *ast::Node, cond: ast::CondExpr) -> il::Val
    throws (LowerError)
{
    let typ = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let thenBlock = try createBlock(self, "cond#then");
    let elseBlock = try createBlock(self, "cond#else");

    if isAggregateType(typ) {
        let dst = try emitReserve(self, typ);
        let layout = resolver::getTypeLayout(typ);
        try emitCondBranch(self, cond.condition, thenBlock, elseBlock);

        let mergeBlock = try createBlock(self, "cond#merge");
        try switchToAndSeal(self, thenBlock);

        let thenVal = emitValToReg(self, try lowerExpr(self, cond.thenExpr));
        emit(self, il::Instr::Blit { dst, src: thenVal, size: il::Val::Imm(layout.size as i64) });

        try emitJmp(self, mergeBlock);
        try switchToAndSeal(self, elseBlock);

        let elseVal = emitValToReg(self, try lowerExpr(self, cond.elseExpr));
        emit(self, il::Instr::Blit { dst, src: elseVal, size: il::Val::Imm(layout.size as i64) });

        try emitJmp(self, mergeBlock);
        try switchToAndSeal(self, mergeBlock);

        return il::Val::Reg(dst);
    } else {
        try emitCondBranch(self, cond.condition, thenBlock, elseBlock);

        let resultType = ilType(self.low, typ);
        let resultReg = nextReg(self);
        let mergeBlock = try createBlockWithParam(
            self, "cond#merge", il::Param { value: resultReg, type: resultType }
        );
        try switchToAndSeal(self, thenBlock);
        try emitJmpWithArg(self, mergeBlock, try lowerExpr(self, cond.thenExpr));
        try switchToAndSeal(self, elseBlock);
        try emitJmpWithArg(self, mergeBlock, try lowerExpr(self, cond.elseExpr));
        try switchToAndSeal(self, mergeBlock);

        return il::Val::Reg(resultReg);
    }
}

/// Convert a binary operator to a comparison op, if applicable.
/// For `Gt`, caller must swap operands: `a > b = b < a`.
/// For `Gte`/`Lte`, caller must swap branch labels: `a >= b = !(a < b)`.
/// For `Lte`, caller must also swap operands: `a <= b = !(b < a)`.
fn cmpOpFrom(op: ast::BinaryOp, unsigned: bool) -> ?il::CmpOp {
    match op {
        case ast::BinaryOp::Eq => return il::CmpOp::Eq,
        case ast::BinaryOp::Ne => return il::CmpOp::Ne,
        case ast::BinaryOp::Lt, ast::BinaryOp::Gt,
             ast::BinaryOp::Gte, ast::BinaryOp::Lte =>
            return il::CmpOp::Ult if unsigned else il::CmpOp::Slt,
        else => return nil,
    }
}

/// Lower a binary operation.
fn lowerBinOp(self: *mut FnLowerer, node: *ast::Node, binop: ast::BinOp) -> il::Val throws (LowerError) {
    // Short-circuit logical operators don't evaluate both operands eagerly.
    if binop.op == ast::BinaryOp::And {
        return try lowerLogicalOp(self, binop, "and#then", "and#else", "and#end", LogicalOp::And);
    } else if binop.op == ast::BinaryOp::Or {
        return try lowerLogicalOp(self, binop, "or#then", "or#else", "or#end", LogicalOp::Or);
    }

    // Handle comparison with a `nil` literal: just check the tag/pointer instead of
    // building a `nil` aggregate and doing full comparison.
    if binop.op == ast::BinaryOp::Eq or binop.op == ast::BinaryOp::Ne {
        let isEq = binop.op == ast::BinaryOp::Eq;
        let leftIsNil = binop.left.value == ast::NodeValue::Nil;
        let rightIsNil = binop.right.value == ast::NodeValue::Nil;

        if leftIsNil {
            return try lowerNilCheck(self, binop.right, isEq);
        } else if rightIsNil {
            return try lowerNilCheck(self, binop.left, isEq);
        }
    }
    // Lower operands.
    let a = try lowerExpr(self, binop.left);
    let b = try lowerExpr(self, binop.right);
    let isComparison = cmpOpFrom(binop.op, false) != nil;

    // The result type for comparisons is always `bool`, while for arithmetic
    // operations, it's the operand type. We set this appropriately here.
    let nodeTy = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let mut resultTy = nodeTy;

    if isComparison {
        let leftTy = try effectiveType(self, binop.left);
        let rightTy = try effectiveType(self, binop.right);
        // Optimize: comparing with a void variant just needs tag comparison.
        if let idx = voidVariantIndex(self.low.resolver, binop.left) {
            return try emitTagCmp(self, binop.op, b, idx, rightTy);
        } else if let idx = voidVariantIndex(self.low.resolver, binop.right) {
            return try emitTagCmp(self, binop.op, a, idx, leftTy);
        }
        // Aggregate types require element-wise comparison.
        // When comparing `?T` with `T`, wrap the scalar side.
        if isAggregateType(leftTy) {
            let mut rhs = b;
            if not isAggregateType(rightTy) {
                rhs = try wrapInOptional(self, rhs, leftTy);
            }
            return try emitAggregateEqOp(self, binop.op, leftTy, a, rhs);
        }
        if isAggregateType(rightTy) {
            let lhs = try wrapInOptional(self, a, rightTy);
            return try emitAggregateEqOp(self, binop.op, rightTy, lhs, b);
        }
        resultTy = leftTy;
    }
    return emitScalarBinOp(self, binop.op, ilType(self.low, resultTy), a, b, isUnsignedType(resultTy));
}

/// Emit an aggregate equality or inequality comparison.
fn emitAggregateEqOp(
    self: *mut FnLowerer,
    op: ast::BinaryOp,
    typ: resolver::Type,
    a: il::Val,
    b: il::Val
) -> il::Val throws (LowerError) {
    let regA = emitValToReg(self, a);
    let regB = emitValToReg(self, b);
    let result = try lowerAggregateEq(self, typ, regA, regB, 0);

    if op == ast::BinaryOp::Ne {
        return emitTypedBinOp(self, il::BinOp::Eq, il::Type::W32, result, il::Val::Imm(0));
    }
    return result;
}

/// Emit a scalar binary operation instruction.
fn emitScalarBinOp(
    self: *mut FnLowerer,
    op: ast::BinaryOp,
    typ: il::Type,
    a: il::Val,
    b: il::Val,
    unsigned: bool
) -> il::Val {
    let dst = nextReg(self);
    let mut needsExt: bool = false;
    match op {
        case ast::BinaryOp::Add => {
            emit(self, il::Instr::BinOp { op: il::BinOp::Add, typ, dst, a, b });
            needsExt = true;
        }
        case ast::BinaryOp::Sub => {
            emit(self, il::Instr::BinOp { op: il::BinOp::Sub, typ, dst, a, b });
            needsExt = true;
        }
        case ast::BinaryOp::Mul => {
            emit(self, il::Instr::BinOp { op: il::BinOp::Mul, typ, dst, a, b });
            needsExt = true;
        }
        case ast::BinaryOp::Div => {
            let op = il::BinOp::Udiv if unsigned else il::BinOp::Sdiv;
            emit(self, il::Instr::BinOp { op, typ, dst, a, b });
            needsExt = true;
        }
        case ast::BinaryOp::Mod => {
            let op = il::BinOp::Urem if unsigned else il::BinOp::Srem;
            emit(self, il::Instr::BinOp { op, typ, dst, a, b });
            needsExt = true;
        }
        case ast::BinaryOp::BitAnd => emit(self, il::Instr::BinOp { op: il::BinOp::And, typ, dst, a, b }),
        case ast::BinaryOp::BitOr => emit(self, il::Instr::BinOp { op: il::BinOp::Or, typ, dst, a, b }),
        case ast::BinaryOp::BitXor => emit(self, il::Instr::BinOp { op: il::BinOp::Xor, typ, dst, a, b }),
        case ast::BinaryOp::Shl => {
            emit(self, il::Instr::BinOp { op: il::BinOp::Shl, typ, dst, a, b });
            needsExt = true;
        }
        case ast::BinaryOp::Shr => {
            let op = il::BinOp::Ushr if unsigned else il::BinOp::Sshr;
            emit(self, il::Instr::BinOp { op, typ, dst, a, b });
        }
        case ast::BinaryOp::Eq => emit(self, il::Instr::BinOp { op: il::BinOp::Eq, typ, dst, a, b }),
        case ast::BinaryOp::Ne => emit(self, il::Instr::BinOp { op: il::BinOp::Ne, typ, dst, a, b }),
        case ast::BinaryOp::Lt => {
            let op = il::BinOp::Ult if unsigned else il::BinOp::Slt;
            emit(self, il::Instr::BinOp { op, typ, dst, a, b });
        }
        case ast::BinaryOp::Gt => { // `a > b` = `b < a`
            let op = il::BinOp::Ult if unsigned else il::BinOp::Slt;
            emit(self, il::Instr::BinOp { op, typ, dst, a: b, b: a });
        }
        case ast::BinaryOp::Lte => { // `a <= b` = `b >= a`
            let op = il::BinOp::Uge if unsigned else il::BinOp::Sge;
            emit(self, il::Instr::BinOp { op, typ, dst, a: b, b: a });
        }
        case ast::BinaryOp::Gte => {
            let op = il::BinOp::Uge if unsigned else il::BinOp::Sge;
            emit(self, il::Instr::BinOp { op, typ, dst, a, b });
        }
        // Logical xor on booleans is equivalent to not equal.
        case ast::BinaryOp::Xor => emit(self, il::Instr::BinOp { op: il::BinOp::Ne, typ, dst, a, b }),
        // Short-circuit ops are handled elsewhere.
        case ast::BinaryOp::And, ast::BinaryOp::Or => panic,
    }
    // Normalize sub-word arithmetic results so high bits are well-defined.
    // The lowering knows signedness, so it can pick the right extension.
    // [`il::Type::W32`] is handled in the backend via 32-bit instructions.
    if needsExt {
        return normalizeSubword(self, typ, unsigned, il::Val::Reg(dst));
    }
    return il::Val::Reg(dst);
}

/// Normalize sub-word values to well-defined high bits.
fn normalizeSubword(self: *mut FnLowerer, typ: il::Type, unsigned: bool, val: il::Val) -> il::Val {
    if typ == il::Type::W8 or typ == il::Type::W16 {
        let extDst: il::Reg = nextReg(self);
        if unsigned {
            emit(self, il::Instr::Zext { typ, dst: extDst, val });
        } else {
            emit(self, il::Instr::Sext { typ, dst: extDst, val });
        }
        return il::Val::Reg(extDst);
    }
    return val;
}

/// Lower a unary operation.
fn lowerUnOp(self: *mut FnLowerer, node: *ast::Node, unop: ast::UnOp) -> il::Val throws (LowerError) {
    let val = try lowerExpr(self, unop.value);
    let t = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let typ = ilType(self.low, t);
    let dst = nextReg(self);
    let mut needsExt: bool = false;

    match unop.op {
        case ast::UnaryOp::Not => {
            // TODO: Is this the best way?
            emit(self, il::Instr::BinOp { op: il::BinOp::Eq, typ, dst, a: val, b: il::Val::Imm(0) });
        }
        case ast::UnaryOp::Neg => {
            emit(self, il::Instr::UnOp { op: il::UnOp::Neg, typ, dst, a: val });
            needsExt = true;
        }
        case ast::UnaryOp::BitNot => {
            emit(self, il::Instr::UnOp { op: il::UnOp::Not, typ, dst, a: val });
            needsExt = true;
        }
    }
    if needsExt {
        return normalizeSubword(self, typ, isUnsignedType(t), il::Val::Reg(dst));
    }
    return il::Val::Reg(dst);
}

/// Lower a cast expression (`x as T`).
fn lowerCast(self: *mut FnLowerer, node: *ast::Node, cast: ast::As) -> il::Val throws (LowerError) {
    let val = try lowerExpr(self, cast.value);

    let srcType = resolver::typeFor(self.low.resolver, cast.value) else {
        throw LowerError::MissingType(cast.value);
    };
    let dstType = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    if resolver::typesEqual(srcType, dstType) {
        return val;
    }
    return lowerNumericCast(self, val, srcType, dstType);
}

/// Check whether a resolver type is a signed integer type.
fn isSignedType(t: resolver::Type) -> bool {
    match t {
        case resolver::Type::I8, resolver::Type::I16, resolver::Type::I32, resolver::Type::I64 => return true,
        else => return false,
    }
}

/// Check whether a resolver type is an unsigned integer type.
fn isUnsignedType(t: resolver::Type) -> bool {
    match t {
        case resolver::Type::U8, resolver::Type::U16, resolver::Type::U32, resolver::Type::U64 => return true,
        else => return false,
    }
}

/// Lower a string literal to a slice value.
///
/// String literals are stored as global data and the result is a slice
/// pointing to the data with the appropriate length.
fn lowerStringLit(self: *mut FnLowerer, node: *ast::Node, s: *[u8]) -> il::Val throws (LowerError) {
    // Get the slice type from the node.
    let sliceTy = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let case resolver::Type::Slice { item, mutable } = sliceTy else {
        throw LowerError::ExpectedSliceOrArray;
    };
    // Build the string data value.
    let ptr = try! alloc::alloc(
        self.low.arena, @sizeOf(il::DataValue), @alignOf(il::DataValue)
    ) as *mut il::DataValue;

    *ptr = il::DataValue { item: il::DataItem::Str(s), count: 1 };

    return try lowerConstDataAsSlice(self, @sliceOf(ptr, 1), 1, true, item, mutable, s.len);
}

/// Lower a builtin call expression.
fn lowerBuiltinCall(self: *mut FnLowerer, node: *ast::Node, kind: ast::Builtin, args: *mut [*ast::Node]) -> il::Val throws (LowerError) {
    match kind {
        case ast::Builtin::SliceOf => return try lowerSliceOf(self, node, args),
        case ast::Builtin::SizeOf, ast::Builtin::AlignOf => {
            let constVal = resolver::constValueFor(self.low.resolver, node) else {
                throw LowerError::MissingConst(node);
            };
            return try constValueToVal(self, constVal, node);
        }
    }
}

/// Lower a `@sliceOf(ptr, len)` or `@sliceOf(ptr, len, cap)` builtin call.
fn lowerSliceOf(self: *mut FnLowerer, node: *ast::Node, args: *mut [*ast::Node]) -> il::Val throws (LowerError) {
    if args.len != 2 and args.len != 3 {
        throw LowerError::InvalidArgCount;
    }
    let sliceTy = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let case resolver::Type::Slice { item, mutable } = sliceTy else {
        throw LowerError::ExpectedSliceOrArray;
    };
    let ptrVal = try lowerExpr(self, args[0]);
    let lenVal = try lowerExpr(self, args[1]);
    let mut capVal = lenVal;
    if args.len == 3 {
        capVal = try lowerExpr(self, args[2]);
    }
    return try buildSliceValue(self, item, mutable, ptrVal, lenVal, capVal);
}

/// Lower a `try` expression.
fn lowerTry(self: *mut FnLowerer, node: *ast::Node, t: ast::Try) -> il::Val throws (LowerError) {
    let case ast::NodeValue::Call(callExpr) = t.expr.value else {
        throw LowerError::ExpectedCall;
    };
    let calleeTy = resolver::typeFor(self.low.resolver, callExpr.callee) else {
        throw LowerError::MissingType(callExpr.callee);
    };
    let case resolver::Type::Fn(calleeInfo) = calleeTy else {
        throw LowerError::ExpectedFunction;
    };
    let okValueTy = *calleeInfo.returnType; // The type of the success payload.

    // Type of the try expression, which is either the return type of the function
    // if successful, or an optional of it, if using `try?`.
    let tryExprTy = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    // Check for trait method dispatch.
    let mut resVal: il::Val = undefined;
    if let case resolver::NodeExtra::TraitMethodCall {
        traitInfo, methodIndex
    } = resolver::nodeData(self.low.resolver, t.expr).extra {
        resVal = try lowerTraitMethodCall(self, t.expr, callExpr, traitInfo, methodIndex);
    } else {
        resVal = try lowerCall(self, t.expr, callExpr);
    }
    let base = emitValToReg(self, resVal); // The result value.
    let tagReg = resultTagReg(self, base); // The result tag.

    let okBlock = try createBlock(self, "ok"); // Block if success.
    let errBlock = try createBlock(self, "err"); // Block if failure.

    let mut mergeBlock: ?BlockId = nil;
    let mut resultSlot: ?il::Reg = nil; // `try` result value will be stored here.

    // Check if the `try` returns a success value or not. If so, reserve
    // space for it.
    let isVoid = tryExprTy == resolver::Type::Void;
    if not isVoid {
        resultSlot = try emitReserve(self, tryExprTy);
    }
    // Branch on tag: zero means ok, non-zero means error.
    try emitBr(self, tagReg, errBlock, okBlock);

    // We can now seal the blocks since all predecessors are known.
    try sealBlock(self, okBlock);
    try sealBlock(self, errBlock);

    // Success path: extract the successful value from the result and store it
    // in the result slot for later use after the merge point.
    switchToBlock(self, okBlock);

    if let slot = resultSlot {
        // Extract the success payload. If the result type differs from the payload
        // type (e.g. `try?` wrapping `T` into `?T`), wrap the value.
        let payloadVal = tvalPayloadVal(self, base, okValueTy, RESULT_VAL_OFFSET);
        let mut okVal = payloadVal;

        if t.returnsOptional and tryExprTy != okValueTy {
            okVal = try wrapInOptional(self, payloadVal, tryExprTy);
        }
        try emitStore(self, slot, 0, tryExprTy, okVal);
    }
    // Jump to merge block if unterminated.
    try emitMergeIfUnterminated(self, &mut mergeBlock);

    // Error path: handle the failure case based on the try expression variant.
    switchToBlock(self, errBlock);

    if t.returnsOptional {
        // `try?` converts errors to `nil` -- store the `nil` and continue.
        if let slot = resultSlot {
            let errVal = try buildNilOptional(self, tryExprTy);
            try emitStore(self, slot, 0, tryExprTy, errVal);
        }
        try emitMergeIfUnterminated(self, &mut mergeBlock);
    } else if t.catches.len > 0 {
        // `try ... catch` -- handle the error.
        let firstNode = t.catches[0];
        let case ast::NodeValue::CatchClause(first) = firstNode.value
            else panic "lowerTry: expected CatchClause";

        if first.typeNode != nil or t.catches.len > 1 {
            // Typed multi-catch: switch on global error tag.
            try lowerMultiCatch(self, t.catches, calleeInfo, base, tagReg, &mut mergeBlock);
        } else {
            // Single untyped catch clause.
            let savedVarsLen = enterVarScope(self);
            if let binding = first.binding {
                let case ast::NodeValue::Ident(name) = binding.value else {
                    throw LowerError::ExpectedIdentifier;
                };
                let errTy = *calleeInfo.throwList[0];
                let errVal = tvalPayloadVal(self, base, errTy, RESULT_VAL_OFFSET);
                let _ = newVar(self, name, ilType(self.low, errTy), false, errVal);
            }
            try lowerBlock(self, first.body);
            try emitMergeIfUnterminated(self, &mut mergeBlock);
            exitVarScope(self, savedVarsLen);
        }
    } else if t.shouldPanic {
        // `try!` -- panic on error, emit unreachable since control won't continue.
        // TODO: We should have some kind of `panic` instruction?
        emit(self, il::Instr::Unreachable);
    } else {
        // Plain `try` -- propagate the error to the caller by returning early.
        // Forward the callee's global error tag and payload directly.
        let callerLayout = resolver::getResultLayout(
            *self.fnType.returnType, self.fnType.throwList
        );
        let calleeErrSize = maxErrSize(calleeInfo.throwList);
        let dst = emitReserveLayout(self, callerLayout);

        emitStoreW64At(self, il::Val::Reg(tagReg), dst, TVAL_TAG_OFFSET);
        let srcPayload = emitPtrOffset(self, base, RESULT_VAL_OFFSET);
        let dstPayload = emitPtrOffset(self, dst, RESULT_VAL_OFFSET);
        emit(self, il::Instr::Blit { dst: dstPayload, src: srcPayload, size: il::Val::Imm(calleeErrSize as i64) });

        try emitRetVal(self, il::Val::Reg(dst));
    }

    // Switch to the merge block if one was created. If all paths diverged
    // (e.g both success and error returned), there's no merge block.
    if let blk = mergeBlock {
        try switchToAndSeal(self, blk);
    } else {
        return il::Val::Undef;
    }
    // Return the result value. For `void` expressions, return undefined.
    // For aggregates, return the slot pointer; for scalars, load the value.
    if let slot = resultSlot {
        if isAggregateType(tryExprTy) {
            return il::Val::Reg(slot);
        }
        return emitLoad(self, slot, 0, tryExprTy);
    } else { // Void return.
        return il::Val::Undef;
    }
}

/// Lower typed multi-catch clauses.
///
/// Emits a switch on the global error tag to dispatch to the correct catch
/// clause. Each typed clause extracts the error payload for its specific type
/// and binds it to the clause's identifier.
fn lowerMultiCatch(
    self: *mut FnLowerer,
    catches: *mut [*ast::Node],
    calleeInfo: *resolver::FnType,
    base: il::Reg,
    tagReg: il::Reg,
    mergeBlock: *mut ?BlockId
) throws (LowerError) {
    let entry = currentBlock(self);

    // First pass: create blocks, resolve error types, and build switch cases.
    let mut blocks: [BlockId; MAX_CATCH_CLAUSES] = undefined;
    let mut errTypes: [?resolver::Type; MAX_CATCH_CLAUSES] = undefined;
    let mut cases: *mut [il::SwitchCase] = &mut [];
    let mut defaultIdx: ?u32 = nil;

    for clauseNode, i in catches {
        let case ast::NodeValue::CatchClause(clause) = clauseNode.value
            else panic "lowerMultiCatch: expected CatchClause";

        blocks[i] = try createBlock(self, "catch");
        addPredecessor(self, blocks[i], entry);

        if let typeNode = clause.typeNode {
            let errTy = resolver::typeFor(self.low.resolver, typeNode) else {
                throw LowerError::MissingType(typeNode);
            };
            errTypes[i] = errTy;

            cases.append(il::SwitchCase {
                value: getOrAssignErrorTag(self.low, errTy) as i64,
                target: blocks[i].n,
                args: &mut []
            }, self.allocator);
        } else {
            errTypes[i] = nil;
            defaultIdx = i;
        }
    }

    // Emit switch. Default target is the catch-all block, or an unreachable block.
    let mut defaultTarget: BlockId = undefined;
    if let idx = defaultIdx {
        defaultTarget = blocks[idx];
    } else {
        defaultTarget = try createBlock(self, "unreachable");
        addPredecessor(self, defaultTarget, entry);
    }
    emit(self, il::Instr::Switch {
        val: il::Val::Reg(tagReg),
        defaultTarget: defaultTarget.n,
        defaultArgs: &mut [],
        cases
    });

    // Second pass: emit each catch clause body.
    for clauseNode, i in catches {
        let case ast::NodeValue::CatchClause(clause) = clauseNode.value
            else panic "lowerMultiCatch: expected CatchClause";

        try switchToAndSeal(self, blocks[i]);
        let savedVarsLen = enterVarScope(self);

        if let binding = clause.binding {
            let case ast::NodeValue::Ident(name) = binding.value else {
                throw LowerError::ExpectedIdentifier;
            };
            let errTy = errTypes[i] else panic "lowerMultiCatch: catch-all with binding";
            let errVal = tvalPayloadVal(self, base, errTy, RESULT_VAL_OFFSET);

            newVar(self, name, ilType(self.low, errTy), false, errVal);
        }
        try lowerBlock(self, clause.body);
        try emitMergeIfUnterminated(self, mergeBlock);

        exitVarScope(self, savedVarsLen);
    }

    // Emit unreachable block if no catch-all.
    if defaultIdx == nil {
        try switchToAndSeal(self, defaultTarget);
        emit(self, il::Instr::Unreachable);
    }
}

/// Emit a byte-copy loop: `for i in 0..size { dst[i] = src[i]; }`.
///
/// Used when `blit` cannot be used because the copy size is dynamic.
/// Terminates the current block and leaves the builder positioned
/// after the loop.
fn emitByteCopyLoop(
    self: *mut FnLowerer,
    dst: il::Reg,
    src: il::Reg,
    size: il::Val,
    label: *[u8]
) throws (LowerError) {
    let iReg = nextReg(self);
    let header = try createBlockWithParam(
        self, label, il::Param { value: iReg, type: il::Type::W32 }
    );
    let body = try createBlock(self, label);
    let done = try createBlock(self, label);

    // Jump to header with initial counter is zero.
    try emitJmpWithArg(self, header, il::Val::Imm(0));

    // Don't seal header yet -- the body will add another predecessor.
    switchToBlock(self, header);
    try emitBrCmp(self, il::CmpOp::Ult, il::Type::W32, il::Val::Reg(iReg), size, body, done);

    // Body: load byte from source, store to destination, increment counter.
    try switchToAndSeal(self, body);

    let srcElem = emitElem(self, 1, src, il::Val::Reg(iReg));
    let byteReg = nextReg(self);
    emit(self, il::Instr::Load { typ: il::Type::W8, dst: byteReg, src: srcElem, offset: 0 });

    let dstElem = emitElem(self, 1, dst, il::Val::Reg(iReg));
    emit(self, il::Instr::Store { typ: il::Type::W8, src: il::Val::Reg(byteReg), dst: dstElem, offset: 0 });

    let nextI = emitTypedBinOp(self, il::BinOp::Add, il::Type::W32, il::Val::Reg(iReg), il::Val::Imm(1));
    // Jump back to header -- this adds body as a predecessor.
    try emitJmpWithArg(self, header, nextI);

    // Now all predecessors of header are known, seal it.
    try sealBlock(self, header);
    try switchToAndSeal(self, done);
}

/// Lower `slice.append(val, allocator)`.
///
/// Emits inline grow-if-needed logic:
///
///     load len, cap from slice header
///     if len < cap: jmp @store
///     else:         jmp @grow
///
///     @grow:
///       newCap = max(cap * 2, 1)
///       call allocator.func(allocator.ctx, newCap * stride, alignment)
///       copy old data to new pointer
///       update slice ptr and cap
///       jmp @store
///
///     @store:
///       store element at ptr + len * stride
///       increment len
///
fn lowerSliceAppend(self: *mut FnLowerer, call: ast::Call, elemType: *resolver::Type) throws (LowerError) {
    let case ast::NodeValue::FieldAccess(access) = call.callee.value
        else throw LowerError::MissingMetadata;

    // Get the address of the slice header.
    let sliceVal = try lowerExpr(self, access.parent);
    let sliceReg = emitValToReg(self, sliceVal);

    // Lower the value to append and the allocator.
    let elemVal = try lowerExpr(self, call.args[0]);
    let allocVal = try lowerExpr(self, call.args[1]);
    let allocReg = emitValToReg(self, allocVal);

    let elemLayout = resolver::getTypeLayout(*elemType);
    let stride = elemLayout.size;
    let alignment = elemLayout.alignment;

    // Load current length and capacity.
    let lenVal = loadSliceLen(self, sliceReg);
    let capVal = loadSliceCap(self, sliceReg);

    // Branch: if length is smaller than capacity, go to @store else @grow.
    let storeBlock = try createBlock(self, "append.store");
    let growBlock = try createBlock(self, "append.grow");
    try emitBrCmp(self, il::CmpOp::Ult, il::Type::W32, lenVal, capVal, storeBlock, growBlock);
    try switchToAndSeal(self, growBlock);

    // -- @grow block ----------------------------------------------------------

    // `newCap = max(cap * 2, 1)`.
    // We are only here when at capacity, so we use `or` with `1` to ensure at least capacity `1`.
    let doubledVal = emitTypedBinOp(self, il::BinOp::Shl, il::Type::W32, capVal, il::Val::Imm(1));
    let newCapVal = emitTypedBinOp(self, il::BinOp::Or, il::Type::W32, doubledVal, il::Val::Imm(1));

    // Call allocator: `a.func(a.ctx, newCap * stride, alignment)`.
    let allocFnReg = nextReg(self);
    emitLoadW64At(self, allocFnReg, allocReg, 0);

    let allocCtxReg = nextReg(self);
    emitLoadW64At(self, allocCtxReg, allocReg, 8);

    let byteSize = emitTypedBinOp(self, il::BinOp::Mul, il::Type::W32, newCapVal, il::Val::Imm(stride as i64));
    let args = try allocVals(self, 3);

    args[0] = il::Val::Reg(allocCtxReg);
    args[1] = byteSize;
    args[2] = il::Val::Imm(alignment as i64);

    let newPtrReg = nextReg(self);
    emit(self, il::Instr::Call {
        retTy: il::Type::W64,
        dst: newPtrReg,
        func: il::Val::Reg(allocFnReg),
        args,
    });

    // Copy old data byte-by-byte.
    let oldPtrReg = loadSlicePtr(self, sliceReg);
    let copyBytes = emitTypedBinOp(self, il::BinOp::Mul, il::Type::W32, lenVal, il::Val::Imm(stride as i64));
    try emitByteCopyLoop(self, newPtrReg, oldPtrReg, copyBytes, "append");

    // Update slice header.
    emitStoreW64At(self, il::Val::Reg(newPtrReg), sliceReg, SLICE_PTR_OFFSET);
    emitStoreW32At(self, newCapVal, sliceReg, SLICE_CAP_OFFSET);

    try emitJmp(self, storeBlock);
    try switchToAndSeal(self, storeBlock);

    // -- @store block ---------------------------------------------------------

    // Store element at `ptr + len * stride`.
    let ptrReg = loadSlicePtr(self, sliceReg);
    let elemDst = emitElem(self, stride, ptrReg, lenVal);
    try emitStore(self, elemDst, 0, *elemType, elemVal);

    // Increment len.
    let newLen = emitTypedBinOp(self, il::BinOp::Add, il::Type::W32, lenVal, il::Val::Imm(1));
    emitStoreW32At(self, newLen, sliceReg, SLICE_LEN_OFFSET);
}

/// Lower `slice.delete(index)`.
///
/// Bounds-check the index, shift elements after it by one stride
/// via a byte-copy loop, and decrement `len`.
fn lowerSliceDelete(self: *mut FnLowerer, call: ast::Call, elemType: *resolver::Type) throws (LowerError) {
    let case ast::NodeValue::FieldAccess(access) = call.callee.value
        else throw LowerError::MissingMetadata;

    let elemLayout = resolver::getTypeLayout(*elemType);
    let stride = elemLayout.size;

    // Get slice header address.
    let sliceVal = try lowerExpr(self, access.parent);
    let sliceReg = emitValToReg(self, sliceVal);

    // Lower the index argument.
    let indexVal = try lowerExpr(self, call.args[0]);

    // Load len and bounds-check: index must be smaller than length.
    let lenVal = loadSliceLen(self, sliceReg);
    try emitTrapUnlessCmp(self, il::CmpOp::Ult, il::Type::W32, indexVal, lenVal);

    // Compute the destination and source for the shift.
    let ptrReg = loadSlicePtr(self, sliceReg);
    let dst = emitElem(self, stride, ptrReg, indexVal);

    // `src = dst + stride`.
    let src = emitPtrOffset(self, dst, stride as i32);

    // Move `(len - index - 1) * stride`.
    let tailLen = emitTypedBinOp(self, il::BinOp::Sub, il::Type::W32, lenVal, indexVal);
    let tailLenMinusOne = emitTypedBinOp(self, il::BinOp::Sub, il::Type::W32, tailLen, il::Val::Imm(1));
    let moveBytes = emitTypedBinOp(self, il::BinOp::Mul, il::Type::W32, tailLenMinusOne, il::Val::Imm(stride as i64));

    // Shift elements left via byte-copy loop.
    // When deleting the last element, the loop is a no-op.
    try emitByteCopyLoop(self, dst, src, moveBytes, "delete");
    // Decrement length.
    let newLen = emitTypedBinOp(self, il::BinOp::Sub, il::Type::W32, lenVal, il::Val::Imm(1));

    emitStoreW32At(self, newLen, sliceReg, SLICE_LEN_OFFSET);
}

/// Lower a call expression, which may be a function call or type constructor.
fn lowerCallOrCtor(self: *mut FnLowerer, node: *ast::Node, call: ast::Call) -> il::Val throws (LowerError) {
    let nodeData = resolver::nodeData(self.low.resolver, node).extra;

    // Check for slice method dispatch.
    if let case resolver::NodeExtra::SliceAppend { elemType } = nodeData {
        try lowerSliceAppend(self, call, elemType);
        return il::Val::Undef;
    }
    if let case resolver::NodeExtra::SliceDelete { elemType } = nodeData {
        try lowerSliceDelete(self, call, elemType);
        return il::Val::Undef;
    }
    // Check for trait method dispatch.
    if let case resolver::NodeExtra::TraitMethodCall { traitInfo, methodIndex } = nodeData {
        return try lowerTraitMethodCall(self, node, call, traitInfo, methodIndex);
    }
    if let sym = resolver::nodeData(self.low.resolver, call.callee).sym {
        if let case resolver::SymbolData::Type(nominal) = sym.data {
            let case resolver::NominalType::Record(_) = *nominal else {
                throw LowerError::ExpectedRecord;
            };
            return try lowerRecordCtor(self, nominal, call.args);
        }
        if let case resolver::SymbolData::Variant { .. } = sym.data {
            return try lowerUnionCtor(self, node, sym, call);
        }
    }
    return try lowerCall(self, node, call);
}

/// Lower a trait method call through v-table dispatch.
///
/// Given `obj.method(args)` where `obj` is a trait object, emits:
///
///     load w64 %data %obj 0          // data pointer
///     load w64 %vtable %obj 8        // v-table pointer
///     load w64 %fn %vtable <slot>    // function pointer
///     call <retTy> %ret %fn(%data, args...)
///
fn lowerTraitMethodCall(
    self: *mut FnLowerer,
    node: *ast::Node,
    call: ast::Call,
    traitInfo: *resolver::TraitType,
    methodIndex: u32
) -> il::Val throws (LowerError) {
    // Method calls look like field accesses.
    let case ast::NodeValue::FieldAccess(access) = call.callee.value
        else throw LowerError::MissingMetadata;

    // Lower the trait object expression.
    let traitObjVal = try lowerExpr(self, access.parent);
    let traitObjReg = emitValToReg(self, traitObjVal);

    // Load data pointer from trait object.
    let dataReg = nextReg(self);
    emit(self, il::Instr::Load {
        typ: il::Type::W64,
        dst: dataReg,
        src: traitObjReg,
        offset: TRAIT_OBJ_DATA_OFFSET,
    });

    // Load v-table pointer from trait object.
    let vtableReg = nextReg(self);
    emit(self, il::Instr::Load {
        typ: il::Type::W64,
        dst: vtableReg,
        src: traitObjReg,
        offset: TRAIT_OBJ_VTABLE_OFFSET,
    });

    // Load function pointer from v-table at the method's slot offset.
    let fnPtrReg = nextReg(self);
    let slotOffset = (methodIndex * resolver::PTR_SIZE) as i32;

    emit(self, il::Instr::Load {
        typ: il::Type::W64,
        dst: fnPtrReg,
        src: vtableReg,
        offset: slotOffset,
    });

    // Check if the method needs a hidden return parameter.
    let methodFnType = traitInfo.methods[methodIndex].fnType;
    let retTy = *methodFnType.returnType;
    let returnParam = requiresReturnParam(methodFnType);

    // Build args: data pointer (receiver) + user args.
    let argOffset: u32 = 1 if returnParam else 0;
    let args = try allocVals(self, call.args.len + 1 + argOffset);

    // Data pointer is the receiver (first argument after hidden return param).
    args[argOffset] = il::Val::Reg(dataReg);

    // Lower user arguments.
    for arg, i in call.args {
        args[i + 1 + argOffset] = try lowerExpr(self, arg);
    }

    // Allocate the return buffer when needed.
    if returnParam {
        if methodFnType.throwList.len > 0 {
            let successType = *methodFnType.returnType;
            let layout = resolver::getResultLayout(
                successType, methodFnType.throwList);

            args[0] = il::Val::Reg(emitReserveLayout(self, layout));
        } else {
            args[0] = il::Val::Reg(try emitReserve(self, retTy));
        }
        let dst = nextReg(self);

        emit(self, il::Instr::Call {
            retTy: il::Type::W64,
            dst,
            func: il::Val::Reg(fnPtrReg),
            args,
        });
        return il::Val::Reg(dst);
    }

    // Scalar call: allocate destination register for non-void return types.
    let mut dst: ?il::Reg = nil;
    if retTy != resolver::Type::Void {
        dst = nextReg(self);
    }
    emit(self, il::Instr::Call {
        retTy: ilType(self.low, retTy),
        dst,
        func: il::Val::Reg(fnPtrReg),
        args,
    });

    if let d = dst {
        if isSmallAggregate(retTy) {
            let slot = emitReserveLayout(
                self,
                resolver::Layout {
                    size: resolver::PTR_SIZE,
                    alignment: resolver::PTR_SIZE
                });

            emit(self, il::Instr::Store {
                typ: il::Type::W64,
                src: il::Val::Reg(d),
                dst: slot,
                offset: 0,
            });
            return il::Val::Reg(slot);
        }
        return il::Val::Reg(d);
    }
    return il::Val::Undef;
}

/// Check if a call is to a compiler intrinsic and lower it directly.
fn lowerIntrinsicCall(self: *mut FnLowerer, call: ast::Call) -> ?il::Val throws (LowerError) {
    // Get the callee symbol and check if it's marked as an intrinsic.
    let sym = resolver::nodeData(self.low.resolver, call.callee).sym else {
        // Expressions or function pointers may not have an associated symbol.
        return nil;
    };
    if not ast::hasAttribute(sym.attrs, ast::Attribute::Intrinsic) {
        return nil;
    }
    // Check for known intrinsic names.
    if mem::eq(sym.name, "ecall") {
        return try lowerEcall(self, call);
    } else if mem::eq(sym.name, "ebreak") {
        return try lowerEbreak(self, call);
    } else {
        throw LowerError::UnknownIntrinsic;
    }
}

/// Lower an ecall intrinsic: `ecall(num, a0, a1, a2, a3) -> i32`.
fn lowerEcall(self: *mut FnLowerer, call: ast::Call) -> il::Val throws (LowerError) {
    if call.args.len != 5 {
        throw LowerError::InvalidArgCount;
    }
    let num = try lowerExpr(self, call.args[0]);
    let a0 = try lowerExpr(self, call.args[1]);
    let a1 = try lowerExpr(self, call.args[2]);
    let a2 = try lowerExpr(self, call.args[3]);
    let a3 = try lowerExpr(self, call.args[4]);
    let dst = nextReg(self);

    emit(self, il::Instr::Ecall { dst, num, a0, a1, a2, a3 });

    return il::Val::Reg(dst);
}

/// Lower an ebreak intrinsic: `ebreak()`.
fn lowerEbreak(self: *mut FnLowerer, call: ast::Call) -> il::Val throws (LowerError) {
    if call.args.len != 0 {
        throw LowerError::InvalidArgCount;
    }
    emit(self, il::Instr::Ebreak);

    return il::Val::Undef;
}

/// Resolve callee to an IL value. For direct function calls, use the symbol name.
/// For variables holding function pointers or complex expressions (eg. `array[i]()`),
/// lower the callee expression.
fn lowerCallee(self: *mut FnLowerer, callee: *ast::Node) -> il::Val throws (LowerError) {
    if let sym = resolver::nodeData(self.low.resolver, callee).sym {
        if let case ast::NodeValue::FnDecl(_) = sym.node.value {
            // First try to look up the symbol in our registered functions.
            // This handles cross-package calls correctly, since packages are
            // lowered in dependency order.
            if let qualName = lookupFnSym(self.low, sym) {
                return il::Val::FnAddr(qualName);
            }
            // Fall back to computing the qualified name from the module graph.
            // This works for functions in the current package.
            let modId = resolver::moduleIdForSymbol(self.low.resolver, sym) else {
                throw LowerError::MissingMetadata;
            };
            return il::Val::FnAddr(qualifyName(self.low, modId, sym.name));
        }
    }
    return try lowerExpr(self, callee);
}

/// Lower a function call expression.
fn lowerCall(self: *mut FnLowerer, node: *ast::Node, call: ast::Call) -> il::Val throws (LowerError) {
    // Check for intrinsic calls before normal call lowering.
    if let intrinsicVal = try lowerIntrinsicCall(self, call) {
        return intrinsicVal;
    }
    let calleeTy = resolver::typeFor(self.low.resolver, call.callee) else {
        throw LowerError::MissingType(call.callee);
    };
    let case resolver::Type::Fn(fnInfo) = calleeTy else {
        throw LowerError::ExpectedFunction;
    };
    let retTy = resolver::typeFor(self.low.resolver, node) else {
        throw LowerError::MissingType(node);
    };
    let returnParam = requiresReturnParam(fnInfo);

    // Lower function value and arguments, reserving an extra slot for the
    // hidden return buffer when needed.
    let callee = try lowerCallee(self, call.callee);
    let offset: u32 = 1 if returnParam else 0;
    let args = try allocVals(self, call.args.len + offset);
    for arg, i in call.args {
        args[i + offset] = try lowerExpr(self, arg);
    }

    // Allocate the return buffer when needed.
    if returnParam {
        if fnInfo.throwList.len > 0 {
            let successType = *fnInfo.returnType;
            let layout = resolver::getResultLayout(successType, fnInfo.throwList);

            args[0] = il::Val::Reg(emitReserveLayout(self, layout));
        } else {
            args[0] = il::Val::Reg(try emitReserve(self, retTy));
        }
        let dst = nextReg(self);
        emit(self, il::Instr::Call {
            retTy: il::Type::W64,
            dst,
            func: callee,
            args,
        });
        return il::Val::Reg(dst);
    }

    // Scalar call: allocate destination register for non-void return types.
    let mut dst: ?il::Reg = nil;
    if retTy != resolver::Type::Void {
        dst = nextReg(self);
    }
    emit(self, il::Instr::Call {
        retTy: ilType(self.low, retTy),
        dst,
        func: callee,
        args,
    });

    // Non-void functions produce a value in a register, while void functions
    // return an undefined value that shouldn't be used.
    if let d = dst {
        if isSmallAggregate(retTy) {
            let slot = emitReserveLayout(
                self,
                resolver::Layout {
                    size: resolver::PTR_SIZE,
                    alignment: resolver::PTR_SIZE
                });
            emit(self, il::Instr::Store {
                typ: il::Type::W64,
                src: il::Val::Reg(d),
                dst: slot,
                offset: 0,
            });
            return il::Val::Reg(slot);
        }
        return il::Val::Reg(d);
    }
    return il::Val::Undef;
}

/// Apply coercions requested by the resolver.
fn applyCoercion(self: *mut FnLowerer, node: *ast::Node, val: il::Val) -> il::Val throws (LowerError) {
    let coerce = resolver::coercionFor(self.low.resolver, node) else {
        return val;
    };
    match coerce {
        case resolver::Coercion::OptionalLift(optType) => {
            if let case ast::NodeValue::Nil = node.value {
                return try buildNilOptional(self, optType);
            }
            return try wrapInOptional(self, val, optType);
        }
        case resolver::Coercion::NumericCast { from, to } => {
            return lowerNumericCast(self, val, from, to);
        }
        case resolver::Coercion::ResultWrap => {
            let payloadType = *self.fnType.returnType;
            return try buildResult(self, 0, val, payloadType);
        }
        case resolver::Coercion::TraitObject { traitInfo, inst } => {
            return try buildTraitObject(self, val, traitInfo, inst);
        }
        case resolver::Coercion::Identity => return val,
    }
}

/// Lower an implicit numeric cast coercion.
///
/// Handles widening conversions between integer types. Uses sign-extension
/// for signed source types and zero-extension for unsigned source types.
fn lowerNumericCast(self: *mut FnLowerer, val: il::Val, srcType: resolver::Type, dstType: resolver::Type) -> il::Val {
    let srcLayout = resolver::getTypeLayout(srcType);
    let dstLayout = resolver::getTypeLayout(dstType);

    if srcLayout.size < dstLayout.size {
        // Widening conversion: sign/zero-extend based on source signedness.
        let dst = nextReg(self);
        let srcIrType = ilType(self.low, srcType);

        if isSignedType(srcType) {
            emit(self, il::Instr::Sext { typ: srcIrType, dst, val });
        } else {
            emit(self, il::Instr::Zext { typ: srcIrType, dst, val });
        }
        return il::Val::Reg(dst);
    }
    if srcLayout.size > dstLayout.size {
        // Narrowing conversion: truncate and normalize to destination width.
        let dstIrType = ilType(self.low, dstType);
        let dst = nextReg(self);

        if isSignedType(dstType) {
            emit(self, il::Instr::Sext { typ: dstIrType, dst, val });
        } else {
            emit(self, il::Instr::Zext { typ: dstIrType, dst, val });
        }
        return il::Val::Reg(dst);
    }
    // Same size: bit pattern is unchanged, value is returned as-is.
    // The backend is responsible for normalizing to the correct width
    // when performing comparisons.
    return val;
}

/// Lower an identifier that refers to a global symbol.
fn lowerGlobalSymbol(self: *mut FnLowerer, node: *ast::Node) -> il::Val throws (LowerError) {
    // First try to get a compile-time constant value.
    if let constVal = resolver::constValueFor(self.low.resolver, node) {
        return try constValueToVal(self, constVal, node);
    }
    // Otherwise get the symbol.
    let sym = resolver::nodeData(self.low.resolver, node).sym else {
        throw LowerError::MissingSymbol(node);
    };
    let mut ty: resolver::Type = undefined;

    match sym.data {
        case resolver::SymbolData::Constant { type, .. } =>
            ty = type,
        case resolver::SymbolData::Value { type, .. } => {
            // Function pointer reference: just return the address, don't load.
            // Functions don't have a separate storage location holding their
            // address; they just exist at an address in the code section.
            if let case resolver::Type::Fn(_) = type {
                return il::Val::Reg(emitFnAddr(self, sym));
            }
            ty = type;
        }
        else => throw LowerError::UnexpectedNodeValue(node),
    }
    let dst = emitDataAddr(self, sym);

    return emitRead(self, dst, 0, ty);
}

/// Lower an assignment to a static variable.
fn lowerStaticAssign(self: *mut FnLowerer, target: *ast::Node, val: il::Val) throws (LowerError) {
    let sym = resolver::nodeData(self.low.resolver, target).sym else {
        throw LowerError::MissingSymbol(target);
    };
    let case resolver::SymbolData::Value { type, .. } = sym.data else {
        throw LowerError::ImmutableAssignment;
    };
    let dst = emitDataAddr(self, sym);

    try emitStore(self, dst, 0, type, val);
}

/// Lower a scope access expression like `Module::Const` or `Union::Variant`.
/// This doesn't handle record literal variants.
fn lowerScopeAccess(self: *mut FnLowerer, node: *ast::Node) -> il::Val throws (LowerError) {
    // First try to get a compile-time constant value.
    if let constVal = resolver::constValueFor(self.low.resolver, node) {
        return try constValueToVal(self, constVal, node);
    }
    // Otherwise get the associated symbol.
    let data = resolver::nodeData(self.low.resolver, node);
    let sym = data.sym else {
        throw LowerError::MissingSymbol(node);
    };
    match sym.data {
        case resolver::SymbolData::Variant { index, .. } => {
            // Void union variant like `Option::None`.
            if data.ty == resolver::Type::Unknown {
                throw LowerError::MissingType(node);
            }
            // All-void unions are passed as scalars (the tag byte).
            // Return an immediate instead of building a tagged aggregate.
            if resolver::isVoidUnion(data.ty) {
                return il::Val::Imm(index as i64);
            }
            let unionInfo = unionInfoFromType(data.ty) else {
                throw LowerError::MissingMetadata;
            };
            let valOffset = unionInfo.valOffset as i32;
            return try buildTagged(self, resolver::getTypeLayout(data.ty), index as i64, nil, resolver::Type::Void, 1, valOffset);
        }
        case resolver::SymbolData::Constant { type, .. } => {
            // Constant without compile-time value (e.g. record constant);
            // load from data section.
            let src = emitDataAddr(self, sym);

            // Aggregate constants live in read-only memory.  Return a
            // mutable copy so that callers that assign through the
            // resulting pointer do not fault.
            if isAggregateType(type) {
                let layout = resolver::getTypeLayout(type);
                let dst = emitReserveLayout(self, layout);
                emit(self, il::Instr::Blit { dst, src, size: il::Val::Imm(layout.size as i64) });

                return il::Val::Reg(dst);
            }
            return emitRead(self, src, 0, type);
        }
        case resolver::SymbolData::Value { type, .. } => {
            // Function pointer reference.
            if let case resolver::Type::Fn(_) = type {
                return il::Val::Reg(emitFnAddr(self, sym));
            }
            throw LowerError::ExpectedFunction;
        }
        else => throw LowerError::UnexpectedNodeValue(node),
    }
}

/// Lower an expression AST node to an IL value.
/// This is the main expression dispatch, all expression nodes go through here.
fn lowerExpr(self: *mut FnLowerer, node: *ast::Node) -> il::Val throws (LowerError) {
    if self.low.options.debug {
        self.srcLoc.offset = node.span.offset;
    }
    let mut val: il::Val = undefined;

    match node.value {
        case ast::NodeValue::Ident(_) => {
            // First try local variable lookup.
            // Otherwise fall back to global symbol lookup.
            if let v = lookupLocalVar(self, node) {
                val = try useVar(self, v);
                if self.vars[v.id].addressTaken {
                    let typ = resolver::typeFor(self.low.resolver, node) else {
                        throw LowerError::MissingType(node);
                    };
                    let ptr = emitValToReg(self, val);
                    val = emitRead(self, ptr, 0, typ);
                }
            } else {
                val = try lowerGlobalSymbol(self, node);
            }
        }
        case ast::NodeValue::ScopeAccess(_) => {
            val = try lowerScopeAccess(self, node);
        }
        case ast::NodeValue::Number(lit) => {
            let mag = -(lit.magnitude as i64) if lit.negative else lit.magnitude as i64;
            val = il::Val::Imm(mag);
        }
        case ast::NodeValue::Bool(b) => {
            val = il::Val::Imm(1) if b else il::Val::Imm(0);
        }
        case ast::NodeValue::Char(c) => {
            val = il::Val::Imm(c as i64);
        }
        case ast::NodeValue::Nil => {
            let typ = resolver::typeFor(self.low.resolver, node) else {
                throw LowerError::MissingType(node);
            };
            if let case resolver::Type::Optional(_) = typ {
                val = try buildNilOptional(self, typ);
            } else if let case resolver::Type::Nil = typ {
                // Standalone `nil` without a concrete optional type. We can't
                // generate a proper value representation.
                throw LowerError::MissingType(node);
            } else {
                throw LowerError::NilInNonOptional;
            }
        }
        case ast::NodeValue::RecordLit(lit) => {
            val = try lowerRecordLit(self, node, lit);
        }
        case ast::NodeValue::AddressOf(addr) => {
            val = try lowerAddressOf(self, node, addr);
        }
        case ast::NodeValue::Deref(target) => {
            val = try lowerDeref(self, node, target);
        }
        case ast::NodeValue::BinOp(binop) => {
            val = try lowerBinOp(self, node, binop);
        }
        case ast::NodeValue::UnOp(unop) => {
            val = try lowerUnOp(self, node, unop);
        }
        case ast::NodeValue::Subscript { container, index } => {
            val = try lowerSubscript(self, node, container, index);
        }
        case ast::NodeValue::BuiltinCall { kind, args } => {
            val = try lowerBuiltinCall(self, node, kind, args);
        }
        case ast::NodeValue::Call(call) => {
            val = try lowerCallOrCtor(self, node, call);
        }
        case ast::NodeValue::Try(t) => {
            val = try lowerTry(self, node, t);
        }
        case ast::NodeValue::FieldAccess(access) => {
            // Check for compile-time constant (e.g., `arr.len` on fixed-size arrays).
            if let constVal = resolver::constValueFor(self.low.resolver, node) {
                match constVal {
                    // TODO: Handle `u32` values that don't fit in an `i32`.
                    //       Perhaps just store the `ConstInt`.
                    case resolver::ConstValue::Int(i) => val = il::Val::Imm(constIntToI64(i)),
                    else => val = try lowerFieldAccess(self, access),
                }
            } else {
                val = try lowerFieldAccess(self, access);
            }
        }
        case ast::NodeValue::ArrayLit(elements) => {
            val = try lowerArrayLit(self, node, elements);
        }
        case ast::NodeValue::ArrayRepeatLit(repeat) => {
            val = try lowerArrayRepeatLit(self, node, repeat);
        }
        case ast::NodeValue::As(cast) => {
            val = try lowerCast(self, node, cast);
        }
        case ast::NodeValue::CondExpr(cond) => {
            val = try lowerCondExpr(self, node, cond);
        }
        case ast::NodeValue::String(s) => {
            val = try lowerStringLit(self, node, s);
        }
        case ast::NodeValue::Undef => {
            let typ = resolver::typeFor(self.low.resolver, node) else {
                throw LowerError::MissingType(node);
            };
            if isAggregateType(typ) {
                // When `undefined` appears as a stand-alone expression,
                /// we need a stack slot for reads and writes.
                let slot = try emitReserve(self, typ);
                val = il::Val::Reg(slot);
            } else {
                val = il::Val::Undef;
            }
        }
        case ast::NodeValue::Panic { .. } => {
            // Panic in expression context (e.g. match arm). Emit unreachable
            // and return a dummy value since control won't continue.
            emit(self, il::Instr::Unreachable);
            val = il::Val::Undef;
        }
        case ast::NodeValue::Assert { .. } => {
            // Assert in expression context. Lower as statement, return `void`.
            try lowerNode(self, node);
            val = il::Val::Undef;
        }
        case ast::NodeValue::Block(_) => {
            try lowerBlock(self, node);
            val = il::Val::Undef;
        }
        case ast::NodeValue::ExprStmt(expr) => {
            let _ = expr;
            val = il::Val::Undef;
        }
        // Lower these as statements.
        case ast::NodeValue::ConstDecl(decl) => {
            try lowerDataDecl(self.low, node, decl.value, true);
            val = il::Val::Undef;
        }
        case ast::NodeValue::StaticDecl(decl) => {
            try lowerDataDecl(self.low, node, decl.value, false);
            val = il::Val::Undef;
        }
        case ast::NodeValue::Throw { .. },
             ast::NodeValue::Return { .. },
             ast::NodeValue::Continue,
             ast::NodeValue::Break => {
            try lowerNode(self, node);
            val = il::Val::Undef;
        }
        else => {
            panic "lowerExpr: node is not an expression";
        }
    }
    return try applyCoercion(self, node, val);
}

/// Translate a Radiance type to an IL type.
///
/// The IL type system is much simpler than Radiance's, only primitive types
/// are used. If a Radiance type doesn't fit in a machine word, it is passed
/// by reference.
///
/// The IL doesn't track signedness - that's encoded in the instructions
/// (e.g., Slt vs Ult).
fn ilType(self: *mut Lowerer, typ: resolver::Type) -> il::Type {
    match typ {
        // Scalars map to their natural widths.
        case resolver::Type::Bool => return il::Type::W8,
        case resolver::Type::I8 => return il::Type::W8,
        case resolver::Type::I16 => return il::Type::W16,
        case resolver::Type::I32 => return il::Type::W32,
        case resolver::Type::U8 => return il::Type::W8,
        case resolver::Type::U16 => return il::Type::W16,
        case resolver::Type::U32 => return il::Type::W32,
        case resolver::Type::I64 => return il::Type::W64,
        case resolver::Type::U64 => return il::Type::W64,
        // Reference types are all pointer-sized (64-bit on RV64).
        case resolver::Type::Pointer { .. } => return il::Type::W64,
        // Aggregates are represented as pointers to stack memory.
        case resolver::Type::Slice { .. } => return il::Type::W64,
        case resolver::Type::Array(_) => return il::Type::W64,
        case resolver::Type::Optional(_) => return il::Type::W64,
        case resolver::Type::Nominal(_) => {
            if resolver::isVoidUnion(typ) {
                return il::Type::W8;
            }
            return il::Type::W64;
        }
        case resolver::Type::Fn(_) => return il::Type::W64,
        case resolver::Type::TraitObject { .. } => return il::Type::W64,
        // Void functions return zero at the IL level.
        case resolver::Type::Void => return il::Type::W32,
        // FIXME: We shouldn't try to lower this type, it should be behind a pointer.
        case resolver::Type::Opaque => return il::Type::W32,
        // FIXME: This should be resolved to a concrete integer type in the resolver.
        case resolver::Type::Int => return il::Type::W32,
        else => panic "ilType: type cannot be lowered",
    }
}