API Reference¶

Complete API reference for DTDE (Distributed Temporal Data Engine).

Table of Contents¶

DbContext Extensions
Configuration API
Query Methods
Cross-Shard Transactions
Entity Configuration
Shard Configuration

DbContext Extensions¶

DtdeDbContext¶

Base class for DTDE-enabled contexts.

namespace Dtde.EntityFramework;

public abstract class DtdeDbContext : DbContext

Properties¶

Property	Type	Description
`TemporalContext`	`ITemporalContext`	Access to temporal context settings
`MetadataRegistry`	`IMetadataRegistry`	Registry of entity metadata
`ShardRegistry`	`IShardRegistry`	Registry of shard definitions

Methods¶

ValidAt\<TEntity>¶

Returns entities valid at a specific point in time.

public IQueryable<TEntity> ValidAt<TEntity>(DateTime asOfDate) where TEntity : class

Parameters: - asOfDate - The point in time to query

Returns: IQueryable<TEntity> filtered to valid entities

Example:

var currentContracts = await db.ValidAt<Contract>(DateTime.Today).ToListAsync();

ValidBetween\<TEntity>¶

Returns entities valid within a date range.

public IQueryable<TEntity> ValidBetween<TEntity>(DateTime startDate, DateTime endDate)
    where TEntity : class

Parameters: - startDate - Start of the range (inclusive) - endDate - End of the range (inclusive)

Returns: IQueryable<TEntity> with entities valid at any point in the range

Example:

var q1Contracts = await db.ValidBetween<Contract>(
    new DateTime(2024, 1, 1),
    new DateTime(2024, 3, 31))
    .ToListAsync();

AllVersions\<TEntity>¶

Returns all versions of entities, bypassing temporal filtering.

public IQueryable<TEntity> AllVersions<TEntity>() where TEntity : class

Returns: IQueryable<TEntity> with all entity versions

Example:

var history = await db.AllVersions<Contract>()
    .Where(c => c.ContractNumber == "CTR-001")
    .OrderBy(c => c.EffectiveDate)
    .ToListAsync();

Configuration API¶

DbContextOptionsBuilder Extensions¶

UseDtde¶

Configures DTDE for the DbContext.

public static DbContextOptionsBuilder UseDtde(
    this DbContextOptionsBuilder optionsBuilder,
    Action<DtdeOptionsBuilder> configureOptions)

Parameters: - configureOptions - Action to configure DTDE options

Returns: The options builder for chaining

Example:

services.AddDbContext<AppDbContext>(options =>
{
    options.UseSqlServer(connectionString);
    options.UseDtde(dtde =>
    {
        dtde.SetMaxParallelShards(10);
        dtde.EnableDiagnostics();
    });
});

UseDtde (with pre-built options)¶

public static DbContextOptionsBuilder UseDtde(
    this DbContextOptionsBuilder optionsBuilder,
    DtdeOptions options)

DtdeOptionsBuilder¶

Builder for configuring DTDE options.

namespace Dtde.EntityFramework.Configuration;

public sealed class DtdeOptionsBuilder

Methods¶

AddShard (Action)¶

Adds a shard using a builder action.

public DtdeOptionsBuilder AddShard(Action<ShardMetadataBuilder> configure)

Example:

dtde.AddShard(s => s
    .WithId("EU")
    .WithShardKeyValue("EU")
    .WithTable("Customers_EU", "dbo")
    .WithTier(ShardTier.Hot));

AddShard (IShardMetadata)¶

Adds a pre-built shard metadata.

public DtdeOptionsBuilder AddShard(IShardMetadata shard)

AddShardsFromConfig¶

Loads shards from a JSON configuration file.

public DtdeOptionsBuilder AddShardsFromConfig(string configPath)

Parameters: - configPath - Path to the JSON configuration file

Example:

dtde.AddShardsFromConfig("shards.json");

JSON Format:

{
  "shards": [
    {
      "shardId": "shard-2024",
      "name": "Year 2024 Data",
      "tableName": "Orders_2024",
      "tier": "Hot",
      "priority": 100,
      "isReadOnly": false,
      "dateRangeStart": "2024-01-01T00:00:00",
      "dateRangeEnd": "2024-12-31T23:59:59"
    }
  ]
}

ConfigureEntity\<TEntity>¶

Configures entity-specific metadata.

public DtdeOptionsBuilder ConfigureEntity<TEntity>(
    Action<EntityMetadataBuilder<TEntity>> configure)
    where TEntity : class

SetMaxParallelShards¶

Sets maximum concurrent shard queries.

public DtdeOptionsBuilder SetMaxParallelShards(int maxParallel)

Parameters: - maxParallel - Maximum concurrent queries (default: 10)

EnableDiagnostics¶

Enables diagnostic logging.

public DtdeOptionsBuilder EnableDiagnostics()

EnableTestMode¶

Enables test mode (single shard, no distribution).

public DtdeOptionsBuilder EnableTestMode()

SetDefaultTemporalContext¶

Sets default temporal context provider.

public DtdeOptionsBuilder SetDefaultTemporalContext(Func<DateTime> provider)

Example:

dtde.SetDefaultTemporalContext(() => DateTime.UtcNow);

Cross-Shard Transactions¶

ICrossShardTransactionCoordinator¶

Coordinates transactions that span multiple database shards.

namespace Dtde.Abstractions.Transactions;

public interface ICrossShardTransactionCoordinator

Properties¶

Property	Type	Description
`CurrentTransaction`	`ICrossShardTransaction?`	The active cross-shard transaction, if any

Methods¶

BeginTransactionAsync¶

Starts a new cross-shard transaction.

Task<ICrossShardTransaction> BeginTransactionAsync(
    CrossShardTransactionOptions? options = null,
    CancellationToken cancellationToken = default)

Parameters: - options - Optional configuration for the transaction - cancellationToken - Cancellation token

Returns: A new cross-shard transaction

Example:

await using var transaction = await coordinator.BeginTransactionAsync();
await transaction.EnlistAsync("shard-eu");
await transaction.EnlistAsync("shard-us");
// Perform operations...
await transaction.CommitAsync();

ExecuteInTransactionAsync¶

Executes an action within a cross-shard transaction with automatic commit/rollback.

Task ExecuteInTransactionAsync(
    Func<ICrossShardTransaction, Task> action,
    CrossShardTransactionOptions? options = null,
    CancellationToken cancellationToken = default)

Parameters: - action - The action to execute within the transaction - options - Optional configuration - cancellationToken - Cancellation token

Example:

await coordinator.ExecuteInTransactionAsync(async tx =>
{
    await tx.EnlistAsync("shard-eu");
    await tx.EnlistAsync("shard-us");

    // Modify data in both shards
    await context.SaveChangesAsync();
});

ExecuteInTransactionAsync\<TResult>¶

Executes a function within a transaction and returns a result.

Task<TResult> ExecuteInTransactionAsync<TResult>(
    Func<ICrossShardTransaction, Task<TResult>> func,
    CrossShardTransactionOptions? options = null,
    CancellationToken cancellationToken = default)

RecoverAsync¶

Recovers in-doubt transactions after a failure.

Task<int> RecoverAsync(CancellationToken cancellationToken = default)

Returns: Number of transactions recovered

ICrossShardTransaction¶

Represents an active cross-shard transaction.

namespace Dtde.Abstractions.Transactions;

public interface ICrossShardTransaction : IAsyncDisposable

Properties¶

Property	Type	Description
`TransactionId`	`string`	Unique transaction identifier
`State`	`TransactionState`	Current transaction state
`IsolationLevel`	`CrossShardIsolationLevel`	Isolation level
`Timeout`	`TimeSpan`	Transaction timeout
`EnlistedShards`	`IReadOnlyCollection<string>`	Enlisted shard IDs

Methods¶

EnlistAsync¶

Enlists a shard in the transaction.

Task EnlistAsync(string shardId, CancellationToken cancellationToken = default)

Parameters: - shardId - The shard ID to enlist

Example:

await transaction.EnlistAsync("shard-eu");

CommitAsync¶

Commits the transaction using two-phase commit.

Task CommitAsync(CancellationToken cancellationToken = default)

RollbackAsync¶

Rolls back the transaction on all enlisted shards.

Task RollbackAsync(CancellationToken cancellationToken = default)

GetParticipant¶

Gets a transaction participant for a specific shard.

ITransactionParticipant? GetParticipant(string shardId)

CrossShardTransactionOptions¶

Configuration options for cross-shard transactions.

namespace Dtde.Abstractions.Transactions;

public class CrossShardTransactionOptions

Properties¶

Property	Type	Default	Description
`Timeout`	`TimeSpan`	30 seconds	Transaction timeout
`IsolationLevel`	`CrossShardIsolationLevel`	`ReadCommitted`	Isolation level
`EnableRetry`	`bool`	`true`	Enable automatic retry
`MaxRetryAttempts`	`int`	`3`	Maximum retry attempts
`RetryDelay`	`TimeSpan`	100ms	Initial retry delay
`UseExponentialBackoff`	`bool`	`true`	Use exponential backoff
`MaxRetryDelay`	`TimeSpan`	10 seconds	Maximum retry delay
`TransactionName`	`string?`	`null`	Optional name for logging
`EnableRecovery`	`bool`	`false`	Enable transaction recovery

Static Presets¶

// Default settings
public static CrossShardTransactionOptions Default { get; }

// Short-lived: 10s timeout, 2 retries, no recovery
public static CrossShardTransactionOptions ShortLived { get; }

// Long-running: 5min timeout, 5 retries, recovery enabled
public static CrossShardTransactionOptions LongRunning { get; }

Example:

var options = new CrossShardTransactionOptions
{
    Timeout = TimeSpan.FromMinutes(2),
    IsolationLevel = CrossShardIsolationLevel.Serializable,
    TransactionName = "FundsTransfer"
};

await coordinator.ExecuteInTransactionAsync(async tx =>
{
    // Operations
}, options);

TransactionState¶

Enum representing transaction states.

public enum TransactionState
{
    Active,       // Transaction is open
    Preparing,    // Two-phase commit in progress
    Committed,    // Successfully committed
    RolledBack,   // Rolled back
    Failed        // Failed during commit
}

CrossShardIsolationLevel¶

Isolation levels for cross-shard transactions.

public enum CrossShardIsolationLevel
{
    ReadUncommitted,
    ReadCommitted,
    RepeatableRead,
    Serializable,
    Snapshot
}

Query Methods¶

IShardedQueryExecutor¶

Interface for executing queries across shards.

namespace Dtde.EntityFramework.Query;

public interface IShardedQueryExecutor

Methods¶

ExecuteAsync\<TEntity>¶

Executes a query across all relevant shards.

Task<IReadOnlyList<TEntity>> ExecuteAsync<TEntity>(
    IQueryable<TEntity> query,
    CancellationToken cancellationToken = default)
    where TEntity : class

Parameters: - query - The LINQ query to execute - cancellationToken - Cancellation token

Returns: Combined results from all shards

ExecuteScalarAsync\<TEntity, TResult>¶

Executes a scalar aggregation across shards.

Task<TResult> ExecuteScalarAsync<TEntity, TResult>(
    IQueryable<TEntity> query,
    Func<IEnumerable<TResult>, TResult> aggregator,
    CancellationToken cancellationToken = default)
    where TEntity : class

Parameters: - query - The LINQ query - aggregator - Function to aggregate shard results - cancellationToken - Cancellation token

Entity Configuration¶

EntityTypeBuilder Extensions¶

Extensions for configuring sharding on entities.

ShardBy¶

Configures property-based sharding.

public static ShardingConfigurationBuilder<TEntity> ShardBy<TEntity, TProperty>(
    this EntityTypeBuilder<TEntity> builder,
    Expression<Func<TEntity, TProperty>> propertyExpression)
    where TEntity : class

Example:

modelBuilder.Entity<Customer>(entity =>
{
    entity.ShardBy(c => c.Region);
});

ShardByHash¶

Configures hash-based sharding.

public static ShardingConfigurationBuilder<TEntity> ShardByHash<TEntity, TProperty>(
    this EntityTypeBuilder<TEntity> builder,
    Expression<Func<TEntity, TProperty>> propertyExpression,
    int shardCount)
    where TEntity : class

Parameters: - propertyExpression - Property to hash - shardCount - Number of hash buckets/shards

Example:

entity.ShardByHash(c => c.Id, shardCount: 8);

ShardByDate¶

Configures date-based sharding.

public static ShardingConfigurationBuilder<TEntity> ShardByDate<TEntity>(
    this EntityTypeBuilder<TEntity> builder,
    Expression<Func<TEntity, DateTime>> dateProperty,
    DateShardInterval interval = DateShardInterval.Year)
    where TEntity : class

Parameters: - dateProperty - DateTime property to shard by - interval - Date interval (Year, Quarter, Month, Day)

Example:

entity.ShardByDate(o => o.CreatedAt, DateShardInterval.Year);

HasTemporalValidity¶

Configures temporal validity for an entity.

public static TemporalConfigurationBuilder<TEntity> HasTemporalValidity<TEntity>(
    this EntityTypeBuilder<TEntity> builder,
    Expression<Func<TEntity, DateTime>> validFrom,
    Expression<Func<TEntity, DateTime?>> validTo)
    where TEntity : class

Parameters: - validFrom - Expression selecting the valid-from property - validTo - Expression selecting the valid-to property (nullable)

Example:

entity.HasTemporalValidity(
    validFrom: c => c.EffectiveDate,
    validTo: c => c.ExpirationDate);

ShardingConfigurationBuilder¶

Builder for sharding configuration.

WithStorageMode¶

Sets the storage mode for shards.

public ShardingConfigurationBuilder<TEntity> WithStorageMode(ShardStorageMode mode)

Parameters: - mode - ShardStorageMode.Tables or ShardStorageMode.Databases

TemporalConfigurationBuilder¶

Builder for temporal configuration.

WithVersioningMode¶

Sets the versioning behavior.

public TemporalConfigurationBuilder<TEntity> WithVersioningMode(VersioningMode mode)

Parameters: - mode - SoftVersion, AuditTrail, or AppendOnly

WithHistoryTable¶

Configures a separate history table (for AuditTrail mode).

public TemporalConfigurationBuilder<TEntity> WithHistoryTable(string tableName)

WithOpenEndedValue¶

Sets the value used for open-ended records.

public TemporalConfigurationBuilder<TEntity> WithOpenEndedValue(DateTime value)

Shard Configuration¶

ShardMetadataBuilder¶

Builder for creating shard metadata.

namespace Dtde.Core.Metadata;

public class ShardMetadataBuilder

Methods¶

WithId¶

Sets the unique shard identifier.

public ShardMetadataBuilder WithId(string shardId)

WithName¶

Sets the display name.

public ShardMetadataBuilder WithName(string name)

WithShardKeyValue¶

Sets the shard key value this shard handles.

public ShardMetadataBuilder WithShardKeyValue(string keyValue)

WithTable¶

Configures table-based sharding.

public ShardMetadataBuilder WithTable(string tableName, string schemaName = "dbo")

WithConnectionString¶

Configures database-based sharding.

public ShardMetadataBuilder WithConnectionString(string connectionString)

WithDateRange¶

Sets the date range this shard covers.

public ShardMetadataBuilder WithDateRange(DateTime start, DateTime end)

WithTier¶

Sets the storage tier.

public ShardMetadataBuilder WithTier(ShardTier tier)

Values: - ShardTier.Hot - Active, fast storage - ShardTier.Warm - Less active data - ShardTier.Cold - Archived data - ShardTier.Archive - Long-term storage

WithPriority¶

Sets query priority (higher = queried first).

public ShardMetadataBuilder WithPriority(int priority)

AsReadOnly¶

Marks the shard as read-only.

public ShardMetadataBuilder AsReadOnly()

Build¶

Creates the shard metadata.

public IShardMetadata Build()

Example:

var shard = new ShardMetadataBuilder()
    .WithId("archive-2020")
    .WithName("2020 Archive")
    .WithTable("Orders_2020", "archive")
    .WithDateRange(new DateTime(2020, 1, 1), new DateTime(2020, 12, 31))
    .WithTier(ShardTier.Archive)
    .AsReadOnly()
    .WithPriority(10)
    .Build();

ShardMetadata Static Methods¶

Factory methods for common scenarios.

ForTable¶

Creates a table-based shard.

public static ShardMetadata ForTable(
    string shardId,
    string tableName,
    string? shardKeyValue = null,
    string schemaName = "dbo")

ForDatabase¶

Creates a database-based shard.

public static ShardMetadata ForDatabase(
    string shardId,
    string name,
    string connectionString,
    string? shardKeyValue = null)

Enumerations¶

ShardStorageMode¶

public enum ShardStorageMode
{
    Tables,     // Multiple tables in same database
    Databases   // Separate databases
}

ShardTier¶

public enum ShardTier
{
    Hot,        // Active data, fast storage
    Warm,       // Less active data
    Cold,       // Archived data
    Archive     // Long-term storage
}

DateShardInterval¶

public enum DateShardInterval
{
    Year,
    Month,
    Quarter,
    Week
}

VersioningMode¶

public enum VersioningMode
{
    SoftVersion,    // Close old, create new
    AuditTrail,     // Copy to history, update current
    AppendOnly      // Never update, always insert
}

Next Steps¶

Configuration - Detailed configuration options
Classes Reference - Complete class documentation
Troubleshooting - Common issues and solutions

← Back to Wiki | Configuration →