Ai.Tlbx.Inference 1.5.0

Ai.Tlbx.Inference

Universal .NET AI inference client for OpenAI, Anthropic, Google, and xAI.

Features

• Multi-provider — OpenAI, Anthropic, Google (AI Studio + Vertex), xAI behind a single interface
• Streaming — IAsyncEnumerable<string> with zero-alloc delta-only design
• Structured output — CompleteAsync<T>() with automatic JSON schema inference
• Tool calling — unified tool loop with streaming support, lives once in the facade
• Embeddings — OpenAI and Google embedding models with batch support
• Image generation — Google Gemini image generation
• Token metering — TokenUsage on every response including cache and thinking tokens
• Resilience — Polly v8 retry with exponential backoff, jitter, and Retry-After respect
• Thinking budget — universal mapping across all providers that support reasoning

Supported Models

OpenAI

Anthropic

Google

xAI

Embeddings

Installation

dotnet add package Ai.Tlbx.Inference

Quick Start

DI Registration

services.AddAiInference(options =>
{
    options.AddOpenAi("sk-...");
    options.AddAnthropic("sk-ant-...");
    options.AddGoogle("AIza...");
    options.AddXai("xai-...");
});

Simple Completion

var response = await client.CompleteAsync(new CompletionRequest
{
    Model = AiModel.ClaudeOpus46,
    Messages = [new ChatMessage { Role = ChatRole.User, Content = "Hello!" }]
});

Console.WriteLine(response.Content);
Console.WriteLine($"Tokens: {response.Usage.TotalTokens}");

Streaming

await foreach (var delta in client.StreamAsync(new CompletionRequest
{
    Model = AiModel.Gpt52,
    Messages = [new ChatMessage { Role = ChatRole.User, Content = "Write a haiku" }]
}))
{
    Console.Write(delta);
}

Structured Output

public sealed record WeatherInfo
{
    public required string City { get; init; }
    public required double Temperature { get; init; }
    public required string Condition { get; init; }
}

var response = await client.CompleteAsync<WeatherInfo>(new CompletionRequest
{
    Model = AiModel.Gemini25Pro,
    Messages = [new ChatMessage { Role = ChatRole.User, Content = "Weather in Berlin?" }]
});

Console.WriteLine($"{response.Content.City}: {response.Content.Temperature}°C, {response.Content.Condition}");

Tool Calling

var tools = new List<ToolDefinition>
{
    new()
    {
        Name = "get_weather",
        Description = "Get current weather for a city",
        ParametersSchema = JsonSchemaGenerator.Generate<WeatherParams>()
    }
};

var result = await client.CompleteWithToolsAsync<string>(
    new CompletionRequest
    {
        Model = AiModel.ClaudeSonnet45,
        Messages = [new ChatMessage { Role = ChatRole.User, Content = "What's the weather in Tokyo?" }]
    },
    tools,
    toolExecutor: async call =>
    {
        var weather = GetWeather(call.Arguments);
        return new ToolCallResult
        {
            ToolCallId = call.Id,
            Result = JsonSerializer.Serialize(weather)
        };
    });

Console.WriteLine(result.Content);
Console.WriteLine($"Tool iterations: {result.Iterations}, Total tokens: {result.Usage.TotalTokens}");

Embeddings

var embedding = await client.EmbedAsync(new EmbeddingRequest
{
    Model = EmbeddingModel.TextEmbedding3Large,
    Input = "The quick brown fox"
});

Console.WriteLine($"Dimensions: {embedding.Embedding.Length}");

Configuration

Logging

services.AddAiInference(options =>
{
    options.AddOpenAi("sk-...");
    options.WithLogging((level, message) =>
    {
        Console.WriteLine($"[{level}] {message}");
    });
});

Custom Retry Policy

var customPipeline = new ResiliencePipelineBuilder<HttpResponseMessage>()
    .AddRetry(new RetryStrategyOptions<HttpResponseMessage>
    {
        MaxRetryAttempts = 2,
        Delay = TimeSpan.FromSeconds(5)
    })
    .Build();

services.AddAiInference(options =>
{
    options.AddOpenAi("sk-...");
    options.WithRetryPolicy(customPipeline);
});

Google Vertex AI

services.AddAiInference(options =>
{
    options.AddGoogle(
        serviceAccountJson: File.ReadAllText("service-account.json"),
        projectId: "my-project",
        location: "us-central1");
});

Thinking Budget

var response = await client.CompleteAsync(new CompletionRequest
{
    Model = AiModel.ClaudeOpus46,
    ThinkingBudget = 10000,
    Messages = [new ChatMessage { Role = ChatRole.User, Content = "Solve this complex problem..." }]
});

Console.WriteLine($"Thinking tokens used: {response.Usage.ThinkingTokens}");

Prompt Caching (Anthropic)

var response = await client.CompleteAsync(new CompletionRequest
{
    Model = AiModel.ClaudeSonnet45,
    EnableCache = true,
    SystemMessage = longSystemPrompt,
    Messages = [new ChatMessage { Role = ChatRole.User, Content = "Question..." }]
});

Console.WriteLine($"Cache read: {response.Usage.CacheReadTokens}, Cache write: {response.Usage.CacheWriteTokens}");

AOT / Trimming

The library is AOT and trimming compatible (IsAotCompatible, IsTrimmable).

For structured output and tool calling with AOT, use the JsonTypeInfo<T> overloads and provide your schema explicitly:

[JsonSerializable(typeof(WeatherInfo))]
internal partial class MyJsonContext : JsonSerializerContext { }

var response = await client.CompleteAsync(
    new CompletionRequest
    {
        Model = AiModel.Gemini25Pro,
        JsonSchema = """{"type":"object","properties":{"city":{"type":"string"},"temp":{"type":"number"}},"required":["city","temp"]}""",
        Messages = [new ChatMessage { Role = ChatRole.User, Content = "Weather in Berlin?" }]
    },
    MyJsonContext.Default.WeatherInfo);

The non-generic methods (CompleteAsync, StreamAsync, EmbedAsync, etc.) are always AOT-safe.

License

MIT