AI

Custom AI Providers¶

A custom AI provider is a Java plugin that lets 3forge Enterprise reach a model backend the built-in providers do not cover.

Overview¶

The built-in AI providers (anthropic, openai, gemini, xai) cover the major hosted model APIs. When you need to reach an internal LLM gateway, a locally-hosted model, or a bespoke wire format, you can write your own provider and register it exactly like a built-in.

A custom provider makes that a first-class option in the Ask AI chatbot and in session.promptAi.

Java Interface¶

Implement com.f1.ami.amicommon.ai.AmiAiProviderPlugin. It extends Enterprise's AmiPlugin, so the plugin framework handles discovery and instantiation.

Note

The interface and every request/response type it deals in (AmiAiRequest, AmiAiResponse, AmiAiMessage, AmiAiStreamHandler, AmiAiCancelToken, …) all live in the shared amicommon module. A provider jar can be built against amicommon alone, with no other Enterprise internals needed on the compile path.

package com.example.ami;

import com.f1.ami.amicommon.AmiPluginAttribute;
import com.f1.ami.amicommon.ai.AmiAiProviderPlugin;
import com.f1.ami.amicommon.ai.AmiAiRequest;
import com.f1.ami.amicommon.ai.AmiAiResponse;
import com.f1.ami.amicommon.ai.AmiAiStreamHandler;
import com.f1.ami.amicommon.ai.AmiAiCancelToken;
import com.f1.container.ContainerTools;
import com.f1.utils.PropertyController;

@AmiPluginAttribute(id = "mygateway", description = "Internal LLM gateway")
public class MyGatewayProvider implements AmiAiProviderPlugin {

    private String endpoint, model;

    // Identity. Must equal the id used in ami.web.ai.providers and the
    // ami.web.ai.<id>.* config namespace, or startup fails.
    @Override 
    public String getPluginId()   { return "mygateway"; }

    // Human-readable name shown in the UI.
    @Override 
    public String getDisplayName() { return "My Gateway"; }

    // Startup hook. Read this provider's own ami.web.ai.mygateway.* keys.
    @Override 
    public void init(ContainerTools tools, PropertyController props) {
        this.endpoint = props.getRequired("ami.web.ai.mygateway.endpoint");
        this.model    = AmiAiProviderPlugin.firstModel(props.get("ami.web.ai.mygateway.model"));
        // No api_key required: this gateway authenticates by URL / network position.
    }

    // Synchronous one-shot completion (backs one-shot calls such as session.promptAi).
    @Override 
    public AmiAiResponse complete(AmiAiRequest request) {
        // ... call the gateway, translate its response into AmiAiResponse ...
    }

    // Streaming completion (backs the chat window). Delegate to the cancellable form.
    @Override 
    public void stream(AmiAiRequest request, AmiAiStreamHandler handler) {
        stream(request, handler, null);
    }

    @Override 
    public void stream(AmiAiRequest request, AmiAiStreamHandler handler, AmiAiCancelToken cancelToken) {
        // ... open the connection (register it on cancelToken if provided),
        //     read the stream, and call handler.onToken / onToolUse / onComplete ...
    }

    // Context-window size, used to decide when history compaction kicks in.
    // Keep it conservative so an unconfigured model compacts a little early
    // rather than overflowing the real window.
    @Override 
    public int getContextWindowTokens() { return 128_000; }
}

Key Points¶

getPluginId() is the identity. It must equal the id used in ami.web.ai.providers and in the ami.web.ai.<id>.* property namespace. Startup fails if they disagree.
No API key is required. Registration comes purely from listing the provider (see below). Your init() reads whatever configuration it needs; a gateway that authenticates by URL or network position simply never reads an api_key.
Configuration is read in init() from absolute ami.web.ai.<id>.* keys via the supplied PropertyController.
complete and stream are the only calls made. stream backs the chat window; complete backs one-shot calls such as session.promptAi. Implement both.

Request and Response Types¶

Everything a provider deals in lives in com.f1.ami.amicommon.ai. All of these types are plain immutable value objects (constructor + getters). A provider's job is to translate between them and its backend's wire format.

The Request: `AmiAiRequest`¶

Passed to both complete and stream. It carries the full state the model needs for one call:

Field	Type	Meaning
`getModel()`	`String`	Model id to call (already resolved from the provider's model list)
`getSystemPrompt()`	`String`	System prompt. Sent as the provider's system field, not as a message
`getMessages()`	`List<AmiAiMessage>`	The conversation so far, oldest first
`getTools()`	`List<AmiAiToolDefinition>`	Tools the model may call. Empty for a plain completion (e.g: `session.promptAi`)
`getMaxTokens()`	`int`	Upper bound on tokens to generate for this call

`AmiAiMessage`: A Conversation Turn¶

Roles are "user" and "assistant". A message is one of 3 shapes (check with isToolUseTurn() / isToolResultTurn()):

Variant	How to build	Carries
Text turn	`new AmiAiMessage(role, content)`	`getContent()`
Assistant tool-use turn	`AmiAiMessage.toolUseTurn(list)`	`getToolUses()`
User tool-result turn	`AmiAiMessage.toolResultTurn(list)`	`getToolResults()`

Map these onto your backend's message model: text turns become normal user/assistant messages, tool-use turns become the assistant's function-call blocks, and tool-result turns become the function-result blocks fed back on the next call.

`AmiAiToolDefinition`: A Tool the Model May Call¶

Field	Type	Meaning
`getName()`	`String`	Tool name
`getDescription()`	`String`	What the tool does (shown to the model)
`getInputSchema()`	`Map<String,Object>`	JSON Schema object describing the tool's parameters

Translate each into your backend's tool/function-declaration format. If your backend does not support tools, you can ignore this list, but then the agentic chatbot cannot call tools through your provider.

The Synchronous Result: `AmiAiResponse`¶

What complete(...) must return:

new AmiAiResponse(stopReason, text, toolUses, inputTokens, outputTokens);

Field	Type	Meaning
`getStopReason()`	`AmiAiStopReason`	Why generation stopped (see below)
`getText()`	`String`	The assistant's text output (may be empty if it only called tools)
`getToolUses()`	`List<AmiAiToolUse>`	Tool calls the model requested (empty if none)
`getInputTokens()`	`int`	Prompt tokens billed
`getOutputTokens()`	`int`	Completion tokens billed

The Streaming Contract: `AmiAiStreamHandler`¶

What stream(...) drives. As data arrives from the backend, call these callbacks on the calling I/O thread:

Callback	When to call it
`onToken(String delta)`	Each text delta as it streams in
`onToolUse(AmiAiToolUse toolUse)`	Once per complete tool-use block the model emits
`onComplete(AmiAiStopReason reason, AmiAiTokenStats stats)`	Exactly once, when the turn finishes successfully
`onError(Exception e)`	If the API call fails or the stream breaks

The contract: stream any number of onToken / onToolUse calls, then terminate the turn with exactly one onComplete or one onError, never both, and never neither. stats may be AmiAiTokenStats.ZERO if your backend does not report usage.

`AmiAiToolUse` and `AmiAiToolResult`¶

A tool call requested by the model, and the result you feed back next turn:

`AmiAiToolUse`		`AmiAiToolResult`
`getId()`	call id	`getToolUseId()`	id of the matching tool-use
`getName()`	tool name	`getToolName()`	tool name (Gemini needs it)
`getInput()`	`Map<String,Object>` args	`getContent()`	result text
		`isError()`	whether it is an error result

`AmiAiStopReason`¶

An enum: TOOL_USE (model wants to call tools), END_TURN (complete response), MAX_TOKENS (hit the limit), ERROR, CANCELLED (aborted mid-stream), MAX_THINK. A provider normally returns TOOL_USE, END_TURN, or MAX_TOKENS.

`AmiAiTokenStats`¶

Per-call token usage reported on onComplete. The 4 fields are disjoint and sum to the billable total:

Field	Meaning
`getInputTokens()`	Non-cached prompt tokens
`getOutputTokens()`	Completion tokens
`getCacheReadTokens()`	Prompt tokens served from cache
`getCacheWriteTokens()`	Prompt tokens written to cache (Anthropic only; others `0`)

Subtract any cached portion from the raw input count before populating this, so the fields stay disjoint. Use AmiAiTokenStats.ZERO when usage is unavailable.

Cancellation: `AmiAiCancelToken`¶

The cancellable stream(request, handler, cancelToken) overload receives a token so the chat's stop button can abort a live stream. As soon as you open the HTTP connection, register it:

HttpURLConnection conn = ...;
if (cancelToken != null) cancelToken.registerConnection(conn);

Another thread calling cancelToken.cancel() then disconnects the socket, which unblocks your stream read at EOF. Check cancelToken.isCancelled() to distinguish an intentional cancel from a real error (suppress the expected read failure and finish with onComplete(AmiAiStopReason.CANCELLED, stats)).

Streaming Skeleton¶

@Override
public void stream(AmiAiRequest request, AmiAiStreamHandler handler, AmiAiCancelToken cancelToken) {
    try {
        HttpURLConnection conn = openConnection(request);   // build request from getModel(), getSystemPrompt(), getMessages(), getTools()
        if (cancelToken != null) cancelToken.registerConnection(conn);

        int in = 0, out = 0;
        for (Event e : readServerSentEvents(conn)) {         // your backend's stream parsing
            if (e.isTextDelta())  handler.onToken(e.text());
            if (e.isToolCall())   handler.onToolUse(new AmiAiToolUse(e.id(), e.name(), e.args()));
            if (e.hasUsage())   { in = e.inputTokens(); out = e.outputTokens(); }
        }
        handler.onComplete(AmiAiStopReason.END_TURN, new AmiAiTokenStats(in, out, 0, 0));
    } catch (Exception e) {
        if (cancelToken != null && cancelToken.isCancelled())
            handler.onComplete(AmiAiStopReason.CANCELLED, AmiAiTokenStats.ZERO);
        else
            handler.onError(e);
    }
}

Registering the Provider¶

Package the class as a .jar, place it on the Web component's classpath (or in a plugin bundle), and add its id, or its fully-qualified class name, to ami.web.ai.providers in the Web component's local.properties. Then supply whatever ami.web.ai.<id>.* configuration your init() reads:

ami.web.ai.enabled            = true
ami.web.ai.providers          = mygateway
ami.web.ai.mygateway.endpoint = https://llm-gw.internal.example.com/v1
ami.web.ai.mygateway.model    = internal-model-name

A custom provider can be mixed with built-ins, and the first entry is the default:

ami.web.ai.providers = mygateway, anthropic

Note

Referencing the provider by fully-qualified class name works too, which is useful when you would rather not rely on classpath discovery of the @AmiPluginAttribute id:

ami.web.ai.providers = com.example.ami.MyGatewayProvider

See Ask AI Chatbot for how the provider list and per-provider properties are consumed.

Custom AI Providers¶

Overview¶

Java Interface¶

Key Points¶

Request and Response Types¶

The Request: AmiAiRequest¶

AmiAiMessage: A Conversation Turn¶

AmiAiToolDefinition: A Tool the Model May Call¶

The Synchronous Result: AmiAiResponse¶

The Streaming Contract: AmiAiStreamHandler¶

AmiAiToolUse and AmiAiToolResult¶

AmiAiStopReason¶

AmiAiTokenStats¶

Cancellation: AmiAiCancelToken¶