Package com.vaadin.flow.component.ai.provider

Class SpringAILLMProvider

java.lang.Object
com.vaadin.flow.component.ai.provider.SpringAILLMProvider
All Implemented Interfaces:
LLMProvider
public class SpringAILLMProvider extends Object implements LLMProvider
Spring AI implementation of LLMProvider.

Supports both streaming and non-streaming Spring AI models. Tool calling is supported through Spring AI's Tool annotation.

Streaming vs. non-streaming: Streaming is enabled by default. To disable it, call setStreaming(false). Streaming mode pushes partial responses to the UI as they arrive, which requires automatic server push or polling to deliver them. Annotate your UI class or application shell with @Push, or enable polling with UI.setPollInterval(), before using streaming mode. A warning is logged at runtime when neither is active.

Blocking the request thread: in non-streaming mode the LLM call blocks the thread that subscribes to the response, which is the UI thread for a prompt triggered from the browser. Call setBackgroundExecution(true) to run the call on a background thread instead, so the request completes and the user's message renders while the LLM works.

Each provider instance maintains its own chat memory. To share conversation history across components, reuse the same provider instance. History restoration (via setHistory(List, Map)) is only supported when using the SpringAILLMProvider(ChatModel) constructor; the SpringAILLMProvider(ChatClient) constructor does not provide access to the internal chat memory.

Note: SpringAILLMProvider is not serializable. If your application uses session persistence, you will need to create a new provider instance after session restore.

Since:
25.1
Author:
Vaadin Ltd

  • Constructor Details

    • SpringAILLMProvider

      public SpringAILLMProvider(org.springframework.ai.chat.model.ChatModel chatModel)
      Constructor with a chat model.
      Parameters:
      chatModel - the chat model, not null
      Throws:
      NullPointerException - if chatModel is null

    • SpringAILLMProvider

      public SpringAILLMProvider(org.springframework.ai.chat.client.ChatClient chatClient)
      Constructor with a chat client. Note: When using this constructor, conversation memory must be configured externally in the ChatClient.
      Parameters:
      chatClient - the chat client, not null
      Throws:
      NullPointerException - if chatClient is null

  • Method Details

    • stream

      public reactor.core.publisher.Flux<String> stream(LLMProvider.LLMRequest request)
      Description copied from interface: LLMProvider
      Streams a response from the LLM based on the provided request. This method returns a reactive stream that emits response tokens as they become available from the LLM. The provider manages conversation history internally, so each call to this method adds to the ongoing conversation context.

      Threading: this method is called on the thread that triggers the prompt, and the returned stream is subscribed to on that same thread. An implementation whose LLM call blocks must therefore schedule that call itself — for example with subscribeOn(Schedulers.boundedElastic()) — otherwise it occupies the UI thread and holds the session lock for the whole turn, and nothing the turn produces reaches the browser until it ends. The built-in providers expose this as a setBackgroundExecution(boolean) setting, since running the turn on the request thread is the simpler default when a turn is short.

      Callers only consume the stream and do not schedule it, so whether a turn runs in the background is decided entirely by the implementation.

      Specified by:
      stream in interface LLMProvider
      Parameters:
      request - the LLM request containing user message, system prompt, attachments, and tools, not null
      Returns:
      a Flux stream that emits response tokens as strings, never null

    • isStreaming

      public boolean isStreaming()
      Gets whether streaming mode is used.
      Returns:
      true if streaming mode is used, false otherwise
      Since:
      25.3

    • setStreaming

      public void setStreaming(boolean streaming)
      Sets whether to use streaming mode. The default is true.
      Parameters:
      streaming - true to use streaming mode, false for non-streaming.

    • isBackgroundExecution

      public boolean isBackgroundExecution()
      Gets whether the LLM call runs on a background thread.
      Returns:
      true if the call runs on a background thread, false if it runs on the thread that asks for the response
      Since:
      25.3

    • setBackgroundExecution

      public void setBackgroundExecution(boolean backgroundExecution)
      Sets whether to run the LLM call on a background thread. The default is false, which runs it on the thread that asks for the response — the UI thread, for a prompt triggered from the browser. The setting has no effect in streaming mode, where the response already arrives on the LLM client's own threads.

      A non-streaming call blocks for the whole turn, every tool call included. On the UI thread that means holding the session lock until the turn ends, so nothing the turn produces reaches the browser and the application appears frozen. Set this to true to run the call on a background thread instead: the request completes immediately, the user's message and the assistant placeholder render, and the response is added when it arrives.

      This requires three things from the application:

      • A way to deliver the response. Annotate the application shell or UI class with @Push, or enable polling with UI.setPollInterval(int). Manual push mode is not enough on its own, because nothing calls ui.push() for you. A warning is logged when neither is active.
      • Thread-safe tools. On a background thread Vaadin thread locals such as UI.getCurrent() and framework contexts such as Spring Security's SecurityContext are not bound, and UI components must not be accessed directly. Wrap component access in ui.access(), or capture what you need in AIController.onRequest(), which still runs on the UI thread. This is the same requirement streaming mode already has.
      • A gated input. The orchestrator processes one prompt at a time. Without background execution, a message submitted while a turn is running waits for the session lock and is processed when the turn ends; with it, the submit is rejected and dropped with a warning — and a connected input has already cleared its text. Disable the input while a turn is running, for example from AIController.onRequest() and AIController.onResponse(Throwable).

      Like the streaming mode, the setting is not preserved when the session is serialized: an application that restores sessions must re-apply it when it recreates the provider.

      Parameters:
      backgroundExecution - true to run the call on a background thread, false to run it on the thread that asks for the response
      Since:
      25.3

    • setHistory

      public void setHistory(List<ChatMessage> history, Map<String,List<AIAttachment>> attachmentsByMessageId)
      Description copied from interface: LLMProvider
      Restores the provider's conversation memory from a list of chat messages with their associated attachments. Any existing memory is cleared before the new history is applied.

      Providers that support setting chat history should override this method.

      This method must not be called while a streaming response is in progress.

      Specified by:
      setHistory in interface LLMProvider
      Parameters:
      history - the list of chat messages to restore, not null
      attachmentsByMessageId - a map from ChatMessage.messageId() to the list of attachments for that message, not null