Package com.vaadin.flow.component.ai.provider

Class LangChain4JLLMProvider

java.lang.Object
com.vaadin.flow.component.ai.provider.LangChain4JLLMProvider
All Implemented Interfaces:
LLMProvider
public class LangChain4JLLMProvider extends Object implements LLMProvider
LangChain4j implementation of LLMProvider.

Supports both streaming and non-streaming LangChain4j models. Tool calling is supported through LangChain4j's Tool annotation.

Streaming vs. non-streaming: The mode is determined by the constructor used. Pass a StreamingChatModel to LangChain4JLLMProvider(StreamingChatModel) for streaming, or a ChatModel to LangChain4JLLMProvider(ChatModel) for non-streaming. Streaming mode pushes partial responses to the UI as they arrive, which requires automatic server push or polling to deliver them. Annotate your UI class or application shell with @Push, or enable polling with UI.setPollInterval(), before using a streaming model. A warning is logged at runtime when neither is active.

Blocking the request thread: a ChatModel call blocks the thread that subscribes to the response, which is the UI thread for a prompt triggered from the browser. Call setBackgroundExecution(true) to run the call on a background thread instead, so the request completes and the user's message renders while the LLM works.

Each provider instance maintains its own chat memory. To share conversation history across components, reuse the same provider instance.

Note: LangChain4JLLMProvider is not serializable. If your application uses session persistence, you will need to create a new provider instance after session restore.

Since:
25.1
Author:
Vaadin Ltd

  • Constructor Details

    • LangChain4JLLMProvider

      public LangChain4JLLMProvider(dev.langchain4j.model.chat.StreamingChatModel chatModel)
      Constructor with a streaming chat model.
      Parameters:
      chatModel - the streaming chat model, not null
      Throws:
      NullPointerException - if chatModel is null

    • LangChain4JLLMProvider

      public LangChain4JLLMProvider(dev.langchain4j.model.chat.ChatModel chatModel)
      Constructor with a non-streaming chat model.
      Parameters:
      chatModel - the non-streaming chat model, not null
      Throws:
      NullPointerException - if chatModel is null

  • Method Details

    • stream

      public reactor.core.publisher.Flux<String> stream(LLMProvider.LLMRequest request)
      Description copied from interface: LLMProvider
      Streams a response from the LLM based on the provided request. This method returns a reactive stream that emits response tokens as they become available from the LLM. The provider manages conversation history internally, so each call to this method adds to the ongoing conversation context.

      Threading: this method is called on the thread that triggers the prompt, and the returned stream is subscribed to on that same thread. An implementation whose LLM call blocks must therefore schedule that call itself — for example with subscribeOn(Schedulers.boundedElastic()) — otherwise it occupies the UI thread and holds the session lock for the whole turn, and nothing the turn produces reaches the browser until it ends. The built-in providers expose this as a setBackgroundExecution(boolean) setting, since running the turn on the request thread is the simpler default when a turn is short.

      Callers only consume the stream and do not schedule it, so whether a turn runs in the background is decided entirely by the implementation.

      Specified by:
      stream in interface LLMProvider
      Parameters:
      request - the LLM request containing user message, system prompt, attachments, and tools, not null
      Returns:
      a Flux stream that emits response tokens as strings, never null

    • isBackgroundExecution

      public boolean isBackgroundExecution()
      Gets whether the LLM call runs on a background thread.
      Returns:
      true if the call runs on a background thread, false if it runs on the thread that asks for the response
      Since:
      25.3

    • setBackgroundExecution

      public void setBackgroundExecution(boolean backgroundExecution)
      Sets whether to run the LLM call on a background thread. The default is false, which runs it on the thread that asks for the response — the UI thread, for a prompt triggered from the browser. The setting has no effect with a StreamingChatModel, whose response already arrives on the LLM client's own threads.

      A ChatModel call blocks for the whole turn, every tool call included. On the UI thread that means holding the session lock until the turn ends, so nothing the turn produces reaches the browser and the application appears frozen. Set this to true to run the call on a background thread instead: the request completes immediately, the user's message and the assistant placeholder render, and the response is added when it arrives.

      This requires three things from the application:

      • A way to deliver the response. Annotate the application shell or UI class with @Push, or enable polling with UI.setPollInterval(int). Manual push mode is not enough on its own, because nothing calls ui.push() for you. A warning is logged when neither is active.
      • Thread-safe tools. On a background thread Vaadin thread locals such as UI.getCurrent() and framework contexts such as Spring Security's SecurityContext are not bound, and UI components must not be accessed directly. Wrap component access in ui.access(), or capture what you need in AIController.onRequest(), which still runs on the UI thread. This is the same requirement a StreamingChatModel already has.
      • A gated input. The orchestrator processes one prompt at a time. Without background execution, a message submitted while a turn is running waits for the session lock and is processed when the turn ends; with it, the submit is rejected and dropped with a warning — and a connected input has already cleared its text. Disable the input while a turn is running, for example from AIController.onRequest() and AIController.onResponse(Throwable).

      Like the streaming mode, the setting is not preserved when the session is serialized: an application that restores sessions must re-apply it when it recreates the provider.

      Parameters:
      backgroundExecution - true to run the call on a background thread, false to run it on the thread that asks for the response
      Since:
      25.3

    • setHistory

      public void setHistory(List<ChatMessage> history, Map<String,List<AIAttachment>> attachmentsByMessageId)
      Description copied from interface: LLMProvider
      Restores the provider's conversation memory from a list of chat messages with their associated attachments. Any existing memory is cleared before the new history is applied.

      Providers that support setting chat history should override this method.

      This method must not be called while a streaming response is in progress.

      Specified by:
      setHistory in interface LLMProvider
      Parameters:
      history - the list of chat messages to restore, not null
      attachmentsByMessageId - a map from ChatMessage.messageId() to the list of attachments for that message, not null