System Prompts, Streaming & Structured Output with ChatClient

System Prompts: A Standing Instruction

Every call in the Intermediate level sent only a user message — whatever persona or behavior the model adopted was entirely up to the model’s own defaults. .system(...) sets a separate, standing instruction that applies to the whole conversation, alongside the per-request user message:

String answer = chatClient.prompt()
    .system("You are a friendly chat bot that answers questions in the voice of a pirate.")
    .user("Tell me a joke")
    .call()
    .content();

If every request through a given ChatClient should share the same persona, that instruction can also be set once, at build time, instead of on every individual call — via defaultSystem(...) on the builder itself:

@Bean
ChatClient chatClient(ChatClient.Builder builder) {
    return builder
        .defaultSystem("You are a friendly chat bot that answers questions in the voice of a pirate.")
        .build();
}

// Every call through this bean now inherits that system
// instruction automatically — callers only ever set .user(...).

Streaming: Flux<String> Instead of a Single String

.call() waits for the model to finish generating the entire response before your code ever sees it — fine for a short reply, less fine for a long one where a user is left staring at nothing for several seconds. .stream() swaps that wait for incremental delivery, returning a reactor.core.publisher.Flux<String> — Project Reactor’s reactive stream type, the same reactive-programming primitive WebClientuses elsewhere in the Spring ecosystem — that emits chunks of the reply as they’re generated:

@GetMapping(value = "/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> chatStream(@RequestParam String message) {
    return chatClient.prompt()
        .user(message)
        .stream()
        .content();
}

Everything before .stream() is identical to the synchronous version — .prompt(), then .user(message), exactly as before. Only the execution mode and the return type change. If richer information than plain text is needed per chunk — token usage, finish reason — a parallel .chatResponse() call in place of .content() returns a Flux<ChatResponse> instead, carrying that metadata alongside each chunk.

Structured Output: Mapping a Reply Into a Java Record

Every example so far treats the model’s reply as text a human reads. Plenty of real use cases actually want structured data your own code can work with directly — a list, a set of fields — without hand-parsing a string the model happened to produce. .entity(...), called in place of .content(), does exactly that — it maps the response directly into a Java type you supply:

public record ActorFilms(String actor, List<String> movies) {}

ActorFilms actorFilms = chatClient.prompt()
    .user("Generate a short filmography for a random actor.")
    .call()
    .entity(ActorFilms.class);

// actorFilms.actor() and actorFilms.movies() are now real Java
// values — no manual JSON parsing of the model's raw text.

A generic collection works too, via ParameterizedTypeReference (needed because Java erases a bare List<ActorFilms> at runtime):

List<ActorFilms> filmographies = chatClient.prompt()
    .user("Generate the filmography of 3 movies each for two different actors.")
    .call()
    .entity(new ParameterizedTypeReference<List<ActorFilms>>() {});

Underneath, this works by giving the model formatting instructions derived from the target type’s shape, then parsing its response back into that type — which means it’s still working from whatever text the model actually generated, not a provider-level guarantee that the output structurally matches. Some providers can additionally enforce structure natively at generation time rather than relying purely on this after-the-fact parsing — reached through an optional spec.useProviderStructuredOutput() setting on the same .entity(...)call — but whether a given provider actually supports that native path is provider-specific, so treat the default, parsing-based behavior shown above as the reliable baseline across providers, this native option as an enhancement where it’s available.

Portability: What Actually Changes If Ollama Isn’t the Provider Anymore

Nothing in any of the code on this page — or the Intermediate level’s controller — mentions Ollama by name. chatClient.prompt(), .system(...), .stream(), .entity(...)— all of it is Spring AI’s own abstraction, not Ollama-specific API surface. That’s the practical payoff of the “universal translator” framing from the Introduction level: swapping from Ollama to a cloud provider is, by Spring AI’s own design, meant to be a dependency swap (a different model starter on the classpath) plus a configuration change (that provider’s own connection properties in place of spring.ai.ollama.*), not a rewrite of the ChatClient calls themselves.

That portability doesn’t erase the real trade-offs between running locally and reaching for a cloud provider — cost, privacy, and needing hardware capable of running a model well are all genuine considerations, covered in full on this site’s dedicated Cloud AI vs Local AI topic. What this page is scoped to is narrower and more mechanical: however that trade-off gets decided, the Java code calling ChatClient is designed to barely notice which side of it you landed on.

Where This Goes Next

Every example across this topic has been a single request, a single reply — the model answers only from whatever it already knew during training, plus whatever fit in one .user(...) message. The next topic in this pillar, RAG with Spring AI, builds directly on the ChatClient foundation from this one, adding the piece needed to answer questions grounded in your own documents: retrieving relevant content from a VectorStore and feeding it into the prompt before the model ever sees the question.

Fun Fact

Flux<String>isn’t a type Spring AI invented for this — it’s Project Reactor’s general-purpose reactive stream type, the exact same one Spring’s reactive web stack has used for years. Reaching for .stream() here means an LLM reply gets handled by the same reactive machinery a Spring developer might already know from an entirely unrelated, non-AI reactive endpoint.

Test Yourself