Spring AI as Its Own Microservice: Why Not Embed It Everywhere?

The Shared Translator Desk Analogy

Imagine a company with several departments — Sales, Support, Legal — each of which occasionally needs a document translated. One approach: every department hires and trains its own translator, buys its own translation software, and maintains its own relationship with a translation vendor. It works, technically. But now there are three separate translation setups to maintain, three separate vendor contracts to renew, and if the company ever wants to switch translation providers or upgrade to a better one, it means redoing that work three times, once per department, probably slightly differently each time.

The other approach: the company sets up one shared Translation Desk. Any department that needs something translated sends the document to that desk and gets the translation back. Sales doesn’t need its own translator; neither does Support or Legal. When the company wants to switch translation vendors or add a new language, that change happens once, at the desk, and every department benefits immediately without touching their own work at all.

A dedicated Spring AI inference service is that shared Translation Desk. Instead of Order Service, Support Service, and Reporting Service each embedding their own ChatClient, their own Ollama/provider configuration, and their own prompt-building logic, one service owns all of that, and every other service simply sends it a request and reads back a response — the same “independent stall in the food court” idea from topic 3’s Introduction, just applied to an LLM capability instead of orders or payments.

Why This Reuses Topic 3’s Reasoning Directly

Nothing about this decision is Spring-AI-specific — it’s the exact same reasoning topic 3’s Introduction already gave for splitting anycapability into its own microservice, applied here to “talk to an LLM” specifically:

• 1

  Single responsibility.One service’s entire job is talking to the model correctly — managing the ChatClientsetup, the system prompt, the provider configuration. Every other service’s code never has to think about any of that; it just makes a normal REST call and gets an answer back.

• 2

  Independent deployability. If the team wants to swap Ollama for a different provider, upgrade a model, or change a prompt template, that change happens in one service and one deployment — not a coordinated redeploy across every service that happens to call an LLM.

• 3

  Independent scaling. LLM calls are typically slower and more resource-intensive than an ordinary REST call. If three different services all end up needing the model heavily, scaling one inference service up (more instances, more GPU/CPU headroom) is simpler than scaling that capability separately, inconsistently, inside every service that happens to embed it.

None of this says embedding ChatClient directly inside a service is wrong — a small system with exactly one service that needs an LLM has little reason to add a whole extra service just to wrap it. The dedicated inference service earns its cost specifically once more than one service needs that capability, which is precisely the situation topic 3 already described microservices architectures growing into over time.

What Doesn’t Change

It’s worth being explicit about what stays exactly the same. Inside the inference service itself, every piece of Spring AI this pillar already covered — the ChatClient fluent chain from topic 4, VectorStore and QuestionAnswerAdvisor from topic 5, @Tool methods from topic 6 — works completely unchanged. The inference service is a Spring Boot application built exactly the way every other topic in this pillar taught, constructor-injecting a ChatClient and exposing it through a @RestController, the same as topic 4’s ChatController or topic 5’s RagController. What’s new is only the fact that this particular @RestController isn’t called directly by an outside client — it’s called by other microservices, over the network, using the exact RestClientpattern topic 3’s Intermediate level already taught for any two Spring Boot services talking to each other. The Intermediate level of this topic builds exactly that.

Fun Fact

This “wrap a capability behind its own service so other services don’t each reinvent it” instinct predates LLMs by decades — it’s the same reasoning that led teams to build a dedicated Notification Service (so nobody hand-rolls their own email/SMS-sending code) or a dedicated Auth Service (so nobody reimplements login logic). An LLM inference service is that same well-worn pattern, just with a ChatClient call instead of a login check behind the API.

Test Yourself

Ready to actually build one — the inference service’s request/response contract, a real @RestController wrapping ChatClient, and the exact RestClient code a different service uses to call it? Continue to the Intermediate level →