Building an Inference Service: Contract, RestClient Call & Gateway Wiring

Designing the Contract: Request and Response DTOs

Before writing the controller, the inference service needs a contract — what a caller sends it, and what it sends back. This is ordinary REST API design, the same discipline topic 2 already taught for any endpoint; nothing here is Spring-AI-specific. What is worth getting right is shaping these DTOs around what topics 4 and 5 already established a ChatClient/RAG call actually needs and returns, so the contract doesn’t invent fields the underlying call has no way to fill:

// What a caller sends in. "question" mirrors the single required input
// every ChatClient call in this pillar has needed since topic 4 —
// .user(message). "useKnowledgeBase" is this service's own addition:
// a plain boolean flag deciding whether to run the request through
// QuestionAnswerAdvisor (topic 5's RAG) or answer from the model alone.
public record InferenceRequest(
    String question,
    boolean useKnowledgeBase
) {}

// What comes back. "answer" is exactly what topic 4's .content() already
// returns. "usedKnowledgeBase" simply echoes back which path the request
// actually took, so a consuming service (or a log, or a UI) can tell
// whether this specific answer was grounded in retrieved documents.
public record InferenceResponse(
    String answer,
    boolean usedKnowledgeBase
) {}

Keeping the request shape this small is deliberate. Every extra field on InferenceRequestis something every calling service now has to know how to fill in correctly — the contract should expose only what a caller genuinely needs to control, and let the inference service keep everything else (which model, which provider, the exact prompt template) as its own internal concern, exactly the encapsulation topic 3’s database-per-service reasoning already applied to data. A caller doesn’t get to reach in and configure the model directly, the same way it doesn’t get to reach into another service’s database directly.

The Inference Service Itself

With the contract settled, the controller is the same shape topics 4 and 5 already built — a constructor-injected ChatClient and VectorStore, exposed through @RestController — just running as its own independent Spring Boot application rather than folded into a bigger one:

@RestController
@RequestMapping("/inference")
public class InferenceController {

    private final ChatClient chatClient;
    private final VectorStore vectorStore;

    // Constructor injection — the exact same mechanics from topic 1,
    // reused one more time in this pillar's very last controller.
    public InferenceController(ChatClient.Builder chatClientBuilder, VectorStore vectorStore) {
        this.chatClient = chatClientBuilder.build();
        this.vectorStore = vectorStore;
    }

    @PostMapping("/ask")
    public InferenceResponse ask(@RequestBody InferenceRequest request) {
        if (request.useKnowledgeBase()) {
            String answer = chatClient.prompt()
                .advisors(QuestionAnswerAdvisor.builder(vectorStore).build())
                .user(request.question())
                .call()
                .content();
            return new InferenceResponse(answer, true);
        }

        String answer = chatClient.prompt()
            .user(request.question())
            .call()
            .content();
        return new InferenceResponse(answer, false);
    }
}

Every line inside ask(...) is content this pillar already taught — the .advisors(QuestionAnswerAdvisor.builder(vectorStore).build()) branch is exactly topic 5’s RagController, and the plain branch is exactly topic 4’s ChatController. This service’s only real contribution is choosing between the two based on the caller’s useKnowledgeBase flag, and packaging both paths behind one contract.

Calling It From Another Service

A different microservice — say, Support Service, handling a customer’s support ticket — calls this exactly the way topic 3’s Intermediate level already taught Order Service calling Inventory Service: RestClient, a base URL pointing at the inference service’s logical name, and a plain synchronous call:

// Living inside Support Service — a completely different Spring Boot
// application from InferenceController above.
public record InferenceRequest(String question, boolean useKnowledgeBase) {}
public record InferenceResponse(String answer, boolean usedKnowledgeBase) {}

@Service
public class InferenceClient {

    private final RestClient restClient;

    // "inference-service" is a logical service name, resolved through
    // service discovery at request time — the exact same shape topic 3
    // used for "http://inventory-service".
    public InferenceClient(RestClient.Builder restClientBuilder) {
        this.restClient = restClientBuilder
            .baseUrl("http://inference-service")
            .build();
    }

    public InferenceResponse ask(String question, boolean useKnowledgeBase) {
        return restClient.post()
            .uri("/inference/ask")
            .body(new InferenceRequest(question, useKnowledgeBase))
            .retrieve()
            .body(InferenceResponse.class);
    }
}

@Service
public class SupportService {

    private final InferenceClient inferenceClient;

    public SupportService(InferenceClient inferenceClient) {
        this.inferenceClient = inferenceClient;
    }

    public String draftReply(String customerQuestion) {
        InferenceResponse response = inferenceClient.ask(customerQuestion, true);
        return response.answer();
    }
}

Support Service never imports Spring AI, never sees a ChatClient, and never configures a model provider. From its point of view, it’s calling an ordinary REST API and getting back a plain InferenceResponse — exactly the encapsulation the Introduction level argued for.

Where This Fits in the Gateway/Discovery Setup

Nothing about registering the inference service or routing to it is special. It joins service discovery exactly the way topic 3 described every service joining: add the Eureka client dependency, point it at the discovery server, and let it register itself under its spring.application.name (inference-service) at startup — the same name InferenceClient above already looks it up by. And if the inference service should also be reachable directly from outside the system — not only from other services — the API gateway gets one more route added to the same routes list topic 3 built for Order Service:

spring:
  cloud:
    gateway:
      server:
        webflux:
          routes:
            - id: order-service-route
              uri: lb://order-service
              predicates:
                - Path=/orders/**
            - id: inference-service-route
              uri: lb://inference-service
              predicates:
                - Path=/inference/**

Same lb:// prefix, same discovery-backed resolution, same shape — an inference service is, to the gateway and to service discovery, just one more backend service with its own name and its own route. Whether outside clients actually get direct access to /inference/**, or the service stays reachable only from inside the system (no gateway route at all, just service-to-service calls), is a decision each system makes based on whether an LLM capability should ever be exposed directly to the outside world at all — this page isn’t asserting one answer is universally correct.

Fun Fact

Nothing above required a single new Spring AI class, annotation, or property beyond what topics 4 and 5 already introduced. Every genuinely new idea on this page — InferenceClient, the gateway route, the discovery registration — is topic 3’s content, applied to a service whose job happens to be “talk to an LLM” instead of “check stock” or “charge a card.” That’s the whole idea this Intermediate level set out to show concretely.

Test Yourself

Every call in this level had a fixed shape: one service asks the inference service a question, and gets one answer back. What happens once the model itself starts making decisions — an agent whose tools aren’t local Java methods at all, but calls to other microservices, each with their own database, their own latency, and their own chance of failing? Continue to the Advanced level →