VectorStore, Document Ingestion & QuestionAnswerAdvisor in Practice

Step 1: The VectorStore Interface

Verified directly against Spring AI’s current (2.0.0) API documentation, VectorStoreis built on a deliberate split between reading and writing — the Fun Fact on the Introduction level mentioned this, and here’s the actual shape:

// The read-only half — similaritySearch only.
public interface VectorStoreRetriever {
    List<Document> similaritySearch(SearchRequest request);

    default List<Document> similaritySearch(String query) {
        return similaritySearch(SearchRequest.builder().query(query).build());
    }
}

// The full interface — adds writing (and extends the read-only one above).
public interface VectorStore extends DocumentWriter, VectorStoreRetriever {
    void add(List<Document> documents);
    void delete(List<String> idList);
    void delete(Filter.Expression filterExpression);
    // ...plus a couple of default methods not shown here
}

A class that only ever needs to search — never to add or delete documents — can depend on the narrower VectorStoreRetriever type instead of the full VectorStore. For a single @RestController that both ingests documents and answers questions against them, though, the full VectorStoretype is what you’ll actually constructor-inject — same dependency injection mechanics as every other topic in this pillar.

Spring AI ships around twenty production-ready VectorStore implementations as of its current release — PGVector, Redis, Elasticsearch, MongoDB Atlas, Pinecone, Qdrant, Weaviate, and more — each one a different concrete class behind the exact same interface. For local development and for this pillar, though, the implementation that matters most is SimpleVectorStore: an in-memory implementation that needs no separate database running at all. Spring AI’s own documentation is direct that it’s meant for testing and demonstration rather than production use — but that’s precisely the same “no cloud dependency, nothing to sign up for” fit this pillar has used Ollama for since topic 4. Every example on this page assumes SimpleVectorStore; the Advisor and VectorStore method calls themselves are identical no matter which implementation is actually behind them.

Step 2: Getting Documents In

A raw document is almost always too large to embed and store as one single unit — the why behind that is covered in full on this site’s dedicated Document Chunking topic, and nothing here repeats that reasoning. Spring AI’s own current splitter for this is TokenTextSplitter, configured through a builder:

List<Document> rawDocuments = List.of(
    new Document("Standard shipping takes 3-5 business days within the continental US."),
    new Document("Items can be returned within 30 days of delivery for a full refund.")
);

TokenTextSplitter splitter = TokenTextSplitter.builder()
    .withChunkSize(800)          // target chunk size, in tokens
    .withMinChunkSizeChars(350)  // don't produce a chunk smaller than this
    .build();

List<Document> chunks = splitter.apply(rawDocuments);

vectorStore.add(chunks);

.apply(rawDocuments) runs the splitter over every document and returns the resulting chunks as their own Document objects, and vectorStore.add(chunks) is the exact current method that embeds each chunk and stores it — Spring AI generates the embedding for you as part of add(...), using whichever embedding model is configured (Ollama’s own embedding models work here, selected the same way topic 4 selected a chat model, via spring.ai.ollama.embedding.model). Nothing in this method call touches an HTTP client, a request body, or a JSON response directly — that’s the entire abstraction paying off.

Real document collections usually aren’t plain Java strings, either — often a PDF, a Word document, or an HTML export. Spring AI’s ETL support includes dedicated DocumentReader implementations for exactly this (a PagePdfDocumentReader for PDFs, among others), which the Advanced level touches on briefly.

Step 3: Getting Documents Back Out

Retrieval is a single method call, configured through SearchRequest’s own builder:

SearchRequest request = SearchRequest.builder()
    .query("How long do I have to return an item?")
    .topK(3)                     // return at most 3 results
    .similarityThreshold(0.7)    // discard weak matches below this score
    .build();

List<Document> results = vectorStore.similaritySearch(request);

topKcaps how many results come back (Spring AI’s own default is 4 if left unset), and similarityThreshold discards anything that doesn’t clear a minimum relevance score — both genuinely useful in practice, since a search that always returns exactly topK results regardless of relevance can end up handing the model context that has nothing to do with the question. A shorthand, vectorStore.similaritySearch("plain query string"), exists too, for the common case where none of those extra settings are needed.

Step 4: Wiring Retrieval Into ChatClient

The pieces above — add documents, search for relevant ones — are already enough to build RAG by hand: run similaritySearch(...), then build a prompt string yourself with the retrieved text folded in, then call chatClient.prompt().user(thatPrompt).call().content() exactly like topic 4 already taught. That manual approach works, and it’s worth understanding, because it’s exactly what Spring AI’s built-in advisor for this pattern does on your behalf.

That advisor is QuestionAnswerAdvisor — add it to a ChatClient call’s .advisors(...) and it runs the retrieval and prompt-assembly step automatically, before the request ever reaches the model:

String answer = chatClient.prompt()
    .advisors(QuestionAnswerAdvisor.builder(vectorStore).build())
    .user("How long do I have to return an item?")
    .call()
    .content();

Everything after .advisors(...) is the exact same fluent chain topic 4 already taught — .user(...), .call(), .content(). The only addition is that one advisor, which — behind the scenes — runs a similaritySearch(...) against the VectorStore it was built with, and rewrites the outgoing request so the model sees the retrieved context alongside the question. The topK and similarity-threshold settings from the previous step can be configured on the advisor itself too, via an optional .searchRequest(...) builder call, if the defaults aren’t the right fit for a given use case.

Spring AI also ships a newer, more modular alternative, RetrievalAugmentationAdvisor, built around a separate VectorStoreDocumentRetriever and composable with optional query-rewriting steps. It’s worth knowing the name exists, since some current tutorials use it instead of QuestionAnswerAdvisor — this pillar teaches QuestionAnswerAdvisor first because its single .builder(vectorStore) call is the simpler, more direct route to the same core pattern.

Putting It Together: A Real Endpoint

Here’s all of it in one place — a @RestController (the annotation from topic 2) with a constructor-injected ChatClient and VectorStore (the dependency injection mechanics from topic 1), exposing an endpoint that takes a question and returns a RAG-grounded answer:

@RestController
@RequestMapping("/ask")
public class RagController {

    private final ChatClient chatClient;
    private final VectorStore vectorStore;

    // Constructor injection, exactly as in every other topic in this pillar —
    // this class declares what it needs, and Spring supplies both beans.
    public RagController(ChatClient.Builder chatClientBuilder, VectorStore vectorStore) {
        this.chatClient = chatClientBuilder.build();
        this.vectorStore = vectorStore;
    }

    @GetMapping
    public String ask(@RequestParam String question) {
        return chatClient.prompt()
            .advisors(QuestionAnswerAdvisor.builder(vectorStore).build())
            .user(question)
            .call()
            .content();
    }
}

A request to /ask?question=How long do I have to return an item? now runs a real similaritySearch(...) against whatever documents were previously add(...)-ed to the injected VectorStore, folds the retrieved text into the request automatically, and only then calls the model — the exact three-stage flow (retrieve, augment, generate) the demo below makes concrete.

Try It: Watch Retrieval Change the Prompt

Pick a question below and see a simulated similaritySearch(...) pull the closest matching snippets from a small sample knowledge base, watch exactly how those snippets get stitched into the augmented prompt, and see the resulting grounded answer (not a live model call — see the note under the demo):

Fun Fact

QuestionAnswerAdvisor doesn’t just insert retrieved text anywhere convenient — Spring AI’s own documentation shows it working through a configurable prompt template, meaning the exact wording of the instruction wrapped around your retrieved context (“answer using only the context below,” for instance) is itself something you can override, not a fixed string buried in the framework.

Test Yourself

That endpoint answers well as long as the whole document collection fits comfortably in one search. What about controlling exactly how documents get split, filtering retrieval down to a specific subset of documents by metadata, and being honest about where this Java implementation’s coverage of RAG actually ends? Continue to the Advanced level →