Chunking Config, Metadata Filtering & Where Spring AI’s RAG Coverage Ends

Chunking Strategy: What TokenTextSplitter Actually Exposes

The trade-offs behind chunk size and split points — too small and a chunk loses context, too large and retrieval gets less precise, split mid-sentence and a chunk can lose its own meaning — are covered in full, conceptually, on this site’s dedicated Document Chunking Advanced topic. What’s specific to Spring AI is which of those trade-offs its own splitter actually lets you control. Verified against TokenTextSplitter’s current builder:

TokenTextSplitter splitter = TokenTextSplitter.builder()
    .withChunkSize(800)            // target chunk size, in tokens (default: 800)
    .withMinChunkSizeChars(350)    // don't emit a chunk smaller than this many characters
    .withMinChunkLengthToEmbed(5)  // discard a chunk shorter than this entirely
    .withMaxNumChunks(10000)       // hard ceiling on chunks produced from one document
    .withKeepSeparator(true)       // keep original separators (e.g. newlines) in each chunk
    .build();

Two things worth being precise about here. First, this splitter measures chunkSize in tokens, using a configurable tokenizer encoding — not characters or words, which matters because token count is what actually determines whether a chunk fits comfortably inside a model’s context window alongside everything else the prompt needs to carry. Second, withKeepSeparator is the one setting here that touches wherea split happens rather than just how big the result is — it’s a narrower lever than the sentence/paragraph-boundary-aware splitting strategies covered conceptually in Document Chunking Advanced, and worth treating as exactly that: one specific splitter’s current configuration surface, not a restatement of the broader chunking-strategy discussion.

Metadata Filtering: Narrowing Retrieval Before Similarity Even Runs

Every retrieval so far searched the entire document collection. Real applications often need to restrict that search first — only this customer’s documents, only this department’s policies, only documents from this year. Spring AI’s SearchRequest supports exactly this through filterExpression(...), verified in two current forms.

The fluent form, built with FilterExpressionBuilder:

FilterExpressionBuilder b = new FilterExpressionBuilder();

SearchRequest request = SearchRequest.builder()
    .query("What is our remote work policy?")
    .filterExpression(
        b.and(b.eq("department", "engineering"), b.gte("year", 2025)).build()
    )
    .build();

Or the same restriction, written as a portable, SQL-like string instead:

SearchRequest request = SearchRequest.builder()
    .query("What is our remote work policy?")
    .filterExpression("department == 'engineering' && year >= 2025")
    .build();

Both forms support the operators you’d expect — ==, !=, >=, in, &&/||, and IS NULL— and Spring AI’s own documentation frames this filter syntax as portable across every VectorStore implementation, so the same filter expression keeps working if the underlying store ever changes from SimpleVectorStoreto a production database. This filter runs as a genuine restriction on the candidate set, not a post-hoc filter applied after ranking — documents that don’t match never factor into the similarity comparison at all.

Ingesting Real Document Formats: TikaDocumentReader

Every example so far started from a plain Java String. Real document collections are rarely that clean — PDFs, Word documents, presentations. Spring AI’s ETL support includes a dedicated TikaDocumentReader, built on Apache Tika, specifically for this:

TikaDocumentReader reader = new TikaDocumentReader(resource); // a PDF, DOCX, PPTX, or HTML Resource
List<Document> rawDocuments = reader.read();

List<Document> chunks = splitter.apply(rawDocuments);
vectorStore.add(chunks);

The pipeline shape doesn’t actually change from what the Intermediate level already showed — read, split, add. The only difference is that TikaDocumentReader does the work of turning a PDF or Word document into plain Documentobjects first, so the splitter never has to know or care that the original file wasn’t plain text to begin with. Spring AI also ships format-specific readers for narrower cases — a dedicated PDF reader, a Markdown-aware reader, and others — worth knowing exist if a collection is entirely one format and doesn’t need Tika’s broader coverage.

Where This Java Implementation’s Coverage Actually Ends

This site’s RAG Pipeline Architecture Advanced topic covers two patterns that go beyond a single retrieve-then-generate pass: agentic RAG (where the model itself decides whether to retrieve again, or rewrite the query, rather than a fixed one-shot retrieval), and GraphRAG (retrieving over a graph of entities and relationships, built at indexing time, rather than pure vector similarity). Both are real, current, and well worth reading conceptually if you haven’t already.

What this topic can responsibly say about Spring AI’s own relationship to those two patterns is narrower than it might be tempting to claim. Everything covered across this topic — VectorStore, QuestionAnswerAdvisor, the newer RetrievalAugmentationAdvisor — is built around the standard linear pipeline: one retrieval pass (optionally with a query-rewriting step in front of it), then one generation pass. That pipeline is genuinely well-supported and, for the large majority of RAG use cases, exactly the right tool. Spring AI’s Advisor API and VectorStoreabstraction are also the kind of building blocks a developer could use to hand-build something closer to an agentic retrieval loop or a graph-backed retrieval step on top of — but that would be a developer composing Spring AI’s primitives into a custom pipeline, not a named, built-in GraphRagAdvisoror agentic-RAG orchestrator shipped by the framework itself. This topic can’t confirm Spring AI ships either of those as a dedicated, out-of-the-box feature, so it isn’t claimed here — if that changes in a future Spring AI release, it belongs in its own update, not an assumption made ahead of the evidence.

Where This Goes Next

Retrieval is one way to give a ChatClientcall information it wouldn’t otherwise have. The next topic in this pillar, Function/Tool Calling in Spring AI, covers a different one: letting the model call real Java methods directly, rather than only reading text handed to it in advance. Both ultimately extend what a single ChatClient call can do, and both are worth having in the same toolbox — the next topic covers tool calling on its own terms, rather than assuming a specific relationship between the two beyond that.

Fun Fact

Apache Tika, the library behind TikaDocumentReader, predates Spring AI by well over a decade — it’s a long-standing Apache project built for exactly this kind of “detect the format, extract the text” job, originally for search-indexing use cases that had nothing to do with LLMs at all. Spring AI reaches for an already-mature, purpose-built library here rather than reimplementing PDF and Office document parsing itself.

Test Yourself