Vertex AI RAG Engine Serverless Practical Guide: Criteria to review before Spanner, how to design metadata search

Publication date: 2026-04-14 | Category: Development information

1) One-line problem definition

Key takeaway: The sooner you want to attach a RAG, the sooner you have to set operational boundaries and filter design before the vector database.

The first problem that many teams encounter when introducing Augmented Search Generation (RAG) is not model performance but Knowledge repository operational complexity. Documents accumulate, but if it is unclear which storage to use, how to divide authority boundaries, and whether only specific document groups can be searched, as the number of files increases, operating costs become more important than accuracy.

This article is intended for backend developers, platform engineers, and AI function PMs who are considering adopting the Vertex AI RAG Engine. The scope is Comparison of serverless mode and Spanner mode, metadata search design, and actual API flow. Excluding tuning your own ranking model or building a large-scale offline evaluation pipeline.

2) Conclusion first

Key summary: For initial adoption, serverless is the default, and it is safer to raise Spanner only when regulation or strong isolation is necessary.

The default I recommend is Serverless mode + explicit metadata schema + small file unit corpus separation. Google's official documentation also describes serverless as the cheapest and recommended starting point. Instead, it does not mean, “Because it is serverless, you can upload it without any preparation.” Actual success depends on how well you first define your filtering criteria before searching.

• Teams that are a good fit: Teams that need to produce a RAG MVP within 2 weeks, teams with few infrastructure operation staff, and teams with continuously increasing document sources
• For overkill: Regulated industries requiring strong data isolation, CMEK, and dedicated performance tiers
• Key judgment axis: Not just storage, but “who will find what document and under what conditions”

3) Core structure decomposition

Key summary: It is easy to understand when viewed by dividing the RAG Engine into a document container, metadata as a filter key, and Vector DB as a search engine.

Based on official documentation, the metadata search hierarchy is organized in the order RagCorpus → RagDataSchema → RagFile → RagMetadata. To put it simply, RagCorpus is a document archive, RagDataSchema is a rule that declares “filter keys such as year, department, and product can be used in this archive,” RagFile is the actual document, and RagMetadata is a tag attached to each document.

In serverless mode, the data for RAG management and the vector search engine must be viewed separately. According to Google documentation, serverless mode handles RAG resource management and orchestration in a fully managed manner, and embedding indexing and vector search use the Vertex AI Vector Search 2.0 collection by default by provisioning it to the project. In other words, rather than “all you need is a RAG engine,” it is closer to a structure that simplifies the operation side of and standardizes the search side to Vector Search 2.0.

4) Explanation of design intent

Key takeaways: The purpose of this structure is less to maximize performance than to reduce initial operational burden while leaving room for expansion.

The reason serverless mode is important is because it allows you to postpone database capacity planning, tier selection, and complex scaling decisions. Conversely, Spanner mode satisfies needs such as dedicated infrastructure and CMEK, but comes with tier selection and cost management responsibilities. Google documents also state that Spanner is suitable for workloads that require dedicated infrastructure, isolation, and CMEK support, while serverless is suitable for fast onboarding and fully managed expansion.

This design has clear tradeoffs.

• What you get: Quick start, infrastructure abstraction, simplification of the basic search flow
• What you give up: Fine-grained storage isolation control, dedicated performance guarantees, and regulatory options like CMEK
• Practical interpretation: As the number of files increases, “which documents should be excluded from search” becomes more important, so the addition of metadata search this April is not a simple function addition, but an enhancement of operability.

5) Evidence and comparison

Key takeaway: The comparison is not between Spanner and an external vector database, but rather “initial operational simplicity vs. regulatory/isolation needs”.

If we combine the official document evidence, the judgment criteria are organized as follows.

• Cost: Serverless has a simple resource management and orchestration cost structure, and it is easy to view Vector DB usage separately.
• Operation: Since Spanner requires you to manage tiers and containment strategies yourself, it may be an overinvestment for small teams.
• Accuracy: Thanks to this metadata search function, filters such as department, period, and product group can be applied before the search, making it easier to increase the “accuracy of finding only the right documents”.
• Scalability: Vector Search 2.0 provides collection-based storage, automatic embedding, and hybrid search and reranking directions, serving as a foundation for growing RAG data.

6) Actual operation flow / step-by-step execution method

Key summary: Determining corpus separation criteria and filter keys before inserting documents can greatly reduce recollection costs later.

1. Start from corpus boundaries.
   Divide by search intent, for example: policy-docs, product-manuals, support-runbooks. Business purpose criteria last longer than file type criteria.
2. Declare the metadata schema first.
   Example: department(STRING), doc_year(INTEGER), product_line(STRING), is_public(BOOLEAN). Operating tags without schema ultimately leads to filter confusion.
3. Attach metadata after uploading the file.
   When the number of documents exceeds 1,000, metadata-based filters are much more reliable than file name rules.
4. Forces a filter before the search request.
   Example: A customer support bot only uses is_public=true and product_line='A' Allowed.
5. Mode switching is understood as a view switching, not data movement.
   According to the official documentation, Serverless and Spanner data are isolated from each other, so you should check whether you need to plan for re-indexing or re-uploading before switching.

#1) Metadata schema example
curl -X POST   -H "Authorization: Bearer $(gcloud auth print-access-token)"   -H "Content-Type: application/json"   "https://${LOCATION}-aiplatform.googleapis.com/v1beta1/projects/${PROJECT_ID}/locations/${LOCATION}/ragCorpora/${RAG_CORPUS_ID}/ragDataSchemas:batchCreate"   -d '{
    "requests": [
      {"rag_data_schema": {"key": "department", "schema_details": {"type": "STRING"}}},
      {"rag_data_schema": {"key": "doc_year", "schema_details": {"type": "INTEGER"}}},
      {"rag_data_schema": {"key": "is_public", "schema_details": {"type": "BOOLEAN"}}}
    ]
  }'

#2) Example of file metadata attachment
curl -X POST   -H "Authorization: Bearer $(gcloud auth print-access-token)"   -H "Content-Type: application/json"   "https://${LOCATION}-aiplatform.googleapis.com/v1beta1/projects/${PROJECT_ID}/locations/${LOCATION}/ragCorpora/${RAG_CORPUS_ID}/ragFiles/${RAG_FILE_ID}/ragMetadata:batchCreate"   -d '{
    "requests": [
      {"rag_metadata": {"user_specified_metadata": {"key": "department", "value": {"str_value": "support"}}}},
      {"rag_metadata": {"user_specified_metadata": {"key": "doc_year", "value": {"int_value": 2026}}}},
      {"rag_metadata": {"user_specified_metadata": {"key": "is_public", "value": {"bool_value": true}}}}
    ]
  }'

In practice, “let’s put all the documents in and then filter them later” is the most expensive option. The reason is that when a search quality issue occurs, re-collection, re-tagging, and verification must be done one more time.

7) Mistakes/Pitfalls

Key takeaway: Most failures are caused by blurred document boundaries and tagging schemes, not models.

• Mistake 1: Splitting the corpus only based on the org chart
  Prevention: Separate the corpus based on the search scenario. Recovery: View usage logs and reorganize your corpus by search intent.
• Mistake 2: Using different metadata keys for different document teams
  Prevention: Do not allow department and dept at the same time and fix dictionary schema. Recovery: Unify keys and retag mismatched documents with a batch normalization script.
• Mistake 3: Misunderstanding the transition between Serverless and Spanner as a data migration
  Prevention: Understand that data in both modes is isolated, as the official documentation states, and include visibility revalidation in the transition checklist. Recovery: Compare the corpus list and search results before and after conversion to check for omissions.
• Mistake 4: Process public/private filters only at application layer
  Prevention: Default metadata filter before search. Recovery: Separate sensitive document corpus and force default filter

8) Strengths and limitations

Key takeaways: Serverless is fast to start with, but it doesn't solve all your regulatory needs.

• Strengths: Quick introduction, fully managed starting point, scalability linked to Vector Search 2.0, metadata-based precision filtering
• Limitations: Absence of CMEK, unsuitable for strong storage isolation needs, need to design transition to data isolation between modes
• Counterexample: In environments with very strict data boundaries, such as finance, public, and healthcare, a Spanner or separate storage strategy may be more appropriate.

9) Points to study more deeply

Key takeaways: The next step is to experiment with search quality and operability over the model.

• Vector Search 2.0 collection structure and schema design
• When to perform hybrid search and re-ranking
• Connection of corpus-level access control and application permission model
• Create a metadata filter experiment table based on incorrect answer cases

10) Execution Checklist + Author’s Perspective

Key takeaway: Even if you use serverless, you must design the search policy yourself.

• Has the corpus been divided into 3 drafts or less based on search intent?
Have you documented required metadata keys such as
• department, doc_year, is_public?
• Have you set filter defaults to prevent sensitive documents from getting mixed up in the default search path?
• Does the team understand that data will not be automatically moved when switching between serverless and Spanner?
• Is search quality evaluation not only based on “answer quality” but also on “percentage of incorrectly retrieved documents”?
• If your goal is to produce an MVP within 2 weeks, did you reduce the scope to serverless instead of Spanner?

Definition of Done: If the corpus boundary, metadata schema, and basic filter policy are documented, and there are 0 prohibited document exposures based on 20 test questions, the first introduction is considered complete.

My recommendation is clear. Most teams start serverless and should design metadata search together early on. Conversely, choosing Spanner from the beginning without regulatory reasons is often overkill. However, if complete isolation or CMEK for each customer is contractually required, it is better to consider Spanner or a separate storage architecture first rather than being forced to endure serverless.

Reference material

• Google Cloud Vertex AI release notes (Confirmation date: 2026-04-14, 2026-04-03 serverless deployment mode, 2026-04-06 check metadata search update)
• Google Cloud Docs - Deployment modes in Vertex AI RAG Engine (Confirmation date: 2026-04-14)
• Google Cloud Docs - Filter with metadata search (Confirmation date: 2026-04-14)
• Google Cloud Docs - Vector Search 2.0 overview (Confirmation date: 2026-04-14)