Pinecone’s fetch and query operations, while both retrieving vectors, serve fundamentally different purposes, and understanding this distinction is key to optimizing your vector database performance.
Let’s see them in action. Imagine you have a collection of documents, each represented by a vector.
First, fetch is like looking up a specific item by its exact name. You know the id of the vector you want, and you want to retrieve its associated metadata and vector data directly.
from pinecone import Pinecone, ServerlessSpec
import os
# Initialize Pinecone (replace with your actual API key and environment)
api_key = os.environ.get("PINECONE_API_KEY")
environment = os.environ.get("PINECONE_ENVIRONMENT") # e.g., "gcp-starter"
pc = Pinecone(api_key=api_key, environment=environment)
# Assuming you have an index named 'my-index'
index_name = "my-index"
# Create an index if it doesn't exist (for demonstration)
if index_name not in pc.list_indexes().names:
pc.create_index(
name=index_name,
dimension=8, # Example dimension
metric="cosine",
spec=ServerlessSpec(cloud='aws', region='us-east-1')
)
index = pc.Index(index_name)
# Upsert some sample data
index.upsert(
vectors=[
("vec1", [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8], {"genre": "scifi"}),
("vec2", [0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1], {"genre": "fantasy"}),
("vec3", [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1], {"genre": "scifi"}),
]
)
# --- Fetching a specific vector ---
# You know the ID 'vec1' and want its data.
fetch_response = index.fetch(ids=["vec1"])
print("--- Fetch Response ---")
print(fetch_response)
# Expected output will contain the vector 'vec1' and its metadata
fetch is incredibly fast because it bypasses the similarity search algorithm. It’s a direct lookup by primary key. You use fetch when you have a specific item’s id and need to retrieve its exact content or metadata. This is common when you’ve already identified a relevant document (perhaps from a previous search or a direct user request) and now need to display its full details or perform further processing on its associated data.
Now, query is about finding what’s similar to a given vector. You provide a query vector, and Pinecone returns the vectors in your index that are closest to it based on the chosen similarity metric (cosine, dotproduct, euclidean).
# --- Querying for similar vectors ---
# You have a query vector and want to find the top 2 most similar vectors.
query_vector = [0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85] # A vector similar to 'vec1'
query_response = index.query(
vector=query_vector,
top_k=2,
include_metadata=True
)
print("\n--- Query Response ---")
print(query_response)
# Expected output will contain the top_k most similar vectors to the query_vector
query involves a computationally intensive Approximate Nearest Neighbor (ANN) search. It’s the workhorse for recommendation systems, semantic search, and anomaly detection, where you need to discover items related to an input, rather than retrieve a precisely known item. You use query when you want to answer questions like "what are the most similar items to this one?" or "find me documents related to this topic."
The core problem fetch solves is efficient retrieval of known entities. The core problem query solves is discovery of related entities.
The mental model is: fetch is a direct address lookup, query is a "find me neighbors" operation.
If you’ve used fetch and are getting performance issues, double-check that you’re not trying to fetch a large number of vectors that would be better served by a batch query with a dummy vector if you only need metadata, or that the IDs you’re fetching actually exist. Conversely, if query is slow, ensure your top_k is reasonable and that you aren’t requesting metadata for every single result if you only need the IDs.
What most people don’t realize is that query can also be used for metadata filtering. You can provide a filter argument to query to restrict the search space before the ANN search is performed, significantly speeding up relevant results. For example, index.query(vector=query_vector, top_k=5, filter={"genre": "scifi"}) will only search within vectors tagged with "scifi". This is crucial for narrowing down results when your index is large and your queries are specific.
The next step after mastering fetch and query is exploring Pinecone’s upsert and delete operations for managing your data.