Skip to main content

Embeddings for Search and RAG

Level: Intermediate · Category: API reference

Tags: Embeddings · Semantic search · Retrieval-augmented generation

POST https://openai.rc.asu.edu/v1/embeddings turns text into vectors, so you can find passages by meaning rather than by keyword: semantic search over papers or notes, de-duplication, clustering, and the retrieval step of retrieval-augmented generation (RAG). Keys and setup are on LLM API Access.

ModelDimensionsMax input
qwen3-embedding-8b4096 (or fewer, see below)32,768 tokens
qwen3-embedding-4b2560 (or fewer)32,768 tokens

Vectors from different models aren't comparable: embed your documents and your queries with the same model, and re-embed everything if you switch.

Semantic search in a few lines​

import math
from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY and OPENAI_BASE_URL
MODEL = "qwen3-embedding-8b"

docs = [
    "Use sbatch to submit a batch script to the Slurm scheduler.",
    "Scratch files that are not accessed for 90 days are deleted.",
    "Request GPUs with the --gres option, for example --gres=gpu:a100:1.",
    "Mamba environments keep Python packages separate for each project.",
]

def embed(texts):
    response = client.embeddings.create(model=MODEL, input=texts)
    return [item.embedding for item in response.data]

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / math.sqrt(sum(x * x for x in a) * sum(y * y for y in b))

doc_vectors = embed(docs)

# Qwen3 embedding models retrieve better when the query states the task.
query = "Instruct: Given a question, retrieve passages that answer it\nQuery: How do I ask for a GPU?"
query_vector = embed([query])[0]

for score, doc in sorted(((cosine(query_vector, v), d) for v, d in zip(doc_vectors, docs)), reverse=True):
    print(f"{score:.3f}  {doc}")
0.604  Request GPUs with the --gres option, for example --gres=gpu:a100:1.
...

Tips:

  • Send batches. input takes a list; send tens to a few hundred passages per request rather than one per request.
  • Prefix queries, not documents. Qwen3 embedding models expect queries in the Instruct: <task>\nQuery: <text> form shown above, and documents as plain text.
  • Store vectors in a file, a database, or a vector store such as FAISS, Chroma or pgvector, and embed new documents only once.
  • Chunk long documents into passages of a few hundred words before embedding; a whole paper in one vector matches queries poorly.

Smaller vectors​

Both models accept dimensions to return shorter vectors (from 4096 or 2560 down to as few as 64), which take less storage and compare faster at some cost in accuracy:

response = client.embeddings.create(model="qwen3-embedding-8b", input=["Slurm job arrays"], dimensions=256)
print(len(response.data[0].embedding))  # 256

Use the same dimensions for documents and queries.

Use it from LangChain or LlamaIndex​

Point the OpenAI embeddings class at the RC endpoint. In LangChain:

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="qwen3-embedding-8b",
    base_url="https://openai.rc.asu.edu/v1",
    check_embedding_ctx_length=False,  # send text, not OpenAI tokenizer ids
)

check_embedding_ctx_length=False matters: without it, LangChain splits text with OpenAI's tokenizer and sends token ids, which these models don't understand.