Skip to main content

Quickstart

Make your first Seer log — measure retrieval quality from unlabeled traffic.

Beta Access Required

Seer is in private beta. Email ben@seersearch.com to request an API key.

tip

Prefer zero-friction integration? Use the @seer_trace decorator to wrap your retrieval function and auto-log query+context.

Quick Start​

Step 1. Install the Seer SDK:

pip install seer-sdk

Step 2. Get an API key from the Seer Console and export it:

export SEER_API_KEY="seer_live_your_key_here"

Step 3. Create quickstart.py and paste:

from seer import SeerClient

client = SeerClient()  # reads SEER_API_KEY from env

# Your retrieval step
def retrieve(query: str) -> list[dict]:
    # Replace with your real retriever
    return [
        {"text": "Christopher Nolan directed Inception.", "score": 0.95},
        {"text": "Nolan is British-American.", "score": 0.89}
    ]

query = "Who directed Inception and what is their nationality?"
context = retrieve(query)

client.log(
    task=query,                           # the user query
    context=context,                      # list of passage dicts
    metadata={
        "env": "prod",                    # environment tag
        "feature_flag": "retrieval-v1",   # for A/B testing
    },
)

print("Logged to Seer!")
# Events are auto-flushed when the process exits

Step 4. Run it:

python quickstart.py

Using the Decorator​

Add logging around your retrieval function without changing call sites:

from seer import seer_trace

@seer_trace(
    task_arg="query",           # which arg is the query
    context_from_return=True,   # return value is the context
)
def retrieve(query: str) -> list[dict]:
    # Your retriever
    return [
        {"text": "Christopher Nolan directed Inception.", "score": 0.95},
        {"text": "Nolan is British-American.", "score": 0.89}
    ]

# Decorator logs automatically when function is called
results = retrieve("Who directed Inception and what is their nationality?")
print("Decorated call logged to Seer!")

What Happens Next​

As logs arrive, Seer evaluates each retrieval to compute metrics:

  • Recall: An evaluator model enumerates the minimal disjoint requirements (K) required to answer the question and checks which are supported by your retrieved context. recall = covered_requirements / K. Seer flags queries with recall < 1.0.
  • Precision: what proportion of retrieved documents in context are useful to answer the question/task. unique docs judged "supporting" / docs in context
  • F1, nDCG: Derived from the recall + precision signals (and optional scores if provided in context items).

Use the Seer dashboard to:

  • Compare variants (change testing): Filter by feature_flag to see A/B impacts.
  • Monitor production: Filter by env or any metadata field.

Context Format​

The context parameter accepts either:

  • Simple strings: ["passage 1", "passage 2"]
  • Passage objects: [{"text": "passage 1", "score": 0.9}, ...]

When using objects, include at least the text field. Optional fields like score, id, and source enable richer analytics.

See Context & Event Schema for the full specification.

Fire-and-Forget Mode​

By default, the SDK operates in fire-and-forget mode:

  • log() returns immediately (non-blocking)
  • Events are queued and sent asynchronously
  • Events auto-flush when your process exits normally

You only need to call client.flush() if:

  • Using os._exit() or process pools
  • Running a short-lived script that exits too quickly
  • You need to ensure events are sent before a specific point
# Only needed in special cases
client.flush()  # block until all queued events are sent

Next Steps​