How teams build production-ready LLM apps

The platform to prototype, evaluate, improve, and observe your LLM app or feature.

Developers use Inductor to ship high-quality LLM-powered functionality
rapidly and reliably.

Try for free Contact us

Custom Playgrounds

Immediately start experimenting and collaborating as you build

Auto-generate a custom playground for your LLM app
Run securely in your environment, with your code and data sources
Share instantly
Iteratively develop test suites for systematic evaluation and improvement

Try for free

Test suites

Systematically test your LLM application to debug and ensure quality

Rapidly customize quality evaluation for your use case
Auto-generate shareable UIs for human evaluation, and automate with rigorous LLM-powered evaluation
Construct, evolve, and share test cases
Automatic orchestration of test suite execution

Try for free

Experiment orchestration

Systematically improve your LLM app to go from prototype to production

Automate your experimentation with Inductor Hyperparameters – rapidly find the LLM app design that delivers the results that you need, across choice of model, prompt, retrieval augmentation, or anything else
Automatic versioning and tracking of all results

Try for free

Production observability

Monitor your live traffic and close the loop with your development workflow – to understand usage, resolve issues, and further improve

Rich filtering to rapidly identify issues and areas for improvement
Apply quality measures to your live traffic for continuous evaluation
Run and analyze A/B tests
Automatically log your live traffic, including LLM app inputs, outputs, and internals

Try for free

Going from an idea or demo to an LLM app that is actually production-ready (i.e., high quality, trustworthy, cost-effective) is difficult and time-consuming. To do that, you need to be able to answer questions like the following, backed by data – can you?

How good is our LLM app?
How can we improve our LLM app?
How do we collect feedback and collaborate?
Are we ready to ship our LLM app?
How is our LLM app behaving on live traffic?

Deliver AI apps as reliably and efficiently as web apps

Get to market faster

Ensure trust and reliability

Develop and ship systematically

Reduce LLM costs

Eliminate undifferentiated heavy lifting

Get started

Show me the code!

Auto-generate a custom playground

Run a custom, shareable playground in your environment and securely use your private data

# In app.py (your existing LLM app code)

import inductor
from openai import OpenAI

client = OpenAI()

def document_qa(question: str):
    """Answer a question about our documents."""
    document_snippets = retrieve_from_vector_db(
        question, num_snippets=10)

    system_message = (
        "You are a helpful document QA assistant. "
        "Using only the following document snippets, answer any "
        "question asked.\n\n"
        "Document snippets:\n\n" +
        "\n\n".join(document_snippets))

    response = client.chat.completions.create(
        messages=[
            {"role": "system", "content": system_message},
            {"role": "user", "content": question}],
        model="gpt-4o")

    return response.choices[0].message.content

# In your terminal

$ inductor playground app:document_qa

Create and run test suites

Systematically test and evaluate your LLM app to debug and ensure quality

# In test_suite.py

import app
import inductor

test_suite = inductor.TestSuite(
    id_or_name="document_qa", llm_program=app.document_qa)

test_suite.add(
    # Add test cases
    inductor.TestCase(
        inputs={"question": "What is the name of the company?"}),
    inductor.TestCase(
        inputs={"question": ("Explain how to use the company's "
                             "product in detail.")}),

    # Add quality measure
    inductor.QualityMeasure(
        name="Answer is correct and relevant",
        evaluator="HUMAN",
        evaluation_type="BINARY",
        spec="Is the answer correct AND relevant to the question asked?"))
    # You can set evaluator="LLM" or evaluator="FUNCTION" to use
    # LLM-powered or programmatic evaluations, respectively.

if __name__ == "__main__":
    test_suite.run()

# In test_suite.yaml

# Test suite configuration
- config:
    name: document_qa
    llm_program_fully_qualified_name: app:document_qa

# Test cases
- test_case:
    inputs:
        question: What is the name of the company?
- test_case:
    inputs:
        question: Explain how to use the company's product in detail.

# Quality measure
- quality:
    name: Answer is correct and relevant
    evaluator: HUMAN
    evaluation_type: BINARY
    spec: Is the answer correct AND relevant to the question asked?

# In your terminal (to run test suite)

$ python test_suite.py

# In your terminal (to run test suite)

$ inductor test test_suite.yaml

Automate experimentation

Use hyperparameters to test anything: prompts, models, RAG strategies, and more

# In app.py (your existing LLM app code)

def document_qa(question: str):
    """Answer a question about our documents."""
    document_snippets = retrieve_from_vector_db(
        question,
        num_snippets=inductor.hparam("num_snippets", default_value=10))

    prompt_header = inductor.hparam(
        "prompt_header",
        default_value=(
            "You are a helpful document QA assistant. "
            "Using only the following document snippets, answer any "
            "question asked."))

    system_message = (
        f"{prompt_header}\n\n"
        "Document snippets:\n\n" +
        "\n\n".join(document_snippets))

    response = client.chat.completions.create(
        messages=[
            {"role": "system", "content": system_message},
            {"role": "user", "content": question}],
        model="gpt-4o")

    return response.choices[0].message.content

# In test_suite.py

# Add hyperparameters
test_suite.add(

    inductor.HparamSpec(
        hparam_name="num_snippets",
        hparam_type="NUMBER",
        values=[5, 10, 15]),

    inductor.HparamSpec(
        hparam_name="prompt_header",
        hparam_type="TEXT",
        values=[
            ("You are a helpful document QA assistant. "
             "Using only the following document snippets, answer any "
             "question asked."),
            ("You are a document QA assistant. Use the following "
             "document snippets to respond to questions concisely, "
             "rejecting any unrelated or unanswerable questions.")]))

# In test_suite.yaml

# Hyperparameters
- hparam:
    name: num_snippets
    type: NUMBER
    values: [5, 10, 15]
- hparam:
    name: prompt_header
    type: TEXT
    values:
        - |
            You are a helpful document QA assistant.
            Using only the following document snippets, answer any
            question asked.
        - |
            You are a document QA assistant. Use the following
            document snippets to respond to questions concisely,
            rejecting any unrelated or unanswerable questions.

# In your terminal

$ python test_suite.py

# In your terminal

$ inductor test test_suite.yaml

Monitor your live traffic

Understand usage, resolve issues, and further improve with automated production logging

# In app.py (your existing LLM app code)

@inductor.logger
# You can also pass quality measures and hyperparameter specs to the
# inductor.logger decorator to do continuous live evaluations and A/B testing.
def document_qa(question: str):
    """Answer a question about our documents."""
    document_snippets = retrieve_from_vector_db(
        question, num_snippets=10)
    inductor.log(document_snippets, name="Document snippets")

    system_message = (
        "You are a helpful document QA assistant. "
        "Using only the following document snippets, answer any "
        "question asked.\n\n"
        "Document snippets:\n\n" +
        "\n\n".join(document_snippets))

    response = client.chat.completions.create(
        messages=[
            {"role": "system", "content": system_message},
            {"role": "user", "content": question}],
        model="gpt-4o")
    inductor.log(response.usage.model_dump(), name="Model token usage")

    return response.choices[0].message.content

Show me the product! Watch a product demo with Inductor's founder to learn more.

Watch Demo

Get started in 5 minutes

Get started with zero code modifications. Enterprise-ready by default.

Integrates instantly

Run Inductor on your LLM app with zero code modifications. Optionally add 1-3 lines of code to get next-level capabilities.

Works seamlessly with your existing stack

Inductor works with any model, and any way of writing LLM apps – from LLM APIs to open source models to custom models.

Stay in your environment

Your code doesn’t leave your environment, in both development and production. Use Inductor seamlessly in any code editor or notebook.

Runs in our cloud account or yours

Inductor is architected from the ground up to run either hosted by us, or self-hosted in your cloud account.