Skip to content
Cloudflare Docs

Bring your own generation model

When using AI Search, AI Search leverages a Workers AI model to generate the response. If you want to use a model outside of Workers AI, you can use AI Search for search while leveraging a model outside of Workers AI to generate responses.

Here is an example of how you can use an OpenAI model to generate your responses. This example uses Workers Binding.

JavaScript
import { openai } from "@ai-sdk/openai";
import { generateText } from "ai";


export default {
  async fetch(request, env) {
    // Parse incoming url
    const url = new URL(request.url);


    // Get the user query or default to a predefined one
    const userQuery =
      url.searchParams.get("query") ??
      "How do I train a llama to deliver coffee?";


    // Search for documents in AI Search
    const searchResult = await env.AI.autorag("my-rag").search({
      query: userQuery,
    });


    if (searchResult.data.length === 0) {
      // No matching documents
      return Response.json({ text: `No data found for query "${userQuery}"` });
    }


    // Join all document chunks into a single string
    const chunks = searchResult.data
      .map((item) => {
        const data = item.content
          .map((content) => {
            return content.text;
          })
          .join("\n\n");


        return `<file name="${item.filename}">${data}</file>`;
      })
      .join("\n\n");


    // Send the user query + matched documents to openai for answer
    const generateResult = await generateText({
      model: openai("gpt-4o-mini"),
      messages: [
        {
          role: "system",
          content:
            "You are a helpful assistant and your task is to answer the user question using the provided files.",
        },
        { role: "user", content: chunks },
        { role: "user", content: userQuery },
      ],
    });


    // Return the generated answer
    return Response.json({ text: generateResult.text });
  },
};