Prompt API Guide

Here's a simple guide on how you can use Prompt API in your application.

Requirements

The Prompt API is available from Chrome 140 and above. Additional requirements:

• Operating System: Windows 11, macOS 13+, Linux, or ChromeOS 16389+
• Language Detector and Translator API work only on Chrome Desktop
• Storage: At least 22 GB of free space
• GPU: Strictly more than 4 GB of VRAM, OR
• CPU: 16 GB of RAM or more and 4 CPU cores or more
• Note: Not available on mobile devices

What can you build with Prompt API?

The Prompt API allows you to send natural language requests to Gemini Nano. You can build:

• Simple chatbot interfaces like ChatGPT We built one for you to try out.
• Agents for calendar event creation based on event details on web pages
• Contact extraction from web pages
• And many more use cases...

Checking Availability

The Prompt API provides a global LanguageModel object that is available on the window object when Chrome supports the Prompt API. Before using the Prompt API, you need to check if it's available on the user's system:

// Check if LanguageModel is available
if (typeof window.LanguageModel !== 'undefined') {
  const availability = await window.LanguageModel.availability();
  // Handle availability status
} else {
  // Prompt API is not supported
}

Or simply: run this in the console:

await LanguageModel.availability();

The availability will be either:

• available - Model is ready to use
• downloadable - Model can be downloaded
• downloading - Model is currently downloading

If it throws an error or LanguageModel is not defined on the window object, it means the Prompt API is not available.

Creating a Session

Before the model can be downloaded, there must be a user interaction, such as a click, tap, or key press.

To create a session using the window.LanguageModel object:

const session = await LanguageModel.create();

const session = await LanguageModel.create({
  monitor(m) {
    m.addEventListener('downloadprogress', (e) => {
      console.log(`Downloaded ${e.loaded * 100}%`);
    });
  },
});

Session Parameters

You can check available parameters:

// Check available parameters
const params = await window.LanguageModel.params();
// {defaultTopK: 3, maxTopK: 128, defaultTemperature: 1, maxTemperature: 2}

You can also pass initial prompts when creating a session:

const session = await LanguageModel.create({
  initialPrompts: [
    {
      role: 'system',
      content: 'You are a helpful assistant.',
    },
  ],
});

Appending Messages

You can add context to a session without generating a response:

await session.append([
  {
    role: 'user',
    value: 'What is the weather in Tokyo?',
  },
]);

Note: append does not generate a response, it only adds context dynamically to the session.

Generating Responses

To generate a response, you can use either prompt or promptStreaming:

const result = await session.prompt('Write me a poem!');
console.log(result);

const stream = session.promptStreaming('Write me an extra-long poem!');
for await (const chunk of stream) {
  console.log(chunk);
}

Structured Output

To generate structured output, you can send a schema object as a responseConstraint:

const schema = {
  "type": "boolean"
};

const result = await session.prompt('Is this a good idea?', {
  responseConstraint: schema,
});
console.log(JSON.parse(result));
// true

Cleaning Up

To terminate a session, call:

session.destroy();

Additional Resources

For more information, check out the official Chrome Prompt API documentation.