Create an Action

In this guide, you will create and execute a simple action on a local browser. If you want to create an action on a Bytebot-managed remote instance instead, click here.

Bytebot’s Act Function

With Bytebot’s act function, you can convert a natural language prompt into an array of BrowserActions. To accomplish this, use the .act(prompt: string, page: Page, options?: Object) function. The function is async and accepts the following optional configurations:

AttributeRequiredDescription
parametersOptionalAn Object where each string key is removed from the prompt and anonymized before leveraging AI models. The key is swapped in after against the Bytebot-managed placeholder. This option can be used to protect sensitive data.

To begin, start by writing a concise and clear prompt inside an async block:

1const prompt = "click on the Get Started button";

Then, call act, passing in the prompt text and the page:

1import { BytebotClient } from '@bytebot/sdk';
2import puppeteer from "puppeteer";
3import "dotenv/config";
4
5async function run() {
6  const browser = await puppeteer.launch();
7  const page = await browser.newPage();
8  await page.goto("https://developer.chrome.com/", {
9    waitUntil: "networkidle0",
10  });
11
12  const bytebot = new BytebotClient({
13    apiKey: process.env.BYTEBOT_API_KEY,
14  });
15
16  bytebot.puppeteer.startSession();
17
18  const prompt = "click on the Get Started button";
19  const actions = await bytebot.puppeteer.act({
20    prompt: prompt,
21    page: page,
22  });
23  console.log("Actions", actions);
24
25  bytebot.puppeteer.endSession();
26}
27
28run().catch(console.error);

If you run this code, you should get a printout in your console similar to:

$Actions [
>  {
>    type: 'Click',
>    xpath: '/html/body/section/section/main/devsite-content/article/div[3]/section[1]/div/header/div[2]/a'
>  }
>]

The actions array is a set of BrowserAction objects, each with a type (e.g., “Click” or “Hover”) and an xpath (a reference to a specific DOM element). The pages array is akin to the open tabs on the browser, each with a pageId and url.

Bytebot’s Execute Function

Now that we have an array of BrowserAction objects, we need to manually execute them. You can accomplish that using the .execute(actions: BrowserAction[], page: Page) function:

1import { BytebotClient } from '@bytebot/sdk';
2import puppeteer from "puppeteer";
3import "dotenv/config";
4
5async function run() {
6  const browser = await puppeteer.launch();
7  const page = await browser.newPage();
8  await page.goto("https://developer.chrome.com/", {
9    waitUntil: "networkidle0",
10  });
11
12  const bytebot = new BytebotClient({
13    apiKey: process.env.BYTEBOT_API_KEY,
14  });
15
16  bytebot.puppeteer.startSession();
17
18  const prompt = "click on the Get Started button";
19  const actions = await bytebot.puppeteer.act({
20    prompt: prompt,
21    page: page,
22  });
23  console.log("Actions", actions);
24
25  //execute the actions
26  await bytebot.puppeteer.execute({ actions: actions, page: page });
27  await browser.close();
28
29  bytebot.puppeteer.endSession();
30}
31
32run().catch(console.error);

Now, the previously generated BrowserActions have been executed on the local instance.

Use Parameters to Handle Sensitive Content

Because Bytebot uses generally available AI models, it is important to safeguard any sensitive content from being shared with them. This is made possible by Bytebot’s parameters option. Parameters allow for a query to be run with placeholders, with the sensitive data substituted after the browser execution.

To use parameters, just include it as an attribute to the options object:

1import { BytebotClient } from '@bytebot/sdk';
2import puppeteer from "puppeteer";
3import "dotenv/config";
4
5async function run() {
6  const browser = await puppeteer.launch();
7  const page = await browser.newPage();
8  await page.goto("https://dribbble.com/session/new", {
9    waitUntil: "networkidle0",
10  });
11
12  const bytebot = new BytebotClient({
13    apiKey: process.env.BYTEBOT_API_KEY,
14  });
15
16  bytebot.puppeteer.startSession();
17
18  const prompt = "Log with email EMAIL and password PASSWORD";
19
20  const parameters = {
21    EMAIL: process.env.USER_EMAIL,
22    PASSWORD: process.env.USER_PASSWORD,
23  };
24
25  const actions = await bytebot.puppeteer.act({
26    prompt: prompt,
27    page: page,
28    options: {
29      parameters: parameters,
30    },
31  });
32  console.log("Actions", actions);
33
34  bytebot.puppeteer.endSession();
35}
36
37run().catch(console.error);

Now, the browser actions will be generated without sharing sensitive data (in this case, email and password).