Using a Local Browser

Create a Multi-Step Query

When creating multi-step queries with a local browser, there are two rules to remember:

  1. The act function will produce one or more void actions, while the extract function will produce exactly one extract action. You cannot extract multiple elements from a single extract call.
  2. Sequential actions (that depend on each other) need to be split into separate act and execute blocks. This is because a browser’s state needs to be updated first before a subsequent action could be generated based on the new state. However, independent non-sequential actions can be combined into a single prompt.

These rules are best explained through a few examples.

Example 1: Independent actions through one prompt

If you have multiple actions that do not depend on each other, they can be combined into a single act call. For example, if you need to select two radioboxes that are already visible:

1import { BytebotClient } from '@bytebot/sdk';
2import puppeteer from "puppeteer";
3import "dotenv/config";
4
5async function run() {
6 const browser = await puppeteer.launch();
7 const page = await browser.newPage();
8await page.goto("https://www.ycombinator.com/companies", {
9 waitUntil: "networkidle0",
10});
11
12 const bytebot = new BytebotClient({
13 apiKey: process.env.BYTEBOT_API_KEY,
14 });
15
16 bytebot.puppeteer.startSession();
17
18 const prompt = "Filter by Top Companies and filter by B2B companies.";
19 const actions = await bytebot.puppeteer.act({
20 prompt: prompt,
21 page: page,
22 });
23 console.log("Actions", actions);
24
25 //execute the actions
26 await bytebot.puppeteer.execute({ actions: actions, page: page });
27 await browser.close();
28
29 bytebot.puppeteer.endSession();
30}
31
32run().catch(console.error);

Here, sorting and filtering can be executed in whatever order. Accordingly, they can be bundle into a single execution.

Example 2: Sequential actions through multiple prompts

Instead of clicking on two category filters, imagine clicking on a category filter and a subsequent sub-category filter. These actions need to be done sequentially. Therefore, they need to be executed in separate act calls.

1import { BytebotClient } from '@bytebot/sdk';
2import puppeteer from "puppeteer";
3import "dotenv/config";
4
5async function run() {
6 const browser = await puppeteer.launch();
7 const page = await browser.newPage();
8 await page.goto("https://www.ycombinator.com/companies", {
9 waitUntil: "networkidle0",
10 });
11
12 const bytebot = new BytebotClient({
13 apiKey: process.env.BYTEBOT_API_KEY,
14 });
15
16 bytebot.puppeteer.startSession();
17
18 const prompts = [
19 "Select the Top Companies",
20 "Select B2B Companies",
21 "Select analytics companies from the B2B submenu",
22 ];
23
24 for (const prompt of prompts) {
25 const actions = await bytebot.puppeteer.act({
26 prompt: prompt,
27 page: page
28 });
29 //execute the actions
30 await bytebot.puppeteer.execute(actions, page);
31 }
32 await browser.close();
33
34 bytebot.puppeteer.endSession();
35}
36
37run().catch(console.error);

Example 3: Multiple extractions

Because extract can only create and execute a single extract browser action, a separate function call is needed for every extraction.

1import { BytebotClient } from '@bytebot/sdk';
2import puppeteer from "puppeteer";
3import "dotenv/config";
4
5async function run() {
6 const browser = await puppeteer.launch();
7 const page = await browser.newPage();
8await page.goto("https://developer.chrome.com/", {
9 waitUntil: "networkidle0",
10});
11
12
13 const bytebot = new BytebotClient({
14 apiKey: process.env.BYTEBOT_API_KEY,
15 });
16
17 bytebot.puppeteer.startSession();
18
19 const schemas = [
20 Text("The main CTA button"),
21 Attribute("The link of the main CTA"),
22 ];
23
24 var extractions = [];
25
26 for (const schema of schemas) {
27 const extractActions = await bytebot.puppeteer.extract({
28 sessionId: browser.sessionId,
29 schema: schema,
30 });
31
32 //execute the actions
33 extractions.push(await bytebot.puppeteer.execute(extractActions, page));
34 }
35
36 await browser.close();
37
38 bytebot.puppeteer.endSession();
39}
40
41run().catch(console.error);

Example 4: Mixing actions and extractions

Actions and extractions can be mixed. For example:

1import { BytebotClient } from '@bytebot/sdk';
2import puppeteer from "puppeteer";
3import "dotenv/config";
4
5async function run() {
6 const browser = await puppeteer.launch();
7 const page = await browser.newPage();
8 await page.goto("https://www.ycombinator.com/companies", {
9 waitUntil: "networkidle0",
10 });
11
12 const bytebot = new BytebotClient({
13 apiKey: process.env.BYTEBOT_API_KEY,
14 });
15
16 bytebot.puppeteer.startSession();
17
18 // alter page*
19 const prompt = "Filter by Top Companies and filter by B2B";
20 const actions = await bytebot.puppeteer.act({
21 prompt: prompt,
22 page: page
23 });
24 await bytebot.puppeteer.execute(actions, page);
25
26 // extract content
27 const extractActions = await bytebot.puppeteer.extract({
28 sessionId: browser.sessionId,
29 schema: Text("The first company's name"),
30 });
31 const name = await bytebot.puppeteer.execute(extractActions, page);
32
33 await browser.close();
34
35 bytebot.puppeteer.endSession();
36}
37
38run().catch(console.error);

Next Steps

Learn more about Bytebot’s recommended tips, or discover Bytebot’s BrowserAction types in-depth.