Scripting & Syntax Guide

This document details how to use JavaScript syntax within task scripts for logical control, data processing, and variable passing.

1. Overview

The browser extension features a built-in secure JavaScript Sandbox Environment for executing dynamic logic within tasks.

Key application scenarios include:

1. Data Interpolation: Using state variables in input fields or URLs.
2. Conditional Logic: Determining whether to execute a step (condition) or whether to loop (while).
3. Data Processing: Cleaning, calculating, or restructuring scraped data in evaluate steps.

⚠️ Important: All scripts and expressions run in a Web Worker sandbox isolated from the page. They cannot access the DOM (i.e., browser objects like window, document, console.log are unavailable) and cannot make network requests (fetch, XMLHttpRequest).

---

2. The state Object

state is the global object that persists throughout the entire task lifecycle, used to store all data.

• Initial State: Determined by the task's initialState.
• Read: Use state.variableName directly in scripts.
• Write:
  • Write the result of an Action to state via outputVar.
  • Modify properties of state directly within an evaluate script (e.g., state.count = 1).

---

3. Template Syntax

In most fields that accept string parameters (such as text in type, selector in click, url in waitForUrl), you can use the double curly brace syntax to insert variable values.

• Syntax: {{state.variablePath}}
• Examples:
  • Assuming state is { "user": { "name": "Alice", "id": 123 }, "baseUrl": "https://example.com" }
  • "{{state.user.name}}" -> "Alice"
  • "{{state.baseUrl}}/profile/{{state.user.id}}" -> "https://example.com/profile/123"

Note: Template syntax only supports simple property access. Complex JavaScript expressions (e.g., {{state.a + state.b}}) are not supported. If calculation is required, please use the evaluate step.

---

4. Boolean Expressions

Used in condition (step execution condition) and while (loop condition) in loop Actions.

• Requirement: Must be a single-line JavaScript expression that returns a boolean value (true or false).
• Context: Can access the state object directly.
• Examples:
  • state.count < 5 (Numeric comparison)
  • state.status === "success" (String comparison)
  • state.items.length > 0 (Array length)
  • state.user && state.user.isLogin (Logical operations)
  • !state.isFinished (Negation)

---

5. Evaluate Script

Used in the evaluate Action to execute a complete piece of JavaScript code for data processing.

• Features:
  1. Handle complex logic.
  2. Clean data (e.g., string slicing, regex replacement).
  3. Mathematical calculations.
  4. Array operations (filter, map, sort).
  5. Update state.
• Variable Access:
  • state: (Read/Write) The global state object. Modifying properties of this object directly updates the task state.
  • input: (Read-only) If inputVar is configured in the evaluate step, the value of that variable is passed in as input.
• Return Value:
  • You can use the return statement to return a value.
  • If outputVar is configured in the JSON, the return value of the script will be written to state[outputVar].

5.1 Scripting Cookbook

The following examples demonstrate common logic patterns used in evaluate actions.

1. Basic State Updates

Increment Counter: Useful for paging loops.

// Approach A: Modify state directly
state.page = (state.page || 0) + 1;

// Approach B: Update outputVar via return value (Recommended)
// JSON: { "action": "evaluate", "script": "return input + 1;", "inputVar": "page", "outputVar": "page" }
return input + 1;

2. Array Operations

Merge Scrape Results: Combine results from multiple scrape actions into a master array.

// Initialize array (if it doesn't exist)
if (!state.allProducts) {
    state.allProducts = [];
}
// Merge current page data (state.currentBatch)
if (state.currentBatch && state.currentBatch.length > 0) {
    state.allProducts = state.allProducts.concat(state.currentBatch);
}
// Return merged length for debugging
return state.allProducts.length;

Deduplication: Remove duplicates based on a specific field (e.g., id or link).

var uniqueMap = {};
var uniqueList = [];

// Assuming state.data is an array containing duplicates
state.data.forEach(function(item) {
    if (!uniqueMap[item.link]) { // Use link as the unique key
        uniqueMap[item.link] = true;
        uniqueList.push(item);
    }
});

state.data = uniqueList; // Update state

Find Specific Element: Find an item that matches a condition from a list.

// Find the first product with price less than 100
var cheapItem = state.products.filter(function(p) {
    return p.price < 100;
})[0];

state.targetProduct = cheapItem || null;

3. String & Regex

Extract Numbers (Price Processing): Extract the number 1234.50 from a string like "$1,234.50".

// Assuming input is "$1,234.50"
if (!input) return 0;
// Remove all non-digit and non-dot characters
var cleanStr = input.replace(/[^\d.]/g, ''); 
return parseFloat(cleanStr);

Generate Random Filename/ID: Used for unique identifiers when saving data.

// Generates "data_20231201_8392.json"
var dateStr = new Date().toISOString().slice(0, 10).replace(/-/g, '');
var randomSuffix = Math.floor(Math.random() * 10000);
return 'data_' + dateStr + '_' + randomSuffix + '.json';

URL Parameter Extraction: Extract a specific parameter value from the current URL.

// Assuming input is a full URL "https://site.com/view?id=123&type=a"
var match = input.match(/[?&]id=([^&]+)/);
return match ? match[1] : null;

4. Date & Time

Calculate Relative Date: For example, calculate the date "3 days ago" for a query.

var d = new Date();
d.setDate(d.getDate() - 3); // Subtract 3 days
// Format as YYYY-MM-DD
var year = d.getFullYear();
var month = ('0' + (d.getMonth() + 1)).slice(-2);
var day = ('0' + d.getDate()).slice(-2);
return year + '-' + month + '-' + day;

Randomize Wait Time: Generate a random wait time (milliseconds) for a sleep step.

// Generate a number between 2000 and 5000
var min = 2000;
var max = 5000;
state.randomDelay = Math.floor(Math.random() * (max - min + 1)) + min;

5. Complex Data Cleaning

Clean Scrape Results: Iterate through a scraped object array and fix field formats.

// state.rawItems is the raw output from scrape
state.cleanItems = state.rawItems.map(function(item) {
    return {
        title: item.title.trim(),
        // Convert 'In Stock' -> true, 'Out of Stock' -> false
        inStock: item.availability.toLowerCase().indexOf('in stock') !== -1,
        // Convert relative path to absolute path
        url: item.url.startsWith('http') ? item.url : 'https://example.com' + item.url,
        scrapedAt: Date.now()
    };
});

---

6. Supported JavaScript Syntax

The sandbox environment is based on a JavaScript interpreter and supports most ES5 and some ES6 syntax features.

Supported

• Basic Types: Number, String, Boolean, Array, Object, null, undefined.
• Control Flow: if/else, switch, for, while, do/while.
• Functions: function declarations, anonymous functions.
• Array Methods: map, filter, reduce, forEach, indexOf, push, pop, slice, splice, etc.
• String Methods: split, replace, substring, trim, toLowerCase, etc.
• Math Object: Math.random(), Math.floor(), Math.max(), etc.
• JSON: JSON.parse(), JSON.stringify().
• Date: new Date(), Date.now().
• ES6: const, let, Arrow functions => (Dependent on interpreter version; using var and function is recommended for maximum compatibility).

Unsupported/Restricted

• ❌ DOM API: window, document, alert, console (console.log may work in some environments but should not be relied upon), localStorage.
• ❌ Async Operations: Promise, async/await, setTimeout, setInterval (evaluate must return results synchronously).
• ❌ Network Requests: fetch, XMLHttpRequest.
• ❌ Module System: import, require.

---

7. Best Practices

1. Keep it Simple: evaluate scripts should focus purely on data processing.
2. Avoid Long-Running Operations: Scripts run on the main thread or Worker; complex long loops may cause the task to freeze.
3. Null Checks: Check if the parent object exists before accessing deep object properties to avoid errors.
   • ✅ if (state.user && state.user.name) ...
   • ❌ if (state.user.name) ... (Errors if user is null)
4. Use outputVar: Prefer updating state via return values and outputVar rather than modifying the deep structure of state directly. This makes the logic clearer.