Input Schema

Scrapeless reads the INPUT_SCHEMA.json file located in the root directory of the project and automatically generates a user interface based on this file for entering the parameters required by the Actor. This simplifies configuration for developers and makes input options more accessible to non-developers.

Main purposes:

Input Validation Used to ensure that data passed to the Actor conforms to the predefined structure and validation rules, enhancing stability and reliability during execution.
UI Generation The platform automatically generates a graphical configuration interface based on the input schema, making it easier for users to fill out and manage input parameters.
Integration Support Scrapeless can automatically generate invocation code and connectors based on the input schema, simplifying how external systems call and integrate with the Actor.

Input Schema defines the input parameters for an Actor. It is a JSON object composed of various field types supported by the Scrapeless platform. Based on the Input Schema, Scrapeless automatically parses the parameters and generates a corresponding user interface.

Below is a UI generated from an Input Schema.

The definition of the Input Schema is as follows:

{
  "title": "Google Trends Input",
  "type": "object",
  "required": [
    "q",
    "data_type"
  ],
  "properties": {
    "q": {
      "title": "Search Query",
      "description": "Parameter defines the query or queries you want to search. You can use anything that you would use in a regular Google Trends search. The maximum number of queries per search is 5 (this only applies to `interest_over_time` and `compared_breakdown_by_region` data_type, other types of data will only accept 1 query per search).",
      "type": "string",
      "default": "Mercedes-Benz,BMW X5"
    },
    "data_type": {
      "title": "Data Type",
      "description": "The supported types are: `autocomplete`,`interest_over_time`,`compared_breakdown_by_region`,`interest_by_subregion`,`related_queries`,`related_topics`.",
      "type": "string",
      "default": "interest_over_time",
      "enum": [
        {
          "label": "Auto complete",
          "value": "autocomplete"
        },
        {
          "label": "Interest over time",
          "value": "interest_over_time"
        },
        {
          "label": "Compared breakdown by region",
          "value": "compared_breakdown_by_region"
        },
        {
          "label": "Interest by region",
          "value": "interest_by_subregion"
        },
        {
          "label": "Related queries",
          "value": "related_queries"
        },
        {
          "label": "Related topics",
          "value": "related_topics"
        }
      ]
    },
    "date": {
      "title": "Date",
      "description": "The supported dates are: `now 1-H`, `now 4-H`, `now 1-d`, `now 7-d`, `today 1-m`, `today 3-m`, `today 12-m`, `today 5-y`, `all`.\n\nYou can also pass custom values:\n\nDates from 2004 to present: `yyyy-mm-dd yyyy-mm-dd` (e.g. `2021-10-15 2022-05-25`)\nDates with hours within a week range: `yyyy-mm-ddThh yyyy-mm-ddThh` (e.g. `2022-05-19T10 2022-05-24T22`). Hours will be calculated depending on the tz (time zone) parameter.",
      "type": "string",
      "default": "today 1-m"
    },
    "hl": {
      "title": "Language",
      "description": "Parameter defines the language to use for the Google Trends search. It's a two-letter language code. (e.g., `en` for English, `es` for Spanish, or `fr` for French).",
      "type": "string",
      "default": "en"
    },
    "tz": {
      "title": "Time Zone",
      "description": "time zone offset. default is `420`.",
      "type": "string",
      "default": "420"
    }
// ...
 
  }
}

The input object data transmitted in the UI interface is as follows:

{
    "q": "Mercedes-Benz,BMW X5",
    "data_type": "interest_over_time",
    "date": "today 1-m",
    "hl": "en",
    "tz": "420",
    "geo": "",
    "cat": "",
    "property": "",
    "is_return_raw_html": "1"
}

Next, let’s learn more about the relevant specifications of Input Schema.

Specification

Structure

{
  "title": "Title",
  "type": "object",
  "required": [], // Which fields are required
  "properties": {} // define the input fields here
}

Property	Type	Required	Description
`title`	String	Yes	Any text to describe your input schema.
`type`	String	Yes	This is a fixed value, always “object”.
`required`	Sring	Yes	An array of strings, each string is the name of a required property.
`properties`	String	Yes	An object, the keys are the names of the properties, the values are the properties definitions.

Fields

Each field entered needs to be defined as properties. Fields may have integer, string, array, boolean, object and other types. Details are as follows:

Property	Value	Required	Description
`type`	One of `integer`, `string`, `array`, `boolean`, `object`	Yes	Field type constraints cannot be mixed.
`title`	String	Yes	Field title
`description`	String	Yes	Field descriptions are displayed as help text in the UI. You can use Markdown format to write these descriptions.
`default`	Consistent with `type`	No	Field default value
`unit`	String	No	Field display unit
`minimum`	Number	No	Field minimum value
`maximum`	Number	No	Field maximum value
`enum`	Array	No	Field enumeration values (enum options)

Render Examples

String

Input Example: We defined a standard input field with the following properties: type is string, title is Username, description is Username is the username of the user, default value is I'm a default value, and the maximum length is 20.

{
  "title": "Test Input",
  "type": "object",
  "required": [
    "username"
  ],
  "properties": {
    "username": {
      "title": "Username",
      "description": "Username is the username of the user",
      "type": "string",
      "default": "I'm a default value",
       "maximum": 20
    }
  }
}

Rendering result:

Country selection example: In this example, we define a country selection input field. Its type is string, the default value is china, and an enumeration of possible values is provided.

{
  "title": "Test Input Select",
  "type": "object",
  "required": [
    "country"
  ],
  "properties": {
    "country": {
      "title": "Country",
      "type": "string",
      "description": "Country selection",
      "default": "china",
      "enum": [
        {
          "label": "China",
          "value": "china"
        },
        {
          "label": "United States",
          "value": "usa"
        },
        {
          "label": "United Kingdom",
          "value": "uk"
        }
      ]
    }
  }
}

Rendering result:

Integer

Example: In the schema below, the input type is constrained to integer, with both maximum and minimum values defined.

{
  "title": "Test Input Number",
  "type": "object",
  "required": [
    "age"
  ],
  "properties": {
    "age": {
      "title": "Age",
      "type": "integer",
      "description": "Age is the age of the user",
      "default": 18,
      "minimum": 12,
      "maximum": 100
    }
  }
}

Rendering result:

Boolean

Example: In the example below, the input type is constrained to boolean, and its default value is set to true.

{
  "title": "Test Input Boolean",
  "type": "boolean",
  "required": [
    "print_log"
  ],
  "properties": {
    "print_log": {
      "title": "Print Log",
      "type": "boolean",
      "description": "Print Log is the print log of the user",
      "default": true
    }
  }
}

Rendering result:

Array

Example: In the example below, the input type is constrained to array, and a default value is provided.

{
  "title": "Test Input Array",
  "type": "array",
  "required": [
    "user_tags"
  ],
  "properties": {
    "user_tags": {
      "title": "User Tags",
      "type": "array",
      "description": "User Tags is the tags of the user",
      "default": [
        "user",
        "admin"
      ]
    }
  }
}

Rendering result:

Object

Example: In the example below, the input type is constrained to object, and a default value is provided.

{
  "title": "Test Input Object",
  "type": "object",
  "required": [
    "user_info"
  ],
  "properties": {
    "user_info": {
      "title": "User Info",
      "type": "object",
      "description": "User Info is the info of the user",
      "default": {
        "name": "John Doe",
        "age": 20,
        "email": "john.doe@example.com"
      }
    }
  }
}

Rendering result:

Fundamentals