Using the Crawlbot or Bulk API querystring parameter

Crawlbot and the Bulk API serve as controllers for sending pages to the appropriate Diffbot API for processing/extraction. By default, these will be generic requests to the appropriate API and will return the default fields from that API.

For example, Bulk or Crawlbot URLs handed to the Article API will be equivalent to calling http://api.diffbot.com/v3/article?url=[url]

You can adjust individual API fields returned or the parameters of extraction API requests via the Crawlbot or Bulk API querystring field.

For example, to specify certain fields and adjust the timeout value in your Article API requests, enter the following in the querystring field:
Querystring field
This will pass &timeout=10000&fields=title,text,meta in each Article API request.