Can multiple Diffbot Extract APIs be used in a single crawl?

No. Only one Extract API may be used at a time with a single crawl.

Suggest Edits

If you wish to process multiple types of pages through separate APIs, your options are:

Use the Analyze API

The Analyze API will automatically determine the page-type of each page crawled, and structure the data from supported pages. This content can be filtered using DQL API, or the JSON downloaded in full and filtered using the type field.

Set up Multiple Crawls

To explicitly use multiple APIs for crawling a single site, you'll need to set-up multiple crawls, each using an independent API. You can use Crawl's multiple mechanisms for controlling/narrowing your crawls to ensure that each separate crawl job only processes the right type of pages.

Updated almost 2 years ago