Using Zapier with Crawlbot or Bulk API jobs

You can integrate your Crawlbot or Bulk API notifications with the fantastic Zapier service, which allows you to take various actions in other applications when your crawl or bulk job completes.

To create a Diffbot ‘zap,’ choose Diffbot as your Trigger app and select “New Crawl Notification” or “New Bulk Job Notification.”


After you select an action app, Zapier will provide you a unique webhook URL. Paste this URL into the Webhook field of the Crawlbot/Bulk API interface, or via API using the notifyWebhook parameter, and Zapier will be notified whenever your job or crawl round completes. You can reuse your Zapier webhook URL for all of your crawl or bulk jobs.

Some things you can do with your webhook notification:

  • Save your JSON output directly to a file (Dropbox, OneDrive, etc.)
  • Create a ticket in your favorite ticket-tracking tool with the crawl information
  • Receive notifications over IM, Hipchat, Twitter, etc.

Let us know if you create a great zap!