2016-06-17

  • Added support for custom headers to the Crawlbot and Bulk Job interfaces. Read more.
  • Added beta field inferredCategory_beta to the Product API, which provides an automatically-determined category for any extracted product.
  • The Bulk Service will no longer normalize URLs before processing them—all pages will be sent to the specified Diffbot API as-is.
  • Various improvements to performance and quality of Diffbot's rendering engine.
  • Corrected an issue in Custom APIs where a replacement rule would errantly trim blank spaces.