2018-01-05 Crawlbot API: Added the useCanonical argument to allow disabling of canonical URL deduplication on specific crawls. See more.