Feature Requests & Ideas

For bug reports and questions, please use the in-app chat or support@mabl.com.

BlackList for LinkCrawler & collapsing identical fails from Anchor TAGS

The link crawler today will post a 404 error when mabl reaches a page like Google or Facebook that have bot blockers in place. The links are then marked broken and added to the broken link report which make it hard to go through. 

Can a blacklist be created that stops mabl from crawling certain domains? Also could the ability be added to collapse identical failures that can arise from mabl crawling anchor tags?

Thanks! 

  • Guest
  • Apr 18 2019
  • For Consideration
  • Attach files
  • Harry Wigman commented
    30 Apr 16:19

    I'd like it if you could set a white-list as well. Say I'm testing example.com, I add that URL and it only includes pages from that site.

Copyright © 2019 mabl Inc. All rights reserved.