Skip to content
Contact Support
  • There are no suggestions because the search field is empty.

Controlled Crawls Explained

Controlled Crawls let you filter which pages are included or excluded, so you get exactly what you need and skip the rest.

How to Use Controlled Crawl Filters

After entering your crawl details, use these filters in the Advanced Crawl Options section:

Require (One Keyword)

  • Only include pages that contain this word.

  • Example: Only pages with "blog" in the URL.

Exclude (One or Multiple Keywords)

  • Leave out pages with these words in the URL.

  • Example: Exclude "contact" pages.


Why Use Filters?

  • Capture only what matters

  • Avoid extra, irrelevant pages

  • Save time by reducing page count