Joel's SharePoint Architecture Blog

Crawl Rules vs Crawler Impact Rules

View this post on Instagram

#spthingaday 069 – SharePoint Search. Crawl Rules vs Crawler Impact Rules. #SharePoint, #SharePointSearch.. SharePoint Search has two similar sounding configurable options that are easy to confuse. They do two very different jobs. Crawl Rules allows you configure how SharePoint crawls within a Content Source. You use them to tell SharePoint what, and as whom to crawl. For instance "when you crawl this path, use these credentials" …or "exclude this path from crawling." Crawler Impact Rules tells SharePoint how hard to crawl a path within a Content Source. For example, if you've set up SharePoint to index itself efficiently, you can create a rule that states: "crawl this path as fast as you can, with up to 8 documents at a time" However, if you're trying to include in your index content from a rickety old SharePoint farm in your shiny, new one, you might want to create a more limiting Crawler Impact Rule: e.g. "crawl this path with at most one document at a time, and leave 60 seconds between each one." #microsoft #microsoftoffice #microsoftsharepoint #microsofttraining #officetraining #freetraining #trainingvideos #contentmanagement #enterprisecontentmanagement

A post shared by JFDI Consulting #spthingaday (@jfdiconsulting) on Feb 15, 2018 at 6:25am PST

SharePoint Search has two similar sounding configurable options that are easy to confuse.

They do two very different jobs.

Crawl Rules allows you configure how SharePoint crawls within a Content Source.

You use them to tell SharePoint what, and as whom to crawl.

For instance “when you crawl this path, use these credentials” …or “exclude this path from crawling.” Crawler Impact Rules tells SharePoint how hard to crawl a path within a Content Source.

For example, if you’ve set up SharePoint to index itself efficiently, you can create a rule that states: “crawl this path as fast as you can, with up to 8 documents at a time”

However, if you’re trying to include in your index content from a rickety old SharePoint farm in your shiny, new one, you might want to create a more limiting Crawler Impact Rule:

e.g. “crawl this path with at most one document at a time, and leave 60 seconds between each one.”

Configuring Content Sources

View this post on Instagram

#spthingaday 068 – SharePoint Search. Configuring Content Sources. #SharePoint, #SharePointSearch.. When you create a Content Source, you can specify the kind of thing you want SharePoint to crawl. Each Content Source has a type, and can be one of: SharePoint Content User Profile Content File Shares Exchange Public Folders Web Content Lotus Notes Custom – which requires 3rd party or custom developed code. You can have multiple paths per Content Source. You can also specify a Full Crawl and Incremental Crawl schedule. For SharePoint-based Content Sources only, you can enable Continuous Crawl. This one checkbox allows SharePoint to launch overlapping Incremental Crawls – by default 15 minutes apart. You can bring this down to 5 minutes apart using PowerShell. The smaller the interval, the fresher the content in your index. #microsoft #microsoftoffice #microsoftsharepoint #microsofttraining #officetraining #freetraining #trainingvideos #contentmanagement #enterprisecontentmanagement

A post shared by JFDI Consulting #spthingaday (@jfdiconsulting) on Feb 15, 2018 at 1:20am PST

When you create a Content Source, you can specify the kind of thing you want SharePoint to crawl.

Each Content Source has a type, and can be one of:

SharePoint Content

User Profile Content

File Shares

Exchange Public Folders

Web Content

Lotus Notes

Custom – which requires 3rd party or custom developed code.

You can have multiple paths per Content Source.

You can also specify a Full Crawl and Incremental Crawl schedule.

For SharePoint-based Content Sources only, you can enable Continuous Crawl.

This one checkbox allows SharePoint to launch overlapping Incremental Crawls – by default 15 minutes apart.

You can bring this down to 5 minutes apart using PowerShell.

The smaller the interval, the fresher the content in your index.