Reviewing Index Results and Logs

This tab lists the pages successfully indexed on the last run. It is the first place to look after a build.

• If a page you expected is absent, the crawler never reached it or was told to skip it. Check link depth and the exclude-path list.
• If pages you did not expect are present, check the exclude-path list and, where applicable, the external domain list.
• If the list is much shorter than your site, the build stopped early — check the Logs tab.

The Clear button in the panel heading discards this record. BWA consults it at the start of each build and skips any page whose content has not changed since it was last indexed, so with the record gone the next build reindexes every page it reaches. Use it after any change that should affect pages already indexed — a stop list edit, a category change, or a weighting change.

Note

Clear affects only the domain and language you are currently working in, and only this record of what was processed. It does not delete the index, and it does not clear the Redirects or Not Found reports. Search continues to work normally in the meantime; the next build simply does more work than it otherwise would. On a site with several domains or languages, clear each one you want fully rebuilt.

A stop list edit clears this record for you, because a changed stop list can only be applied by reindexing every page.

Lists URLs that redirected somewhere else during the crawl. A handful is normal on any established site. A large number is worth attention: every redirect consumes a step of the crawler's link depth, so a site with layers of redirects may not reach its deeper content within the configured depth.

Lists links the crawler followed that did not resolve. Because the crawler walks your site the way a visitor does, this is a broken-link report for the whole site — useful well beyond search itself.

Results are grouped by the page containing the broken link:

The panel heading shows when the report was produced, and Download exports the list so it can be circulated to whoever maintains the content.

The Logs tab lists the log files written by past builds. Select one to read it in the viewer beneath.

Logs are only written when Enable Logging is switched on. If you are investigating a build problem and the log is empty or stale, switch logging on and rebuild before drawing conclusions.

Note
Logging is verbose. Leave it on while tuning a site, and switch it off once builds are stable, to limit disk usage.

When search is not returning what you expect, work in this order:

1. Is the page in Index Results? If yes, the problem is with the query or with result display, not with indexing.
2. If it is absent, is it reachable by following links from the home page within the configured depth?
3. Does any entry on the exclude-path list appear anywhere in its URL?
4. Does the page appear under Redirects or Not Found?
5. Does the page require a signed-in session? See HTTP Headers and Cookies.
6. If the page is indexed but a specific word is not findable, check the stop list, then check whether that part of the page sits inside an ignore block.
7. Read the build log for that URL.