SearchIndexer..::..BuildIndex Method
Assembly: AdvantageCSP.Keyoti.SiteIndexer (in AdvantageCSP.Keyoti.SiteIndexer.dll)
Syntax
public SearchIndexPluginIndexResult BuildIndex()
Return Value
The outcome of the run. Exceptions are caught and reported through Error()()()() rather than allowed to escape.Remarks
The host calls this once per plug-in instance, and one instance exists per domain per run. The sequence is: resolve the start URL with GetStartUrl(), hand it to DocumentIndex.Import wrapped in a WebsiteBasedIndexableSourceRecord built from the base's PathsToExclude/PathsToInclude, close the index, then reopen it to sweep out stale documents and optimize.
Everything interesting happens underneath Import — it blocks until the crawl finishes, raising the dispatcher events handled in Dispatcher_Action(Object, ActionEventArgs). By the time it returns, ProcessedLinks holds every URL this pass kept, so any document already in the index whose URI is absent from that set has disappeared from the site and is removed. That sweep is gated on ScrapeComplete precisely because a crawl that never reached ImportFinished()()()() would have a partial ProcessedLinks and would delete live content.
BreakLock() in the finally is not optional. The base implementation deletes <IndexDirectory>\lock unconditionally, whereas the host refuses to start a domain whose lock file is less than two days old — skip the call and a single crash leaves the domain unindexable until the lock ages out.
Two things a custom indexer should do differently. GetStartUrl() returns String when the domain has no primary URL configured, and it is fed straight into the source record here; guard it and return a failed result instead. And clearing ProcessedLinks to null at the end makes any later access throw and a second BuildIndex() call impossible — that only survives because the host calls this method exactly once.
Examples
public override SearchIndexPluginIndexResult BuildIndex() { SearchIndexPluginIndexResult result = new SearchIndexPluginIndexResult() { Success = false }; DocumentIndex documentIndex = new DocumentIndex(Configuration); try { string startUrl = GetStartUrl(); if (string.IsNullOrEmpty(startUrl)) { result.Message = "Search domain has no primary url; nothing to crawl."; result.Error = new Exception(result.Message); return result; } documentIndex.Import(new WebsiteBasedIndexableSourceRecord(startUrl, PathsToExclude, PathsToInclude)); documentIndex.Flush(); documentIndex.Optimize(); documentIndex.Flush(); documentIndex.Close(); result.Success = true; result.Message = "Success"; } catch (Exception ex) { result.Message = "Failed"; result.Error = ex; SearchHelper.WriteToErrorTable(Configuration, "BuildIndex", "Failed Import", ex); } finally { if (!documentIndex.IsClosed) documentIndex.Close(); BreakLock(); } return result; }