Homec4science

Allow index extensions to skip indexing if the object has not changed

Authored by epriestley <git@epriestley.com> on Dec 21 2015, 20:04.

Description

Allow index extensions to skip indexing if the object has not changed

Summary:
Fixes T9890. This allows IndexExtensions to emit an object version.

Before we build indexes, we check if the indexed version is the same as the current version. If it is, we just don't call that extension.

T9890 has a case where this is useful: a script went crazy and posted thousands of comments to a single task.

Without versioning, that results in the same comments being indexed over and over again. With versioning, most of the queue could just exit without doing any work.

Test Plan:

  • Added a sleep(1) to the actual indexing, used bin/search index --background to queue up a lot of tasks, ran them with bin/phd debug task, saw them complete very quickly with only one actual index operation performed.
  • Used bin/search index --trace and bin/search index --trace --background to observe the behavior of queries against the index version store, which looked sensible.
  • Made comments/transactions, saw versions update.
  • Used bin/remove destroy, verified index versions were purged.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T9890

Differential Revision: https://secure.phabricator.com/D14845

Details

Committed
epriestley <git@epriestley.com>Dec 22 2015, 02:27
Pushed
aubortJan 31 2017, 17:16
Parents
rPH23c42486e457: Rename "SearchEngine" to "FulltextStorageEngine"
Branches
Unknown
Tags
Unknown

Event Timeline

epriestley <git@epriestley.com> committed rPHa761f733848b: Allow index extensions to skip indexing if the object has not changed (authored by epriestley <git@epriestley.com>).Dec 22 2015, 02:27