Homec4science

Improve Conduit performance for custom fields

Authored by epriestley <git@epriestley.com> on Jul 31 2016, 17:19.

Description

Improve Conduit performance for custom fields

Summary:
Ref T11404. Depends on D16350.

Currently, custom fields can issue "N+1" queries in some cases, so querying 100 revisions issues 100 extra queries.

This affects all *.search endpoints for objects with custom fields, and some older endpoints (notably differential.query).

This change bulk loads "normal" custom fields, which gets rid of some of these queries. Instead of loading fields for each object, we build a big list of all fields and load them all at once.

The next change will tackle the remaining inefficient edge queries.

Test Plan:

  • Configured a custom field with normal database storage in Differential.
  • Ran differential.query, looking at custom fields in results for correctness.
  • Ran differential.revision.search, looking at custom fields in results for correctness.
  • In both cases, observed queries drop from 3N to 2N (all the "normal" custom field stuff got bulk loaded).

Reviewers: yelirekim, chad

Reviewed By: chad

Maniphest Tasks: T11404

Differential Revision: https://secure.phabricator.com/D16351

Details

Committed
epriestley <git@epriestley.com>Jul 31 2016, 20:15
Pushed
aubortJan 31 2017, 17:16
Parents
rPH6e57582aff82: Allow `*.search` Conduit API methods to have data bulk-loaded by extensions
Branches
Unknown
Tags
Unknown

Event Timeline

epriestley <git@epriestley.com> committed rPHb8f75f95116e: Improve Conduit performance for custom fields (authored by epriestley <git@epriestley.com>).Jul 31 2016, 20:15