/otwarchive

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

AO3-5280 - Make the scheduled reindexing job index ExternalWorks and Series. #3202

Open
wants to merge 2 commits into
from
+27 −20

Conversation

Reviewers

@houndci-bot houndci-bot

Assignees
No one assigned
Labels
Awaiting review Priority: Broken on Test (High)
Projects
None yet
Milestone
No milestone
4 participants
@tickinginstant @houndci-bot @redsummernight @sarken
@tickinginstant
Contributor

tickinginstant commented Dec 10, 2017

Issue

https://otwarchive.atlassian.net/browse/AO3-5280

Purpose

This pull request does three things to try to fix the indexing of ExternalWorks and Series:

  1. Add Series and ExternalWork to the list of classes indexed by ScheduledReindexJob.
  2. Refactor the logic of IndexQueue and IndexSubqueue so that IndexSubqueue won't handle ExternalWorks or Series (since it would cause IndexSubqueue.klass to fail with an error).
  3. Remove the unnecessary id: "external_work-#{id}" field (and similar) from bookmarkable_json, since it apparently causes ES to error when re-indexing a document. (Credit to WendyBeth for figuring out this issue and the fix in #3122.)

Testing

Testing instructions are in the bug report.

tickinginstant added some commits Dec 10, 2017

@tickinginstant
AO3-5280 Index Series and ExternalWorks in queue.
064c7ec
@tickinginstant
AO3-5280 Cut bookmarkable id field that breaks ES.

Some checks haven’t completed yet

1 pending and 3 successful checks
hound — Hound is busy reviewing changes...
Scrutinizer — 1 new issues
Details
codeclimate — 1 fixed issue
Details
continuous-integration/travis-ci/pr — The Travis CI build passed
Details
869d767
app/models/indexing/index_queue.rb
+ klass = klass.classify # convert to the uppercase version
+
+ ids.in_groups_of(BATCH_SIZE, false).each_with_index do |id_batch, i|
+ if $rollout.active?(:start_new_indexing)
@houndci-bot

houndci-bot Dec 10, 2017

Do not introduce global variables.

app/models/indexing/index_queue.rb
+ end
+
+ # After the ES6 upgrade, delete this whole unless block.
+ unless $rollout.active?(:stop_old_indexing)
@houndci-bot

houndci-bot Dec 10, 2017

Use next to skip iteration.
Do not introduce global variables.

spec/models/index_queue_spec.rb
+
+ # Once the upgrade is complete, this check can be deleted.
+ expect(IndexSubqueue).to receive(:create_and_enqueue).exactly(
+ $rollout.active?(:stop_old_indexing) ? 0 : 1
@houndci-bot

houndci-bot Dec 10, 2017

Do not introduce global variables.

@redsummernight redsummernight added the Awaiting review label Dec 10, 2017

@sarken sarken added the Priority: Broken on Test (High) label Dec 10, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment