/otwarchive

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

AO3-5209 Don't change the appearance of non-work & bookmark filters #3117

Merged
merged 3 commits into from Oct 23, 2017

Conversation

Reviewers

@houndci-bot houndci-bot

@elzj elzj

Assignees
No one assigned
Labels
Awaiting review Priority: Broken on Test (High)
Projects
None yet
Milestone
No milestone
3 participants
@sarken @elzj @houndci-bot
@sarken
Owner

sarken commented Oct 23, 2017

Issue

https://otwarchive.atlassian.net/browse/AO3-5209

Purpose

Restores the old HTML, JavaScript, and CSS for the Elasticsearch 0.90 filters. Reworks the CSS for the new filters so it doesn't negatively affect the styling of the filters on the Collections index.

Testing

Compare the work and bookmark filters when using Elasticsearch 0.90 on staging to those on production, and compare the collections filters to those on production.

sarken added some commits Oct 22, 2017

@sarken
AO3-5209 Use old JavaScript and views for old Elasticsearch filters (… 
…styling fixes to come)
a8cbb42
@sarken
AO3-5209 Restyle old filters
b470687
@sarken
AO3-5209 Ensure new work and bookmark filter styling does not conflic… 
…t with exisiting collection filter styles

All checks have passed

4 successful checks
Scrutinizer — No new issues
Details
codeclimate — All good!
Details
continuous-integration/travis-ci/pr — The Travis CI build passed
Details
hound — 4 violations found.
36c3bf6

@sarken sarken added the Awaiting review label Oct 23, 2017

public/javascripts/filters_old.min.js
@@ -0,0 +1 @@
+function showFilters(){var e=$j("dd.tags");e.each(function(e,r){var t=$j(r).find("input"),i=$j(r),n=$j("."+i.attr("id")+"_open"),s=$j("."+i.attr("id")+"_close");t.each(function(e,t){$j(t).is(":checked")&&($j(r).show(),$j(n).hide(),$j(s).show())})})}function setupNarrowScreenFilters(){var e=$j("form.old-filters"),r=$j("#outer"),t=$j("#go_to_filters"),i=$j("#leave_filters");t.click(function(t){t.preventDefault(),e.removeClass("narrow-hidden"),r.addClass("filtering"),e.find(":focusable").first().focus(),e.trap()}),i.click(function(i){i.preventDefault(),r.removeClass("filtering"),e.addClass("narrow-hidden"),t.focus()})}$j(document).ready(function(){showFilters(),setupNarrowScreenFilters()});
@houndci-bot

houndci-bot Oct 23, 2017

Line is too long.
Expected an assignment or function call and instead saw an expression.
Missing semicolon.
'$j' is not defined.

public/stylesheets/site/2.0/18-zone-searchbrowse.css
+/* INTERACTION: OLD FILTERS */
+/* ES UPGRADE TRANSITION: REMOVE WHEN DONE */
+
+form.old-filters {
@sarken

sarken Oct 23, 2017

Owner

old-filters 100% violates our class naming conventions, but this is definitely temporary, so I figure we can live with it.

public/javascripts/filters_old.min.js
@@ -0,0 +1 @@
+function showFilters(){var e=$j("dd.tags");e.each(function(e,r){var t=$j(r).find("input"),i=$j(r),n=$j("."+i.attr("id")+"_open"),s=$j("."+i.attr("id")+"_close");t.each(function(e,t){$j(t).is(":checked")&&($j(r).show(),$j(n).hide(),$j(s).show())})})}function setupNarrowScreenFilters(){var e=$j("form.old-filters"),r=$j("#outer"),t=$j("#go_to_filters"),i=$j("#leave_filters");t.click(function(t){t.preventDefault(),e.removeClass("narrow-hidden"),r.addClass("filtering"),e.find(":focusable").first().focus(),e.trap()}),i.click(function(i){i.preventDefault(),r.removeClass("filtering"),e.addClass("narrow-hidden"),t.focus()})}$j(document).ready(function(){showFilters(),setupNarrowScreenFilters()});
@sarken

sarken Oct 23, 2017

Owner

I'm only restoring the minified version of the old filter JavaScript because I couldn't think of a good reason to restore the non-minified version.

@sarken sarken added the Priority: Broken on Test (High) label Oct 23, 2017

@elzj

elzj approved these changes Oct 23, 2017
View changes

@elzj elzj merged commit 46596a3 into otwcode:master Oct 23, 2017
4 checks passed

4 checks passed

Scrutinizer No new issues
Details
codeclimate All good!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
hound 4 violations found.

zz9pzza added a commit that referenced this pull request Nov 27, 2017

@elzj @zz9pzza
Re-merge Elasticsearch branch (#3179) 
* AO3-4815 Update Elasticsearch and filter interface (#3104)

* Upgrades faraday (ran bundle update elasticsearch)

* Adds elasticsearch-model to directly replace tire model

Uses rollout to indicate which module to include in the searchable modules

* Allows application to connect to two ES instances

* Switches version of ES in use by Indexer and Query depending on rollout feature

* Fixes feature failure in importing

The image is rendered with https://, not http://

* Adds awesome_print gem, to make inspecting json easier

* Adds class to recreate indices in upgraded ES instance & reindex from remote

* Removes elasticsearch-model

* Removes rollout code from models

* WIP - Making app compatible with ES 5.5

* WIP - Pulls in elasticsearch overhaul code

Starting to debug elasticsearch overhaul code, both in its incompatibilities
with current ES and its underlying bugs.

* Gets search/works_anonymous feature specs to pass with new ES code

* Gets search/works_tags features to pass with new es code

* Fixes search/works_info spec failures

* Fixes test failures in search/work_stats

* Updates ES version read by Travis

* Updates ES instance to point to the url Travis will recognize

* Fixes some failing feature specs for search

* Fixes admins/admin_works specs

* Fixes bookmark_create and collection_dashboard feature specs

* Fixes importing/archivist feature specs

* Fixes other_a/orphan_work feature specs

Still using Tire, and reindexing with both Tire and elasticsearch will not break
anything and will be useful when running the test suite twice against the two
different ES versions

* Fixes all other other_a feature failures

* Fixes works_anonymous feature failures

* Fixes tag_wrangling_relationships feature failures

* Fixes users feature failures

* Fixes works feature failures

* Adds the new ES indexing code wherever Tire is use in test setup

* Fixes works default rails actions spec failures

* Fixes elasticsearch_upgrade_spec failures and refactors ES refresh code

* Fixes work query spec

* Adds some minor refactoring to tests

* Adds bookmark indexing/query code & fixes associated tests

* Updates BookmarkSearch references to BookmarkSearchForm

* Fixes bookmark_create feature failures

* Fixes admin/admin_works feature failures

* Fixes bookmarks feature spec failures

* Fixes collections feature specs

* Fixes other_a and search failures

* Fixes users feature specs

* Removes refrences to tire from tests and specs for now

* Fixes search steps problem when reindexing models with any? false

* Fixes reindexing models without any records in specs

* Removes outdated use of WorkSearch model in works controller

* Fixes typo

* Fixes failing specs caused by fixes to failing features

* Fixes bugs in bookmark query

* Some feature failure cleanup

* Adds code for Tag search using updated ES

* Adds pseud search code

* Small test fixes

* Fixes some failing tests, and attempts to address Travis issues

* Adds fixes for Travis failures that are repeatable locally

* Fixes problem where bookmarks without works can't be searched for

* Reindexes indexes everywhere they are updated in the features directory

This should remove inconsistent, nearly-unrepeatable errors caused by the fact
that Elasticsearch is asynchronous. Yay!

* Fixes bookmark query spec failure

* Refactors some of the index updating logic in features

* Fixes default rails actions for works spec

* Isolates Tire reindexing to one file in features

* Fixes tags and wrangling test failures

* Distinguishes ES versions in specs

* Fixes some typos

* Updates to catch old ES version no matter the environment

* Attempting a run with old ES version

* Fixes Tire import

* Fixes spec helper method

* Separates ES wrappers based on ES version in controllers

* Fixes use of new ES code in helpers for use_old_search?

* Fixes old ES code usage in bookmarks controller

* Fixes search using old ES for bookmarks

* Removes tests that were catching a bug that is no longer a problem

* Fixes works failures for old es version

* Fixes features failing on Travis

* Fixes spec failures for old ES version

* Sets Travis to dl new version of ES to see how build does

* Attempt with ES 6.0 alpha

* Adds fix for bookmark deletion bug

* Adds missing details to work indexer

* Adds rollout to indexing

* Swaps out 'use_old_search?' with 'use_new_search?'

* Fixes param names in tests

* Adds rollout to application record use_new_search? method

* Fixes small details & removes unused code

* Fixes typo

* Fixes typo

* Fixes typo

* Fixes lack of question mark for rollout.active? method

* Runs Travis build on old ES and sets elasticsearch version with rollout

* Fixes inccorect method invocation

* Fixes incorrect method invocation in people controller

* Fixes works stats feature failures

* Adds fixes for failing specs

* Uses appropriate count method for old es version

* Removes debug line from feature

* Upgrades nokogiri due to security concern

* Makes it so app will respond appropriately to ES version

To run on Travis, all three ES urls need to be the same:
ENV['ELASTICSEARCH_URL'] &
ENV['ELASTICSEARCH_1_URL'] &
ENV['UPGRADED_ELASTICSEARCH_URL']

All need to point to the same thing. Whenever running an instance of the app
with two versions of ES, ENV['ELASTICSEARCH_URL'] and ENV['ELASTICSEARCH_1_URL']
need to point to the old version and ENV['UPGRADED_ELASTICSEARCH_URL'] needs to
point to the new version.

* Second attempt with 6.0.0-alpha1 ES

* For test with ES 6.0.0-alpha1

* Fixes Travis run in 6.0.0-alpha1 ES

* Untar

* Try starting both

* Simple test run

* Differentiate the builds

* Differentiate the builds

* Restore all the tests

* Fixes call to new elasticsearch api perform_request

* Updates to test suite to watch how upgraded es version works on its own

* refactors out index updating methods in feature specs

* Adds rollout specs

* Fixes bug in work search form object

* Matches config to travis

* Removes all but rspec tests from travis for now

* Sets up tests & rollout methods use expected behavior

* Fixes switch from string to text or keyword for index definitions

* Updates index definitions to be compatible with 6.0.0.beta2 ES version

* Changes localhost to home IP for old ES urls

* Does not require old es to work when old es is gone

* Switches ES bool queries to true/false

* Fixes spec failures for ES Group 1

* Adds specific test to ensure old elasticsearch does not index when it's not present

* Moves to new elasticsearch document indexing when old es is gone

* Fixes problem with helper method for specs

* Ensures app never attempts to reindex an old es index in the new es instance

* Makes it so that a user doesn't have to be present if use_new_search is 100% enabled

* Runs ES=1

* Runs ES=2

* Runs ES=3

* Runs ES=4

* Back to rspec

* Removes command to rebuild three times upon failure

* Updates set_rollout script to use global  variable

* Adds rollout definitions to test suite to see if that addresses Travis build problem

* Runs build with ES=1

* Runs all builds

* Adds same assurances to features as specs that ES will work w/ both versions

* Fixes imported image URL in feature spec

* Fixes reference to tire methods when tire is gone in feature support methods

* Runs only ES=4

* Moves es/rollout code for feature specs

* Fixes boolean representation in work search form

* Fixes boolean in ES search in bookmark search form

* Fixes bookmarks feature failures

* Fixes works_restricted feature failures

* Runs all four ES groups

* Runs ES=1 and ES=4

* Comments out test that's breaking due to third-party url bug

* Makes Travis only run 2 trials

1 - With the old Elasticsearch, in which we test that the old Elasticsearch
continues to work while start_new_indexing is true (individual tests also test
that the SearchForm classes continue to work as expected, meaning that the new
Elasticsearch also works in this case when use_new_search is true)

2 - With the new Elasticsearch

* Adds old exlclusion filter code

* Adds tests & code for work filtering with exclusion tags

* Addresses Travis test failures

* Adds bookmark exclusion filters

* Downgrades back to 5.6.1 (latest stable) ES version

* Adds single_mapping => true to ES indices

* Fixes failure to index bookmarks when they do not have a bookmarkable object

* Adds comments to help when removing old code

* Removes rollout manipulation from Travis

This was not actually setting features. All Travis needs to do is install one or
both versions of ES depending on the test group being run.

* Adds test cases for use_new_search? controller method

* Adds tests & fixes bugs for filtering

* Removes default 'true' from use_new_search method

Reverts old styles and scripts to master

* Fixes typos

* Fixes more typos

* Fixes rollout test for works controller

* Fixes works controller mistake

* Ports over ALL js and css changes that effect filters

* Removes useless method

* Removes file that has nothing to do with new search

* Fixes failing filter tests

* Pulls in fix for autocomplete bug from master

* AO3-4815: Fill out new batch indexing code

* Fixes merge clobbering of css & js files

* AO3-4815: Add new ES to codeship script and use local.yml instead of … (#3110)

* AO3-4815: Add new ES to codeship script and use local.yml instead of editing the regular config

* Fix ES download url and load local config when testing

* AO3-4815: Ensure correct results when browsing bookmarks by user (#3109)

* AO3-5213 Index authors on works correctly for sorting (#3114)

AO3-5213 Index authors on works correctly for sorting

* AO3-5207 Add StatCounterIndexer and fix order of reindexing. (#3111)

* AO3-5207 First go at StatCounter reindexing.

* AO3-5207 Add RSpec tests to make sure it works.

* AO3-5207 Small fixes to pass tests.

* AO3-5207 Make sure to call original when stubbing.

* AO3-5207 Finally fixed.

* AO3-5212 Index correct user ids for works. Also throw in a couple of updated work type tag ids (#3112)

* AO3-5209 Don't change the appearance of non-work & bookmark filters (#3117)

* AO3-5209 Use old JavaScript and views for old Elasticsearch filters (styling fixes to come)

* AO3-5209 Restyle old filters

* AO3-5209 Ensure new work and bookmark filter styling does not conflict with exisiting collection filter styles

* AO3-5216 Fix setting authenticity tokens when cached pages load (#3118)

This was the fix for AO3-5153, which got lost in a merge.

* AO3-5214 Only run the tests on Travis once for now (#3116)

* AO3-5208: Fix filtering behavior (#3113)

* AO3-5208: Fix filtering behavior by drilling down and setting facets appropriately.

* AO3-5208: Don't error when there are no filters

* AO3-5208: Ensure excluded tags are visible in filters

* AO3-5208: Make sure bookmark facet query knows about exclusions. Also don't index filters and other bookmarkable fields on the bookmark itself since that's what we're trying to avoid.

* AO3-5208: Dry up some tag-related query methods, reduce number of db requests, only exclude children of Character tags, and make exclusion work for bookmark tags again

* AO3-5208: Update some tests to match new behavior

* AO3-5208: Put excluded bookmark tags in the right filter bucket

* AO3-5208: Remove sub tag exclusion code, since the meta tag id is already indexed on works

* AO3-5208: Comment out tests for types of exclusion filtering we aren't currently doing for bookmarks

* AO3-5208: Enable querying on both parent and child bookmark fields and reflect queries in bookmark filters

* AO3-5208: Update tests for bookmark search syntax change and bookmarker indexing encompassing user name

* AO3-5208: Make 'other tags' filtering work for noncanonical tags as well

* AO3-5223: Fix id on bookmarks exclude field

* AO3-5208: Remove child exclusion from filters and enable inclusion for noncanonical tags

* AO3-5220 and AO3-5224: Fix searching bookmarks by notes and tag names and index bookmarkable date for sorting

* AO3-5220: Update old bookmark search code for notes fix, fix spacing in module

* AO3-5225 Fix sidebar counts and scope by user id if a user is present (#3119)

* AO3-5225: Fix sidebar counts and scope by user id if a user is present

* AO3-5225: Update user dashboard feature to check counts in multiple places

* AO3-5231: Fix formatting of search text and don't chop up tag searches

* AO3-5231: Remove duplication and add specs

Some checks haven’t completed yet

1 pending and 1 successful checks
continuous-integration/codeship — Build in progress
Details
continuous-integration/travis-ci/push — The Travis CI build passed
Details
e2c284a

sarken added a commit that referenced this pull request Nov 27, 2017

@elzj @sarken
AO3-5254: Upgrade to Elasticsearch 6 (#3183) 
* AO3-4815 Update Elasticsearch and filter interface (#3104)

* Upgrades faraday (ran bundle update elasticsearch)

* Adds elasticsearch-model to directly replace tire model

Uses rollout to indicate which module to include in the searchable modules

* Allows application to connect to two ES instances

* Switches version of ES in use by Indexer and Query depending on rollout feature

* Fixes feature failure in importing

The image is rendered with https://, not http://

* Adds awesome_print gem, to make inspecting json easier

* Adds class to recreate indices in upgraded ES instance & reindex from remote

* Removes elasticsearch-model

* Removes rollout code from models

* WIP - Making app compatible with ES 5.5

* WIP - Pulls in elasticsearch overhaul code

Starting to debug elasticsearch overhaul code, both in its incompatibilities
with current ES and its underlying bugs.

* Gets search/works_anonymous feature specs to pass with new ES code

* Gets search/works_tags features to pass with new es code

* Fixes search/works_info spec failures

* Fixes test failures in search/work_stats

* Updates ES version read by Travis

* Updates ES instance to point to the url Travis will recognize

* Fixes some failing feature specs for search

* Fixes admins/admin_works specs

* Fixes bookmark_create and collection_dashboard feature specs

* Fixes importing/archivist feature specs

* Fixes other_a/orphan_work feature specs

Still using Tire, and reindexing with both Tire and elasticsearch will not break
anything and will be useful when running the test suite twice against the two
different ES versions

* Fixes all other other_a feature failures

* Fixes works_anonymous feature failures

* Fixes tag_wrangling_relationships feature failures

* Fixes users feature failures

* Fixes works feature failures

* Adds the new ES indexing code wherever Tire is use in test setup

* Fixes works default rails actions spec failures

* Fixes elasticsearch_upgrade_spec failures and refactors ES refresh code

* Fixes work query spec

* Adds some minor refactoring to tests

* Adds bookmark indexing/query code & fixes associated tests

* Updates BookmarkSearch references to BookmarkSearchForm

* Fixes bookmark_create feature failures

* Fixes admin/admin_works feature failures

* Fixes bookmarks feature spec failures

* Fixes collections feature specs

* Fixes other_a and search failures

* Fixes users feature specs

* Removes refrences to tire from tests and specs for now

* Fixes search steps problem when reindexing models with any? false

* Fixes reindexing models without any records in specs

* Removes outdated use of WorkSearch model in works controller

* Fixes typo

* Fixes failing specs caused by fixes to failing features

* Fixes bugs in bookmark query

* Some feature failure cleanup

* Adds code for Tag search using updated ES

* Adds pseud search code

* Small test fixes

* Fixes some failing tests, and attempts to address Travis issues

* Adds fixes for Travis failures that are repeatable locally

* Fixes problem where bookmarks without works can't be searched for

* Reindexes indexes everywhere they are updated in the features directory

This should remove inconsistent, nearly-unrepeatable errors caused by the fact
that Elasticsearch is asynchronous. Yay!

* Fixes bookmark query spec failure

* Refactors some of the index updating logic in features

* Fixes default rails actions for works spec

* Isolates Tire reindexing to one file in features

* Fixes tags and wrangling test failures

* Distinguishes ES versions in specs

* Fixes some typos

* Updates to catch old ES version no matter the environment

* Attempting a run with old ES version

* Fixes Tire import

* Fixes spec helper method

* Separates ES wrappers based on ES version in controllers

* Fixes use of new ES code in helpers for use_old_search?

* Fixes old ES code usage in bookmarks controller

* Fixes search using old ES for bookmarks

* Removes tests that were catching a bug that is no longer a problem

* Fixes works failures for old es version

* Fixes features failing on Travis

* Fixes spec failures for old ES version

* Sets Travis to dl new version of ES to see how build does

* Attempt with ES 6.0 alpha

* Adds fix for bookmark deletion bug

* Adds missing details to work indexer

* Adds rollout to indexing

* Swaps out 'use_old_search?' with 'use_new_search?'

* Fixes param names in tests

* Adds rollout to application record use_new_search? method

* Fixes small details & removes unused code

* Fixes typo

* Fixes typo

* Fixes typo

* Fixes lack of question mark for rollout.active? method

* Runs Travis build on old ES and sets elasticsearch version with rollout

* Fixes inccorect method invocation

* Fixes incorrect method invocation in people controller

* Fixes works stats feature failures

* Adds fixes for failing specs

* Uses appropriate count method for old es version

* Removes debug line from feature

* Upgrades nokogiri due to security concern

* Makes it so app will respond appropriately to ES version

To run on Travis, all three ES urls need to be the same:
ENV['ELASTICSEARCH_URL'] &
ENV['ELASTICSEARCH_1_URL'] &
ENV['UPGRADED_ELASTICSEARCH_URL']

All need to point to the same thing. Whenever running an instance of the app
with two versions of ES, ENV['ELASTICSEARCH_URL'] and ENV['ELASTICSEARCH_1_URL']
need to point to the old version and ENV['UPGRADED_ELASTICSEARCH_URL'] needs to
point to the new version.

* Second attempt with 6.0.0-alpha1 ES

* For test with ES 6.0.0-alpha1

* Fixes Travis run in 6.0.0-alpha1 ES

* Untar

* Try starting both

* Simple test run

* Differentiate the builds

* Differentiate the builds

* Restore all the tests

* Fixes call to new elasticsearch api perform_request

* Updates to test suite to watch how upgraded es version works on its own

* refactors out index updating methods in feature specs

* Adds rollout specs

* Fixes bug in work search form object

* Matches config to travis

* Removes all but rspec tests from travis for now

* Sets up tests & rollout methods use expected behavior

* Fixes switch from string to text or keyword for index definitions

* Updates index definitions to be compatible with 6.0.0.beta2 ES version

* Changes localhost to home IP for old ES urls

* Does not require old es to work when old es is gone

* Switches ES bool queries to true/false

* Fixes spec failures for ES Group 1

* Adds specific test to ensure old elasticsearch does not index when it's not present

* Moves to new elasticsearch document indexing when old es is gone

* Fixes problem with helper method for specs

* Ensures app never attempts to reindex an old es index in the new es instance

* Makes it so that a user doesn't have to be present if use_new_search is 100% enabled

* Runs ES=1

* Runs ES=2

* Runs ES=3

* Runs ES=4

* Back to rspec

* Removes command to rebuild three times upon failure

* Updates set_rollout script to use global  variable

* Adds rollout definitions to test suite to see if that addresses Travis build problem

* Runs build with ES=1

* Runs all builds

* Adds same assurances to features as specs that ES will work w/ both versions

* Fixes imported image URL in feature spec

* Fixes reference to tire methods when tire is gone in feature support methods

* Runs only ES=4

* Moves es/rollout code for feature specs

* Fixes boolean representation in work search form

* Fixes boolean in ES search in bookmark search form

* Fixes bookmarks feature failures

* Fixes works_restricted feature failures

* Runs all four ES groups

* Runs ES=1 and ES=4

* Comments out test that's breaking due to third-party url bug

* Makes Travis only run 2 trials

1 - With the old Elasticsearch, in which we test that the old Elasticsearch
continues to work while start_new_indexing is true (individual tests also test
that the SearchForm classes continue to work as expected, meaning that the new
Elasticsearch also works in this case when use_new_search is true)

2 - With the new Elasticsearch

* Adds old exlclusion filter code

* Adds tests & code for work filtering with exclusion tags

* Addresses Travis test failures

* Adds bookmark exclusion filters

* Downgrades back to 5.6.1 (latest stable) ES version

* Adds single_mapping => true to ES indices

* Fixes failure to index bookmarks when they do not have a bookmarkable object

* Adds comments to help when removing old code

* Removes rollout manipulation from Travis

This was not actually setting features. All Travis needs to do is install one or
both versions of ES depending on the test group being run.

* Adds test cases for use_new_search? controller method

* Adds tests & fixes bugs for filtering

* Removes default 'true' from use_new_search method

Reverts old styles and scripts to master

* Fixes typos

* Fixes more typos

* Fixes rollout test for works controller

* Fixes works controller mistake

* Ports over ALL js and css changes that effect filters

* Removes useless method

* Removes file that has nothing to do with new search

* Fixes failing filter tests

* Pulls in fix for autocomplete bug from master

* AO3-4815: Fill out new batch indexing code

* Fixes merge clobbering of css & js files

* AO3-4815: Add new ES to codeship script and use local.yml instead of … (#3110)

* AO3-4815: Add new ES to codeship script and use local.yml instead of editing the regular config

* Fix ES download url and load local config when testing

* AO3-4815: Ensure correct results when browsing bookmarks by user (#3109)

* AO3-5213 Index authors on works correctly for sorting (#3114)

AO3-5213 Index authors on works correctly for sorting

* AO3-5207 Add StatCounterIndexer and fix order of reindexing. (#3111)

* AO3-5207 First go at StatCounter reindexing.

* AO3-5207 Add RSpec tests to make sure it works.

* AO3-5207 Small fixes to pass tests.

* AO3-5207 Make sure to call original when stubbing.

* AO3-5207 Finally fixed.

* AO3-5212 Index correct user ids for works. Also throw in a couple of updated work type tag ids (#3112)

* AO3-5209 Don't change the appearance of non-work & bookmark filters (#3117)

* AO3-5209 Use old JavaScript and views for old Elasticsearch filters (styling fixes to come)

* AO3-5209 Restyle old filters

* AO3-5209 Ensure new work and bookmark filter styling does not conflict with exisiting collection filter styles

* AO3-5216 Fix setting authenticity tokens when cached pages load (#3118)

This was the fix for AO3-5153, which got lost in a merge.

* AO3-5214 Only run the tests on Travis once for now (#3116)

* AO3-5208: Fix filtering behavior (#3113)

* AO3-5208: Fix filtering behavior by drilling down and setting facets appropriately.

* AO3-5208: Don't error when there are no filters

* AO3-5208: Ensure excluded tags are visible in filters

* AO3-5208: Make sure bookmark facet query knows about exclusions. Also don't index filters and other bookmarkable fields on the bookmark itself since that's what we're trying to avoid.

* AO3-5208: Dry up some tag-related query methods, reduce number of db requests, only exclude children of Character tags, and make exclusion work for bookmark tags again

* AO3-5208: Update some tests to match new behavior

* AO3-5208: Put excluded bookmark tags in the right filter bucket

* AO3-5208: Remove sub tag exclusion code, since the meta tag id is already indexed on works

* AO3-5208: Comment out tests for types of exclusion filtering we aren't currently doing for bookmarks

* AO3-5208: Enable querying on both parent and child bookmark fields and reflect queries in bookmark filters

* AO3-5208: Update tests for bookmark search syntax change and bookmarker indexing encompassing user name

* AO3-5208: Make 'other tags' filtering work for noncanonical tags as well

* AO3-5223: Fix id on bookmarks exclude field

* AO3-5208: Remove child exclusion from filters and enable inclusion for noncanonical tags

* AO3-5220 and AO3-5224: Fix searching bookmarks by notes and tag names and index bookmarkable date for sorting

* AO3-5220: Update old bookmark search code for notes fix, fix spacing in module

* AO3-5225 Fix sidebar counts and scope by user id if a user is present (#3119)

* AO3-5225: Fix sidebar counts and scope by user id if a user is present

* AO3-5225: Update user dashboard feature to check counts in multiple places

* AO3-5231: Fix formatting of search text and don't chop up tag searches

* AO3-5231: Remove duplication and add specs

* Upgrade elasticsearch version on travis

* Remove single type mapping which is no longer needed

* Update has_parent query syntax

* Update syntax for parent and child filters, which are now only supported as queries

* Update bookmark query spec to match new syntax

* Upgrade codeship

Some checks were not successful

1 failing and 1 successful checks
continuous-integration/travis-ci/push — The Travis CI build failed
Details
continuous-integration/codeship — Build succeeded
Details
2dae45d

sarken added a commit that referenced this pull request Nov 30, 2017

@elzj @sarken
AO3-5221: Bookmarkable indexing (#3186) 
* AO3-4815 Update Elasticsearch and filter interface (#3104)

* Upgrades faraday (ran bundle update elasticsearch)

* Adds elasticsearch-model to directly replace tire model

Uses rollout to indicate which module to include in the searchable modules

* Allows application to connect to two ES instances

* Switches version of ES in use by Indexer and Query depending on rollout feature

* Fixes feature failure in importing

The image is rendered with https://, not http://

* Adds awesome_print gem, to make inspecting json easier

* Adds class to recreate indices in upgraded ES instance & reindex from remote

* Removes elasticsearch-model

* Removes rollout code from models

* WIP - Making app compatible with ES 5.5

* WIP - Pulls in elasticsearch overhaul code

Starting to debug elasticsearch overhaul code, both in its incompatibilities
with current ES and its underlying bugs.

* Gets search/works_anonymous feature specs to pass with new ES code

* Gets search/works_tags features to pass with new es code

* Fixes search/works_info spec failures

* Fixes test failures in search/work_stats

* Updates ES version read by Travis

* Updates ES instance to point to the url Travis will recognize

* Fixes some failing feature specs for search

* Fixes admins/admin_works specs

* Fixes bookmark_create and collection_dashboard feature specs

* Fixes importing/archivist feature specs

* Fixes other_a/orphan_work feature specs

Still using Tire, and reindexing with both Tire and elasticsearch will not break
anything and will be useful when running the test suite twice against the two
different ES versions

* Fixes all other other_a feature failures

* Fixes works_anonymous feature failures

* Fixes tag_wrangling_relationships feature failures

* Fixes users feature failures

* Fixes works feature failures

* Adds the new ES indexing code wherever Tire is use in test setup

* Fixes works default rails actions spec failures

* Fixes elasticsearch_upgrade_spec failures and refactors ES refresh code

* Fixes work query spec

* Adds some minor refactoring to tests

* Adds bookmark indexing/query code & fixes associated tests

* Updates BookmarkSearch references to BookmarkSearchForm

* Fixes bookmark_create feature failures

* Fixes admin/admin_works feature failures

* Fixes bookmarks feature spec failures

* Fixes collections feature specs

* Fixes other_a and search failures

* Fixes users feature specs

* Removes refrences to tire from tests and specs for now

* Fixes search steps problem when reindexing models with any? false

* Fixes reindexing models without any records in specs

* Removes outdated use of WorkSearch model in works controller

* Fixes typo

* Fixes failing specs caused by fixes to failing features

* Fixes bugs in bookmark query

* Some feature failure cleanup

* Adds code for Tag search using updated ES

* Adds pseud search code

* Small test fixes

* Fixes some failing tests, and attempts to address Travis issues

* Adds fixes for Travis failures that are repeatable locally

* Fixes problem where bookmarks without works can't be searched for

* Reindexes indexes everywhere they are updated in the features directory

This should remove inconsistent, nearly-unrepeatable errors caused by the fact
that Elasticsearch is asynchronous. Yay!

* Fixes bookmark query spec failure

* Refactors some of the index updating logic in features

* Fixes default rails actions for works spec

* Isolates Tire reindexing to one file in features

* Fixes tags and wrangling test failures

* Distinguishes ES versions in specs

* Fixes some typos

* Updates to catch old ES version no matter the environment

* Attempting a run with old ES version

* Fixes Tire import

* Fixes spec helper method

* Separates ES wrappers based on ES version in controllers

* Fixes use of new ES code in helpers for use_old_search?

* Fixes old ES code usage in bookmarks controller

* Fixes search using old ES for bookmarks

* Removes tests that were catching a bug that is no longer a problem

* Fixes works failures for old es version

* Fixes features failing on Travis

* Fixes spec failures for old ES version

* Sets Travis to dl new version of ES to see how build does

* Attempt with ES 6.0 alpha

* Adds fix for bookmark deletion bug

* Adds missing details to work indexer

* Adds rollout to indexing

* Swaps out 'use_old_search?' with 'use_new_search?'

* Fixes param names in tests

* Adds rollout to application record use_new_search? method

* Fixes small details & removes unused code

* Fixes typo

* Fixes typo

* Fixes typo

* Fixes lack of question mark for rollout.active? method

* Runs Travis build on old ES and sets elasticsearch version with rollout

* Fixes inccorect method invocation

* Fixes incorrect method invocation in people controller

* Fixes works stats feature failures

* Adds fixes for failing specs

* Uses appropriate count method for old es version

* Removes debug line from feature

* Upgrades nokogiri due to security concern

* Makes it so app will respond appropriately to ES version

To run on Travis, all three ES urls need to be the same:
ENV['ELASTICSEARCH_URL'] &
ENV['ELASTICSEARCH_1_URL'] &
ENV['UPGRADED_ELASTICSEARCH_URL']

All need to point to the same thing. Whenever running an instance of the app
with two versions of ES, ENV['ELASTICSEARCH_URL'] and ENV['ELASTICSEARCH_1_URL']
need to point to the old version and ENV['UPGRADED_ELASTICSEARCH_URL'] needs to
point to the new version.

* Second attempt with 6.0.0-alpha1 ES

* For test with ES 6.0.0-alpha1

* Fixes Travis run in 6.0.0-alpha1 ES

* Untar

* Try starting both

* Simple test run

* Differentiate the builds

* Differentiate the builds

* Restore all the tests

* Fixes call to new elasticsearch api perform_request

* Updates to test suite to watch how upgraded es version works on its own

* refactors out index updating methods in feature specs

* Adds rollout specs

* Fixes bug in work search form object

* Matches config to travis

* Removes all but rspec tests from travis for now

* Sets up tests & rollout methods use expected behavior

* Fixes switch from string to text or keyword for index definitions

* Updates index definitions to be compatible with 6.0.0.beta2 ES version

* Changes localhost to home IP for old ES urls

* Does not require old es to work when old es is gone

* Switches ES bool queries to true/false

* Fixes spec failures for ES Group 1

* Adds specific test to ensure old elasticsearch does not index when it's not present

* Moves to new elasticsearch document indexing when old es is gone

* Fixes problem with helper method for specs

* Ensures app never attempts to reindex an old es index in the new es instance

* Makes it so that a user doesn't have to be present if use_new_search is 100% enabled

* Runs ES=1

* Runs ES=2

* Runs ES=3

* Runs ES=4

* Back to rspec

* Removes command to rebuild three times upon failure

* Updates set_rollout script to use global  variable

* Adds rollout definitions to test suite to see if that addresses Travis build problem

* Runs build with ES=1

* Runs all builds

* Adds same assurances to features as specs that ES will work w/ both versions

* Fixes imported image URL in feature spec

* Fixes reference to tire methods when tire is gone in feature support methods

* Runs only ES=4

* Moves es/rollout code for feature specs

* Fixes boolean representation in work search form

* Fixes boolean in ES search in bookmark search form

* Fixes bookmarks feature failures

* Fixes works_restricted feature failures

* Runs all four ES groups

* Runs ES=1 and ES=4

* Comments out test that's breaking due to third-party url bug

* Makes Travis only run 2 trials

1 - With the old Elasticsearch, in which we test that the old Elasticsearch
continues to work while start_new_indexing is true (individual tests also test
that the SearchForm classes continue to work as expected, meaning that the new
Elasticsearch also works in this case when use_new_search is true)

2 - With the new Elasticsearch

* Adds old exlclusion filter code

* Adds tests & code for work filtering with exclusion tags

* Addresses Travis test failures

* Adds bookmark exclusion filters

* Downgrades back to 5.6.1 (latest stable) ES version

* Adds single_mapping => true to ES indices

* Fixes failure to index bookmarks when they do not have a bookmarkable object

* Adds comments to help when removing old code

* Removes rollout manipulation from Travis

This was not actually setting features. All Travis needs to do is install one or
both versions of ES depending on the test group being run.

* Adds test cases for use_new_search? controller method

* Adds tests & fixes bugs for filtering

* Removes default 'true' from use_new_search method

Reverts old styles and scripts to master

* Fixes typos

* Fixes more typos

* Fixes rollout test for works controller

* Fixes works controller mistake

* Ports over ALL js and css changes that effect filters

* Removes useless method

* Removes file that has nothing to do with new search

* Fixes failing filter tests

* Pulls in fix for autocomplete bug from master

* AO3-4815: Fill out new batch indexing code

* Fixes merge clobbering of css & js files

* AO3-4815: Add new ES to codeship script and use local.yml instead of … (#3110)

* AO3-4815: Add new ES to codeship script and use local.yml instead of editing the regular config

* Fix ES download url and load local config when testing

* AO3-4815: Ensure correct results when browsing bookmarks by user (#3109)

* AO3-5213 Index authors on works correctly for sorting (#3114)

AO3-5213 Index authors on works correctly for sorting

* AO3-5207 Add StatCounterIndexer and fix order of reindexing. (#3111)

* AO3-5207 First go at StatCounter reindexing.

* AO3-5207 Add RSpec tests to make sure it works.

* AO3-5207 Small fixes to pass tests.

* AO3-5207 Make sure to call original when stubbing.

* AO3-5207 Finally fixed.

* AO3-5212 Index correct user ids for works. Also throw in a couple of updated work type tag ids (#3112)

* AO3-5209 Don't change the appearance of non-work & bookmark filters (#3117)

* AO3-5209 Use old JavaScript and views for old Elasticsearch filters (styling fixes to come)

* AO3-5209 Restyle old filters

* AO3-5209 Ensure new work and bookmark filter styling does not conflict with exisiting collection filter styles

* AO3-5216 Fix setting authenticity tokens when cached pages load (#3118)

This was the fix for AO3-5153, which got lost in a merge.

* AO3-5214 Only run the tests on Travis once for now (#3116)

* AO3-5208: Fix filtering behavior (#3113)

* AO3-5208: Fix filtering behavior by drilling down and setting facets appropriately.

* AO3-5208: Don't error when there are no filters

* AO3-5208: Ensure excluded tags are visible in filters

* AO3-5208: Make sure bookmark facet query knows about exclusions. Also don't index filters and other bookmarkable fields on the bookmark itself since that's what we're trying to avoid.

* AO3-5208: Dry up some tag-related query methods, reduce number of db requests, only exclude children of Character tags, and make exclusion work for bookmark tags again

* AO3-5208: Update some tests to match new behavior

* AO3-5208: Put excluded bookmark tags in the right filter bucket

* AO3-5208: Remove sub tag exclusion code, since the meta tag id is already indexed on works

* AO3-5208: Comment out tests for types of exclusion filtering we aren't currently doing for bookmarks

* AO3-5208: Enable querying on both parent and child bookmark fields and reflect queries in bookmark filters

* AO3-5208: Update tests for bookmark search syntax change and bookmarker indexing encompassing user name

* AO3-5208: Make 'other tags' filtering work for noncanonical tags as well

* AO3-5223: Fix id on bookmarks exclude field

* AO3-5208: Remove child exclusion from filters and enable inclusion for noncanonical tags

* AO3-5220 and AO3-5224: Fix searching bookmarks by notes and tag names and index bookmarkable date for sorting

* AO3-5220: Update old bookmark search code for notes fix, fix spacing in module

* AO3-5225 Fix sidebar counts and scope by user id if a user is present (#3119)

* AO3-5225: Fix sidebar counts and scope by user id if a user is present

* AO3-5225: Update user dashboard feature to check counts in multiple places

* AO3-5221: Set up a pathway from bookmarked classes to our indexers

* AO3-5221: Fix code that got re-added on merge

* AO3-5221: Do a little method delegation

* AO3-5221: Make sure stat counter indexing still works

* AO3-5221: Update comments

All checks have passed

2 successful checks
continuous-integration/codeship — Build succeeded
Details
continuous-integration/travis-ci/push — The Travis CI build passed
Details
08a676d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment