AO3-4359 Mass import bookmarks #2442

Merged
merged 34 commits into from Jul 10, 2016

3 participants

@ariana-paris
ariana-paris commented Apr 28, 2016 edited

https://otwarchive.atlassian.net/browse/AO3-4359

Adds bookmark importing capability to the mass import API. Open Doors sometimes import sites which include works that are links to stories on another site which is not being imported (and therefore hasn't granted permission to scrape in the story contents). This will allow those works to be imported as bookmarks on the Archive, referencing the story's currently location rather than importing it.

In order to achieve this, I have also:

  • refactored the existing code so that the authentication handling and work imports are in separate files
  • rationalised the API end points so that instead of api/v1/import, we now have api/v1/works and api/v1/bookmarks (paving the way for, say, collections, users or, idk, videos in future, whatever comes up next)
  • added an original_id field to all the API requests so the remote site can pass in its own id for the story/bookmark it's importing and use that to match up the Archive's response to the relevant item (currently, the OD temporary sites rely on the responses being in the same order as the requests which is risky)
  • converted routes.rb to use Ruby 1.9 hashes where possible
ariana-paris added some commits Oct 25, 2015
@ariana-paris ariana-paris AO3-4359 - Add plumbing for bookmark imports 37b8df0
@ariana-paris ariana-paris AO3-4359 - Simplify return codes and import bookmarks c889df4
@ariana-paris ariana-paris AO3-4359 - Move works importing code into the works controller d35995f
@ariana-paris ariana-paris AO3-4359 - Handle items collection in incoming json d26cd15
@ariana-paris ariana-paris Merge branch 'master' into AO3-4359-Mass-import-bookmarks
# Conflicts:
#	app/controllers/api/v1/import_controller.rb
#	spec/api/api_spec.rb
d0aec91
@ariana-paris ariana-paris AO3-4359 - Merge with master branch (WIP - broken tests) 6beecba
@ariana-paris ariana-paris AO3-4359 - Add plumbing for bookmark imports 8f97f69
@ariana-paris ariana-paris AO3-4359 - Simplify return codes and import bookmarks 43de8d8
@ariana-paris ariana-paris AO3-4359 - Move works importing code into the works controller 2593b07
@ariana-paris ariana-paris AO3-4359 - Handle items collection in incoming json ce02679
@ariana-paris ariana-paris AO3-4359 - Merge with master branch (WIP - broken tests) 065c23e
@ariana-paris ariana-paris AO3-4359 - Document expected Json in code 8f3b4ca
@ariana-paris ariana-paris AO3-4383 - Include title e3cadb5
@ariana-paris ariana-paris AO3-4359 - Attempt to correct the tests (WIP) d69c22d
@ariana-paris ariana-paris Merge remote-tracking branch 'origin/AO3-4359-Mass-import-bookmarks' …
…into AO3-4359-Mass-import-bookmarks

# Conflicts:
#	app/controllers/api/v1/bookmarks_controller.rb
#	app/controllers/api/v1/works_controller.rb
#	spec/api/api_spec.rb
52291e7
@ariana-paris ariana-paris AO3-4359 - Fix incomplete merge with master and related tests de23cbe
@ariana-paris ariana-paris AO3-4359 - Split out API specs for better clarity e772b47
@ariana-paris ariana-paris AO3-4359 - Reinstate change to deploy.rb overwritten by merge 4fde7e9
@ariana-paris ariana-paris AO3-4359 - Fix local warning about errors and clamp down on timezones…
… in date test
3b81147
@ariana-paris ariana-paris AO3-4359 - Reset mocks after use to better isolate tests 19b1fd3
@ariana-paris ariana-paris AO3-4359 - Flatten request json to hide internal Archive details 25e04bc
@ariana-paris ariana-paris AO3-4359 - Add original_ref field so remote ids can be used on the AP…
…I client side
1805c56
@ariana-paris ariana-paris AO3-4359 - Rename original_ref to original_id a398315
@ariana-paris ariana-paris AO3-4359 - Remove debugging method 8e68a86
@ariana-paris ariana-paris AO3-4359 - Refactor routes.rb to use Ruby 1.9 hashes
6de14d9
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/models/story_parser_spec.rb
@@ -143,12 +136,22 @@ class StoryParser
end
describe "#download_and_parse_chapters_into_story" do
+

Extra empty line detected at block body beginning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/api/api_works_spec.rb
valid_headers
- parsed_body = JSON.parse(response.body)
- expect(parsed_body.first["status"]).to eq "ok"
- expect(parsed_body.first["work_url"]).to eq work_url(@work)
- expect(parsed_body.first["created"]).to eq @work.created_at.as_json
+ parsed_body = JSON.parse(response.body, symbolize_names: true)
+ expect(parsed_body.first[:status]).to eq "ok"
+ expect(parsed_body.first[:work_url]).to eq work_url(@work)
+ expect(parsed_body.first[:created]).to eq @work.created_at.as_json
+ end
+
+ it "should return the original reference if one was provided" do
+ post "/api/v1/works/urls",
+ { original_urls: [
+ { id: "123", url: "foo" }

Use 2 spaces for indentation in an array, relative to the start of the line where the left square bracket is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/api/api_works_spec.rb
end
+

Extra blank line detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/api/api_helper.rb
@@ -0,0 +1,67 @@
+require 'spec_helper'
+require 'webmock/rspec'
+
+module ApiHelper
+

Extra empty line detected at module body beginning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/api/api_bookmarks_spec.rb
+ pseud_id = @user.default_pseud.id
+ post "/api/v1/bookmarks/import",
+ { archivist: @user.login,
+ bookmarks: [ bookmark ]
+ }.to_json,
+ valid_headers
+ first_bookmark = Bookmark.find_all_by_pseud_id(pseud_id).first
+ bookmark_response = JSON.parse(response.body, symbolize_names: true)[:bookmarks].first
+ assert_equal bookmark_response[:archive_url], bookmark_url(first_bookmark)
+ end
+
+ WebMock.allow_net_connect!
+ end
+
+ describe "Invalid API bookmark import" do
+ before do

Use 2 (not 4) spaces for indentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/api/api_bookmarks_spec.rb
+ @user = create(:user)
+ end
+
+ after do
+ WebMock.reset!
+ end
+
+ it "should return 400 Bad Request if an invalid URL is specified" do
+ post "/api/v1/import",
+ { archivist: @user.login,
+ bookmarks: [ bookmark.merge!( { url: "" } ) ] }.to_json,
+ valid_headers
+ assert_equal 400, response.status
+ end
+
+ it "should return 400 Bad Request if no bookmarks are specified" do

Inconsistent indentation detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/api/api_bookmarks_spec.rb
+ end
+
+ WebMock.allow_net_connect!
+ end
+
+ describe "Invalid API bookmark import" do
+ before do
+ mock_external
+ @user = create(:user)
+ end
+
+ after do
+ WebMock.reset!
+ end
+
+ it "should return 400 Bad Request if an invalid URL is specified" do

Inconsistent indentation detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/api/api_bookmarks_spec.rb
+ end
+
+ describe "Invalid API bookmark import" do
+ before do
+ mock_external
+ @user = create(:user)
+ end
+
+ after do
+ WebMock.reset!
+ end
+
+ it "should return 400 Bad Request if an invalid URL is specified" do
+ post "/api/v1/import",
+ { archivist: @user.login,
+ bookmarks: [ bookmark.merge!( { url: "" } ) ] }.to_json,

Redundant curly braces around a hash parameter.
Space inside square brackets detected.
Space inside parentheses detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
spec/api/api_bookmarks_spec.rb
@@ -0,0 +1,132 @@
+require 'spec_helper'
+require 'api/api_helper'
+
+describe "API BookmarksController" do
+ include ApiHelper
+
+ # Override is_archivist so all users are archivists from this point on
+ class User < ActiveRecord::Base
+ def is_archivist?
+ true
+ end
+ end
+
+ bookmark = {
+ id: "123",

Use 2 spaces for indentation in a hash, relative to the start of the line where the left curly brace is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
config/routes.rb
@@ -460,7 +460,11 @@
namespace :api do
namespace :v1 do
- resources :import, only: [:create], defaults: { format: :json }
+ resources :bookmarks, only: [:create], defaults: { format: :json}

Space inside } missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@ariana-paris ariana-paris AO3-4359 - Tidy up formatting in tests
b5f74d8
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
config/deploy.rb
@@ -65,7 +65,7 @@
namespace :deploy do
desc "Restart the unicorns"
task :restart do
- find_servers(:roles => :app).each do |server|
+ find_servers(:roles => [ :app, :web ]).each do |server|

Use the new Ruby 1.9 hash syntax.
Space inside square brackets detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
app/controllers/api/v1/works_controller.rb
+ original_id: external_work[:id],
+ original_url: original_url,
+ messages: work_messages
+ }
+ end
+
+ # Work-level error handling for requests that are incomplete or too large
+ def work_errors(work)
+ status = :bad_request
+ errors = []
+ urls = work[:chapter_urls]
+ if urls.nil?
+ errors << "This work doesn't contain chapter_urls. Works can only be imported from publicly-accessible URLs."
+ elsif urls.length >= ArchiveConfig.IMPORT_MAX_CHAPTERS
+ errors << "This work contains too many chapter URLs. A maximum of #{ ArchiveConfig.IMPORT_MAX_CHAPTERS }" +
+ "chapters can be imported per work."

Align the operands of an expression spanning multiple lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
app/controllers/api/v1/works_controller.rb
+ url: work_url,
+ original_id: external_work[:id],
+ original_url: original_url,
+ messages: work_messages
+ }
+ end
+
+ # Work-level error handling for requests that are incomplete or too large
+ def work_errors(work)
+ status = :bad_request
+ errors = []
+ urls = work[:chapter_urls]
+ if urls.nil?
+ errors << "This work doesn't contain chapter_urls. Works can only be imported from publicly-accessible URLs."
+ elsif urls.length >= ArchiveConfig.IMPORT_MAX_CHAPTERS
+ errors << "This work contains too many chapter URLs. A maximum of #{ ArchiveConfig.IMPORT_MAX_CHAPTERS }" +

Use \ instead of + or << to concatenate those strings.
Space inside string interpolation detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on an outdated diff Apr 28, 2016
app/controllers/api/v1/works_controller.rb
+ status = :bad_request
+ errors = []
+ urls = work[:chapter_urls]
+ if urls.nil?
+ errors << "This work doesn't contain chapter_urls. Works can only be imported from publicly-accessible URLs."
+ elsif urls.length >= ArchiveConfig.IMPORT_MAX_CHAPTERS
+ errors << "This work contains too many chapter URLs. A maximum of #{ ArchiveConfig.IMPORT_MAX_CHAPTERS }" +
+ "chapters can be imported per work."
+ end
+ status = :ok if errors.empty?
+ [status, errors]
+ end
+
+ # send invitations to external authors for a given set of works
+ def send_external_invites(works, archivist)
+ external_authors = works.collect(&:external_authors).flatten.uniq

Prefer map over collect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on the diff Apr 28, 2016
app/controllers/api/v1/works_controller.rb
private
+ # Set messages based on success and error flags
+ def response_message(messages)
+ if @some_success && @some_errors

Use the return of the conditional for variable assignment and comparison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
ariana-paris added some commits Apr 29, 2016
@ariana-paris ariana-paris Merge branch 'master' of git://github.com/otwcode/otwarchive into AO3…
…-4359-Mass-import-bookmarks
dd86654
@ariana-paris ariana-paris AO3-4359 - Undo inadvertent change to deploy.rb and address Hound com…
…ments
2033515
@houndci-bot houndci-bot commented on the diff May 1, 2016
app/controllers/api/v1/bookmarks_controller.rb
+ external_bookmarks.each do |external_bookmark|
+ bookmarks_responses << import_bookmark(archivist, external_bookmark)
+ end
+
+ # set final response code and message depending on the flags
+ messages = response_message(messages)
+ end
+
+ render status: status, json: { status: status, messages: messages, bookmarks: bookmarks_responses }
+ end
+
+ private
+
+ # Set messages based on success and error flags
+ def response_message(messages)
+ if @some_success && @some_errors

Use the return of the conditional for variable assignment and comparison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot and 1 other commented on an outdated diff May 1, 2016
app/controllers/api/v1/base_controller.rb
+ # if no works are supplied. If there is neither a valid archivist nor valid works, a 400 is returned
+ # with both errors as a message
+ def batch_errors(archivist, import_items)
+ status = :bad_request
+ errors = []
+
+ unless archivist && archivist.is_archivist?
+ status = :forbidden
+ errors << "The 'archivist' field must specify the name of an Archive user with archivist privileges."
+ end
+
+ if import_items.nil? || import_items.empty?
+ errors << "No items to import were provided."
+ elsif import_items.size >= ArchiveConfig.IMPORT_MAX_WORKS_BY_ARCHIVIST
+ errors << "This request contains too many items to import. A maximum of #{ ArchiveConfig.IMPORT_MAX_WORKS_BY_ARCHIVIST }" +
+ "items can be imported in one go by an archivist."

Align the operands of an expression spanning multiple lines.

@sarken
Organization for Transformative Works member
sarken added a note May 13, 2016

Really minor complaint -- can this be rephrased to "at one time"/"at once"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@houndci-bot houndci-bot commented on the diff May 1, 2016
app/controllers/api/v1/base_controller.rb
+ # Top-level error handling: returns a 403 forbidden if a valid archivist isn't supplied and a 400
+ # if no works are supplied. If there is neither a valid archivist nor valid works, a 400 is returned
+ # with both errors as a message
+ def batch_errors(archivist, import_items)
+ status = :bad_request
+ errors = []
+
+ unless archivist && archivist.is_archivist?
+ status = :forbidden
+ errors << "The 'archivist' field must specify the name of an Archive user with archivist privileges."
+ end
+
+ if import_items.nil? || import_items.empty?
+ errors << "No items to import were provided."
+ elsif import_items.size >= ArchiveConfig.IMPORT_MAX_WORKS_BY_ARCHIVIST
+ errors << "This request contains too many items to import. A maximum of #{ ArchiveConfig.IMPORT_MAX_WORKS_BY_ARCHIVIST }" +

Use \ instead of + or << to concatenate those strings.
Space inside string interpolation detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@sarken
Organization for Transformative Works member

bats eyelashes Can you include a link to the issue in the message at the top, please? (And edit the issue to have 0.9.34 as the release for now, as long as you're there?) Thanks!

@ariana-paris

Doh, I have no explanation for why I keep forgetting to put the ticket in the pull request.

@sarken sarken and 1 other commented on an outdated diff May 13, 2016
app/controllers/api/v1/bookmarks_controller.rb
+ bookmarks_responses << import_bookmark(archivist, external_bookmark)
+ end
+
+ # set final response code and message depending on the flags
+ messages = response_message(messages)
+ end
+
+ render status: status, json: { status: status, messages: messages, bookmarks: bookmarks_responses }
+ end
+
+ private
+
+ # Set messages based on success and error flags
+ def response_message(messages)
+ if @some_success && @some_errors
+ messages << "At least one bookmark was not created. Please check the bookmark array for further information."
@sarken
Organization for Transformative Works member
sarken added a note May 13, 2016

Will an archivist know what the "bookmark array" is? (I ask mostly because I'm not looking at the page and I don't know.)

Good point - that's an implementation detail that doesn't need to be in the message. The developer who implements the API will presumably turn the JSON array into something more user friendly, so I'll rephrase that as "individual bookmark messages".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@ariana-paris ariana-paris AO3-4359 - Rephrase user-visible messages to remove implementation de…
…tails
4d37439
@shalott shalott commented on the diff May 24, 2016
app/controllers/api/v1/bookmarks_controller.rb
@@ -0,0 +1,117 @@
+class Api::V1::BookmarksController < Api::V1::BaseController
@shalott
Organization for Transformative Works member
shalott added a note May 24, 2016

Shouldn't this be ExternalBookmarksController? We may well want to add API for regular bookmarks at some point

With all my focus on the Open Doors use case, I hadn't thought of the two bookmark types possibly being used in future - I'll rename this and have a think about whether the URL end-point needs to be changed as well. (i.e. would it still make sense to POST both external and internal bookmarks to api/v1/bookmarks, or should there be a separate api/v1/external-bookmarks, and if so what happens to GET requests when we get around to them? Should both types be returned by a call to bookmarks?)

@shalott
Organization for Transformative Works member
shalott added a note May 24, 2016

Hmm, I was thinking about External Works, I realize now. And the actual bookmark object is the same, so all bookmark creation makes sense in BookmarksController, but where the process of creating external works belongs... IDK, probably it should be here after all and just made conditional on whether the url passed is for an AO3 work or not, that seems most parallel to the existing process since External Works controller just makes the new external work object and passes off the actual creation to the Bookmarks controller via the form.

As discussed previously, I've added some error handling. As a general rule, I like to delegate as much as possible to the back-end when writing an API, as the model should be performing essential data validation before anything goes into the database, and this theoretically ensures consistency between the UI and the API.

However, the ExternalWorks model doesn't check for the presence of a fandom and returns a weird message for a missing author, so I've intercepted some invalid requests in the API controller to return helpful messages from there instead, and added tests to check that this is happening.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@shalott shalott and 1 other commented on an outdated diff May 24, 2016
spec/api/api_bookmarks_spec.rb
+ { archivist: @user.login,
+ bookmarks: [ bookmark, bookmark ]
+ }.to_json,
+ valid_headers
+ assert_equal 200, response.status
+ end
+
+ it "should create bookmarks associated with the archivist" do
+ pseud_id = @user.default_pseud.id
+ post "/api/v1/bookmarks/import",
+ { archivist: @user.login,
+ bookmarks: [ bookmark, bookmark ]
+ }.to_json,
+ valid_headers
+ bookmarks = Bookmark.find_all_by_pseud_id(pseud_id)
+ assert_equal bookmarks.count, 2
@shalott
Organization for Transformative Works member
shalott added a note May 24, 2016

I'm confused by these two last specs. I assumed from the first one that duplicate bookmarks wouldn't be created, and the spec was therefore testing that it would return 200 even if only one bookmark were created. Then the second one seems to expect both to be created? In which case the first spec doesn't seem to be testing "when only some are created".

My gut feeling is an exact duplicate should ideally not be created? So bookmarks.count should be 1 not 2?

Good point about the "some" bookmarks spec. When I originally wrote the works controller, I returned different HTTP responses for these different states (all, none or some created). But as they're application errors, not HTTP errors, I now think 200 is the code to use. Anyway, I copy-pasted those into the bookmarks spec and forgot to clean up. I'll remove that test altogether.

The second one does expect duplicates to be created because there doesn't seem to be a de-duping mechanism in the Bookmark model. If I bookmark the same work multiple times, I get multiple bookmarks (even though only one is listed under the work, interestingly). This may indeed be a unintended flaw in the Bookmark model, but it feels as if fixing it is beyond the scope of this particular ticket which aims to reproduce the web behaviour in the API (though a separate one which fixes the model and this test sounds like a good idea!)

@shalott
Organization for Transformative Works member
shalott added a note May 24, 2016

FWIW, my inclination would be it's more urgent to avoid creating the duplicates here because eg if an import fails halfway you would ideally like to be able to re-run without worrying about creating duplicate bookmarks?

Good point. I'll see if it's easy to do without delving too intimately into the Bookmark model - for example, by searching for a bookmark owned by the archivist which already has that URL (sounds easy enough but we'll see!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
ariana-paris added some commits May 24, 2016
@ariana-paris ariana-paris Merge branch 'master' of git://github.com/otwcode/otwarchive into AO3…
…-4359-Mass-import-bookmarks
5d7a3bb
@ariana-paris ariana-paris AO3-4359 - Remove pointless test 1e57672
@ariana-paris ariana-paris AO3-4359 - Don't create duplicate bookmarks given the same archivist …
…and external URL
5a99e50
@houndci-bot houndci-bot commented on an outdated diff May 25, 2016
app/controllers/api/v1/bookmarks_controller.rb
+ url = bookmark_request[:external][:url]
+ if url.nil?
+ errors << "This bookmark does not contain a URL to an external site."
+ end
+
+ archivist_bookmarks = Bookmark.find_all_by_pseud_id(archivist.default_pseud.id)
+
+ unless archivist_bookmarks.empty?
+ archivist_bookmarks.each do |bookmark|
+ if bookmark.bookmarkable_type == "ExternalWork" && ExternalWork.find(bookmark.bookmarkable_id).url == url
+ errors << "There is already a bookmark for this archivist and the URL #{url}"
+ end
+ end
+ end
+
+

Extra blank line detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@shalott shalott and 1 other commented on an outdated diff Jun 1, 2016
app/controllers/api/v1/bookmarks_controller.rb
@@ -0,0 +1,128 @@
+class Api::V1::BookmarksController < Api::V1::BaseController
+ respond_to :json
+
+ def create
+ archivist = User.find_by_login(params[:archivist])
+ external_bookmarks = params[:bookmarks]
@shalott
Organization for Transformative Works member
shalott added a note Jun 1, 2016

here is where I think the naming is confusing me -- these are just bookmarks and they happen to be of external works. Can we just call them bookmarks throughout? it looks to me like this code would actually work just fine for a NON external work too.

I'm not sure it would work for the other types as is, because the external_bookmark method in the API bookmark_controller coerces the JSON input into a structure with an external hash in it, which IIRC is the structure passed back by the normal bookmark_controller when creating a bookmark for an external work. I think this brings the external= virtual attribute in the Bookmark model into play.

Having said that, I didn't inspect the other bookmark types, and it's been a while since I unpicked how bookmarks are created, so you might well be right. I'll rename that variable to make it less external-centric which will make it easier to adapt this if we have a use case for creating other types of bookmarks in future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@ariana-paris ariana-paris AO3-4359 - Rename external_bookmark variable to be less specific
f833083
@houndci-bot houndci-bot commented on an outdated diff Jun 1, 2016
spec/api/api_bookmarks_spec.rb
+ end
+
+ describe "Invalid API bookmark import" do
+ before do
+ mock_external
+ @user = create(:user)
+ end
+
+ after do
+ WebMock.reset!
+ end
+
+ it "should return 400 Bad Request if an invalid URL is specified" do
+ post "/api/v1/import",
+ { archivist: @user.login,
+ bookmarks: [ bookmark.merge!(url: "") ]

Space inside square brackets detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@shalott shalott commented on an outdated diff Jun 1, 2016
app/controllers/api/v1/bookmarks_controller.rb
+
+ # Set messages based on success and error flags
+ def response_message(messages)
+ if @some_success && @some_errors
+ messages << "At least one bookmark was not created. Please check the individual bookmark results for further information."
+ elsif !@some_success && @some_errors
+ messages << "None of the bookmarks were created. Please check the individual bookmark results for further information."
+ else
+ messages << "All bookmarks were successfully created."
+ end
+ messages
+ end
+
+ # Returns a hash
+ def import_bookmark(archivist, params)
+ bookmark_request = external_bookmark(archivist, params)
@shalott
Organization for Transformative Works member
shalott added a note Jun 1, 2016

Sorry to be nitpicky some more: it feels to me like even here it makes more sense to have this remain generic (eg "build_bookmark_request(user, params)") because the only place where it actually becomes specific to external works is in the currently named external_bookmark method and bookmark_errors (which is already named generically) -- and in those two places, I would suggest adding a comment explaining that for the moment this ONLY works for bookmarks for external works. There is basically no difference except if the "external" stuff is passed AFAIK.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@shalott shalott commented on an outdated diff Jun 1, 2016
app/controllers/api/v1/bookmarks_controller.rb
+ unless archivist_bookmarks.empty?
+ archivist_bookmarks.each do |bookmark|
+ if bookmark.bookmarkable_type == "ExternalWork" && ExternalWork.find(bookmark.bookmarkable_id).url == url
+ errors << "There is already a bookmark for this archivist and the URL #{url}"
+ end
+ end
+ end
+
+ status = :ok if errors.empty?
+ [status, errors]
+ end
+
+ # Map Json request to Bookmark request
+ def external_bookmark(archivist, params)
+ {
+ pseud_id: archivist.default_pseud.id,
@shalott
Organization for Transformative Works member
shalott added a note Jun 1, 2016

and last question here: what kind of cleaning is happening on this user input and where? I might suggest also a test or two in the spec of submitting bad/malicious content to make sure it doesn't get passed through.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@ariana-paris ariana-paris AO3-4359 - Add handling for common errors that aren't handled by the …
…ExternalWork model
d631c11
@houndci-bot houndci-bot commented on the diff Jun 5, 2016
spec/api/api_bookmarks_spec.rb
+ end
+
+ WebMock.allow_net_connect!
+ end
+
+ describe "Invalid API bookmark import" do
+ before do
+ mock_external
+ @user = create(:user)
+ end
+
+ after do
+ WebMock.reset!
+ end
+
+

Extra blank line detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@shalott
Organization for Transformative Works member

Looks good to me!

@sarken sarken merged commit af3a2ec into otwcode:master Jul 10, 2016

2 checks passed

Details continuous-integration/travis-ci/pr The Travis CI build passed
hound 16 violations found.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment