/otwarchive

Development Environment

redsummernight edited this page Dec 10, 2017 · 68 revisions

Vagrant is a tool that allows us to easily create and install a fully-configured development environment.

Table of Contents

System Requirements

Initial Setup

  • Install version 4.3.8 or greater of VirtualBox (Vagrant supports up to 5.1)
  • Install version 1.5.0 or greater of Vagrant

Reinstalling Vagrant

NOTE: If you are installing a new Vagrant, completely remove the old one and install from scratch. In the folder with your Vagrantfile, run: 

   vagrant destroy

and then proceed with the below instructions.

First Run

In a command window, make a new directory for the project and in that directory in a command window run: 

    vagrant init aooo/ao3_1 
    vagrant up --provider virtualbox

This will download and boot the virtual machine.

Note: The download can take a long time -- make sure to set your computer not to go to sleep or else the download will be interrupted.

The output will look something like: 

    Bringing machine 'default' up with 'virtualbox' provider...
    ==> default: Importing base box 'aooo/ao3_1'...
    ==> default: Matching MAC address for NAT networking...
    ==> default: Checking if box 'aooo/ao3_1' is up to date...
    ==> default: Setting the name of the VM: vagrant_test_default_1435761128429_81089
    ==> default: Fixed port collision for 22 => 2222. Now on port 2200.
    ==> default: Clearing any previously set network interfaces...
    ==> default: Preparing network interfaces based on configuration...
        default: Adapter 1: nat
    ==> default: Forwarding ports...
        default: 22 => 2200 (adapter 1)
    ==> default: Booting VM...
    ==> default: Waiting for machine to boot. This may take a few minutes...
        default: SSH address: 127.0.0.1:2200
        default: SSH username: vagrant
        default: SSH auth method: private key
        default: Warning: Connection timeout. Retrying...
        default:
        default: Vagrant insecure key detected. Vagrant will automatically replace
        default: this with a newly generated keypair for better security.
        default:
        default: Inserting generated public key within guest...
        default: Removing insecure key from the guest if its present...
        default: Key inserted! Disconnecting and reconnecting using new SSH key...
    ==> default: Machine booted and ready!
    ==> default: Checking for guest additions in VM...
    ==> default: Mounting shared folders...
        default: /vagrant => /Users/mac/Desktop/vagrant_test

Now you have downloaded the Vagrant virtual machine, you can connect to it using the following command: 

    vagrant ssh

This will connect you to the Ubuntu virtual machine as the user "vagrant".

Clone the GitHub Repository

Next, you need to check out a copy of the Archive code to work with.

First, if you're new to Git and GitHub:

Now you'll need to fork the otwarchive repository: And now it's time to clone the files from your fork onto your machine so you can work with them.

If you intend to edit the Archive files using tools on your host machine, then perform the following step in the folder where you downloaded the Vagrantfile. The files will then be mirrored in the /vagrant folder on the Vagrant guest machine (note the leading slash in /vagrant). 

   git clone [email protected]:YOURUSERNAME/otwarchive.git
   Cloning into 'otwarchive'...
   remote: Reusing existing pack: 78163, done.
   remote: Counting objects: 15, done.
   remote: Compressing objects: 100% (12/12), done.
   remote: Total 78178 (delta 5), reused 7 (delta 3)
   Receiving objects: 100% (78178/78178), 41.37 MiB | 745 KiB/s, done.
   Resolving deltas: 100% (56038/56038), done.

If you are going to edit the Archive files directly on the guest virtual machine, you can place them wherever is convenient.

Once you have cloned the repository, initialize git and add the release version of the Archive upstream so you can keep up to date: 

  git init
  git remote add upstream git://github.com/otwcode/otwarchive.git
  git fetch upstream

Install Necessary Gems

After connecting with vagrant ssh you want to change into the directory on the Vagrant machine that mirrors the directory you just set up: 

   cd /vagrant/otwarchive/

You will now be in the Archive directory with the current master branch checked out.

Update Ruby and gems

If the Archive's version of Ruby has changed since the last time the Vagrant was updated, you may need to install a new version of Ruby and/or update some gems. You would see an error message like this after cd into the otwarchive directory: 

  ruby-2.3.4 is not installed

If so, update Ruby: 

  rvm install ruby-2.3.4 # or whatever version is needed

After you update Ruby, you will need to install some additional gems: 

  rvm --default use 2.3.4
  rvm gemset use global 
  gem install rvm-capistrano
  gem install bundler
  gem install god
  gem install unicorn

bundle install

Once Ruby and the gems are all updated if necessary: 

   bundle install

You will see: 

   Using rake (10.1.0)
   ...[long list of gems]...
   Your bundle is complete!
   Use `bundle show [gemname]` to see where a bundled gem is installed.

Copy vagrant specific config files 

   cp ~/conf/* /vagrant/otwarchive/config/

Update your database 

  bundle exec rake db:migrate

Start Your Unicorns

After connecting with vagrant ssh: 

   start_my_unicorns

You will see: 

starting new unicorn master
setting /home/vagrant/app/current to /vagrant/otwarchive
copying config database.yml local.yml redis.yml
  -=t`", /`-.___.
    (_/, `-._.r="
      /|/`--,`}
      \I'  ,//  
 
vagrant@ubuntu:/vagrant/otwarchive$

Update the skins from the css 

   rake skins:load_site_skins

Cache the Skins

In order to have the Archive skins available, you have to cache them.

After connecting with vagrant ssh, start a console with: 

    bundle exec rails c

In the console paste the command: 

   Skin.where('cached = true').where('id NOT IN (?)', [887, 890, 928, 929]).each(&:cache!)

Exit console with exit

Filter counts 

  bundle exec rake Tag:reset_filter_counts

Browse the Archive

Point your browser to http://127.0.0.1:8081/

If you get a "connection denied" message, check your Vagrantfile contains: 

  config.vm.network "forwarded_port", guest: 8081, host: 8081

If you get a "connection denied" message, you may need to configure VirtualBox to forward your port. (This has been an issue under OS X in at least one case.)

  • Open VirtualBox on your computer
  • Right-click on the AO3 box in the list of boxes and choose Settings
  • Click on Network
  • Click on the Port Forwarding button
  • Forward host port 8081 to guest port 8081 and OK
If you continue to have issues, make sure that your version of VirtualBox matches the version of the Guest Additions installed in the vagrant box (you should be notified when you run vagrant up if it doesn't). If not, download an older version of VirtualBox from their site.

Load Autocomplete Data

To load the autocomplete data into redis, you have to use the rake task. After connecting with vagrant ssh: 

   bundle exec rake autocomplete:load_data

This will take some time to complete. You can continue working on the Archive in the meantime.

Create User and Admin Accounts

Refer to Development Data for help with creating admin and user accounts to use on your development environment.

Keeping Up To Date (vagrant provision)

You can keep your Vagrant install up to date with fixes and improvements by running 

    vagrant provision

in the same directory as the Vagrantfile, after running vagrant up.

Updating Your Checkout

Vagrant provision won't update your git tree with new upstream data - you'll need to do this yourself. You'll also need to run a few bundle commands after executing the pull operation. This is the general flow for updating your source tree (do this from your vagrant directory): 

    vagrant ssh
    cd /vagrant/otwarchive
    git pull upstream master
    bundle install
    bundle exec rake db:migrate
    bundle exec rake After

Once you've done this, your git tree will be in sync with the upstream otwarchive, and you're ready to start your unicorns.

Starting and Stopping the App

Starting 

  start_my_unicorns

You will see: 

   vagrant@ubuntu:/vagrant/otwarchive$

Stopping 

  kill_my_unicorns

Run Resque Jobs

After connecting with vagrant ssh: 

  start_my_workers

And 

  kill_my_workers

Note: The version of the Archive code being run by the Resque jobs will not automatically reload new code as you write. So if you change the way any background job works, you will have to kill your workers and start them again before the changes will be reflected in your results.

Tests

After connecting with vagrant ssh, to run all tests, use: 

  run_cucumber

This script will set up local settings files required to run tests, recreate the test database, upgrade the gems, run all RSpec tests, and run all Cucumber tests.

This will take a while. Instead, once the system is set up by run_cucumber, you can run individual tests.

If you find that all Cucumber tests are failing immediately, and you see uninitialized constant REDIS_AUTOCOMPLETE in the verbose test output, you can fix it with this change to redis.yml.

Vagrant Snapshots

After setting up the environment properly, you can run 

    vagrant snapshot push

This will save snapshot of the current vagrant VM. You may revert to it using 

    vagrant snapshot pop

It can save you the time of recreating the entire VM in case you manage to ruin your working environment.

Troubleshooting

Elasticsearch Problems

If your Vagrant copy of Elasticsearch has problems and the Archive can't connect to it, you may need to restart Elasticsearch.

After connecting with vagrant ssh: 

  sudo /etc/init.d/elasticsearch restart

During the Elasticsearch upgrade, you can pick a version: 

  systemctl restart elasticsearch-1
  systemctl restart elasticsearch-5
  systemctl restart elasticsearch-6

Clear Memcached

If you need to dump your Memcached store: 

  rails c development

This will load your Rails console in the development environment. You then need to issue the command to dump the store: 

  Rails.cache.clear

Stylesheet 404s

If you notice the layout on your dev environment does not adjust for smaller screen sizes or if you get an error like the following: 

  131 ActionController::RoutingError (No route matches [GET] "/stylesheets/site/2.0/25-role-handheld.css"):
  131 ActionController::RoutingError (No route matches [GET] "/stylesheets/site/2.0/26-role-aural.css"):
  129 ActionController::RoutingError (No route matches [GET] "/stylesheets/site/2.0/27-role-print.css"):

You may need to reload the skins: 

  rake skins:load_site_skins

Mangled JS or CSS files

If you are saving JS or CSS files on the host machine and you notice they are being served truncated, or with bad characters, you can fix it by editing /etc/nginx/nginx.conf in the Vagrant machine.

Change the line that contains sendfile on; to sendfile off;. (source)

Share Your Work with Others

If you have any questions regarding code development, please don't hesitate to send an email to [email protected] and we will try to get back to you as soon as possible!