Planet Drupal

Upgrading Drupal’s Viewport module to Drupal 8 (Part 3 – Final)

Posted by A Dev From The Plains on October 16, 2016 at 9:44am

Image taken from https://drupalize.me

With parts one and two of the series covered, this 3rd (and final!) part will cover some other important aspects of Drupal development that I wanted to pay attention to while working on the Drupal 8 version of the viewport module.

Essentially, the kind of aspects that are easy to forget (or ignore on purpose) when a developer comes up with a working version or something “good enough” to be released, and thinks there’s no need to write tests for the existing codebase (we’ve all done it at some point).

This post will be a mix of links to documentation and resources that were either useful or new to me, and tips about some of the utilities and classes available when writing unit or functoinal tests.

Testing

Since I wrote both functional tests and unit tests for the viewport module, this section is split in two parts, for the sake of clarity and structure.

- Functional tests

Before getting into the bits and blobs of functional test classes, I decided read a couple articles on the matter, just to see how much testing had changed in the last year(s) in Drupal 8. This article on the existing types of Drupal 8 tests was a good overview, as well as Acquia’s lesson on unit and functional tests.

Other than that, there were also a few change notices I went through in order to understand what was being done with the testing framework and why. TL;DR: Drupal runs away from simpletest and moves to PHPUnit to modernise the test infrastructure. That started by adding the PHPUnit test framework to Drupal core. Also, new classes were added that leveraged PHPUnit instead of the existing Simpletest-based classes. Specifically, a new KernelTestBase was added, and also a BrowserTestBase class, replacing the well-known WebTestBase.

I decided to base all my tests in the PHPUnit classes exclusively, knowing that Simpletest will just die at some point. One of the key requirements for this was to put all test classes in a different path: {module-folder}/tests/src/{test-type}/{test-classes}. Also, the @group annotation was still required, as with Simpletest, so that tests of specific groups or modules can be executed alone, without all the test suite running.

The first thing I noticed when I started to write the first Functional test class, ViewportPermissionsTest, was that the getInfo() method was no longer needed, since it’s been removed in favor of PHPDoc, which means all the info there is retrieved from the documentation block of the test class.

With PHPUnit introduced in Drupal 8, a lot of new features have been added to test classes through PHP Annotations. Without getting into much details, two of them that called my attention when trying to debug a test, were the @runTestsInSeparateProcesses and @preserveGlobalState annotations. Most of the time the defaults will work for you, although there might be cases where you may want to change them. Note that running tests in separate processes is enabled by default, but some performance issues have been reported on this feature for PHPUnit.

I also came across some issues about the lack of a configuration schema defined for my module, when trying to execute tests in the ViewportTagPrintedOnSelectedPagesTest class, which led me to find another change notice (yeah… too many!) about tests enforcing strict configuration schema adherence by default. The change notice explains how to avoid that (if really necessary), but given that’s not a good practice, you should just add a configuration schema to all your modules, if applicable.

Some other scattered notes of things I noticed when writing functional tests:

drupalPostForm() $edit keys need to be the HTML name of the form field being tested, not the field name as defined in the Form API $form array. Same happens with drupalPost().
When working on a test class, $this->assertSession()->responseContains() should be used to check HTML contents returned on a page. The change notice about BrowserTestBase class, points to $this->assertSession()->pageTextContains(), but that one is useful only for actual contents displayed on a page, and not the complete HTML returned to the browser.

- Unit Tests

The main difference between unit tests and functional tests (speaking only of structure in the codebase), is that unit tests need to be placed under {module-name}/tests/src/Unit, and they need to extend the UnitTestCase class.

Running PHPUnit tests just requires executing a simple command from the core directory of a Drupal project, specifying the testsuite or the group of tests to be executed, as shown below:

php ../vendor/bin/phpunit –group=viewport

php ../vendor/bin/phpunit –group=viewport –testsuite=unit

As mentioned in the link above, there has to be a properly configured phpunit.xml file within the core folder. Also, note that Drupal.org’s testbot runs tests in a different way. As detailed in the link above, tests can be executed locally in the same way the bot does, to ensure that the setup will allow a given module to receive automated testing support once contributed to Drupal.org.

PHPUnit matchers: In PHPUnit parlance, a matcher is ultimately an instance of an object that implements the PHPUnit_Framework_MockObject_Matcher_Invocation interface. In summary, it helps to define the result that would be expected of a method call in an object mock, depending on the arguments passed to the method and the number of times it’s been called. For example:

$this->pathMatcher->expects($this->any())
      ->method('getFrontPagePath')
      ->will($this->returnValue('/frontpage-path'));

That’s telling the pathMatcher mock (don’t confuse with a PHPUnit matcher), to return the value “/frontpage-path” whenever the method getFrontPagePath() is called. However, this other snippet:

$this->pathMatcher->expects($this->exactly(3))
      ->method('getFrontPagePath')
      ->will($this->returnValue('/third-time'));

It’s telling the mock to return the value “/third-time”, only when the getFrontPagePath() method is executed exactly 3 times.

will(): As seen in the examples above, the will() method can be used to tell the object mock to return different values on consecutive calls, or to get the return value processed by a callback, or fetch it from a values map.

These concepts are explained in more detail in the Chapter 9 of the PHPUnit manual.

Coding Standards and Code Sniffer

With the module fully working, and tests written for it, the final stage was to run some checks on the coding standards, and ensure everything is according to Drupal style guides. Code Sniffer makes this incredibly easy, and there’s an entire section dedicated to it in the Site Building Guide at Drupal.org, which details how to install it and run it from the command line.

Finally, and even though I didn’t bother changing the README.txt contents, I also noticed the existence of a README Template file available in the online documentation, handy when contributing new modules. With all the codebase tidied up and proper tests in place, the module was finally good to go.

That’s it! This ends the series of dev diaries about porting the Viewport module to D8. I hope you enjoyed it! There might be a similar blog post about the port process of the User Homepage module, so stay tuned if you’re interested in it.

Links and (re)sources

As usual, I’ll leave a list of the articles and other resources I read while working on the module upgrade.

Which D8 test is right for me?: link.
Acquia’s lesson on Unit and Functional tests: link.
Simpletest Class, File and Namespace structure (D8): link (note Simpletest wasn’t used in the end in the module port).
Converting D7 SimpleTests to D8: link.
Configuration Schema and Metadata: link.
Drupal SimpleTest coding standards: link
PHPUnit in Drupal 8: link.
Agile Unit Testing: link.
Test Doubles in PHPUnit: link.
Drupal coding standards: link.
README Template file: link.

Change Records and topic discussions

No Drupal core security release planned for Wednesday, October 19

Posted by Drupal core announcements on October 15, 2016 at 3:11am

The monthly security release window for Drupal 8 and 7 core will take place on Wednesday, October 19.

Unless there is an unexpected security emergency, however, this window will not be used and there will be no Drupal 8 or 7 core releases on that date.

Drupal 8 site owners should note that the Drupal 8.1.x branch is no longer supported; sites running on an 8.1.x release should therefore update to the Drupal 8.2.x branch as soon as possible in preparation for any future security releases. (The most recent release on that branch is 8.2.1, as of the date of this post.)

Similarly, Drupal 7 site owners should update to the 7.51 release (the most recent Drupal 7 release as of this post), which includes the larger changes from Drupal 7.50 as well.

Other upcoming core release windows include:

Wednesday, November 2 (patch release window)
Wednesday, November 16 (security release window)

Drupal 6 is end-of-life and will not receive further releases.

For more information on Drupal core release windows, see the documentation on release timing and security releases, as well as the Drupal core release cycle overview.

DrupalEasy Podcast 187 - Breaking BADCamp (Anne Stefanyk, Heather Rodriguez, Jon Peck - BADCamp)

Posted by DrupalEasy on October 15, 2016 at 2:25am

Direct .mp3 file download.

BADCamp organizers Anne Stefanyk, Heather Rodriguez, and Jon Peck join Mike Anello and Ryan Price to discuss the biggest and baddest (pun intended) non-DrupalCon event in all the land(s)! Recent Drupal-related news, picks of the week, and five questions are - as always - included free with this episode.

Interview

https://2016.badcamp.net

DrupalEasy News

Book Review: Drupal 8 Development Cookbook

Three Stories

Drupal file upload by anonymous or untrusted users into public file systems -- PSA-2016-003 - security "update" for Drupal core warning site admins about the dangers of allowing anonymous file uploads.
Getting accurate 'Available updates' for Drupal 6 - blog post by David Snopek.
Implement User Enhancements on Drupal.org and port Dreditor features.

Here’s to our Drupal Association Supporting Partners! – Q3

Posted by Drupal Association News on October 14, 2016 at 6:54pm

There are many inspiring companies that are giving their time to contribute code, help educate others, share their experiences, and be part of the Drupal community. Some of these companies have taken one step further, becoming Supporting Partners with the Drupal Association.

Supporting Partners fund Drupal.org and the engineering team who make the improvements you continuously see. Without our Supporting Partners, the work we do would not be possible.

Here is what is going on now.

Q3 2016 Roadmap

Establish a Technical Advisory Committee (TAP)
Evolve the D.O front page to support the adoption journey.
Identify cost savings
Stable release of the Composer façade
Documentation Migration
Project Application Process Revamp
Drupal 8 User Guide
The search function is now an icon in the header
You can now directly download PDF invoices of DrupalCon ticket orders, right from http://events.drupal.org
Security shield icon
Virtualized Infrastructure

We would like to acknowledge the companies that continue to show their endless support. Here is a list of new and renewing companies just this quarter:

Our Supporting Partners, Technology Supporters, and Hosting Supporters help us make Drupal.org great. They also become some of the best-known Drupal contributors. Read about the program benefits and if you want to give back to the Project, please contact our Executive Director, Megan Sanicki, for details.

4 Common Developer Concerns in a Site Migration

Posted by Zivtech on October 14, 2016 at 4:43pm

Websites have a shelf life of about 5 years, give or take. Once a site gets stale, it’s time to update. You may be going from one CMS to another, i.e., WordPress to Drupal, or you may be moving from Drupal 6 to Drupal 8. Perhaps the legacy site was handcrafted, or it may have been built on Squarespace or Wix.

Content is the lifeblood of a site. A developer may be able to automate the migration, but in many cases, content migration from an older site may be a manual process. Indeed, the development of a custom tool to automate a migration can take weeks to create, and end up being far costlier than a manual effort.

Before setting out, determine if the process is best accomplished manually or automatically. Let’s look at the most common concerns for developers charged with migrating content from old to new.

1. It’s All About Data Quality

Old data might not be very structured, or even structured at all. A common bad scenario occurs when you try to take something that was handcrafted and unstructured and turn it into a structured system. Case in point would be an event system managed through HTML dumped into pages.

There's tabular data, there are dates, and there are sessions; these structured things represent times and days, and the presenters who took part. There could also be assets like video, audio, the slides from the presentation, and an accompanying paper.

What if all that data is in handcrafted HTML in one big blob with links? If the HTML was created using a template, you might be able to parse it and figure out which fields represent what, and you can synthesize structured data from it. If not, and it's all in a slightly different format that's almost impossible to synthesize, it just has to be done manually.

2. Secret Data Relationships

Another big concern is a system that doesn't expose how data is related.

You could be working on a system that seems to manage data in a reasonable way, but it's very hard to figure out what’s going on behind the scenes. Data may be broken into components, but then it does something confusing.

A previous developer may have used a system that's structured, but used a page builder tool that inserted a text blob in the top right corner and other content in the bottom left corner. In that scenario, you can't even fetch a single record that has all the information in it because it's split up, and those pieces might not semantically describe what they are.

3. Bad Architecture

Another top concern is a poorly architected database.

A site can be deceptive because it has structured data that describes itself. The system could find stuff as each element was requested, but then it is really hard to find the list of elements and load all of the data in a coordinated way.

It's just a matter of your architecture. It’s important to have a clearly structured, normalized database with descriptively named columns. And you need consistency, with all the required fields actually in all the records.

4. Automated Vs. Manual Data Migration

Your migration needs to make some assumptions about what data it’s going to find and how it can use that to connect to other data.

Whether there are 6 or 600,000 records of 6 different varieties, it's the same amount of effort to automate a migration. So how do you know if you should be automating, or just cutting and pasting?

Use a benchmark. Migrate five pieces of content and time out how long that takes. Multiply by the number of pieces of content in the entire project to try to get a baseline of what it would take to do it manually. Then estimate the effort to migrate in an automated fashion. Then double it. Go with the number that’s lower.

One of the reasons to pick a system like Drupal is that the data is yours. It's an open platform. You can read the code and look at the database. You can easily extract all of the data and take it wherever you want.

If you’re with a hosted platform, that may not be the case. It's not in the hosted platform’s best interest to give you a really easy way to extract everything so you can migrate it somewhere else.

If you're not careful and you pick something because it seems like an easy choice now, you run the risk of getting locked in. That can be really painful because the only way to get everything out is to cut and paste. It’s still technically a migration. It's just not an automated one.

Nothing About Us Without Us - Diversity of the Web with Nikki Stevens

Posted by Acquia Developer Center Blog on October 14, 2016 at 4:42pm

Nikki Stevens and Jeffrey A. "jam" McGuire talk at Drupal Camp Costa Rica 2016

Following her inspiring keynote on diversity, inclusion, and equality at Drupal Camp Costa Rica 2016, I got the chance to speak with Nikki Stevens on front of my mic and camera. Along with sharing some fun Drupal memories like the time young Nikki broke production when Steve Rude was out for the day, a lot of this conversation is about the benefits diversity brings and how we all can improve our own organisations and communities.

One eye-opening moment for me was when Nikki talked about how many diversity surveys are themselves flawed by the assumptions of their authors. Seemed perfectly obvious to me once I got there. Nikki and Nick Lewis have been working on a better diversity survey by crowdsourcing the survey itself.

Help write the diversity survey!

Go to the Diversity of the Web community-drafted survey and make your own contributions and suggestions now!

For more background information on the survey and the challenges of involvement, identity, and measuring it all, read its origin story on Nikki's blog: Nothing About Us Without Us.

Interview Video - 19 min.

Transcript

A full transcript of our conversation will appear here shortly!

Skill Level: Beginner Intermediate Advanced

Announcing Florida DrupalCamp’s Second Featured Speaker, Drupal Association Director Megan Sanicki!

Posted by DrupalEasy on October 14, 2016 at 4:37pm

The 9th annual Florida DrupalCamp (FLDC) will be held February 17,18, and 19th, 2017 in Orlando, Florida; DrupalEasy is proud to be a sponsor.

We’re super excited to announce Megan Sanicki as our second featured speaker. Megan is the Executive Director of the Drupal Association (DA), and will talk about all the things that the DA does to promote and serve Drupal.

Megan joins Helena Zubkow as a featured speaker for FLDC. Helena is an accessibility expert for Lullabot and will be leading a double-length session on how to make Drupal sites accessible. The third (and final) featured speaker will be announced in the coming weeks.

Bigger and better!

FLDC is expanding to 3 full days of activities in 2017. On Friday, February 17, we'll be offering several free training events (details coming soon) as well as sprint opportunities and a social event. Saturday, February 18 will be a day full of sessions followed by an afterparty. Sunday, February 19 will include a half-day of sessions and additional sprints.

Sponsor Florida Drupalcamp!

FLDC is accepting sponsorships. More information can be found at https://www.fldrupal.camp/sponsors/become-sponsor.

Session Submissions are Open!

Have new Drupal or web development knowledge that you’d like to share? We’d love to have you! Submit your session proposal at https://www.fldrupal.camp/sessions

Entity validation in Drupal 8 - part 1 - how validation works

Posted by lakshminp.com on October 14, 2016 at 4:10pm

Drupal 8 has its entity validation separate and decoupled from the typical validation given by its form API. This is done for a lot of reasons. For one, entities might get added from other non UI means, like via the REST API, or programmatically, while importing data from an external source. Under these circumstances, the entity validation API comes in handy.

Drupal 8's validation API uses the Symfony validator component.Each validation mechanism can be at the entity level(composite), field level or entity property level. Validation can be specified by multiple means.

1.While creating the entity as a part of the annotation.

Ex: the Comment entity has a validation constraint which imposes a restriction where the name of the anonymous comment author cannot match the name of any registered user. This is implemented using CommentNameConstraint and specified in the Comment entity annotation.

 *   bundle_entity_type = "comment_type",
 *   field_ui_base_route  = "entity.comment_type.edit_form",
 *   constraints = {
 *     "CommentName" = {}
 *   }
 * )
 */
class Comment extends ContentEntityBase implements CommentInterface {

2.Inside the entity class's baseFieldDefinitions().

Ex: The User entity has a constraint where each user name should be a unique value.

$fields['name'] = BaseFieldDefinition::create('string')
  ->setLabel(t('Name'))
  ->setDescription(t('The name of this user.'))
  ->setRequired(TRUE)
  ->setConstraints(array(
    // No Length constraint here because the UserName constraint also covers
    // that.
    'UserName' => array(),
    'UserNameUnique' => array(),
  ));

We will see what BaseFieldDefinition means in a future post. For now, all you have to understand is, the above line places a validation constraint that the name property of every user object should be unique.

3.Entity validation constraints can be placed on existing entities from other modules via hooks.

This implements hook_entity_type_alter.

function my_module_name_entity_type_alter(array &$entity_types) {
  $node = $entity_types['node'];
  $node->addConstraint('CustomPluginName', ['plugin', 'options']);
}

We shall be creating one such validation constraint on the node entity shortly.

A validation component consists of 2 parts.

The constraint class, which is a concrete implementation of Constraint, implemented as a Drupal 8 plugin.
The validation class, a concrete implementation of ConstraintValidator.

The constraint contains the metadata/rules required for the validation, the messages to show as to what exactly got invalidated, and a pointer to the validation class, whose default value is a "Validator" string appended to the fully qualified constraint class name.

/**
 * Returns the name of the class that validates this constraint.
 *
 * By default, this is the fully qualified name of the constraint class
 * suffixed with "Validator". You can override this method to change that
 * behaviour.
 *
 * @return string
 */
public function validatedBy()
{
    return get_class($this).'Validator';
}

The validation class contains the actual validation implementation. For example, a "unique name" constraint's validator will iterate through all entities in the database to ensure that the name of the entity being validated is not used by any other entity. The validator class also has access to the constraint class metadata, messages etc. It should, at minimum, implement the validate method, which takes in the object to be validated(string, entity etc.) and the associated constraint. Upon failing the validation, this method returns an object of type ConstraintViolationInterface. This gives all the information as to why the validation failed, where exactly it failed, the invalid value etc.

Let's see how a node can be validated and the validation errors consumed with the below example.

use Drupal\node\Entity\Node;

$node = Node::create([ 'title' => 'New article', 'type' => 'article']);
$node->field_email = 'foobar';
$violations = $node->validate();
if ($violations->count() > 0) {
  foreach($violations as $violation) {
    print_r($violation->getMessage()->render());
    print("\n");
    print_r($violation->getPropertyPath());
    print("\n");
    print_r($violation->getInvalidValue());
    print("\n");
  }
 }

Assuming you have an email field which goes by the machine name field_email, if you run this code using drush scr command in a Drupal 8 setup, your output should be very similar to this.

$ drush scr node-validate.php
This value is not a valid email address.
field_email.0.value
foobar

The getPropertyPath give the field name and the delta as to where the violation occurs.

Now that we got a hang of how entity validation works, let's create our own validation constraint in the next post.

The White House is Open Sourcing a Drupal Chat Module for Facebook Messenger

Posted by Acquia Developer Center Blog on October 14, 2016 at 3:02pm

Today, Jason Goldman, Chief Digital Officer of the White House, announced that the White House is open-sourcing a Drupal module that will enable Drupal 8 developers to quickly launch a Facebook Messenger bot.

The goal, Goldman said, is to make it easier for citizens to message the President via Facebook.

This is the first-ever government bot on Facebook messenger.

Tags: acquia drupal planet

Content Modeling in Drupal 8

Posted by ThinkShout on October 14, 2016 at 3:00pm

In many modern frameworks, data modeling is done by building out database tables. In Drupal, we use a web-based interface to build our models. This interface makes building the database accessible for people with no database experience. However, this easy access can lead to overly complex content models because it’s so easy to build out advanced structures with a few hours of clicking. It’s surprising how often Drupal developers are expected to be content modeling experts. Rachel Lovinger wrote this great overview of content modeling for the rest of us who aren’t experts yet.

Data Modeling Goal

Our goal when modeling content in Drupal is to build out the structure that will become our editor interface and HTML output. We also need to create a model that supports the functionality needed in the website. While accomplishing this, we want to reduce the complexity of our models as much as possible.

Getting Started

One of the first things to do when building a Drupal site is build content types. So, before you start a site build, start with either a content model or a detail page wireframe. This spreadsheet from Palantir will help you. The home page design may look amazing, but it’s unhelpful for building out content types. Get the detail pages before you start building.

Why Reduce Complexity?

The more content types you create, the more effort it will take to produce a site. Furthermore, the more types you have, the more time it will take to maintain the site in the future. If you have 15 content types and need to make a site-wide change, you need to edit 15 different pages.

The more pages you need to edit, the more mistakes you will make in choosing labels, settings, and formatters. Lastly, content can’t easily be copied from one type to another, which makes moving content around your site harder when there are many content types. So, the first thing you’ll want to do with your content model is collapse your types into as few types as feasible. How many is that?

5 Content Types is Enough

Drupal has many built in entities like files, taxonomy, users, nodes, comments, and config. So, the vast majority of sites don’t need any more than 5 content types. Instead of adding a new content type for every design, look for ways to reuse existing types by adding fields and applying layouts to those fields.

Break Up the Edit Form

Drupal 8 allows you to have different form displays for a single content type. With either Form Mode Control or Form Mode Manager, you can create different edit experiences for the same content type without overloading the admin interface.

By reducing the complexity of the content model, we decrease maintenance cost, improve the consistency of the website, and simplify the editing experience. Now that you’ve got some content modeling basics, look for opportunities to reduce and reuse content types in your Drupal projects. Content editors will thank you.

Friday 5: 5 Advantages of Component-Driven Theming

Posted by Mediacurrent on October 14, 2016 at 2:44pm