In the last few months, we introduced features to improve this aspect of SensioLabsInsight: GitHub pull requests integration, analysis status favicons, documentation on how to compute statuses for your analyses, etc.

To go even further, we have added support for notifications. Notifications are simple configurations you can add to your projects to be notified directly in your favorite messaging app if an analysis succeeded or failed. For instance, you could add the following configuration to your project to be notified in Slack:

notifications:
    - type: slack
      url: https://hooks.slack.com/services/XXXXXXXXX/XXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXX
      channel: '#insight'

You can learn more about notifications in the dedicated section of the documentation.

For now, we support Slack, HipChat and custom WebHooks. We plan to add support for other apps so don't hesitate to write a comment under this blog post to ask for one!

As always, if you have any issue, feel free to contact us using the "Contact us" button on the left side of each page of SensioLabsInsight.

Enjoy!

Symfony Flex is a Composer plugin providing several major improvements over the Standard Edition, the classic way of creating Symfony projects. By default, it is extremely slim to let you add only what you need. It also automatically configures and uninstalls packages when you need or don't need them anymore. Finally, it is suited to much more use cases than just the simple web application scenario.

As more and more applications start using Symfony Flex, we have added support for it in SensioLabsInsight. To analyze a Symfony Flex application, create it with the Symfony 3/4 web project project type and everything will work as expected, out of the box!

Enjoy!

The built-in support of this technique was added in Symfony 3.3 and Symfony Flex, the new way to create projects in Symfony 3.4, will use it by default.

As more applications are migrating towards the usage of environment variables, we added support for .env.dist files in SensioLabsInsight.

To use it, create a .env.dist file in the working directory of your project and all environment variables defined in it will be accessible during Composer installation and during the analysis.

This feature helps supporting Symfony 4 and Symfony Flex on SensioLabsInsight and it also means that any project you have may now use environment variables easily (even non-Symfony projects).

Enjoy!

At SensioLabsInsight, we embrace this by promoting the usage of the .sensiolabs.yml file in your repository to configure how your project should be analyzed over the web based configuration.

However, due to how we run analysis internally, some options such as the PHP version or its extensions were only available in the web configuration.

We improved this experience by adding support for all the configuration options in the YAML configuration file. Configuring SensioLabsInsight is now possible entirely from the .sensiolabs.yml file!

Enjoy!

However, there is always room for improvements, especially for special cases which are a bit more difficult to setup like when your code cannot be pulled automatically by Insight.

To improve this, we added a new project creation page and updated project creation forms.

The new creation page aims to help you choose the right creation method for your project. As some features differ from one method to another, we now give you advices on how to create your project the proper way:

Creation forms are also updated to be more expressive about the role of each field:

Enjoy!

The favicon for in progress analysis

The favicon for succeeded analysis

The favicon for failed analysis

Enjoy!

To help you explore SensioLabsInsight documentation seamlessly, we added a search engine based on Algolia.

Find it on https://insight.sensiolabs.com/docs.

Enjoy!

As SensioLabsInsight needs access to your code, configuring it depends on where your code is hosted and how you run your test suites. Combined with a wide range of configuration settings, it might be time consuming to optimize its integration with your projects.

Knowing all this, we rewrote the documentation, aiming it to be clearer, more precise and better organized to suit most common needs.

For instance, you can read a new Getting started tutorial explaining in details the concepts of SensioLabsInsight, new documentation categories dedicated to the different platforms supported by SensioLabsInsight and a new reference page listing all SensioLabsInsight's options.

Read it on https://insight.sensiolabs.com/docs.

A few months ago, we introduced pull requests analysis and commit status on GitHub. You can now configure conditions that check the quality of your project during your development process.

New failure conditions attached to pull requests ensure the quality of your project stay manageable on each individual pull request.

We continue in this direction by adding a new analysis view dedicated to GitHub pull requests. This new report displays violations related to the files modified in the pull request.

This feature is a major improvement for anyone using SensioLabsInsight with GitHub as it will be much easier to find issues introduced in each pull request. We hope this feature will become your new ally during your next Pull Request reviews!

A smooth experience for GitHub users on SensioLabsInsight

This new feature is a step towards a goal we are targeting since a few weeks: bringing a great and smooth user experience for GitHub users on SensioLabsInsight.

This will lead us to many more modifications, including some not only targeting GitHub. We hope these changes will help you to increase your productivity and to focus as much as possible on what really matters: your business logic.

In the meantime, I would like to share with you a small screencast I made to show how quick it can be to set up SensioLabsInsight for a Symfony project hosted on GitHub. Enjoy!

For example, to ensure that the quality of your project does not decrease on pull requests, you can use a configuration like the following:

commit_failure_conditions:
    - "project.severity.critical > 0"
    - "project.severity.major > 0"
    - "project.severity.minor > 0"
    - "project.severity.info >= 15"

Failure conditions and Github statuses were already a big step forward in the integration of SensioLabsInsight into your development pipeline. We made it even more powerful with failure conditions related to the current pull request.

Pull requests related failure conditions

The concept is quite simple but very useful: in addition of the project variable, you can now use the pr variable that gives access to the violations related to files modified by the pull request.

This configuration can be useful for instance in the case of legacy projects needing a migration towards better practices: instead of having a huge list of issues, you can now use SensioLabsInsight to improve your project quality steps by steps.

Here is the reference of this new variable:

commit_failure_conditions:
    - "project.severity.critical > 0"
    - "project.severity.major > 0"

    # Fail by severities count introduced by the current pull request
    # - "pr.severity.critical > 0"
    # - "pr.severity.major > 0"
    # - "pr.severity.minor > 0"
    # - "pr.severity.info >= 15"
    #
    # Fail by categories count introduced by the current pull request
    # - "pr.category.architecture > 0"
    # - "pr.category.bugrisk > 0"
    # - "pr.category.codestyle > 0"
    # - "pr.category.deadcode > 0"
    # - "pr.category.performance > 0"
    # - "pr.category.readability > 0"
    # - "pr.category.security > 0"
    #
    # Fail by total count introduced by the current pull request
    # - "pr.violations > 150"

We hope this feature will help you improve the quality of your projects, enjoy!

__autoload

The __autoload is now deprecated in favor of spl_autoload_register as it is much more flexible allowing chained autoloaders to be registered and as these two autoload registration methods are incompatible.

function __autoload($className) {
    include $className.'.php';
}

// Should be replaced by

spl_autoload_register(function($className) {
    include $className.'.php';
});

create_function

create_function was a technique used before PHP 5.3 and based on eval to create lambda functions. Due to its nature, this function is not performant and is not needed anymore. It should not be used anymore.

create_function('$a, $b', 'return strlen($b) - strlen($a);');

// Should be replaced by

function($a, $b) {
    return strlen($b) - strlen($a);
};

gmp_random

gmp_random generates a random number between 0 and 2**($n*BITS_PER_LIMB)-1, where BITS_PER_LIMB is platform specific and not accessible for userland code. Therefore, this function is tied to the platform and should not be used.

PHP 5.6 introduced gmp_random_bits() and gmp_random_range() functions to solve this issue. These functions should always be preferred over gmp_random().

gmp_random();

// Should be replaced by

gmp_random_range($min, $max); // or
gmp_random_bits($bits);

each

each is a technique that can be used to iterate over an array, similarly to foreach. However, it is inferior to foreach in every way, being 10 times slower and posing problems to the engine. Therefore, foreach should always be preferred.

while (list($key, $val) = each($array)) {
    echo $key.' => '.$val.'<br />';
}

// Should be replaced by

foreach ($array as $key => $value) {
    echo $key.' => '.$val.'<br />';
}

Don’t hesitate to drop us a message if you face any issue.

Enjoy!

The challenge of keeping clean code

Keeping code clean during the first phase of a project is usually not too difficult: the team does not change much, the architecture was well thought to solve the original problem and the code is written in big chunks, following the same conventions at each modification.

On the other hand, all along the project lifespan, the team changes, the original problem evolves and the code has to adapt to these changements. This naturally leads to technical debt.

As developers, technical debt is our worst enemy: it brings bugs and lack of trust in the code, it makes the project's architecture fragile and ultimately it dramatically slows down development speed. While keeping code clean is therefore essential, it's usually very difficult to measure it and act early enough.

That's where SensioLabsInsight shines: by providing an objective and thorough analysis of your project's code, it will give you an idea on when and how you have to act to avoid technical debt.

The SensioLabsInsight commit status on GitHub: the ultimate tool in your hands

At SensioLabs, we are developers, just like you. We understand how important Continous Integration is and especially how its integration in your development process has to be as smooth as possible.

Today, we bring SensioLabsInsight to you by integrating it in your development process directly as a Github status on your pull requests, just like your other Continous Integration tools.

This feature is enabled by default on every Github project and has been backported to every repository, even created before today. You should already get commit statuses on your pull requests!

Configure it to your needs

The commit status is processed by analysing your code and evaluating a few conditions called "failure conditions": if one of them is evaluated as true, the commit is displayed as failed on Github.

To change the failure conditions, you can add the key commit_failure_conditions to you project configuration (in the web interface) or to your .sensiolabs.yml file (in your repository).

Here is the reference of this configuration key:

commit_failure_conditions:
    # By severities count (default configuration, any change will override it)
    - "project.severity.critical > 0"
    - "project.severity.major > 0"

    # # By other severities count
    # - "project.severity.minor > 0"
    # - "project.severity.info >= 15"
    #
    # # By categories count
    # - "project.category.architecture > 0"
    # - "project.category.bugrisk > 0"
    # - "project.category.codestyle > 0"
    # - "project.category.deadcode > 0"
    # - "project.category.performance > 0"
    # - "project.category.readability > 0"
    # - "project.category.security > 0"
    #
    # # By project grade (none, bronze, silver, gold, platinum)
    # - "project.grade >= gold"
    #
    # # By total violations count
    # - "project.violations > 150"

We're very happy to release this feature, please help us make it better by giving your feedback!

New rule for missing visibillity on class methods and properties

One of the main goals of SensioLabsInsight is to promote the best practices used by professional PHP applications. With this in mind, we introduced a new rule to ensure a project always uses visibility flags (public/protected/private) on its class methods and properties.

There are a few reasons to always use visibility flags:

• it helps to show the intent of classes
• it provides coding guidelines on how the code should be used
• it reduces the need to validate properties when used internally

class Example
{
    // This will now trigger a violation
    static $foo = 4;

    // This will now trigger a violation
    function bar() {
        return 0;
    }

    // This is valid
    public static $foob = 4;

    // This is valid
    private function barb() {
        return 0;
    }
}

New rule for PHP 7 reserved words

PHP 7 introduced a list of new reserved keywords (and PHP 7.1 is continuing to do so) which should not be used as classes, interfaces or traits names to avoid issues when migrating.

This rule will warn you if a class, an interface or a trait in your code uses a reserved keyword.

// Invalid class name
class Int {}

// Invalid interface name
interface Bool {}

// Invalid trait name
trait Void {}

Return type hints are now properly taken into account

Until now, return type hints were not taken into account as the usage of a class or an interface and false positive "Unused use statement" violations were triggered.

This is fixed as return type hints are now handled properly by SensioLabsInsight.

use Foo\Bar;

// This won't trigger a "Unused use statement" violation anymore
function myFunction(): Bar {
    return otherFunction();
}

Allow the usage of the Event annotation

The usage of the Event annotation in Symfony core code is progressively increasing to provide a useful autocompletion basis for development environments.

Until now, this annotation was considered as a regular Doctrine annotation and a violation was triggered as it was not imported with a use. We fixed this behavior to accept this annotation as a native one (on the same level as @param and @return).

Updated deprecated functions, classes and methods for PHP 7 and Symfony 3.2

Following the changes of PHP and Symfony in term of deprecations, we updated our rules to ease your work in migrating towards new versions.

There’s two ways to do that. Either by downloading it via GitHub’s API, or via a git clone. The second option can be much more expensive in time.

When we released SensioLabsInsight, GitHub had one limitation: 60 call per hour on the API. Given the huge number of analyses we run each hour, you can imagine that we didn’t have a choice but to git clone.

The great news, to some extent, is that we weren’t the only ones to face this issue. And the Drupal Community was the one to trigger the change. Nils and Jordi got in touch with the GitHub team, who had done some great job on their infrastructure meanwhile. They were able to drop the API rate limiting.

Right away, we changed our ComposerInstallScript.php file. Here's the diff:

What’s in it for you? Faster analyses!

Enjoy,

What changes? Nothing. It’s just working.

Don’t hesitate to drop us a message to support@sensiolabs.com if you face any issue.

Enjoy,

End of 2015, two major breakthroughs happened in the PHP world: PHP7 and Symfony3. We of course had to support it in SensioLabsInsight.

You can now simply analyse Symfony 3 projects, just like you used to analyse Symfony2, Laravel, Drupal, etc.

Follow our getting started tutorial, and select Symfony3 Web Project at step 2.

Stay tuned for PHP7 support!

Don’t hesitate to drop us a message to support@sensiolabs.com if you face any issue.

Enjoy,

At SensioLabsInsight we're committed to break this law as much as possible. In addition to upgrading our servers hardware on a regular basis, we are constantly optimizing our code analyses engine.

Most of the times, these code optimizations provide a small reduction in the time it takes to analyze projects. However, a few days ago we introduced an engine optimization that has reduced analysis time up to a third of the previous time.

The following chart plots the analysis duration for a project that contains about 180,000 lines of code and tens of code quality violations. The red line marks the moment when the new analysis engine was introduced:

Faster analyses mean that you'll get earlier feedback whenever your codebase introduces code quality violations. Therefore, your development team would be able to tackle quality problems as soon as they are introduced.

In normal circumstances this error is rarely seen in the analyses performed by our users, but a recent change introduced by the 2.5 version of the DBAL project is breaking lots of projects on SensioLabsInsight.

Basically, when calling the getDatabasePlatform() method in DBAL 2.5, that call needs to establish a connection in order to evaluate the appropriate database platform class if Doctrine\DBAL\Connection is not already connected.

The workaround is to explicitly configure the database server version used by Doctrine. In Symfony applications, this is as easy as declaring the server_version parameter of Doctrine DBAL in the app/config/config.yml file. The value of this parameter must be the version number of your database server. For example, if you use MySQL, execute mysql -V to get the version and configure it as follows:

# app/config.yml
doctrine:
    dbal:
        server_version: 5.6

In addition to making your application bootable again for SensioLabsInsight, this new parameter will save your application one database call/connection on production environment.

This graph shows the estimated amount of time that it would take for each project to fix all the problems found by the SensioLabsInsight analysis service.

Some of you made a lot of interesting questions and comments about the graph and we'd like to clarify some things in this article.

Important information is missing

First of all, as any other graph, there is a lot of missing information that should be considered to better analyze the results. For starters, project age is a determining factor in its quality. WordPress and phpBB were released more than 10 years ago, before most of the modern PHP features were available. This put those projects in a competitive disadvantage over other more modern applications such as Composer or Doctrine.

Another important factor missing on the graph is the project size. Projects with large codebases, such as WordPress or Drupal, are always going to have more technical debt than small projects such as Silex or Twig. However, large projects usually have lots of contributing developers and thus, technical debt could be more easily assessed on large projects.

SensioLabsInsight is not a Symfony service

SensioLabsInsight is a service provided by SensioLabs, the company behind the Symfony project. Since all the Symfony ecosystem products (Symfony, Silex, Twig) got excellent results on the analysis, some of you expressed concerns about the analysis impartiality.

First, SensioLabsInsight is not a Symfony-focused service. Our motto is "PHP Quality Done right" and we analyze any PHP project or library applying the same analysis rules.

Nevertheless, if you analyze Symfony applications, we apply additional rules specially designed for that framework. And the same goes for Drupal, Laravel and Silex projects. We apply special rules for those projects, meaning that their analyses are stricter.

But if your project is not based on Drupal, Laravel, Silex or Symfony, the special rules are not applied and therefore, the analysis results aren't impacted. Moreover, if some of the analysis rules affect your project in an unfair way, you can ignore those rules from the technical debt calculation.

Tracking technical debt evolution

Some developers interpreted this graph as an absolute score grading the quality of a project. This would mean for instance that Drupal is better than WordPress and Laravel is better than CakePHP. This is simply not true.

This graph only shows the still photo of the total technical debt of some PHP projects at a given time. The truly interesting metric is the evolution of the technical debt. If this value increases in time, the technical debt is out of control and using that project may be risky for companies. If the technical debt decreases in time, the project will eventually reach a manageable debt level.

Analyze your own projects

SensioLabsInsight lets you analyze your PHP projects for free and that's the best way to fully understand how the code quality analyses work and to get a first glimpse of your technical debt.

Analyzing the technical debt of your PHP projects may be useful for comparing them with those of your competitors, to check if your contractors are doing a good job and as an objective quality review before delivering some project.

To get the most out of SensioLabsInsight, read the Getting Started tutorial, install the PHPStorm plugin and don't forget to proudly show the result using our cool medal widgets.

Introducing the Lite Individual Plan

A lot of freelancers reached us to ask for a SensioLabsInsight plan with less enterprise features and a more affordable price. That's why today we're introducing a new Lite Individual Plan which provides all the major features for one private project starting at 6 euros/month for the yearly subscription.

Open documentation for Symfony rules

Our code analysis rules are the result from the intensive practice by our experts in auditing customer quality. When we detect a violation, we're not simply telling you: « There's something wrong with your code ». We're telling you why, how to solve it, and support it with references and clear examples.

That's why documentation is one of the best features of SensioLabsInsight. Starting today, we've decided to open the Symfony-related rules documentation to all users.

Unlimited collaborators for Business Plans

We noticed that the collaborators feature showed great interest from the Business plans users. As these plans are used by companies with large development teams, most of them reached the maximum number of collaborators limit too quickly. As a result, starting today we are allowing all business plans an unlimited amount of collaborators.

New plans switching and upgrading

If you are already a monthly subscriber, at the end of your monthly subscription, you'll have to either keep going with your current plan, or to subscribe to one of the new plans. Users who subscribed yearly will automatically be upgraded until the end of their current subscription. Check out the new plans and pricing page for a complete overview of each plan.

And if you feel like you need to talk with us or have more questions about these new plans, don't hesitate to get in touch with our sales representative, Ludovic Duval: ludovic.duval@sensiolabs.com

When dealing with software, the total debt amount at a given moment is not that important as its evolution in time. SensioLabsInsight premise is to analyze your code quality on a continuous basis to easily track the evolution of your technical debt.

In order to simplify the tracking of this evolution, we are proud to introduce the new PHPStorm plugin for SensioLabsInsight. This new plugin seamlessly integrates SensioLabsInsight analyses and reports into your favorite IDE.

Once installed, you'll see new markers that provide insights about the code quality violations introduced by each line of code:

Moreover, this plugin displays a new panel at the bottom of PHPStorm window that provides direct access to the most common tasks on SensioLabsInsight:

Learn how to install this plugin and discover all its features reading our new tutorial about How to integrate SensioLabsInsight into PHPStorm and the screencast published at PHPStorm's official blog.

Static method should not contain $this reference

Static methods of PHP classes should not use $this but self to avoid errors at runtime. Although this is an easy rule to follow, it's not uncommon to introduce the following errors when refactoring your application code:

<?php

class FileUtils
{
    private $contents = array();

    // This method used to be static
    // But to handle caching, it had to become an instance method
    public function getContents($filePath)
    {
        if (isset($this->contents[$filePath])) {
            return $this->contents[$filePath];
        }
        if (!file_exists($filePath)) {
            throw new \InvalidArgumentException(
                sprintf('Required file "%s" does not exist', $filePath)
            );
        }

        return $this->contents[$filePath] = file_get_contents($filePath);
    }

    public static function countNbLines($filePath)
    {
        // when getContents was turned to non-static, the developer updated
        // the call below. Unfortunately, they forgot to remove the static
        // keyword in the countNbLines() definition
        return substr_count($this->getContents($filePath), "\n");
    }
}

Imagine overviewing this error until it's too late and your application is deployed in production! That's exactly what SensioLabsInsight is for: we keep on adding rules preventing bugrisks in production, to make your code always more robust.

Interfaces names should end with "Interface"

One of the main goals of SensioLabsInsight is to promote the good practices used by professional PHP applications. In the past we couldn't enforce those code checks because they could affect your analysis results.

However, now that the chocolate medals are gone we'll start adding lots of these code style checks on a regular basis.

The first one of these rules checks whether your interface classes include the Interface suffix:

<?php

// Wrong interface name
interface Storage { ... }

// Wrong interface name
interface StorageInt { ... }

// Correct interface name
interface StorageInterface { ... }

Although this is not the only way to name interfaces, it's probably the easiest one to follow consistently in your codebase. Some developers prefer to use names such as Loggable and Cacheable. The problem with this approach is that sometimes is really hard to come up with a good name that follows this strategy.

In any case, as any other SensioLabsInsight rule, you can tweak its behavior. To do so, edit your project configuration and add the interface_name_pattern option to define the regular expression used to validate the name of the interfaces:

rules:
    # ...
    php.interface_has_no_interface_suffix:
        enabled: true
        interface_name_pattern: /(Interface|Exception|able)$/

You can also disable all violations reported by this rule adding the following option to your project configuration:

rules:
    # ...
    php.interface_has_no_interface_suffix:
        enabled: false

False positives, tweaks and bugfixes

As always, we keep on tuning these and the other rules for false positives. Don't hesitate to report us any false positives on the new rules.

These days it's common to use the same GitHub account both for personal and professional projects. Indeed, most of our users belong to at least one GitHub organization, where they can collaborate on projects owned by other users.

Starting today, when you click on the Add Project button, the project listing will show not only your personal projects, but also the projects created by all the GitHub organizations you belong to (up to 100 projects):

If you are one of the administrators of the selected project, the analysis will begin as soon as you click on the Analyze button. The reason is that when you are an administrator, SensioLabsInsight can automatically register a new SSH deploy key for you in the repository.

In case you are not an administrator, you'll see instead the following message:

As SensioLabsInsight cannot register a deploy key in these cases, you either have to fork the project to your private account or you have to use the Private Git option to analyze the project. The latter is the most convenient way and that's why SensioLabsInsight shows you the needed configuration.

We hope that this new feature will allow you to analyze the quality of all your organizations' projects and we'd love to get your feedback about it!

It turns out that about 11% of all our analyses ended up with chocolate medals. That's much higher than we thought. Also, our users took their analysis medals quite seriously. They complained very loudly if a false positive violation prevented them from reaching a given medal. "No problem," we used to reply, "just ignore the false positive and you'll get the medal you want". The invariable answer sounded like: "But that will give me a chocolate medal. Chocolate medals are cheap, everyone can get one for free in a cereal box. I want a genuine medal!".

That revealed that our medals had a great value in the eyes of our users. But that was also a huge burden. A statical code analysis tool always has false positives. For Insight, every false positive had to be fixed in the analysis engine to prevent chocolate medals. We ended up tweaking our engine very thoroughly, completely rewriting some rules to take into account all the false positives reported by our users.

In retrospect, the chocolate medal idea was a mistake. When users consider a given violation as a false positive, we can't possibly interfere with that - they know their codebase, we don't. When users tweak / disable some rules because, in their organization, different coding standards apply, we can't tell them they're wrong. Dealing with false positives took us away from adding new rules to the analysis engine, and adding more features to Insight. Our rules must apply to 80% of the cases, not 100% - that's just way too expensive.

Today, we're retiring chocolate medals. Even if you tweak project configurations, even if you ignore violations, you will always get a genuine medal. And if you want to benchmark your projects with other projects, don't stop at the medal. Also look at the score, the number of ignored violations, and the project configuration.

That doesn't mean that we'll stop fixing false positives in our rules. But we'll certainly spend less time doing that, and more time implementing new rules and new features.

You will see the score under the medal, written on a ribbon:

20 is Bad, 95 is Excellent

Each project now has a score on 100, based both on the remediation cost and the number of lines of code. Here is the score we computed on of a few public projects:

• Silex: 94/100
• Symfony2: 86/100
• Faker: 77/100
• SwiftMailer: 46/100
• Doctrine ORM: 45/100
• Twig: 27/100
• Joomla: 27/100

We're still tweaking our grading algorithm, but that already gives you an idea of where your project might be.

How We Compute The Insight Score

We first thought about using exclusively the remediation cost to compute the grade. But it turns out that large projects often have a heavy remediation cost, even though their codebase isn't necessarily of worse quality. It's just that the more you write code, the more chances you have to write bad code. Using only the remediation cost would make large projects have a lower score, and that would not be fair. We needed a scoring system to grade a project relative to projects of the same size.

Going further, we made a few statistics on all the analyses computed by Insight. On average, the remediation cost increases by one hour each time a projects gets bigger by about 60 lines of code. That means that the average remediation cost computed by Insight on the thousands of projects already analyzed is of one minute per line of code.

Whether a project is above or below that average is a great sign of its quality. That's why the Insight score is based on a remediation cost / lines of code ratio. The best projects get a 100, the worst get a 20, because we don't want to give a zero!

Try It And Send Us Your Feedback

We hope that this new score will help you assess your quality progression in a fine-grained way. Do you think it's fair for your project? Do you think it's not? Send us your feedback about this new feature!

We Hear You

Insight has always been agnostic as to where your code is hosted. Whether it's in a self-hosted Git server, in a public BitBucket project, or in a private GitHub repository, we analyze it. You just need to allow Insight to pull your code.

But that implies a lot of friction in the project addition process. We've received various feedbacks about that process, none was good. We've compared what we do with similar services, and Insight is notoriously harder to dig into. We've used our tool over and over, and we've been exasperated just like you to have to create an account, setup SSH keys, go through up to 10 steps before the analysis actually starts.

To say the least, adding a new project on Insight has always been a rough path. But adding a new project is the very first thing new users do. The impressions we gave during this first contact weren't encouraging.

Introducing GitHub and BitBucket Single-Sign-On via OAuth

We've just pushed the ability to add projects via GitHub and BitBucket directly, using their OAuth APIs. As a result, if your code is hosted by one of these platforms, you don't have to touch the keyboard anymore. In now takes about five clicks for a newcomer to launch an analysis. Five clicks, and less than a minute.

And once you've authorized SensioLabsInsight to access your repositories hosted on GitHub or BitBucket, analyzing yet another project becomes extremely simple. Our user base uses GitHub a lot, so I hope that this huge improvement will ease your life.

Adding a Project Has Never Been Easier

We've also improved the usability of the project addition workflow in general, using a "wizard" approach. We've minimized the amount of information required to get started. We've simplified the process for every use case, including for those who host their code behind a firewall, and who need to push code to SensioLabsInsight rather than let us pull the code.

We've done lots of user tests to validate our new design. We believe that adding a new project is not painful anymore, and it's even so easy that we've analyzed more and more projects ourselves!

Quality Made Simple

Our purpose has always been to improve PHP code quality in general. Without the pain of setting up a CI server yourself, and using a unique set of PHP and Symfony checks that no other platform provides, you can spot major weaknesses in your code in no time.

Now is the perfect time to give SensioLabsInsight a try. Tell us what you think about out first contact workflow, and about the service in general.

Choose The Branch From The Analyze Button

Whenever you need to analyze another branch than the default one, click on the left part of the "Analyze" button to reveal the branch choice input. Then type the name of the branch you want, and click on "Analyze".

The branch dropdown is present on all occurrences of the "analyze" button.

See Non-Master Branch Analyses In The List

Every analysis made on a branch different than the default one shows the branch name close to the project name. That way, you can immediately spot those non-master analyses, which is especially useful if you're using a SCM hook to analyze all pushes to the master repository, including on feature branches. You can set the default branch for a project in the project settings if it's not master.

By the way, did you notice that the name of the person who triggered the analysis is displayed above the project name? That way, you can know when a collaborator, or the API, launched an analysis for you.

The mention of the branch is also displayed in the full report sidebar, where you can see the title of the last commit in the analyzed branch.

Detection Of Parent Analysis

Each analysis is compared to its "parent" analysis to determine if there are new violations (red arrows), or fixed violations (green arrows). Determining the parent of an analysis in the master branch is easy: it's the last analysis on the master branch. But how about when you analyze another branch?

We've implemented an algorithm using the git history, to find the first ancestor of the analyzed branch already analyzed by Insight. This helps you figure out the real quality evolution of a branch:

• If this is the first analysis in the branch, the analysis is compared to the latest analysis on master
• otherwise, the analysis is compared to the previous analysis on the same branch

Ignore Branches In SCM Hook

If you use a SCM hook on GitHub, you may not want to have your project analyzed each time you push to the gh-pages. The default project configuration has been updated to take gh-pages into account, using a new setting called ignore_branches:

ignore_branches:
    - gh-pages

You can add more branches to ignore so that some pushes to your projects don't automatically trigger an Insight analysis.

We're still working on the branch analysis feature - make sure you come back in the next weeks to see the changes!

Setting Up the Environment

Jenkins integration is performed through the SensioLabsInsight API. The easiest way to use this API is via the PHAR version of its command line tool. Download it from http://get.insight.sensiolabs.com/insight.phar, or execute the following commands:

# download it with cURL
$ curl -o insight.phar -s http://get.insight.sensiolabs.com/insight.phar

# download it with wget
$ wget http://get.insight.sensiolabs.com/insight.phar

Check that the insight.phar file has been correctly downloaded by executing the following command that lists all your projects configured on Insight:

$ php insight.phar projects

The first time you execute insight.phar, you will be prompted for your SensioLabsInsight API token and user UUID. You can find this information in the API/SDK tab of your SensioLabsInsight account preferences:

The insight.phar file will optionally save these credentials for you to reuse them for the next commands. To override these stored credentials, use the --api-token and --user-uuid command options.

Integrating SensioLabsInsight Reports into Jenkins Builds

First, make sure that you have installed the PMD plugin, which is required to process the SensioLabsInsight reports generated in PMD format. Then, create a new build command that executes the following:

#!/bin/sh
php insight.phar analysis --format=pmd <project-UUID> > insight-pmd.xml

Finally, add a new post-build action in Jenkins to publish the PMD analysis results stored in the insight-pmd.xml file.

And that's all! From there on, whenever a new build is initiated by Jenkins, a new analysis will be triggered in SensioLabsInsight and the results will be integrated directly into your Jenkins output.

Click on the report details to view the complete list of violations, including the file and line of code where each one was found:

Improving Your Deployment Process

Deploying code that contains security exploits and performance bottlenecks can pose a serious threat to your company. Integrating SensioLabsInsight into your deployment process via Jenkins prevents you from deploying bad code into production servers.

Start by adding a new fail-condition option to the analysis command and then set the condition that will cause the build to fail:

$ php insight.phar analysis <project-UUID> --format="pmd" \
      --fail-condition="counts.security > 0"

The counts.security > 0 expression is a good starting point to prevent the deployment of dangerous code, but you can use any other condition built with the ExpressionLanguage component.

The two variables that can be used in the expressions are called analysis and counts. The analysis variable stores all the information about the analysis as defined by the Analysis.php model class. The count variable stores the number of violations grouped by category. Combining both variables you can build very powerful expressions:

// the project has a too low grade
analysis.grade in ['none', 'bronze']

// the project has a lot of violations, including some performance issues
analysis.nbViolations > 50 and counts.performance > 0

// the technical debt of the project is too high
analysis.remediationCost > 100

Our first approach to count the number of lines of code was naive. This brought us a lot of feedback about unfair "Symfony controller action method should not be too long", or "PHP classes should be short" violations. What if the code had a lot of comments? What if your developers followed a coding standard adding more newline characters than PSR-2?

We've recently released a new line counting algorithm, but before explaining it, let's look at some code.

<?php
/**
 * @Method("GET")
 * @Route(
 *     "/books/{bookId}",
 *     name="project"
 * )
 */
public function bookAction($bookId)
{
    $book = $this->getDoctrine()->getManager()
        ->getRepository('Book')
        ->find($bookId);

    if (!$book) {
        throw $this->createNotFoundException('Book not found.');
    }

    return array(
        'book' => $book
    );
}

How much lines of code do you think this method has? 12? 14? 21? Let's look at the same code, written differently:

<?php
/**
 * @Method("GET")
 * @Route("/books/{bookId}", name="project")
 */
public function bookAction($bookId) {
    $project = $this->getDoctrine()->getManager()->getRepository('Book')->find($bookId);
    if (!$book) { throw $this->createNotFoundException('Book not found.'); }
    return array('book' => $book);
}

It's the exact same code, yet it has only 5 lines of code without comments, 9 with comments. And what about this version?

<?php
/**
 * @Method("GET")
 * @Route(
 *     "/books/{bookId}",
 *     name="project"
 * )
 */
public function bookAction($bookId)
{
    /** @var $book \Library\Model\Book */
    $book = $this->getDoctrine()
        ->getManager()
        ->getRepository('Book')
        ->find($bookId)
    ;

    if (!$book) {
        // FIXME: redirect to the book creation page instead
        throw $this->createNotFoundException('Book not found.');
    }

    return array(
        'book' => $book
    );
}

Now you understand why it's a difficult problem. Ignoring comments and newlines isn't a big deal, but counting lines in a consistent manner across all coding styles can't just be done in a simple way. Even using third-party line counter scripts won't help us, since they will count the three extracts above in a different way.

SensioLabsInsight already parses your entire codebase to analyze it. The analysis engine turns PHP scripts into an Abstract Syntax Tree (AST), an object-oriented representation of all the statements, variables, and expressions in your code. That means that during the analysis, our engine has a normalized version of the code at hand. The only thing that we need to do is to serialize this AST in a consistent manner, and count the lines of the resulting string.

For the code extract above, the normalized version serialized by the Insight engine looks like this:

<?php
public function bookAction($bookId)
{
    $book = $this->getDoctrine()->getManager()->getRepository('Book')->find($bookId);
    if (!$book) {
        throw $this->createNotFoundException('Book not found.');
    }
    return array('book' => $book);
}

Whatever coding style is used, Insight will always count 8 lines of code for this piece of code. So don't be surprised if the line counts displayed in a violation don't exactly reflect the actual number of lines in your code. Don't be surprised either if a few violations disappear during your next analysis: this new counting algorithm generally counts less lines than before, and therefore less methods and classes cross the treshold of our "method too long" or "class too long" violations.

We'll be tweaking our serialization algorithm to better match PSR-2 in the future. And rest assured that when Insight count the lines in your code, it does it in a fair way.

SensioLabsInsight is a quality assurance tool. We've put a lot of thinking into this, to determine when SensioLabsInsight should alert you, and what feedback it should provide. You can already configure a SensioLabsInsight analysis after every commit, but that may bring up too much noise. While not enough notifications implies missed opportunities to detect and solve a major issue, too many notifications may result in ignored alerts.

Today, we're introducing a new major feature that we think you're going to love: Monthly Reports. Once a month, SensioLabsInsight analyzes all the projects you've registered, compares the result with the previous monthly analysis, and summarizes the differences.

Once a month, you will receive a single email from SensioLabsInsight for all your projects, containing a bird's eye view of the software quality evolution over the past month. It shows the evolution of the technical debt per project, the lost and earned medals, the number of new and fixed violations. We've designed this email to go right to the point; it provides just the information you need. Here is an example of how it looks:

Monthly analyses are triggered automatically, once for every user and for every project. These analyses appear with a special ribbon in your dashboard, so don't be surprised if you see analyses that you didn't trigger yourself:

Even without actively controlling your projects on SensioLabsInsight, you will be able to monitor the quality evolution, and make the right decision at the right time. Incidentally, since we keep on adding new rules every week, that means that you will be alerted if one of your projects has new violations due to these new rules, even if the code doesn't change.

We are rolling out the feature for every user progressively, so check your inbox for a new SensioLabsInsight email this month. If you're eager to see what it looks like, head for the Public Analyses section and check already executed monthly analyses. And don't hesitate to give us your feedback about the Monthly Report feature!

Disabled Twig escaping

Disabling Twig escaping for an entire application is possible, but it's strongly discouraged. From now on, any Symfony application with such a huge XSS vulnerability in config.yml will raise a critical security violation.

twig:
    # ...
    autoescape: false

Use of raw filter or autoescape off in Twig templates

Another possible XSS vulnerability comes from abuse of the |raw filter. Whenever the data piped to this filter comes from a user input, a Cross-Site Scripting attack is possible. Therefore SensioLabsInsight raises a critial security violation each time |raw is used, or when a variable is output inside a {% autoescape off %} block.

{{ user.header|raw }}

{% autoescape off %}
{{ user.header }}
{% endautoescape %}

Injection of the request service for Symfony 2.4 applications

The soon-to-be-released Symfony 2.4 deprecates the injection of the request service in favor of the request_stack service, mush more adapted to sub-requests (read more about it in the Symfony blog).

services:
  foo:
    class: Foo\Bar
    calls:
      - [setRequest, ['@request=']]

SensioLabsInsight can now detect, when you use Symfony 2.4, every occurrence of a request service injection. This will ease your upgrading process and give you no excuse to keep on using the flawed request service.

Unreachable code

Do you sometimes move a return statement up during a debugging session, and forget to remove it before committing? Yeah, that happens a lot. SensioLabsInsight can now detect unreachable code, blocked behind a return, a die(), a throw, a continue, or an exit. It's often a simple mistake, but it may cause a serious bug nevertheless.

<?php
class Foo
{
  public function computeStuff($value)
  {
    // added for debugging
    return;

    if ($value > 0) {
      // actual stuff computing
    }
  }
}

Of course, SensioLabsInsight won't bug you with unreachable code used for consistency's sake, such as the break statements in the following snippet:

<?php
switch ($value) {
  case 'foo':
    return 0;
    break;
  case 'bar':
    return 1;
    break;
  case default:
    return null;    
}

PHP Syntax Error

One of the first rule we developed was actually disabled because of a forgotten return (that happens to everybody). Using Insight on Insight helped us track this unreachable code and reenable the rule.

This rule is very simple: it uses the php command to check the syntax of all committed PHP files in your projects. That way you can spot a syntax error even before you execute that code!

<?php
// can you spot the syntax error in the next line?
echo 'hello, world!";

False positives, tweaks and bugfixes

As always, we keep on tuning these rules for false positives. Don't hesitate to mention us any false positives on the new rules using the "Feedback" button!

This week, we've also removed false violations on the use of the $_GET superglobal, which should obviously be allowed on applications not using Symfony. We've also changed the "Commented out code" severity to minor, so that you can get a silver medal even with one or two lines of PHP code commented. And we've fixed a lot of small issues to make your quality analysis experience even better.

And when computing the violation diff, SensioLabsInsight smartly guesses the "previous" analysis, even if you are using sha1 references instead of branch names like 1.0.

Those new features are very useful if you want to continuously check the evolution of a project quality as it lets you run analyses on all branches of a project and even for pull requests in an independent way. The branch names for the analyzed commit are displayed on the analysis page:

And triggering an analysis automatically every time you push some code is really easy thanks to our API and our PHP SDK.

If you are using Github, go the Account page, and under the "API/SDK" tab, there is a special section about Github at the bottom. Copy the webhook URL and add it as a "Services Hook" on the Github settings page for all projects you want to continuously analyze with SensioLabsInsight:

If you are using Jenkins or any other continuous integration platform, you can trigger an analysis by using the following URL pattern: https://XXX:YYY@insight.sensiolabs.com/api/projects/ZZZ/analyses.

where XXX is your user id, YYY is your API token, and ZZZ is the project id. How do you know the values for a specific project? Again, the API page gives you everything you need to get started fast. Select the project you want to analyze in the select box, and copy/paste the URL it gives you below:

You can then POST on this URL via Curl:

curl -X POST -H "Accept: application/vnd.com.sensiolabs.insight+xml" "https://XXX:YYY@insight.sensiolabs.com/api/projects/ZZZ/analyses"

Happy quality!

{
    "require": {
        "ext-mongo": ">=1.2.12,<1.6-dev"
    }
}

When running an analysis on a project, the rules that are activated depends on a number of different factors like the type of your project and the ability to boot the Kernel (at least for a Symfony2 application.) And booting a project is only possible after running the composer install command.

So, if MongoDB is not available or if the MongoDB PHP extension is not installed on the machine used to run the analysis, Composer will fail and a large number of interesting rules will be automatically disabled.

Good news is that enabling MongoDB is as simple as adding this snippet of code in your project configuration:

pre_composer_script: |
    yes '' | pecl install mongo

php_ini: |
    extension=mongo.so

Of course, this snippet can be used if you need any other C extension that is not available by default.

For more tips about SensioLabsInsight, have a look at our FAQ.

False Positives

Most of your feedback concerned rules showing false positives on your projects. This was particularly frustrating as it usually prevented you to get a good medal without ignoring a rule (which turns the medal into a chocolate medal). Static analysis is a complex task, and we needed these false positives that you pointed us to refine our analyzer. We've fixed most of the false positives you reported, including:

• Reports of unused variable, or unused method parameter that were actually used
• Incorrect unused use statement violations (the alias being used in a docbloc, or in a string)
• Data found in configuration that were considered sensitive by mistake
• Wrong violations on unfixed composer dependency
• Violation on logging verbosity in production that were not considering complex loggers
• And many more uncommon use cases

We'll continue to refine our analyzer to be sure to raise only violations that make sense in the context of your projects.

New Rules

We have given a lot of love to existing rules (including adding documentation to those lacking it). But most important, we have added several new rules that we think will be a great help in your development process.

• Detect missing use statements (and wrong usage of throw new Exception without a leading slash)
• Detect function calls in for loops (bad for performance)
• Check that Doctrine mapping data is valid
• Detect TODO and FIXME comments
• Detect PHP code in public repository (for instance by plugins like ckeditor)
• Detect unfixed dependency version in composer
• Detect out of maintenance dependencies
• Detect injections of the Doctrine EntityManager and of the Dependency Injection Container

These rules are automatically enabled for all future analyses. If you analyze a project again, don't be surprised that new violations appear: that's because we keep on adding new rules every week!

Usability improvements

A large share of your feedback concerned the overall usability of the web interface. You reported many small glitches that were fixed quickly and continuously. If you haven't been on SensioLabsInsight for a while, you might be surprised by how much the interface has changed since you first came. Usability highlights include:

• The dashboard is now the homepage for connected users
• In the full report, the "new" tab is now a filter on the sidebar. That allows you to find the same violation count on the summary and on the full report
• The number of new and fixed violations now appear for each analysis in the dashboard
• You can easily navigate to the next and previous analysis in a full report
• Some hard to understand and badly worded sentences were rewritten to be understood by any English speaker
• An FAQ to answer most common questions
• The project configuration page was entirely redesigned to show only basic configuration at first
• You can now ignore all the violations of a given rule, and disable this rule for future analyses

We know we have a lot more to do, and we'll keep on iterating on the design and interactions. Come back regularly to see our progress.

New features

Lastly, we've been hard at work adding the features that you missed the most. You can now:

• Turn the dashboard into a timeline of events, showing not only analyses, but also new violations, projects you now collaborate to...
• Analyze a private repository (either by pushing it to a SensioLabs Git repository, or by sharing your public key)
• Display information about the version of the application analyzed (because the date was obviously not enough...)
• Run an analysis on a branch
• Add a search engine of public project analyses
• Display and filter violation by author
• Chart the quality evolution over time
• Receive email notifications
• Export a quality widget for your project

And under the hood, SensioLabsInsight has been optimized and stabilized to make it ready for production. This was made possible by all the tests you've been doing on the platform, so again thanks a million times to the beta testers.

Keep on analyzing and sending us feedback, together we'll build an awesome product!

First of all, since we're in the beta stage, the product is far from perfect. However, we value feedback, as early as possible. So please share your experience with us using the "feedback" button present on every page - whether for bugs, missing features, UI problems, and also of course if you think that the product fits your needs.

If you read this, it means you have been granted an account with five free private projects and unlimited Open-Source projects. When creating a project, only check the "Open-Source" checkbox if you want other people to get access to your analyses. If you need more than five private projects, contact us to explain your case and we will try to get you more. Also, choose the type of your project carefully as our analysis engine behaves differently based on this setting.

The product evolves very fast as we deploy a new version every day. So don't hesitate to come back often and re-analyse your projects, even if you did not made big changes to your code. You will notice that SensioLAbs Insight helps not only assessing the current state of your code, but also its evolution trends.

That's it! Go to insight.sensiolabs.com, connect with your SensioLabsConnect account, and analyse your first project! We are thrilled to have you be part of our private beta and we are looking forward to receiving your feedback on our work.