Assertions

An assertion is an expression that must return a Boolean (true if the assertion succeeds, false otherwise). An assertion is made of metric values, variables, operators, and units:

1
2
3
"main.peak_memory < 10mb"
"metrics.sql.queries.count == 0"
"main.peak_memory < vars.max_memory"

In the above example, main.peak_memory and metrics.sql.queries.count are metric values, 10mb uses a unit to make the expression more readable, and vars.max_memory is a variable for which the value is defined in the environment.

Metric Values

Blackfire exposes metrics that are associated with the current profile. The value of one dimension of a metric can be used in assertions; for instance, the count value of the sql.queries metric is stored in metrics.sql.queries.count.

Learn more about all build-in metrics exposed in assertions by default and how to create your own.

The available dimensions for metrics are the following ones:

  • count
  • wall_time
  • cpu_time
  • memory
  • peak_memory
  • io
  • network_in
  • network_out

Variables

Variables are useful when you configure several Blackfire environments to run performance tests on various machines hosting the same application. They enable to use the same .blackfire.yml, to be deployed on several servers that have different purposes, configuration or specifications.

Variables can be defined in the environment configuration and used in an expression by prefixing the key with vars.. Variable values can also use units.

Example: You have configured two different environments in Blackfire:

  • Pre-production hits your pre-production servers, that run on PHP 7
  • Production hits your production servers, that run on PHP 5.5

The same application runs on all servers, but as PHP 7 uses a lot less memory, you cannot set the same maximum value for your assertions on memory.

Your .blackfire.yml will look like this when using environment variables:

1
2
3
4
5
tests:
    "Pages shouldn't use too much memory":
        path: "/.*"
        assertions:
            - "main.memory < 2Mb * var('memory_coefficient')"

Then, both in your Pre-production and Production Blackfire environments, you will create the memory_coefficient variable:

  • For Pre-production, memory_coefficient == 1
  • For Production, memory_coefficient == 2

The var function also takes a default value as a second argument in case the variable is not defined in all your environments.

1
2
3
4
5
tests:
    "Pages shouldn't use too much memory":
        path: "/.*"
        assertions:
            - "main.memory < 2Mb * var('memory_coefficient', 1)"

Caution

The vars.xxx notation, supported in old versions of Blackfire, is now deprecated and you should migrate your expression to use the var() function instead.

Development vs Production

When an assertion is run from an environment, the is_dev() function returns false when the environment is configured for production usage.

Operators

The following operators are supported in assertions:

  • + addition
  • - subtraction
  • * multiplication
  • / division
  • % modulus
  • ** pow
  • == equal
  • != not equal
  • < less than
  • > greater than
  • <= less than or equal to
  • >= greater than or equal to
  • not or !
  • and or &&
  • or or ||

Units

To make assertions more readable, you can add a unit to numbers. A unit is just a multiplier applied to the raw number associated with it.

The default time unit is the millisecond; when using 10 in an assertion for a time value, that evaluates to 10 milliseconds. But using a unit (like ms or s) makes assertions more explicit; 10s is interpreted as 10 seconds.

The default memory unit is the byte. The following units are also available: kb, kib, mb, mib, gb, gib.

Blackfire also supports the following generic units: k, ki, m, mi, g, gi.

Comparisons

Assertions on profile comparisons can also be written thanks to the percent and diff functions:

If you want to assert that a metric value does not increase by more than a given percent between two profiles, use the percent function:

1
"percent(main.wall_time) < 10%"

You can also test the evolution of a metric value in absolute terms:

1
"diff(metrics.sql.queries.count) < 2"

In this example, the assertion checks that the newer profile has less than 2 additional SQL statements compared to the previous one.