Chromium Blog
News and developments from the open source browser project
Updating JavaScript Benchmarks for Modern Browsers
Wednesday, May 4, 2011
Benchmarks are incredibly important for influencing the direction of JavaScript engines. Over the past two years, JavaScript has gotten dramatically faster across all major browsers, particularly in areas that popular benchmarks measure. As it turns out, browser developers are a competitive bunch. However, as JS engines get faster, benchmarks must evolve in order to keep pushing them in the right direction.
We’ve been constantly updating our
V8 benchmark suite
to force us to get faster in areas that are important to web developers. We’ve fixed bugs and added new tests for
regular expressions
and
memory management
. We’re very focused on JavaScript performance, so we scrutinize our benchmark carefully to make sure that it’s as useful a measuring stick as possible.
The two other widely cited JS benchmarks are
SunSpider
from Apple, and
Kraken
, a new benchmark from Mozilla.
SunSpider was one of the first suites of tests, first appearing in 2007. It’s done a lot to help improve JS engines, but many of the tests in the suite are less relevant to the web in 2011. Even for the more relevant tests, JavaScript has gotten so fast that many finish in only a few milliseconds. This just isn’t long enough to figure out which engine is faster--a golf cart and a Tesla will finish a 10-foot race in nearly the same time.
To make the benchmark more meaningful, we’ve experimented by making the race longer by running each of the tests in SunSpider 50 times consecutively. While repeating a trivial test many times isn’t a great solution, it does provide an opportunity for some optimization. With this change, the results begin to reflect Chrome’s true performance. It’s more than 30% faster on the SunSpider suite overall and up to 4x faster on some of the individual tests.
Kraken, a more modern benchmark, is in better shape. Unfortunately, the
canonical
version of the benchmark has not been updated to reflect all the latest changes which address at least
one important bug
. As a result, the benchmark is less useful and has even (mis)led us to spend time making some
irrelevant optimizations
in Chrome. To help us focus on the right things, we’re now testing the latest version of Kraken built directly from Mozilla’s source code repository.
We’re posting a
modified version of SunSpider
and the
latest version of Kraken
to make it easy to run the benchmarks in your own browser and compare results.
Posted by Mike Belshe, Software Engineer
Labels
accessibility
1
benchmarks
1
beta
1
blink
1
chrome apps
3
Chrome Frame
1
chrome web store
26
chromeframe
3
chromeos
3
chromium
3
cloud print
1
dart
8
devtools
11
extensions
23
gdd
1
googlechrome
12
html5
11
incognito
1
javascript
3
linux
2
mac
1
mobile
2
na
1
native client
8
New Features
5
octane
1
open web
2
releases
2
rlz
1
security
30
spdy
2
ssl
2
v8
5
web intents
1
webaudio
3
webgl
7
webkit
5
webp
5
webrtc
4
websockets
5
webtiming
1
Archive
2016
Jun
May
Apr
Mar
Feb
Jan
2015
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2014
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2013
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2012
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2011
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2010
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2009
Dec
Nov
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Feed
Google
on
Follow @ChromiumDev
Give us feedback in our
Product Forums
.
