Google will no longer support the Octane browser benchmark after the company determined that the test was actually becoming detrimental to web performance.
"Using Octane to measure V8 performance didn’t capture important use cases for the modern web, such as loading frameworks quickly, supporting large applications with new patterns of state management, or ensuring that ES2015+ features are as fast as their ES5 equivalents," Google writes in the blog post .
Cheating for a better score
It also reached a point where developers were cheating to get higher scores on the test. This, Google says, resulted in real-world performance issues. One instance, the company points out, developers were taking advantage of a bug that gave them a 15% performance bump in Octane. It did nothing on the actual web.
In balance, the test penalized some optimizations that were actually good for real-world performance, optimizations that were discovered in the past few years and that Octane didn't take into account.
For these and many other reasons, Google is ending support for Octane. "Unfortunately, similar issues exist in other static or synthetic benchmarks," Google writes, issuing a sort of warning to developers against using such tools to mark their work.