tl;dr: Always test your own application, and trust, but verify every benchmark you see. PHP 7 is actually faster than HHVM in many cases, neck-in-neck in others, and slightly slower in others. Both PHP 7 and HHVM blow PHP ≤ 5.6 out of the water.
Skip to benchmark results:
- Drupal 8 Results (concurrency 10)
- Drupal 8 Results (concurrency 1)
- Drupal 8 Results (‘bare metal’, concurrency 1)
- Wordpress 4.4 Results (‘bare metal’, concurrency 1)
- Drupal 7 Results (concurrency 10)
Introduction and Methodology
As PHP 7 became a reality through this past year, there were scores of benchmarks pitting PHP 7 against 5.6 and HHVM using applications and frameworks like Drupal, Wordpress, Joomla, Laravel, October, etc.
One benchmark that really stood out to me (in that it seemed so wrong for Drupal, based on my experience) was The Definitive PHP 7.0 & HHVM Benchmark from Kinsta. Naming a benchmark that way certainly makes the general PHP populace take it seriously!
The results are pretty damning for PHP 7:
In the comments on that post, Thomas Svenson mentioned:
Standard installation for Drupal 8 has cache on as default. If you did not turn that off, then it is probably a reason to why the PHP 7 boost isn’t bigger.
Would be interesting to see the result comparing the benchmark with/without caching enabled in Drupal 8. Should potentially reveal something interesting.
This was my main concern too, as there wasn’t enough detail in the benchmarking article to determine what exactly was the system under test. Therefore, I’ll submit my own PHP 7 vs HHVM benchmark here, using the following versions:
- Ubuntu 14.04
- Drupal 8.0.1
- Nginx 1.4.6
- MySQL 5.5.46
- PHP 5.6.16, PHP 7.0.1, or HHVM 3.11.0
All tests were run using Drupal VM version 2.1.2 with VMware Fusion 8.1.0, on my mid-2013 MacBook Air 13″ 1.7 GHz i7 with 8GB of RAM. Using the above notes, you can exactly replicate this benchmarking environment should you desire. All tests were run five times, the first two results were discarded (because they often reflect times when some caches are still warming), and the latter three were averaged.
After installing Drupal 8.0.1 with the standard installation profile (this is done automatically by Drupal VM), I logged in as the admin user (user 1), then grabbed the admin user’s session cookie, and ran the following two commands:
<br />
# Benchmark Drupal 8 home page out of the box with default caching options enabled.<br />
ab -n 750 -c 10 http://drupalvm.dev/
# Benchmark Drupal 8 /admin page logged in as user 1.
ab -n 750 -c 10 -C “SESSxyz=value” http://drupalvm.dev/admin
Drupal 8 results (concurrency 10)
| Environment | D8 Caching | Requests/second | Percent difference |
|---|---|---|---|
| PHP 5.6.16 | Enabled (home, anonymous) | 214.39 req/s | ~ |
| PHP 7.0.1 | Enabled (home, anonymous) | 407.10 req/s | 62% faster than 5.6 |
| HHVM 3.11.0 | Enabled (home, anonymous) | 260.19 req/s | 19% faster than 5.6 |
| PHP 5.6.16 | Bypassed (/admin, user 1) | 20.09 req/s | ~ |
| PHP 7.0.1 | Bypassed (/admin, user 1) | 39.26 req/s | 65% faster than 5.6 |
| HHVM 3.11.0 | Bypassed (/admin, user 1) | 34.41 req/s | 53% faster than 5.6 |
…and some graphs of the above data:
PHP 5.6, PHP 7, and HHVM running Drupal 8.0.1, cached
PHP 5.6, PHP 7, and HHVM running Drupal 8.0.1, uncached
Drupal 8 results (concurrency 1)
Sometimes, the use of concurrency (-c 10 in the above case)(to simulate concurrent users hitting the site at the same time, can cause benchmarks to be slightly inaccurate. The reason I usually use a level of concurrency is so the benchmark more closely mirrors real-world usage, and tests the full stack a little better (because PHP by itself is nice to benchmark, but very few sites are run on top of PHP alone!).
Learn More 4501
Learn More 4502
Learn More 4503
Learn More 4504
Learn More 4505
Learn More 4506
Learn More 4507
Learn More 4508
Learn More 4509
Learn More 4510
Learn More 4511
Learn More 4512
Learn More 4513
Learn More 4514
Learn More 4515
Learn More 4516
Learn More 4517
Learn More 4518
Learn More 4519
Learn More 4520
Learn More 4521
Learn More 4522
Learn More 4523
Learn More 4524
Learn More 4525
Learn More 4526
Learn More 4527
Learn More 4528
Learn More 4529
Learn More 4530
Learn More 4531
Learn More 4532
Learn More 4533
Learn More 4534
Learn More 4535
Learn More 4536
Learn More 4537
Learn More 4538
Learn More 4539
Learn More 4540
Learn More 4541
Learn More 4542
Learn More 4543
Learn More 4544
Learn More 4545
Learn More 4546
Learn More 4547
Learn More 4548
Learn More 4549
Learn More 4550
Learn More 4551
Learn More 4552
Learn More 4553
Learn More 4554
Learn More 4555
Learn More 4556
Learn More 4557
Learn More 4558
Learn More 4559
Learn More 4560
Learn More 4561
Learn More 4562
Learn More 4563
Learn More 4564
Learn More 4565
Learn More 4566
Learn More 4567
Learn More 4568
Learn More 4569
Learn More 4570
Learn More 4571
Learn More 4572
Learn More 4573
Learn More 4574
Learn More 4575
Learn More 4576
Learn More 4577
Learn More 4578
Learn More 4579
Learn More 4580
Learn More 4581
Learn More 4582
Learn More 4583
Learn More 4584
Learn More 4585
Learn More 4586
Learn More 4587
Learn More 4588
Learn More 4589
Learn More 4590
Learn More 4591
Learn More 4592
Learn More 4593
Learn More 4594
Learn More 4595
Learn More 4596
Learn More 4597
Learn More 4598
Learn More 4599
Learn More 4600
Learn More 4601
Learn More 4602
Learn More 4603
Learn More 4604
Learn More 4605
Learn More 4606
Learn More 4607
Learn More 4608
Learn More 4609
Learn More 4610
Learn More 4611
Learn More 4612
Learn More 4613
Learn More 4614
Learn More 4615
Learn More 4616
Learn More 4617
Learn More 4618
Learn More 4619
Learn More 4620
Learn More 4621
Learn More 4622
Learn More 4623
Learn More 4624
Learn More 4625
Learn More 4626
Learn More 4627
Learn More 4628
Learn More 4629
Learn More 4630
Learn More 4631
Learn More 4632
Learn More 4633
Learn More 4634
Learn More 4635
Learn More 4636
Learn More 4637
Learn More 4638
Learn More 4639
Learn More 4640
Learn More 4641
Learn More 4642
Learn More 4643
Learn More 4644
Learn More 4645
Learn More 4646
Learn More 4647
Learn More 4648
Learn More 4649
Learn More 4650
Learn More 4651
Learn More 4652
Learn More 4653
Learn More 4654
Learn More 4655
Learn More 4656
Learn More 4657
Learn More 4658
Learn More 4659
Learn More 4660
Learn More 4661
Learn More 4662
Learn More 4663
Learn More 4664
Learn More 4665
Learn More 4666
Learn More 4667
Learn More 4668
Learn More 4669
Learn More 4670
Learn More 4671
Learn More 4672
Learn More 4673
Learn More 4674
Learn More 4675
Learn More 4676
Learn More 4677
Learn More 4678
Learn More 4679
Learn More 4680
Learn More 4681
Learn More 4682
Learn More 4683
Learn More 4684
Learn More 4685
Learn More 4686
Learn More 4687
Learn More 4688
Learn More 4689
Learn More 4690
Learn More 4691
Learn More 4692
Learn More 4693
Learn More 4694
Learn More 4695
Learn More 4696
Learn More 4697
Learn More 4698
Learn More 4699
Learn More 4700
Learn More 4701
Learn More 4702
Learn More 4703
Learn More 4704
Learn More 4705
Learn More 4706
Learn More 4707
Learn More 4708
Learn More 4709
Learn More 4710
Learn More 4711
Learn More 4712
Learn More 4713
Learn More 4714
Learn More 4715
Learn More 4716
Learn More 4717
Learn More 4718
Learn More 4719
Learn More 4720
Learn More 4721
Learn More 4722
Learn More 4723
Learn More 4724
Learn More 4725
Learn More 4726
Learn More 4727
Learn More 4728
Learn More 4729
Learn More 4730
Learn More 4731
Learn More 4732
Learn More 4733
Learn More 4734
Learn More 4735
Learn More 4736
Learn More 4737
Learn More 4738
Learn More 4739
Learn More 4740
Learn More 4741
Learn More 4742
Learn More 4743
Learn More 4744
Learn More 4745
Learn More 4746
Learn More 4747
Learn More 4748
Learn More 4749
Learn More 4750
Learn More 4751
Learn More 4752
Learn More 4753
Learn More 4754
Learn More 4755
Learn More 4756
Learn More 4757
Learn More 4758
Learn More 4759
Learn More 4760
Learn More 4761
Learn More 4762
Learn More 4763
Learn More 4764
Learn More 4765
Learn More 4766
Learn More 4767
Learn More 4768
Learn More 4769
Learn More 4770
Learn More 4771
Learn More 4772
Learn More 4773
Learn More 4774
Learn More 4775
Learn More 4776
Learn More 4777
Learn More 4778
Learn More 4779
Learn More 4780
Learn More 4781
Learn More 4782
Learn More 4783
Learn More 4784
Learn More 4785
Learn More 4786
Learn More 4787
Learn More 4788
Learn More 4789
Learn More 4790
Learn More 4791
Learn More 4792
Learn More 4793
Learn More 4794
Learn More 4795
Learn More 4796
Learn More 4797
Learn More 4798
Learn More 4799
Learn More 4800
Learn More 4801
Learn More 4802
Learn More 4803
Learn More 4804
Learn More 4805
Learn More 4806
Learn More 4807
Learn More 4808
Learn More 4809
Learn More 4810
Learn More 4811
Learn More 4812
Learn More 4813
Learn More 4814
Learn More 4815
Learn More 4816
Learn More 4817
Learn More 4818
Learn More 4819
Learn More 4820
Learn More 4821
Learn More 4822
Learn More 4823
Learn More 4824
Learn More 4825
Learn More 4826
Learn More 4827
Learn More 4828
Learn More 4829
Learn More 4830
Learn More 4831
Learn More 4832
Learn More 4833
Learn More 4834
Learn More 4835
Learn More 4836
Learn More 4837
Learn More 4838
Learn More 4839
Learn More 4840
Learn More 4841
Learn More 4842
Learn More 4843
Learn More 4844
Learn More 4845
Learn More 4846
Learn More 4847
Learn More 4848
Learn More 4849
Learn More 4850
Learn More 4851
Learn More 4852
Learn More 4853
Learn More 4854
Learn More 4855
Learn More 4856
Learn More 4857
Learn More 4858
Learn More 4859
Learn More 4860
Learn More 4861
Learn More 4862
Learn More 4863
Learn More 4864
Learn More 4865
Learn More 4866
Learn More 4867
Learn More 4868
Learn More 4869
Learn More 4870
