Test with the new <a href="https://www.raspberrypi.org/forums/viewtopic.php?f=117&

Upstream issue filed: <a class="issue-link js-issue-link" data-error-text="Failed to l

I'd also like to run an informal <a href="https://www.phoronix-test-suite.com/?k=downl

One interesting difference: <div class="snippet-clipboard-content notranslate posi

Phoronix: <div class="snippet-clipboard-content notranslate position-relative over

Test Performance and Functionality on Raspberry Pi OS 64-bit,about geerlingguy/drupal-pi

Comments (21)

geerlingguy commented on June 12, 2024 1

Well now, I just realized I want to have one more benchmark to highlight the vast chasm between a Pi's performance and a modern super-fast x86-64 processor like the i9 in my laptop, along with way faster memory (32GB of it), and a zillion-times-faster disk.

Therefore:

Comparison to AWS t3.small and macOS i9 2.4

Benchmark	32-bit Pi	64-bit Pi	AWS t3.small	i9 2.4 GHz Mac
Install Drupal	100.99 s	102.65 s	18.22 s	16.59 s
First page load	5.42 s	4.43 s	1.25 s	0.97 s
`ab` auth test	28.01 req/s	25.76 req/s	36.38 req/s	122.53 req/s
`wrk` anon test	104.91 req/s	100.26 req/s	293.13 req/s	340.64 req/s

For EC2, I installed docker following these directions on Amazon Linux 2. I tried on a free tier t2.micro instance, but Composer commands need at least 1.2 GB of memory to run at all.

Comparison between Pi model 4 versions

32-bit Raspberry Pi OS

Benchmark	1 GB	2 GB	4 GB	8 GB
Install Drupal	105.17 s	100.28 s	102.19 s	100.99 s
First page load	5.44 s	5.06 s	4.78 s	5.42 s
`ab` auth test	28.10 req/s	27.30 req/s	28.01 req/s	28.01 req/s
`wrk` anon test	103.52 req/s	101.22 req/s	106.12 req/s	104.91 req/s

64-bit Raspberry Pi OS Beta

Benchmark	1 GB	2 GB	4 GB	8 GB
Install Drupal	105.33 s	105.87 s	TODO s	102.65 s
First page load	5.39 s	4.31 s	TODO s	4.43 s
`ab` auth test	25.88 req/s	25.56 req/s	TODO req/s	25.76 req/s
`wrk` anon test	95.77 req/s	97.99 req/s	TODO req/s	100.26 req/s

Compute Module 3+

Benchmark	32-bit Pi	64-bit Pi
Install Drupal	166.62 s	184.05 s
First page load	7.63 s	8.65 s
`ab` auth test	16.98 req/s	13.65 req/s
`wrk` anon test	51.07 req/s	40.77 req/s

from drupal-pi.

geerlingguy commented on June 12, 2024

Upstream issue filed: geerlingguy/ansible-role-docker_arm#10

from drupal-pi.

geerlingguy commented on June 12, 2024

~~Testing again with ansible-playbook -i inventory -c local main.yml -e "docker-version=5:19.03.9~3-0~debian-buster"~~

That didn't work, because it's docker_version. Oops! Anyways, better to override in a config.yml anyways...

from drupal-pi.

geerlingguy commented on June 12, 2024

I'd also like to run an informal https://www.phoronix-test-suite.com/?k=downloads to see if there's a notable difference between the armv7l and arm64 version.

from drupal-pi.

geerlingguy commented on June 12, 2024

One interesting difference:

$ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
HYPRIOT_DEVICE="Raspberry Pi"
HYPRIOT_IMAGE_VERSION="v1.12.0"

# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

from drupal-pi.

geerlingguy commented on June 12, 2024

The mysql arm image I'm using from Hypriot hasn't been updated in two years, and isn't compatible with arm64... so maybe consider one of the following for mysql_docker_image:

# If this works, it could be used for both AMD64 and ARM64.
mysql_docker_image: mysql/mysql-server:8.0

# This would be more compatible with MySQL 5.6/5.7...
mysql_docker_image: arm64v8/mariadb:10.4

Note that making the change as a default would break any existing users if they are not on Raspberry Pi OS 64-bit and/or have an older Pi that can't run it.

from drupal-pi.

geerlingguy commented on June 12, 2024

Phoronix:

sudo apt-get install -y php7.3-cli php7.3-xml
wget https://phoronix-test-suite.com/releases/phoronix-test-suite-9.6.1.tar.gz
tar -xvf phoronix-test-suite-9.6.1.tar.gz
cd phoronix-test-suite
./phoronix-test-suite system-info # make sure it's working, agree to terms
./phoronix-test-suite list-recommended-tests # see what tests are recommended

# Run a few benchmarks (run as root to avoid prompts; don't do this on an instance you care about, it'll install a lot of stuff!)
export FORCE_TIMES_TO_RUN=3
./phoronix-test-suite benchmark pts/john-the-ripper
./phoronix-test-suite benchmark pts/compress-7zip
./phoronix-test-suite benchmark pts/iperf # requires separate server: `iperf -s`
./phoronix-test-suite benchmark pts/ramspeed

from drupal-pi.

geerlingguy commented on June 12, 2024

Pi 4, 64-bit

(Note: Long-term test runs with a bare Pi (no fan, no cooling heatsink) resulted in the Pi CPU overheating (vcgencmd measure_temp) and throttling (vcgencmd get_throttled) quite a bit. I had to put the Pi in my modified Pi Case with Fan and it was much more consistent.

john-the-ripper

Test: Blowfish
Average: 1223 Real C/S
Deviation: 3.50%
Samples: 15

Test: MD5
Average: 32790 Real C/S
Deviation: 0.16%
Samples: 5

iperf

Note: Done manually iperf -s on Mac, iperf -c [ip] on Pi, phoronix was erroring out.

917 Mbits/sec

(Gave up after a few hours; Phoronix suites were having some issues (lots of results had high variance or were erroring out), so I'm going to use other benchmarking for simplicity's sake.)

from drupal-pi.

geerlingguy commented on June 12, 2024

Pi 4 64-bit

All benchmarks run three times, averaged result in table.

Benchmark	Result
`iperf`	917 Mbits/sec
`stress-ng`	1552 bogo ops/s

# stress-ng --cpu 4 --cpu-method fft --cpu-ops 50000 --metrics-brief
stress-ng: info:  [13789] defaulting to a 86400 second (1 day, 0.00 secs) run per stressor
stress-ng: info:  [13789] dispatching hogs: 4 cpu
stress-ng: info:  [13789] cache allocate: using defaults, can't determine cache details from sysfs
stress-ng: info:  [13789] successful run completed in 32.50s
stress-ng: info:  [13789] stressor       bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info:  [13789]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [13789] cpu               50000     32.44    129.16      0.00      1541.21       387.12

Pi 4 32-bit

Benchmark	Result
`iperf`	937 Mbits/sec
`stress-ng`	1377 bogo ops/s

# stress-ng --cpu 4 --cpu-method fft --cpu-ops 50000 --metrics-brief
stress-ng: info:  [980] defaulting to a 86400 second (1 day, 0.00 secs) run per stressor
stress-ng: info:  [980] dispatching hogs: 4 cpu
stress-ng: info:  [980] cache allocate: using defaults, can't determine cache details from sysfs
stress-ng: info:  [980] successful run completed in 36.53s
stress-ng: info:  [980] stressor       bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info:  [980]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [980] cpu               50000     36.34    145.26      0.00      1376.07       344.21

from drupal-pi.

geerlingguy commented on June 12, 2024

For comparison on my Mac with it's i9, stress-ng (slightly different test, processor, OS, etc.) gives 209,486 (compared to ~50,000 for a float on the Pi). So a tad bit faster :P

from drupal-pi.

geerlingguy commented on June 12, 2024

Testing with https://gist.github.com/geerlingguy/570e13f4f81a40a5395688667b1f79af now.

Just noting—when I run the above script I currently get a prompt Would you like to save these test results (Y/n): n... need to either add an expect/echo, or figure out some other way to tell phoronix to stop asking so many darn questions!

from drupal-pi.

geerlingguy commented on June 12, 2024

Raspberry Pi 4 - 8GB

Benchmark	32-bit	64-bit	% difference
x264 2019-12-17	5.30 fps	5.69 fps	7.10%
LAME MP3 Encoding 3.100	41.42 s	29.33 s	34.18%
PHPBench 0.8.1	102343	110403	7.58%

Pi OS 64-bit

    x264 2019-12-17
    H.264 Video Encoding
    Frames Per Second > Higher Is Better
    pi-arm64-2 . 5.69

    LAME MP3 Encoding 3.100
    WAV To MP3
    Seconds < Lower Is Better
    pi-arm64-2 . 29.33 

    PHPBench 0.8.1
    PHP Benchmark Suite
    Score > Higher Is Better
    pi-arm64-2 . 110403

Pi OS 32-bit

    x264 2019-12-17
    H.264 Video Encoding
    Frames Per Second > Higher Is Better
    pi-arm . 5.30 

    LAME MP3 Encoding 3.100
    WAV To MP3
    Seconds < Lower Is Better
    pi-arm . 41.42

    PHPBench 0.8.1
    PHP Benchmark Suite
    Score > Higher Is Better
    pi-arm . 102343

from drupal-pi.

geerlingguy commented on June 12, 2024

Also linking to (currently private repo): geerlingguy/turing-pi-cluster#11

from drupal-pi.

geerlingguy commented on June 12, 2024

Raspberry Pi Compute Module 3+

Benchmark	32-bit	64-bit	% difference
x264 2019-12-17	2.17 fps	2.34 fps	7.54%
LAME MP3 Encoding 3.100	80.74 s	64.36 s	22.58%
PHPBench 0.8.1	41496	48684	15.94%

Pi OS 64-bit

    x264 2019-12-17
    H.264 Video Encoding
    Frames Per Second > Higher Is Better
    pi-cm-arm64 . 2.34 

    LAME MP3 Encoding 3.100
    WAV To MP3
    Seconds < Lower Is Better
    pi-cm-arm64 . 64.36 

    PHPBench 0.8.1
    PHP Benchmark Suite
    Score > Higher Is Better
    pi-cm-arm64 . 48684

Pi OS 32-bit

    x264 2019-12-17
    H.264 Video Encoding
    Frames Per Second > Higher Is Better
    rpi-arm32-cm . 2.17 

    LAME MP3 Encoding 3.100
    WAV To MP3
    Seconds < Lower Is Better
    rpi-arm32-cm . 80.74 

    PHPBench 0.8.1
    PHP Benchmark Suite
    Score > Higher Is Better
    rpi-arm32-cm . 41496

from drupal-pi.

geerlingguy commented on June 12, 2024

The results speak for themselves—64-bits is much faster for pure CPU-based operations. But in the real world, things are a bit more murky.

I will still need to get Drupal Pi running on a standalone Pi 4 8GB to do some final benchmarks there with ab/wrk, but at least in the Pi Clusters I'm operating, Drupal is actually slower when memory-constrained running on 64-bit because it can't handle as many threads under Apache due to memory pointers being larger!

If we can allocate 4+ GB containers (e.g. on new Pi 4 8GB models) then I think the picture will be much different. Anyways, onward to the next benchmarks!

from drupal-pi.

geerlingguy commented on June 12, 2024

I'm going to do a few tests with Drupal Pi on the 8 GB vs 4 GB vs 2 GB Pi 4. Since there's no constraint on memory/CPU it should just consume as much as it can. I might need to tweak Apache settings to let it create more threads for requests, though.

Drupal Performance Tests (drupal-pi OOTB config)

Benchmark	32-bit	64-bit	% difference
Install Drupal	99.54 s	74.00 s	29.43%
First page load	4.98 s	3.77 s	27.65%
`ab` auth test	27.60 req/s	31.90 req/s	14.45%
`wrk` anon test	104.51 req/s	110.80 req/s	5.84%

Note: I was so surprised at these results that I ran them all again, a few more times, and it turns out I must've had an anomaly or some unaccounted-for variable in the tests. In reality the differences are much closer, and in all cases except first page load, the 32-bit OS version wins by a small margin—see later results in this thread. I'm leaving these results for historical purposes.

Reminder: These tests were run on the exact same Pi, using the exact same configuration and Docker images, using the exact same 32GB microSD card, the only difference was running Raspberry Pi OS 32-bit 2020-05-27 versus 64-bit 2020-05-27 beta.

That's... a pretty hefty speedup when you have a ton of RAM to play with! Note that these tests used 1-2 GB of RAM at peak, while system load jumped to 25.00+, but having the greater memory overhead and 64-bit OS made a huge difference:

The actual tests I ran (for completeness)

# Install Drupal.
docker-compose exec drupal bash -c 'composer require drush/drush'
time docker-compose exec drupal bash -c 'vendor/bin/drush site:install standard --db-url="mysql://drupal:$DRUPAL_DATABASE_PASSWORD@$DRUPAL_DATABASE_HOST/drupal" --site-name="Drupal Pi Test" -y'

# First page load.
time curl http://localhost/

# `ab` auth test
ab -n 100 -c 10 -C "SESSxyz=XYZ" http://10.0.100.52/

# `wrk` anon test
wrk -t4 -c100 -d30 http://10.0.100.52/

from drupal-pi.

geerlingguy commented on June 12, 2024

Well that's annoying; with the linuxserver/mariadb:latest image I am getting:

The database server version 10.1.44-MariaDB-0ubuntu0.18.04.1 is less than the minimum required version 10.3.7.

The CLI install using drush outputs a deceptively misleading message:

# time docker-compose exec drupal bash -c 'vendor/bin/drush site:install standard --db-url="mysql://drupal:$DRUPAL_DATABASE_PASSWORD@$DRUPAL_DATABASE_HOST/drupal" --site-name="Drupal Pi Test" -y'

 // You are about to DROP all tables in your 'drupal' database. Do you want to continue?: yes.                          

 [notice] Starting Drupal installation. This takes a while.

In install.core.inc line 2298:
                       
  Database settings:   
                       
  Array

I'm guessing that Array contains one item, and that item is the message from the UI installer above.

from drupal-pi.

geerlingguy commented on June 12, 2024

Drat: linuxserver/docker-mariadb#47 (comment)

from drupal-pi.

geerlingguy commented on June 12, 2024

Now testing with https://hub.docker.com/r/webhippie/mariadb — seems like it also supports both arches, as well as amd64. Might be a winner.

from drupal-pi.

geerlingguy commented on June 12, 2024

I think I have, at this point, beat this issue into a pulp, and have also made the project cross-compatible with any 32-bit or 64-bit ARM OS now.

Therefore, I shall close it, and then take the data and put it into a new blog post and video, coming soon™.

from drupal-pi.

geerlingguy commented on June 12, 2024

Another strange thing I ran into while testing CM3+:

# docker-compose exec drupal bash -c 'composer require drush/drush'

                                                                                                                       
  [Composer\Downloader\TransportException]                                                                             
  The "https://packages.drupal.org/8/packages.json" file could not be downloaded: php_network_getaddresses: getaddrin  
  fo failed: Temporary failure in name resolution                                                                      
  failed to open stream: php_network_getaddresses: getaddrinfo failed: Temporary failure in name resolution

I could curl that URL with no problem. A reboot of the Pi seemed to fix the problem.

Also, for the wrk load test, the 3+ couldn't cope with 4 threads and 100 connections; this killed the Pi. I had to back it off to 2 threads and 2 connections to get it to work without toppling the Pi and sending system load into the 50s/100s.

from drupal-pi.

Comments (21)

Comparison to AWS t3.small and macOS i9 2.4

Comparison between Pi model 4 versions

32-bit Raspberry Pi OS

64-bit Raspberry Pi OS Beta

Compute Module 3+

Pi 4, 64-bit

john-the-ripper

iperf

Pi 4 64-bit

Pi 4 32-bit

Raspberry Pi 4 - 8GB

Pi OS 64-bit

Pi OS 32-bit

Raspberry Pi Compute Module 3+

Pi OS 64-bit

Pi OS 32-bit

Drupal Performance Tests (drupal-pi OOTB config)

The actual tests I ran (for completeness)

Related Issues (20)

Recommend Projects

Recommend Topics

Recommend Org