php opcache explained
What is php op-cache?
Php is an interpretive language. The interpreter has to read each line of php code, parse and tokenize it – i.e. convert to internal format also called opcode or operation codes. It can then interpret by “running” the opcodes. With php 8, the JIT changes it a bit.
A php project like Magento has a lot of files – so if php had to read each file everytime there is a reference to it, it will spend a lot of time reading the file from disk, parsing it and converting to opcodes. In order to speed up that process, php has opcode cache (or opcache), where the opcodes are kept in cache.
This cache is stored in memory or on disk. Using and configuring memory gives the best performance.
Opcache and JIT
JIT transforms php from a purely interpretive language to a compiling language that generates opcodes that the underlying CPU can run. Unlike pure compiling languages like “C”, php uses a Just-in-time option and generates these CPU opcodes or machine language instrutions and stores them in memory. The opcache module is responsible for this.
Opcache and php-fpm
Typically php is a single threaded application. So, each hit requires a different php process. Typically at the end of the execution, the php interpretor terminates.
php-fpm is the process manager for php. It runs keeps manages a pool of php processes and keeps track of which php process is busy. It has an ability to start and stop processes. Php-fpm communicates with a web server like nginx using the fastcgi protocol.
Php-fpm keeps a single opcache that all the processes in the pool can share.
How much memory does php opcache need?
Php opcache allocates different blocks of memory for different types of data. The amount of memory needed is dependent on the number of files in your project, the number and total size of string literals and the size of each of executable code in all your files.
JIT needs additional memory to store the machine opcodes.
Opcache configuration parameter are stored in /etc/php.d/opcache.ini (or the opcache.ini in your system).
memory (opcache.memory_consumption) : the memory where opcaches are stored
string (opcache.interned_strings_buffer) : string literals is shared in a separate block of memory
keys (opcache.max_accelerated_files) : opcache has a hash table with the filename as key. The keys are stored in this block of memory.
JIT (opcache.jit_buffer_size) : the memory where the JIT generated machine code is stored.
Each has a separate configuration value. The exact value you need depends on the number of files and the number of shared projects using this same php-fpm pool.
Is there an easy way to see how much memory is being used?
A small php program can help :
<?php>
$status = opcache_get_status();
print “Memory (opcache.memory_consumption) : \n“;
print " used_memory=" . $status['memory_usage']['used_memory'] . "\n";
print " total_memory=" . ($status['memory_usage']['free_memory'] + $status['memory_usage']['used_memory']) . "\n";
print " hit_ratio=" . ($status['opcache_statistics']['hits'] / ($status['opcache_statistics']['hits'] + $status['opcache_statistics']['misses']) * 100) . "\n";
print “string (opcache.interned_strings_buffer) : \n“;
print " used_memory=" . $status['interned_strings_usage']['used_memory'] . "\n";
print " total_memory=" . $status['interned_strings_usage']['buffer_size'] . "\n";
print "hit_ratio=" . ($status['opcache_statistics']['hits'] / ($status['opcache_statistics']['hits'] + $status['opcache_statistics']['misses']) * 100) . "\n";
print “keys (opcache.max_accelerated_files): \n”;
print " used_memory=" . $status['opcache_statistics']['num_cached_keys'] . "\n";
print " total_memory=" . $status['opcache_statistics']['max_cached_keys'] . "\n";
print " hit_ratio=" . $status['opcache_statistics']['opcache_hit_rate'] . "\n";
if ($status[jit']['buffer_size']){
print “jit (opcache.jit_buffer_size) : \n“;
print " used_memory=" . ($status[jit']['buffer_size'] - $status['jit][ buffer_free_memory']) . "\n";
print " total_memory=" . $status[jit']['buffer_size'] . "\n";
}else{
print"jit is disabled”
}
Save this as a file name opcachestatus.php in a folder where php scripts can be executed from the browser. (Warning : do not ship this to production. Treat it like phpinfo.php).
If used memory in any section is more than total memory, you will get better performance by increasing the corresponding value in the opcache.ini file.
What happens when the cache memory runs out?
- When memory is full, opcache will essentially do a restart.
- We do not know how the jit memory behaves on being full.
Invalidating opcache
There are only two hard things in Computer Science: cache invalidation and naming things.
- Phil Karlton
Once an item is in cache, it can serve stale content - i.e. it needs to detect a php file has changed. What makes cache invalidation hard, is that there is a tradeoff between serving stale content vs speed.
Opcache gives you options.
- You can ask php-fpm to always check if a file has changed
opcache.validate_timestamps 1;
opcache.revalidate_freq 0; - You can ask php-fpm check if the file has changed atmost once every x seconds – replace x by the number of seconds you want
opcache.validate_timestamps 1;
opcache.revalidate_freq x - You can ask php-fpm to never check until you clear the cache explicitly
opcache.validate_timestamps 0
Opcache for production
- We like these settings
validate_timestamps 0
opcache.max_wasted_percentage 50
opcache.enable_file_override 1
opcache.max_file_size 0
opcache.consistency_checks 0
opcache.preferred_memory_model ‘’
opcache.file_update_protection 0
opcache.huge_code_pages 1
opcache.file_cache_only 0
opcache.file_cache ‘’ - JIT : for magento production server, where the only one application is running, we like to use the following for JIT
opcache.jit=1205
opcache.jit_buffer_size=200M - If using horizontal scaling with a load balancer, if you query opcache settings from a web application, you will get the result of the server the hit was executed from.
This is because each php-fpm server will have its own opcache.
Opcache and luroConnect
luroConnect enables opcache across all our customer servers. On dev/staging, opcache is run with
opcache.validate_timestamps 1;
opcache.revalidate_freq 0;
Our production servers run with
opcache.validate_timestamps 0
We continuously monitor opcache memory usage. Since we use a horizontally scaling architecture, we need to make sure if any app server exceeds the memory limits, all other servers are updated as well. We start with known magento required memory limits and tune to higher if needed.
Since we turn off validate_timestamps, each code deploy results in a reload of the php.
On php 8 servers, we use JIT with
opcache.jit=1205
this means we tell php to compile all functions into JIT code. We do this as all servers are running a single application and that never changes until a deployment is done.
