How Magento can get near 0 downtime deployment
Factor III of the 12 Factor App says “Store config in the environment”.
12 Factor App is what devops lives by – a set of 12 principles written by Adam Wiggins for predictable web app deployments.
Storing configuration in environment, separate from code has the advantages of reliable deployment along with reduced time to deploy. It allows separation of the build stage from the deploy stage, with some deploys being just a change in a softlink to the web root folder.
Historical preview : Magento 1
Magento 1 did not have much of a build process – js and css were not versioned, magnification was “online” first access based as was database upgrade information, configuration was stored in the database.
The most reliable way to go from a dev configuration to a live configuration would require a set of known steps that would work or changes directly to the database.
luroConnect developed its own build and deploy process. In our build step we
- get source code from git
- minify css and js files in the skin and js folders using a grunt based process
- set appropriate file ownership and permissions
During the deploy phase, we
- Copy app/etc/local.xml from a secure deployment configuration area (our environment)
- modify the core config data to add a version string in the skin and js URLs
- access the website once through the index.php to cause the update scripts to run
Deploy process is of course run with the site in maintenance – we prefer to do this at the nginx level. Mostly it is a small blip.
Historical preview – pre Magento 2.2
Early Magento 2 builds were similar – except there was some help from the bin/magento command. Our deploy process did not need to version the static access anymore. Plugin enable / disable was given via config.php. Our deployment environment contained env.php.
However, developers had to manually configure and experiment with some options.
Site bringup required devops to access the admin panel or update the database with custom sql – enabling varnish, setting up CDN with a static URL, etc.
Magento 2.2 and beyond
Magento adopted the direction of the 12 factor app and presented in Magento Live UK 2017 a new set of features that would help in ensuring an ability to split the application configuration and environment configuration. Application configuration was defined in app/etc/config.php which is advised to be in git and hosting environment and secure details are kept in env.php which should not be kept in git.
It is a slightly weak conformance – as commented by 12factor app “This is a huge improvement over using constants which are checked into the code repo, but still has weaknesses: it’s easy to mistakenly check in a config file to the repo; there is a tendency for config files to be scattered about in different places and different formats, making it hard to see and manage all the config in one place. Further, these formats tend to be language- or framework-specific.”
Magento has fixed this in 2 ways
- The language specific aspect is addressed to some extent in Magento by allowing to use bin/magento cli to edit env.php for sensitive data. The config:sensitive:set directly writes to env.php. These commands no not require the database, hence, can be set in a pre-deploy step.
- Use of scoped environment variable names. These would be set in Nginx configuration or an include file such as fastcgi_params.
However, there is no documented way to set database details – except to manually edit the env.php file.
The app:config:dump command
A great help in maintaining a known configuration of the application (which 12factor app suggests be committed to git). This ensures communication between developer to operations.
The app:config:dump command writes to config.php and env.php. While config.php is suggested to be committed to git, env.php should not be committed to git.
If a value is in config.php, the Magento admin panel does not allow the parameter to be edited. This locking helps with giving stability to the application configuration. It ensures the application is developed and tested with a known configuration.
The figure alongside shows the suggested flow.
Suggested flow for using app:config:dump
Why is Magento deployment yet keeping site in maintenance?
However, we find that even after 2 1/2 years of announcement, the acceptance and understanding of these features is weak. Leaving websites in maintenance mode as code is deployed.
Developers are failing to maintain a discipline to own the configuration or devops to understand the application’s build and deploy process.
There are some practical problems as well. An eCommerce manager would like to have control on the live website on say, when backorders would be allowed storewide. Since this is locked in config.php, this request has to go through developers or devops.
luroConnect near 0 downtime deploy
luroConnect’s Magento 2 build is in a pipeline – such as a bitbucket pipeline. A commit triggers the pipeline that does the following
- composer install (with the compose cache to speed this process)
- bin/magento setup:di:compile
- bin/magento setup:static-content:deploy
The contents are then tarred and sent to the staging and production servers.
Upon deploy the contents are untared, deployment related files like env.php are copied, media and var are softlinked. The web root softlink is changed to point to this new release. The process is slightly more complicated when multiple autoscale instances are running, as running instances are replaced with ones with new code.
If required the bin/magento setup:upgrade command is run and only then is it required to keep the site in maintenance.
