devOps for Magento
Not all problems you have in running your Magento site can be solved by developers or operations including hosting and linux system administrators. devOps sits between developers and operations – making solutions that address rather odd problems. They are a sort of oil that you require to keep your site moving along nicely.
You need devops irrespective of the size of site or the number of changes you are making to your site.
Here are some problems that our devops team have helped solve.
Giving ftp access to the live site to external vendors can be dangerous. The work being done without a maintenance flag may bring down temporarily parts of a site giving a bad experience to visitors.
The right way would be to have a staging server where the vendor demonstrates the changes and then a process takes from staging to live using a versioning system like git.
Our devops team can help automating creating a staging site which can be used for a short time. Using code from git and database from the previous night’s backup, the server can be ready in a short time, automatically when you want and on any cloud provider of your choice using their APIs.
All CDNs give API based invalidations. However, each has a different terminology / concept and API calls. With our inotifywait based scripts we easily identify changed css / js files and invalidate CDNs.
An offline minifier is our preferred solution. Built as a grunt script, we would add this as a default step at the end our site update script.
We can also work on a custom nginx with page load module or apache’s page load module or maybe even use the Magento merge and minifier.
When bulk uploading products, images need to be transferred to a specific folder. We would ideally like to give access to only the upload folder.
Our devops team has designed a solution for this. A image upload login is created where images are uploaded. Upon upload, the files are automatically transferred to the final location required by the csv.
Technically, the script uses a feature called inotifywait in linux. This watches the folder on which upload is performed and moves the file to the final destination instantaneously.
It is difficult to ensure all images are optimized before upload. Magento automatically generates different sized images that are needed on different pages. When using CDN, the images need to be optimized before upload to CDN.
Our devops team created a solution to solve this problem. Using the convert utility (part of ImageMagik) and inotifywait feature of linux, the moment a new image is created, it is automatically optimized. By watching the image cache folder, the images are converted typically before the CDN gets them.
Sites need code updates to be smooth. Code updates may come from plugin vendors or from internal developers or contractors.
Having a git repository is a starting point. When code is deployed there are configurations such as database and cache settings which are in files that need to be retained on the live server. You may not trust a developer to have full access to the live server to deploy.
Our devops team has developed a easy to use special and secure login mechanism which you can cause the code to be deployed automatically with the specified files being retained. The access to this login is easily restricted with private keys making it very secure.
For larger environments we have a detailed deployment mechanism using jenkins that gives historical status of deployments and with an ability to rollback a deployment.
Testing a new plugin or getting a project completed requires to build a development and/or staging site. Reliably creating an environment at will is essential to higher productivity and avoiding “it works for me” situations. Use of “localhost” have short term gains with long term pains.
Also taking backup of live data to test development code is detrimental to the performance of the live site.
Our devops automation will allow creation of a server and deploying the latest code and previous nights’ database backup. Once the code is developed, committed and pushed, the developer instance can be safely destroyed.
No one likes wasted resources! Why keep a cloud server running or at full capacity when access is known to be low?
Our devops team can write automated scripts that use the cloud service API to resize or shutdown or powerup a instance at known time or with a single command.