Search Results - nginx

Nginx as a load balancer for Magento

Introduction

During seasonal peaks or as traffic grows, there will be a need to add multiple app servers to your Magento store. A load balancer for Magento becomes essential. We have found that using nginx as a load balancer gives acceptable performance. We have not found many instances where we would recommend a hardware load balancer. Recent tests by nginx confirms this.

We would recommend a different load balancer only for additional features such as autoscaling.

Nginx as a load balancer offers many advantages including

  • uneven upsream servers as nginx can assign weights to each load balancer
  • self healing – takes a upstream server out of a cluster if it stops responding
  • path based load balancing
  • combination of path based and weight based load balancing
  • php upstream servers
  • SSL/TLS termination

This article assumes the process of adding a new app server to a Magento cluster is well understood. Here the focus is on the nginx configuration.

Load Balancer for Magento : Basic architecture

(more…)

May 18, 2016 By hosting, Magento, Nginx 2 Comments Read more...

Nginx or Apache : Best server for Magento

Introduction

Apache server has been for years been the default http server linux hosts use. However, recently there have been many newer “lighter” http servers. This blog article focuses on Magento hosting. Magento is a php based web eCommerce framework. Nginx requires php-fpm to process php requests. So, this comparison is really apache vs nginx + php-fpm. Apache offers MPM (Multi-Processing Module) configurations pre-fork, worker and event. In this discussion we will use the “event” MPM.
This discussion is very popular. Examples include this. We focus on Magento here.

Key Differences between apache and nginx

There are some differences architecturally that make nginx look slightly better for Magento hosting.
(more…)

April 29, 2016 By apache, hosting, Magento, Nginx 0 Comments Read more...

Using AWS Autoscale “warm pools” to reduce costs

AWS Autoscale added a new feature “Warm Pool”.  Let us explore this feature and see how luroConnect uses this to reduce hosting costs.

The autoscale latency problem

Usually, AWS Autoscale will launch a new server with the given AMI image based on the launch configuration or launch template configured. Launching a new server takes about 4 minutes or more. So let us say a scale-out event is configured for launching a server when the CPU across all autoscale instances exceeds 70% for 1 minute. Now, let us say a sale promotion on facebook causes a surge in traffic causes this event to trigger. It takes AWS 4+ minutes to respond and add a new server. If during this 4 minute period, the surge goes past 70% and say reaches 90-100%, it is likely that visitors will see a slowdown or even errors. The 4+ minute period is called the autoscale latency and in designing the scale-out and scale-in parameters, it plays a crucial role.

For a website that sees frequent surge in traffic in short spurts, one would be prompted to use a lower threshold for a scale-out event. A lower threshold will result in frequent triggering of scale-out events.

At the same time the scale-in threshold will also have to be reduced to ensure enough spread between scale-out and scale-in events. A lower spread will result in an unhealthy sequence of a scale-out event adding a resource for it to be immediately removed.

Autoscale designers then tend to add higher number of minimum instances, possibly of larger sizes. That reduces the effectiveness of autoscale – and increases AWS costs.

Lowering the autoscale latency results in a better autoscale system. As the latency reduces, the need for larger number of minimum instances or larger size instances reduces. This results in savings in the AWS bill.

Introducing the warm pool

AWS now introduces the concept of a warm pool. The costs saving of a warm pools come from AWS policy for not charging for instances in stopped state – except for the disks. A warm pool is a set of autoscale instances that are launched but kept in stopped state. When a scale-out event happens, the latency Is now reduced to the boot time of an instance and any initialization needed – we measured adding 3 instances took about 35 seconds to start serving traffic for Magento.

A scale-in policy simply stops the selected instance and add it back to the warm pool.

Warm Pool For Autoscale

How to use a warm pool?

If you are using launch template for your autoscale, creating a warm pool is easy and documented here. If using lifecycle events, newer events have been introudced.

If using a launch configuration, we suggest upgrading to a launch template before using a warm pool. While upgrading to a launch template is easy, it is advisable to read about launch templates as they are a different and a larger concept.

Changing your instance image when in a warm pool

AWS has support for “instance refresh” – a term used by AWS to indicate an update in AMI image for all running and warm pool instances in a single command. However, this update has a crucial flaw – it can keep your website inaccessible for a short time. This is due to AWS terminating an instance before adding one. If an image has to be updated – such as a new code deploy – a custom strategy has to be deployed to ensure the website does not go down.

luroConnect support for warm pool

luroConnect now supports warm pools across all its autoscale plans, with a scripted image update policy that ensures 0 downtime during image change as well as a code deploy strategy that ensures 0 downtime on code deploy.

Issues with AWS Reference architecture and tools for a Magento application

At luroConnect we implemented our autoscaling system after addressing flaws in many implementations we had seen.

As AWS autoscale by default is integrated into AWS load balancers – ELB or ALB. Using AWS reference implementation will put the code in a autoscale instance with nginx or apache with php and the code. Traffic can be routed through the ELB/ALB which will handle SSL and route the traffic to each autoscale instance.

When code has to be updated, a new AMI will be created and AWS instance refresh can be run to update the instances.

You could use AWS CodeDeploy as described here but you need to set it up to make sure Magento setup upgrade can be run when required.

Problems with autoscale implementations for Magento

  1. Issues configuring FPC (Full Page Cache) with this configuration : If varnish is configured on all autoscale instances (as we have seen many implementations do), each server will warm caches on its own. Clearing pages from cache will also be difficult. Using redis as a FPC increases per page latency for cached pages.
  2. Media and var folders are needed to be shared across all servers. NFS is typically used to share. However, the configuration of each autoscale instance has to be such that it can discover and mount the folders from the NFS server.
  3. When a code change has to be deployed, it is not clear how it can be done without causing a downtime of the website. Using AWS Code Deploy requires a complex setup to ensure setup upgrade is run before one of the 0 downtime strategies can be used.
  4. When a new server is launched, conditions to check the health of the website are not easy to write. This results in a few error responses before the server is ready to serve traffic.
  5. It is difficult to use a AWS ALB to route traffic for specific purposes – for example, routing traffic to a wordpress server for /blog urls.

luroConnect Autoscale on AWS : Smooth setup and running.

luroConnect Autoscale solves these problems.

luroConnect lets AWS monitor instances and decide when to add or remove (scale out or scale in) instances. luroConnect autoscale for AWS adds cloudwatch events and lifecycle management generated by AWS Autoscale to ensure a very smooth Autoscaling operation. luroConnect uses nginx as a load balancer and does not require a ALB/ELB to operate. luroConnect Autoscale supports AWS Autoscale with warm instances and has a mechanism to update the AMI when needed without any downtime.

  1. Using nginx as a load balancer allows high flexibility in deciding which urls go to varnish for full page cache and which should be directly served by php. varnish as a full page cache gives the maximum impact of full page caching.
  2. A nfs server holds shareable content of magento – specific media and var folders for example. Using NIS, autofs and NFS, each new app server is able to discover the NFS share.
  3. When a code change has to be deployed, php code using nfs is shared to each app server. A php reload and opcache configuration will ensure the new code is kept in the php opcache memory for all future operations. A php file from NFS share is loaded only once.
  4. Before a server is added to the nginx load balancer, extensive checks are done to ensure the new autoscale instance is ready to take traffic, including warming the opcache.
  5. nginx as a load balancer brings in a lot of flexibilty in routing traffic such as a /blog to a wordpress website, custom rewrites, etc.

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and autoscale to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for Magento.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimized hosting costs
Click here to schedule the call
June 10, 2023 By devops, Horizontal Scaling, hosting, Magento, Nginx 0 Comments Read more...

Do you know what size server you want for your eCommerce site?

Leaving the toughest question unanswered

When signing up for your Magento hosting, the first question you see asked, before you place an order, is what size server you want. It has become so ubiquitous, that everyone just expects to answer it looking at the cost.

But this is much like Mathematics books leaving tough problems as exercise to the readers!

It should not be that way! The size and architecture of the server you need depends on many factors.

Factors to consider

  1. The traffic and pattern. We routinely ask for 2 google analytics graphs – one for a typical day and one for a high sale day. This drives the architecture and server size.
  2. Your hosting stack – are you vanilla magento? or do you use headless / PWA? or use some software for image optimisation on your server?
  3. If the live site is already hosted, current CPU and memory usage.
  4. The size of the magento database.

luroConnect always starts an engagement with a server sizing sheet that is filled on behalf of the merchant. This allows us to propose a hosting plan on the customers cloud account and an appopriate luroConnect support plan.

Take the guesswork out of server sizing with horizontal scaling

A classic 3-tier architecture.

  • The web layer (WAF, apache / nginx /varnish, cron, rabbitmq),
  • the application layer (php, nodejs) and
  • the database layer (mysql, elasticsearch, redis).

Horizontal Scaling :

  • app servers can scale independently – indeed they can be autoscaled.
  • Low traffic websites can fold either the app or the db layers or both into the web layer
  • The db layer can be extended to have master slave
  • A proxy layer can load balance read traffic between master and slave, giving scalability at the database level

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and autoscale to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for Magento.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimized hosting costs
Click here to schedule the call
March 28, 2023 By hosting, Magento, Site Speed 0 Comments Read more...

Case Study : How a simple code change had a major impact on website performance

During black Friday sale, one of our customers’ websites was too slow and unusable for long periods. We were embarrassed, but all our analysis pointed to code. As part of our active support, we help with identifying code bottlenecks. We use many techniques including profiling, but a simple technique for a really slow website is analyzing the php slow log.

Observations

We track slowest URLs and chart them on our dashboard. A typical days chart looked like the one below. Most of the slow URLs were category listing page urls.

What is the php slow log?

The PHP slow request log is where PHP records information about any request that takes more than “x” seconds to execute. The number of seconds is configurable in the php-fpm.d/www.conf file.

The PHP slow request log records stack traces (also known as tracebacks) of each slowly executing script at the moment the request passed “x” seconds in execution. For example the following trace was generated for the website we were optimizing. The first line gives the date and time, the php-fpm pool name and the pid of the fpm process. The second line names the script – for Magento it will be the index.php always. The following lines have the memory address, the function name, the file name and the line number. A stack trace always lists the last call first.

[03-Dec-2020 22:54:57]  [pool www] pid 19267
script_filename = /www/usa/index.php
[0x00007ffa5ec21d30] execute() /home2/customer/77faf58/vendor/magento/zendframework1/library/Zend/Db/Statement/Pdo.php:228
[0x00007ffa5ec21c80] _execute() /home2/customer/77faf58/vendor/magento/framework/DB/Statement/Pdo/Mysql.php:93
[0x00007ffa5ec21be0] _execute() /home2/customer/77faf58/vendor/magento/zendframework1/library/Zend/Db/Statement.php:303
[0x00007ffa5ec21b30] execute() /home2/customer/77faf58/vendor/magento/zendframework1/library/Zend/Db/Adapter/Abstract.php:480
[0x00007ffa5ec21aa0] query() /home2/customer/77faf58/vendor/magento/zendframework1/library/Zend/Db/Adapter/Pdo/Abstract.php:238
[0x00007ffa5ec219d0] query() /home2/customer/77faf58/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:541
[0x00007ffa5ec218a0] _query() /home2/customer/77faf58/vendor/magento/framework/DB/Adapter/Pdo/Mysql.php:615
[0x00007ffa5ec21810] query() /home2/customer/77faf58/vendor/magento/zendframework1/library/Zend/Db/Adapter/Abstract.php:737
[0x00007ffa5ec21760] fetchAll() /home2/customer/77faf58/vendor/magento/framework/Data/Collection/Db/FetchStrategy/Query.php:21
[0x00007ffa5ec216e0] fetchAll() /home2/customer/77faf58/vendor/magento/framework/DB/Query.php:182
[0x00007ffa5ec21660] fetchAll() /home2/customer/77faf58/vendor/magento/framework/Data/AbstractSearchResult.php:233
[0x00007ffa5ec215c0] load() /home2/customer/77faf58/vendor/magento/framework/Data/AbstractSearchResult.php:107
[0x00007ffa5ec21550] getItems() /home2/customer/77faf58/vendor/magento/module-catalog-inventory/Model/StockRegistryProvider.php:143
[0x00007ffa5ec21440] getStockItem() /home2/customer/77faf58/vendor/magento/module-catalog-inventory/Model/StockRegistry.php:88
[0x00007ffa5ec213c0] getStockItem() /home2/customer/77faf58/vendor/magento/module-catalog-inventory/Model/Plugin/AfterProductLoad.php:40
[0x00007ffa5ec212b0] afterLoad() /home2/customer/77faf58/vendor/magento/framework/Interception/Interceptor.php:146
[0x00007ffa5ec21150] Magento\Framework\Interception\{closure}() /home2/customer/77faf58/vendor/magento/framework/Interception/Interceptor.php:153
[0x00007ffa5ec21070] ___callPlugins() /home2/customer/77faf58/generated/code/Magento/Catalog/Model/Product/Interceptor.php:65
[0x00007ffa5ec20fe0] load() /home2/customer/77faf58/app/code/Plazathemes/Bestsellerproduct/Block/Bestsellerproduct.php:112
[0x00007ffa5ec20ef0] getBestsellerProduct() /home2/customer/77faf58/app/code/Mageplaza/CatalogPermissions/Observer/ProductLoadAfter.php:79

How do you trace from a hit to a slow log entry

From nginx we can trace to the slow log using a match in the php-fpm access log. We format nginx and php access log in a way that helps tracing.

The nginx log entry that we traced using the format

$remote_addr|$remote_user|$time_local|$request_time|$upstream_response_time|$request|$status|$body_bytes_sent|$http_referer|$http_user_agent|$host|$upstream_addr

127.0.0.1|-|03/Dec/2020:22:55:01 -0600|8.966|8.967|GET /usa/health-and-rejuvenation/cereals-health-mix HTTP/1.1|200|20458|https://www.ishalife.com/usa/yoga-gear/bedding|Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36|www.ishalife.com|10.120.0.7:9000

php access log format used
access.format = %R - %u %t %m %r%Q%q %s %f %{mili}d %{kilo}M %{total}C%% %p

php access log entry to the traced url (matched by time and response time)

10.120.0.2 - 03/Dec/2020:22:54:52 -0600 GET /usa/index.php 200 /home2/customer/www/usa/index.php 8962.264 4096 48.20% 19267

The %p records the process id of php-fpm that processed this hit. Based on the time and process id, the slow log entry can be accurately traced.

Analysis

The stack trace clearly indicated that an observer was called when the event “ProductLoadAfter” was called. For this website, it slowed down the category display as the call was made for each product. A peep into the code in productloadafter observer was loading “Best Seller Products”. So, for each product in the category, it would load the (almost) identical set of best seller products. The observer object was not even used.

Let us look at the code

public function execute(Observer $observer)
    {
        /** @var AbstractCollection $productCollection */
        
           $productCollection = $this->productBest->getBestsellerProduct();
       
           foreach ($productCollection as $key => $product) {
                /** @var product $product */

                $productObject = $this->productRepo->getById($product->getId());
                if ($this->helperData->validateConditionDefault($productObject, ObjectTypeAction::TYPE_PRODUCT)
                    && !$this->helperData->checkIPPassAllRestrict()
                ) {
                    /** @var DataObject $productObject */
                    if ($productObject->getData('mpcp_usecf_hideaction') === '0'
                        && $this->helperData->getProductHideCatAndWidget()) {
                        $productCollection->removeItemByKey($key);
                    }
                    // do not use config hide action
                    if (!$productObject->getData('mpcp_usecf_hideaction')
                        && $productObject->getData('mpcp_hideaction') === '1') {
                        $productCollection->removeItemByKey($key);
                    }
                }
            }
    }

The solution

Remove the observer as apart from badly written, the result was never used on the website. Code analysis revealed 2 unused plugins.

2 unused plugins and 1 observer call was removed, resulting some category pages loading 10x faster!

Before and After stats

We monitor hits in real time and chart hits / minute and avg response time per minute. In the graphs below, the left axis is hits per minute (plotted in turquoise) and the right axis is average response time in seconds per minute (plotted in black).

BEFORE

AFTER

CPU usage of php servers

BEFORE

AFTER

Database CPU usage across multiple days

Conclusion

The importance on good code and possibly the negative effects of unused functionality was highlighted in this case study.

As users of plugins we need to understand the motivation of plugin vendors to pack in substantial functionality leads to undesired side effects, especially on functionality you don’t want to use. Budget to clean up vendor code before using.

December 29, 2020 By case study, performance, Site Speed 0 Comments Read more...

Interview with Carmen Bremen, Mage-One

Carmen Bremen is an accomplished Magento developer and community member for many years. She is a certified developer and she has been awarded Magento Master She is one of the founding members of Mage-One – a paid long-term-support (LTS) for Magento 1. We chatted with her over email recently.

luroConnect : Tell us about yourself (how many years in Magento, feathers in your cap)
Carmen : My name is Carmen and I met Magento in 2010. At first I thought: I’ll never learn this, took the challenge and didn’t want to give up Magento afterwards. Now Adobe unfortunately announced the end of Magento 1 and maybe that’s why I became part of Mage One. I just don’t want to give up Magento 1 so fast 🙂
​
luroConnect : You must have researched and talked to merchants about M1 end-of-life. Can you summarize your finding?
​
Carmen : Many merchants would like to continue using their Magento 1 shop a little longer. Some have not yet decided which system they want to switch to. Others think that the investment in the shop has to pay off first and others are so used to Magento 1 that they just want to continue using it.
​
luroConnect : What type of patches does one expect from mage-one?
​
Carmen : We will offer security and compatibility patches. We will not develop new features or remove old Magento bugs.
​
luroConnect : Php 7.2 EOL is nearing – will you deliver patches for php version upgrades?
​
Carmen : Of course. The first patch we release will be the one for PHP 7.3.
​
luroConnect : What type of tests do you run before you release your patches?
​
Carmen : We use Cypress for our tests and have been working on numerous different tests for weeks.
​
luroConnect : How many years of mage-one support can we expect?
​
Carmen : We intend to offer our service for at least 5 years.
​
luroConnect : I know this is not legal advice I am asking, but for merchants who process payments by Paypal or Visa, a big question is what would happen after June 2020? Will they lose PCI compliance? (Refer resources if you can). 
Will mage-one satisfy the 6.2 PCI rule.
​
Carmen : From our perspective 100%. We offer a certificate in our customer account that can be presented to payment providers. Nevertheless, the final decision is up to the payment provider itself. However, we already have a statement from PayPal, for example, that you can continue using PayPal with Mage One.

luroConnect Support for Magento 1 past EOL

We have a 4-point plan to support you. Signup Now and we will contact you. No credit card required.

From EUR 29

Mage-One Patches for post EOL Magento 1 support. Will satisfy PCI vendor requirement.

mage-one logo

From EUR 99

Sansec eComscan examines a store for malware, vulnerabilities and unauthorized accounts. Written by well known security expert Willien De Groot, it scans files, databases and 3rd party components of Magento.

From USD 50*

Inbuilt into our Nginx, with M1 rules, protects from OWASP Top 10, with the ability of virtual patching.

From USD 50*

Staging environment to ensure patches are tested before taken live.

Signup now! And we will be in touch with you. No credit card required.

June 25, 2020 By Uncategorized 0 Comments Read more...

How Magento can get near 0 downtime deployment

Factor III of the 12 Factor App says “Store config in the environment”.

12 Factor App is what devops lives by – a set of 12 principles written by Adam Wiggins for predictable web app deployments.

Storing configuration in environment, separate from code has the advantages of reliable deployment along with reduced time to deploy. It allows separation of the build stage from the deploy stage, with some deploys being just a change in a softlink to the web root folder.

Historical preview : Magento 1

Magento 1 did not have much of a build process – js and css were not versioned, magnification was “online” first access based as was database upgrade information, configuration was stored in the database.

The most reliable way to go from a dev configuration to a live configuration would require a set of known steps that would work or changes directly to the database.

luroConnect developed its own build and deploy process. In our build step we

  • get source code from git
  • minify css and js files in the skin and js folders using a grunt based process
  • set appropriate file ownership and permissions

During the deploy phase, we

  • Copy app/etc/local.xml from a secure deployment configuration area (our environment)
  • modify the core config data to add a version string in the skin and js URLs
  • access the website once through the index.php to cause the update scripts to run

Deploy process is of course run with the site in maintenance – we prefer to do this at the nginx level. Mostly it is a small blip.

Historical preview – pre Magento 2.2

Early Magento 2 builds were similar – except there was some help from the bin/magento command. Our deploy process did not need to version the static access anymore. Plugin enable / disable was given via config.php. Our deployment environment contained env.php.

However, developers had to manually configure and experiment with some options.

Site bringup required devops to access the admin panel or update the database with custom sql – enabling varnish, setting up CDN with a static URL, etc.

Magento 2.2 and beyond

Magento adopted the direction of the 12 factor app and presented in Magento Live UK 2017 a new set of features that would help in ensuring an ability to split the application configuration and environment configuration. Application configuration was defined in app/etc/config.php which is advised to be in git and hosting environment and secure details are kept in env.php which should not be kept in git.

It is a slightly weak conformance – as commented by 12factor app “This is a huge improvement over using constants which are checked into the code repo, but still has weaknesses: it’s easy to mistakenly check in a config file to the repo; there is a tendency for config files to be scattered about in different places and different formats, making it hard to see and manage all the config in one place. Further, these formats tend to be language- or framework-specific.”

Magento has fixed this in 2 ways

  1. The language specific aspect is addressed to some extent in Magento by allowing to use bin/magento cli to edit env.php for sensitive data. The config:sensitive:set directly writes to env.php. These commands no not require the database, hence, can be set in a pre-deploy step.
  2. Use of scoped environment variable names. These would be set in Nginx configuration or an include file such as fastcgi_params.

However, there is no documented way to set database details – except to manually edit the env.php file.

The app:config:dump command

A great help in maintaining a known configuration of the application (which 12factor app suggests be committed to git). This ensures communication between developer to operations.

The app:config:dump command writes to config.php and env.php. While config.php is suggested to be committed to git, env.php should not be committed to git.

If a value is in config.php, the Magento admin panel does not allow the parameter to be edited. This locking helps with giving stability to the application configuration. It ensures the application is developed and tested with a known configuration.

The figure alongside shows the suggested flow.

Suggested flow for using app:config:dump

Why is Magento deployment yet keeping site in maintenance?

However, we find that even after 2 1/2 years of announcement, the acceptance and understanding of these features is weak. Leaving websites in maintenance mode as code is deployed.

Developers are failing to maintain a discipline to own the configuration or devops to understand the application’s build and deploy process.

There are some practical problems as well. An eCommerce manager would like to have control on the live website on say, when backorders would be allowed storewide. Since this is locked in config.php, this request has to go through developers or devops.

luroConnect near 0 downtime deploy

luroConnect’s Magento 2 build is in a pipeline – such as a bitbucket pipeline. A commit triggers the pipeline that does the following

  • composer install (with the compose cache to speed this process)
  • bin/magento setup:di:compile
  • bin/magento setup:static-content:deploy

The contents are then tarred and sent to the staging and production servers.

Upon deploy the contents are untared, deployment related files like env.php are copied, media and var are softlinked. The web root softlink is changed to point to this new release. The process is slightly more complicated when multiple autoscale instances are running, as running instances are replaced with ones with new code.

If required the bin/magento setup:upgrade command is run and only then is it required to keep the site in maintenance.

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and autoscale to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for Magento.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimized hosting costs
Click here to schedule the call
May 5, 2020 By devops, hosting, Magento, Nginx, scale, Site Speed 0 Comments Read more...

12 factor app and Magento

Adam Wiggins’ 12 factor app (https://12factor.net) is a highly respected standard for web apps. While written with SaaS applications in mind, let us explore and see how Magento and the ecosystem stands up to these factors.

1. Codebase. One codebase tracked in revision control, many deploys.
Magento is in git and hence a typical Magento project should not have a problem with this.
However, if you use vue-storefront, a popular PWA frontend to Magento, this is broken. Vue-storefront has 2 repos of its own in addition to the Magento repo, all becoming one app.
Another violation happens when a plugin vendor gets ssh access to your live server to fix a plugin issue. Plugin vendors have a serious problem integrating their code into multiple source bases without Magento supporting a versioned plugin architecture out-of-the-box.

2. Dependencies. Explicitly declare and isolate dependencies.
With composer Magento solves this problem.
Violation of plugins is a case in point – many plugins are installed not as composer dependencies. Instead they make it to the merchant repo.

Magento uses php and typical websites are deployed using php-fpm. One may argue that the php-fpm plugins that Magento depends on are not explicitly declared. Leading to the application not working exactly in 2 environments. Another case in point is dependency on php version.

3. Config. Store config in the environment.
12 factor app requires environment variables to be used. Magento has split application and environment configuration between config.php and env.php.
Here is what 12 factor says.
“Another approach to config is the use of config files which are not checked into revision control, such as config/database.yml in Rails. This is a huge improvement over using constants which are checked into the code repo, but still has weaknesses: it’s easy to mistakenly check in a config file to the repo; there is a tendency for config files to be scattered about in different places and different formats, making it hard to see and manage all the config in one place. Further, these formats tend to be language- or framework-specific.”

However, Magento has worked towards this. Specifically, with bin/magento config:set and bin/magento config:sensitive:set commands are a useful way for hosting providers to be 12 factor compliant.

luroConnect has always stored hosting configuration settings separately from the release. Upon deployment of code, the contents of deployment folder are copied. Sometimes they have settings for the application. These include hosting specific as well as sensitive settings. We are moving to using config:set and config:sensitive:set for versions of Magento that support it. We will also move towards storing sensitive variables in secure key stores.

4. Backing services. Treat backing services as attached resources.
“Resources can be attached to and detached from deploys at will.”

While Magento is very good at storing key connections outside the application and database, violations exist in 3rdparty plugins. To “ease” the deployment most store credentials and connectivity details in the database. Another issue is with SMTP plugins, instead of depending on magento’s default use of localhost and let postfix configuration manage the actual email sending, developers see the convenience of storing this information in the database.

Check out this post on SMTP and postfix configurations.

5. Build, release, run. Separate build and run stages.
Magento has been improving the code deployment process. The setup upgrade is the only command that, if needed, requires the site under maintenance.

6. Processes. Execute the app as one or more stateless processes.
Twelve-factor processes are stateless and share-nothing. Any data that needs to persist must be stored in a stateful backing service, typically a database.

Magento is very good on this count if used with nginx and php-fpm.

7. Port binding. Export services as port binding.
“PHP apps might run as a module inside Apache HTTPD” is flagged as a violation if the apache is also used as a webserver.
nginx + php-fpm gives the best isolation and performance of any stack. Php processes can be independently controlled in a server running php-fpm while nginx can be used for routing and handling web requests, terminating SSL, etc.

8. Concurrency. Scale out via a process model.
Magento is very good at this. Aided by php-fpm process model that complies with the 12 factor app, it is possible to build a cluster to handle only checkout urls for example, with routing handled by an application load balancer such as nginx.

9. Disposability. Maximize robustness with fast startup and graceful shutdown.
While Magento and php are good at this, some notes are in order.
A reload of php-fpm by default will kill all php processes even though they may be executing a request. Ensuring no new traffic is coming to the php-fpm, and waiting for draining by checking the status for number of active processes (with a timeout ofcourse) will ensure gracefulness in shutdown.
In order to ensure robustness against sudden death of the php-fpm process, it is best to keep the queue length (listen.backlog) to a small number. Turns out managing the queue to scaleout helps in application performance as well.

10. Dev/prod parity. Keep development, staging, and production as similar as possible.
The 12 factor app describes 3 gaps – time, personnel and tools. Based on our experience, the personnel gap is eliminated by automation. A commit trigger based automated CI/CD pipeline with an automated deploy to staging and production ensures there is no personnel gap.

A development environment with write access to git can be created with a similar infrastructure to help developers debug issues.

11. Logs. Treat logs as event streams.
Magento allows creation of multiple log files. Modern logging such as monolog allows more control of what is and what isn’t logged. Logs are also generated by nginx, php-fpm and other services used.
Streaming logs for querying and analysis is typically done by your hosting provider.

luroConnect uses fluentd to capture logs. Logs are sent to our Insight service, which analyzes data per minute, hour or day.

12. Admin processes. Run admin/management tasks as one-off processes.
Magento supports cron and rabbitmq based processes. In addition, setup upgrade is also used to change the state of the database during deployment.
However, suggested access to developer for “run arbitrary code or inspect the app’s models against the live database” is not recommended by luroConnect due to security and the risks of the application stability with the state being altered arbitrarily.

April 29, 2020 By devops, hosting, Magento, Nginx 0 Comments Read more...

How we hosted a big sale on Magento without a hitch

During an online sale that includes promotion to a list, brands worry about downtime due to “unexpected high load”. We believe that while every system has a rated limit, there can be flexibility added to your infrastructure. With some planning, including autoscale and tuning, a successful sale event can be a very likely outcome. See how we handled one such sale without a hitch.

In our experience promotions via SMS generates very high peaks. In this article we will show our recent experience with a sale day that generated a peak of 30 hits per second to Magento.

The sale

On Saturday 21st March, 2020 one of our customers had a sale event. The merchant – a fashion retail brand – is an omni channel retailer in India with about 145 stores across 45 cities. There have been previous sale events, but this one was different as most stores were closed due to the corona virus measures in place. This meant that more customers were likely to shop online than visit the store.

The sale was promoted via sms and email. The SMS list was about 1 million. Promotional messages were sent in batches of 200k every half hour. Marketing was to start at 10 AM local time.

Monitoring the website

We monitor the nginx log file, analyze and display information on our dashboard. A key component of our dashboard is the plot of number of hits / minute (we count only php hits, static hits like css, js and images are excluded). We also calculate avg response time per minute as well as standard deviation per minute. Both of these have a unit of seconds. We also have alerts setup for many parameters including slowdown and a 5xx error response.

The graph below is for a 24 hour period, including the sale time, which lasted about 5 hours with the effect keeping the site more busy than usual afterwards. Note these are server logs graphs – they look quite different from google analytics!

The traffic & response

The graph gives as an idea of the traffic to the website. The turquoise line is hits per minute served by php.

  • Traffic peaked a about 2000 hits per minute – about 30 hits per second.
  • 3476 items were added to cart during the sale.
  • Average server response was mostly below 1 second.
  • There were 3 errors – all of them related to checkout function (“This payment method is not available”)

Mysql query cache and response time

The store is a Magento 1 website. Just as the traffic / hits to the website started peaking, we got a slow alert – represented by the large black line close to the first traffic peak around 10.45 am in the previous figure. Our prior experience with the website showed that the mysql query cache plays a role in the site performance. Before the sale started, mysql query cache was turned off. On seeing the slowdown alert, we turned on the mysql query cache and saw an immediate improvement in performance. Both avg response per minute and standard deviation per minute improved – for some time.

However, standard deviation – higher number indicates some visitors faced slower response to the website than others – deteriorated with time. After the sale (10pm local time), mysql query cache was turned off – resulting in improved standard deviation, but slightly worse avg response time.

AWS autoscale

The following graph plots AWS autoscale instances over the same 24 hour period. A time based minimum 5 instances was planned during the sale from 9.15 AM to 7.30PM. At the peak 7 instances were required – our autoscale policy adds 2 instances and removes 1 at a time.

luroConnect Architecture for AWS and controls

Our standard scalable architecture uses nginx as a load balancer. The communication between the load balancer to php servers is using fcgi – individual php servers do not have nginx. A NFS server is used to share folders that need sharing – media and var folders in Magento 1. Magento cache and sessions are served from redis. Code is also shared using nfs.

A custom lambda function communicates between autoscale lifecycle hooks and the nginx load balancer adding and remove instances as the AWS autoscale policy decides.

Our cache controls including the ability to turn on and off mysql cache, resize memory allocated to redis cache, resize memory allocated to php-fpm opcache.

Interested in knowing about the advanced architecture of luroConnect?

Fill the form below and we will contact you.


My company owns the Magento site
Yes, I am a developer
I represent a Magento Agency

March 25, 2020 By Uncategorized 0 Comments Read more...

Magento 1 end of life

June 2020 is a few weeks away and many stores are not ready to move away from Magento 1. But wait, you don’t have to update – now or in the long run.

COVID-19 Update : We find many customers have had to delay their Magento 2 launch in these uncertain times. We also know many of them did not have plans to keep Magento 1 uptodate. In fact, we know agencies that have stopped support for Magento 1.

Starting at a low cost of $200 per month with no long term contract. It includes reviewing your current hosting for security, moving your website to the latest Magento 1.9 and latest php supported as well as adding additional security measures to your website. It also includes help signing you up for Mage One or Open Mage projects for support beyond Magento 1 EOL, if required.

Signup now (no credit card required) and we will be in touch with you.

What does end-of-life for Magento 1 mean?

Magento 1 End-of-Life does not mean your website will stop working. It means Adobe will stop giving fixes for Magento 1, even security patches. As php version in use goes out-of-life, no upgrades will be given by Adobe.

However, being an open source platform, your Magento 1 website will not stop working. The code and license do not restrict you from running the website.

Stay on Magento 1 for short or even long term

That is a valid option and many customers are choosing this. Makes sense if

  • You have a lot of investment in the customizations which may be difficult to replicate anywhere
  • You have a stable money generating store and any change looks like a risk
  • Are in the process of migration, but the migration may take some time

What are the options to stay on Magento 1?

  • Use paid support plan from Mage one (https://mage-one.com).
    luroConnect is a partner and we will apply the patches for you as they are released.
  • Use open source Magento 1 fork (https://github.com/OpenMage/magento-lts) with support from the community.

What are the risks?

  • Support from either of the above reduces over time as many websites move out of Magento 1
  • Developer support may reduce as most developers move to Magento 2
  • Plugin vendors have already stopped support or are stopping support.

Magento 1 and PCI

Many merchants received email from or read their post and advice move to another platform after June 2020.

It refers to PCI / DSS Requirement 6 – excerpted here with highlighting for relevance of discussion.

Your Magento 1 store software has many vendors, including Adobe / Magento for the core, but also plugin vendors. Since it is open source, you are free to modify the core and take the responsibility and other requirements may apply.

By switching to an alternative vendor for the Magento 1 core – such as Mage-One or Open Mage, in our non-legal opinion, you are not on Magento 1 any more and do not have Adobe as your vendor. If a plugin vendor does not give security patches to your Magento 1 plugin any more, it is important to take over the plugin code responsibility as a separate contract.

PCI does not have a vendor approval process. However, your vendor may need to justify satisfying some other requirements for safe and secure coding practices.

However, by not recognizing your core application, you may need to talk to PayPal as a merchant to get PCI approval. This would include scans.

luroConnect Support for Magento 1 past EOL

We have built add-on package for Magento 1 EOL support. We have a 4-point plan to support you.

From EUR 29

Mage-One Patches for post EOL Magento 1 support. Will satisfy PCI vendor requirement.

mage-one logo

From EUR 99

Sansec eComscan examines a store for malware, vulnerabilities and unauthorized accounts. Written by well known security expert Willien De Groot, it scans files, databases and 3rd party components of Magento.

From USD 50*

Inbuilt into our Nginx, with M1 rules, protects from OWASP Top 10, with the ability of virtual patching.

From USD 50*

Staging environment to ensure patches are tested before taken live.

* This is in addition to our fees for WAF and staging environments if not included in your support plan. Customer pays for hosting costs of staging server.

How we protect you

  1. File system security to prevent 0-day or new unknown vulnerabilities. Strict file and folder permissions prevent uploads to folders that execute code
  2. Support for Magento 1 Nginx rules not allowing execution of php from skin and  js or php from media. This rule will prevent many malicious code to fail as they depend the ability to upload malicious code and execute.
  3. WAF – Web Application Firewall – with strict Magento 1 rules. This prevents SQL Injection and cross site scripting related attacks from being allowed.
  4. Virtual patching – block URLs that are known to have vulnerabilities. For example, we do not allow saving of the “miscellaneous” header and footer section from being written from the admin login.
  5. Partnership with mage-one to get the latest patches and keep your site uptodate.
  6. Admin login protection via dual password. The first is a basic http challenge. This reduces password guess of the admin URL as 2 passwords have to be guessed.
  7. Password guess prevention by restricting how many failed attempts are allowed in a day from the same IP – implemented at the application server level without changing Magento code.
  8. Staging environment to test patches from open mage or mage one or any other source you may have. Also support php version upgrade first on staging before upgrading production.
  9. Protect source code by using secure deploy process
  10. Secure backup With a proven restore strategy
  11. Support for our secure deploy process that ensures 0 downtime during code deploy and not have git folder in the hosting folder. An ability to rollback by switching to any previous deployed version is an added advantage.
  12. System components upgrade – including php. As versions of php approach their security end-of-life and support for higher versions appear in patches, php version will also be upgraded.
  13. Partnership with Sansec for their eCommscan scanning product.

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.
Click here to schedule the call

Hosting help moving to Magento 2

When moving to Magento 2, to reduce the downtime during the move, luroConnect has plans for you.

  • Staging server support plans.
  • Magento 2 transition plan with minimum downtime. Our care even includes URL rewrite rules to ensure SEO value is not lost during transition.
March 6, 2020 By hosting, Magento, Security 0 Comments Read more...