Magento

Should I move to shopify from Magento?

The platform battle of Magento vs Shopify rages on. Increasingly merchants are moving to Shopify from Magento due to easy of operations and website management.

For many, where the decision was a no-brainer, the move has already been done.
For others, this question rages on, as shopfiy adds more features from itself and its partners in an easy to manage.

This article is written in 2022 for the “others”.

The “others” I refer to have some of these characteristics

  • A working eCommerce business – one that makes money, increasing the risk of a move
  • Migrated to Magento 2 and hence have invested in the technology
  • Customizations that differentiate the store and would be difficult to achieve in Shopify. These may include custom attributes, use of custom options, personalization technology, etc.
  • Headless frontend
  • Worried about data ownership

What are the pain points in Magento that makes merchants look at shopify

  • Managing the Agency or Magento Agencies are expensive!
  • Website speed scores or Magento website speed scores can be bad.
  • Everything is so hard in Magento compared to Shopify

What options you have to ease the pain and stay on Magento?

  • Managing the Agency.In my conversation with merchants unhappy with Magento, the main reason to consider a move is the performance of the Agency – i.e. they have an agency problem, not a platform problem.
    • Agencies have powerful marketing presentations that appeal to the heart of an eCommerce merchant. But, a merchant should pay attention to the compatibility between of the agency to how the business operates before choosing one. Infact, it may be even desirable to manage multiple agency relationships.Some factors to consider.
    • If you have an in-house project manager, you need developers – an strong in development. One of our customers has 2 agencies – a cheaper local and a more established one. Depending on the complexity of the project, they decide which agency will be best. They have their own project manager.
    • Some agencies have a delivery manager that acts as a single point of contact. They work with you on your requirement, discuss with the developers and are responsible for the delivery. Ensure they communicate with all teams – including business and SEO.Some agencies offer a full eCommerce service – including SEO to development. These are very suitable for growing eCommerce arm of a omni channel merchant. Such agencies can work on outcome basis – with a profit share for example.
    • A point of conflict between merchants and agencies is the business model – Agencies like an hourly billing model as it closely matches their biggest cost – the manpower used to execute the project. Many merchants may think of a fixed cost model best suits them – with outcomes well defined. Software on the other hand is very difficult to estimate, especially in an agile development world. Merchants wanting a fixed cost model should be more acceptable to cost escalations due to change requests.
    • Use an expert to interface with an agency. If you have a complex project and do not have an inhouse eCommerce manager, it will be a good idea to hire an consultant to interface with the agency. The consultant can act as a project manager communicating the business needs to developers and ensuring the delivery meets the quality goals.
  • The user interface. From a competitive perspective, UI is the most crucial part that can help you differentiate from your competitors. A more personalized website or an emersive experience will attract more visitors to stay and purchase.But, magento websites are slow on page speed scores, especially those developed on the original Magento 2 theming, which is archaic by todays UI technology standards.The Magento UI stack is evolving. Apart from the many headless options, Hyvä Themes has completely rewritten the theme to be fast and modern.Infact, in my opinion, UI should be the main reason to stay on Magento – the ability to give the best user experience. Rather than have a single page speed goal for the website, have a more user experience view. Even google suggests to have targets for faster website and work towards achieving them.
  • Ease of Shopify vs Magento. A big complaint against Magento is its bloat – too many features, not many, easy to use. A customer recently made this comment when they were trying to enable the watermark in Magento. Turns out, while the watermark feature is shown in Magento’s admin dashboard, enabling it will immediately remove all images – until the developer had accounted for this in their development. The feature bloat meant the developer did not really understand why it would not work. While this feature just works out of the box in shopify.

Magento has a community

Magento has over the years grown to have a very vibrant community. Some estimates put it at over 300,000 strong. Mainly tech professionals, it also consists of many areas of eCommerce business.

For some merchants, it is important to know they (or their online business) is not beholden to a single corporation. A Shopify network for example, though very cohesive, is orchestrated by Shopify. Shopfiy does whet these vendors, but there is no assurance of quality given by Shopify on their behalf. Moreover, Shopfiy can decide to compete with one of the features with any partner in that network, making your favourite plugin obsolete.

Magento on the other hand has independent plugin developers, some of who have made a name for themselves. The agency and merchant have the task of ensuring every plugin they use is good in terms of quality and functionality.

Migration to shopify may not be all roses

On the other hand, shopify’s ease of administration can get in the way of large changes. A process has to be used including using staging site for UAT, git and tools like ThemeKit, Beanstalk and/or the shopify github app. Which brings back the agency and project management question.

Getting feature parity with a Magento website may also not be easy. One merchant moved to Shopify and lost some upsell opportunities as they had used custom options in Magento. They commissioned some custom development and realised that the agency would launch a server and store some data there. This added to a vendor lockin (their agency) they did not want.

luroConnect easing the pain of Magento and Headless/PWA

luroConnect is a managed hosting platform for eCommerce – including Magento and Headless eCommerce.

We believe in serving 2 stake holders in an eCommerce website – the developer / agency and the merchant. We have tools and features to help both. To remove the pain of taking developer code to production. luroConnect supports dev, staging and production environments, with an inbuilt CI/CD for all platforms we support, with low downtime deploys.

The luroConnect stack deploys on the cloud of customers’ choice. The production site is backed by a performance dashboard, configurable alerts and 24×7 support.

Our extensive agency network helps us match a merchant to an agency.

Considering moving from Magento to Shopify? Talk to us. We may just give you a reason to stay back!

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.

Magento and graphql : A slowdown saga

PWA, Magento and graphql : A slowdown saga

Note : I write this article as Magento 2.4.5 is released and fixes some major performance issues that have resulted in the dream of headless and PWA go sour for many early adopters of technologies like PWA Studio and Vue Storefront 2.

PWAs and headless came with the promise of fast navigation and have had a rough ride at best, especially with Magento backends. Early PWAs like Vue Storefront 1 were implemented using Magento Rest API with a api middleware in nodejs that used elastissearch to store cached version of magento catalog for faster access. Magento implemented Graphql to avoid the need for the rest api. Use of direct rest api is ill-advised without a middleware as it exposes a Magento admin token. A middleware would sanitize the request possibly by checking a cookie and also limit the range of queries run. Graphql prevented the use of admin tokens and instead relied on frontend cookies for authentication.Magento’s REST API also results in large datasets. Graphql has a powerful, standardized query syntax to theoretically present a path breaking solution to build Magento headless around. However, early implementations did not fully cover the entire magento data model – Magento even advised the use of REST api when graphql was not sufficient. And when real world websites with large builds and catalogs were deployed, performance issues emerged.

Graphql are ajax calls in Magento that use sessions. In a typical hit processed by Magento, a session is opened, the hit is processed and then the session is closed. (A hit here is either a frontend hit or an ajax call).

Sessions and race conditions

Graphql are ajax calls in Magento that use sessions. In a typical hit processed by Magento, a session is opened, the hit is processed and then the session is closed. A hit is either a frontend hit or an ajax call.

Magento can store information in sessions – cart data for example. In order to prevent 2 hits from writing contents to the session simultaneously – called a race condition, sessions are locked for the duration of the processing a hit. (Take for example a checkout flow and let us assume that the address and payment info is written to session. You would want to ensure both get written. If both “raced”, only one piece of information would be written). Magento identifies an internal session by setting a cookie in the visitors browser. Each hit carries the cookie allowing magento to identify the session. Since a session is unqiue for a visitor, so Magento can process process only one hit per visitor at any given time.

With ajax in general and graphql in particular, many simultaneous hits may go to the server. Magento, through the session locking mechanism, queues them up. The hits are not guaranteed to be processed in the order they were received or any order in particular.

A typical session content in redis looks like

hgetall sess_eba681783f7000bc668cc45005aeaca4
1) "lock"
2) "0"
3) "writes"
4) "5"
5) "data"
6) "_session_validator_data|a:4:{s:11:\"remote_addr\";s:14:\"116.58.200.181\";s:8:\"http_via\";s:0:\"\";s:20:\"http_x_forwarded_for\";s:0:\"\";s:15:\"http_user_agent\";s:117:\"Mozilla/5.0 (Linux; Android 10; RMX2030) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Mobile Safari/537.36\";}_session_hosts|a:1:{s:10:\"host.com\";b:1;}default|a:0:{}customer_base|a:2:{s:11:\"customer_id\";N;s:17:\"customer_group_id\";N;}checkout|a:0:{}"
7) "wait"
8) "0"
9) "pid"
10) "10.0.2.197.host.com|3735"
11) "req"
12) "POST host.com/graphql"

Most of these entries are used internally by the redis session module. Entry 6 is the main session, stored in php serialized format.

If each hit takes only a few milliseconds and the number of ajax calls are not high, the overall response time may not perceptibly slow the site.

But, if even one ajax hit takes time, the page load can slow down. If a customer opens multiple tabs or refreshes the page, the slowdown may be more severe, sometimes leading to errors.

Here is an example. We are analyzing server logs based on IP. Lines 1 and 2 indicate the same link in google was clicked twice – possibly in 2 tabs on the browser. As the page loads we see graphql’s being loaded. The last 2 items are an indication of the problem.

The 2nd last hit is the slow hit – it took 10 seconds to return a result as the item was not in cache. The problem in the last hit was that it was waiting for the 2nd last hit to be processed, although in all probability it was in cache. If the number of items in this queue grows above 6 – for example if a visitor opens tabs and accesses urls not in varnish, a 500 error may be returned, possibly resulting in the UI showing a block not fully loaded at all. A refresh typically seems to solve the problem.

Server can run out of php resources

When too many graphql calls wait, server side can run short of php resources. This time you can see 504 and 502 errors. Adding more php resources for the same server risks running out of RAM. Adding more servers does help with the problem, but that is a waste of resources.

Graphql and caching issues

Graphql in Magento needs many caches to perform well. However, upon cache clear the “galloping horses” problem can bring down a busy website.

Upon cache clear the first access to graphql results in the schema being read. The performance for this action has improved substantially since Magento 2.3, but we yet highlight it as a concern. A cache warming of this is preferred.

Queries like filter in a category page can be slow and hence caching and warming some queries is always a good idea.

Enabling varnish cache is also important – varnish does not use sessions.

Recommended way to use Magento graphql

Graphql needs authorization – it needs to know what customer is asking for the information. It needs it so that the results include say for example the correct tiered pricing for this customer. While sessions include this information, they also store state and hence need the protection of a lock.

Magento supports JWT customer tokens for graphql.
https://devdocs.magento.com/guides/v2.4/graphql/mutations/generate-customer-token.html

Use the generateCustomerToken mutation to create a new customer token.
Example :

mutation {
  generateCustomerToken(
    email: "bobloblaw@example.com"
    password: "b0bl0bl@w"
  )
}

Response

{
  "data": {
    "generateCustomerToken": {
      "token": "ar4116zozoagxty1xjn4lj13kim36r6x"
    }
  }
}

Magento suggests NOT enabling cookie and JWT token authentication at the same time to reduce the chances of encountering problems caused by the differences between the two authorization methods. However, disabling cookie was added a feature only in Magento 2.4.5

However, Magento 2.4.5 varnish sample configuration disables varnish caching for logged in user. This can reduce performance.

Conclusion

Magento performance continues to be a troublesome aspect of the eCommerce platform. At luroConnect we look at performance in great depths.

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and many scaling options to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for eCommerce – specifically Magento and various PWA / Headless technologies.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimize hosting costs

Sansec reports new Magento 1 hack

Over the weekend of Sept 11, 2020, Sansec reported a web skimming attack on Magento 1 stores. It was the largest single day automated attach recorded by Sansec.

What are skimming attacks?

“Skimming” attacks are malicious code added to your website so when a site visitor is entering any personal information including credit card, the content is “skimmed” and sent to the attacker.

The website looks completely normal.

In September 2018, British Airways revealed that 380,000 passenger information had been skimmed from the website. The modus operandi for this attack was access to the code (possibly the version control of a 3rd party javascript module used on the website). The attack went undetected for months.

Some attacks are also called “Magecart” attacks.

In Magento we see 2 popular ways

  • Break the admin password and upload content to “Miscellaneous Header” or “Miscellaneous Footer” sections.
  • Upload a php code file which in turn loads the real malicious code in javascript in the page – either directly into the page or modify a known javascript file.

The current attack is of the second variety.

Stores on luroConnect were not attacked!

The attack as described by Sansec in the article used the Magento connect to bypass Magento admin and upload malicious code in javascript files to facilitate skimming of credit card information.

luroConnect has many rules that helped prevent this attack from affecting any of our Magento 1 websites.

Rule 1 : “/downloader” URL  is not accessible on any live or staging website. We expect code to be deployed through git and expect the developer to use a manual process to install modules. We disallow magento connect based installation in any of our managed websites.

Rule 2 : Our web directory owner and hosting users are different. Hosting user is the user php code runs as. Moreover, /skin folder is not writable by the hosting user.

Rule 3 : We use a static minifier and deploy the code to a folder skin.min which is not in git. The /skin folder itself is never used.

Rule 4 : Staging and dev environments are protected using a HTTP Basic Authentication. Automated attack vectors would need to add a password guesser before they can reach the staging URLs. This is assuming a developer would have relaxed permissions in the dev environment.

Rule 5 : Our platform bars ssh access to the hosting user. This prevents any accidental change in permissions being permanent. Even in the rare case ssh access is given (for debugging purposes), upon relinquishing the access, we sanitize the environment with default permissions.

How to protect your store?

One of the best ways is to sign up for Sansec’s security scanner eComscan

luroConnect is a very secure platform for Magento hosting. We call it layered security – from a secure file system and strict folder permissions, to an inbuilt WAF with configurable rules to partnering with Sansec for security scans.

We host you on your cloud or physical hardware using our stack. Learn more about our plans here.

Supercharge your Magento with a Varnish cluster

What is a varnish cluster?

A varnish cluster is a set of varnish nodes, each in a different geographical location, in front of the same Magento backend.

As shown a Magento hosted in the US East region can serve varnish nodes from across the world. Access to the website from each region is directed towards the nearest varnish, benefiting from lower latency and faster page loads.

If you serve customers in different regions – internationally or across the USA, your store can benefit from a varnish cluster.

Varnish and Magento performance

Magento 2 was architected to work with Varnish for improved performance. A typical webpage – category listing or product detail page – when returned from a cached varnish page (called a HIT in varnish) typically has a TTFB of a few milliseconds. An uncached page (also called a MISS in varnish) results in TTFB of a few seconds. Optimizing a MISS is very crucial, but we will not cover that in this article. We believe varnish is one portion of optimizing your website.

What is Network Latency

Latency is the time it takes for a network packet to go from your computer to the server with a request and come back with a response – assuming the response from the server was available immediately. When a page scores a HIT in varnish, the response is almost immediate from the server. Any  Time To First Byte (TTFB) recorded on the browser can be attributed to network latency.

If you market your website to many regions around the world, and you host varnish in a single location, your visitors may be faced with higher latency. When latency of access is in a few hundred milliseconds, it becomes the bottleneck and needs optimization.

The below information from https://wondernetwork.com/pings/ gives an idea of expected latencies. To read the table use the row header as source and column header as destination. So, a ping latency from Los Angeles to Mumbai would be 267 msec.

What is the cause of latency?

Core latency is a function of distance – even light will take ~40msec to travel 13,000 km – the distance from New York to Mumbai. A network packet travels through wires at that speed, and network wires are much longer than a straight line. Moreover, a response packet has to travel all the way back.

Latency is also caused as traffic goes through network equipment. The number of switches / servers a packet has to go through depends on the network and service providers – yours as well as at the server you are connecting to.

A varnish cluster architecture

Using a GeoIP based DNS service such as AWS Route53 a users request for a domain is redirected to one of several IPs. The Magento backend is in the “default” region – where maximum traffic is expected. A varnish in the regions desired – shown US East and Europe below.

Geo-IP based DNS

A geoip based dns router such as AWS route53 can help direct traffic to the nearest varnish node based on the guessing the country the IP requesting name resolution is coming from. So users on a browser say in Australia would be directed to be served from the varnish node in Australia and one from the west states of California would be directed to the varnish there.

Since IP to region or country can never be so accurate, it should be possible for all varnish nodes to serve customers from any region. Specifically, a language or currency switcher should be available on the website.

Magento 2 supports multiple varnish nodes out of the box

An important feature in using a cache in production is its need to automatically and quickly clear contents on demand from the user and application. For example, when a product goes out of stock, the corresponding page should display the out of stock label. Magento uses a tag-based system to flush appropriate content from the cache. Magento allows setting up multiple varnish hosts and a tag-based cache clear is sent to all the hosts.

bin/magento setup:config:set --http-cache-hosts=<varnish internal ip>:6081,<varnish internal ip 2>:6081

Refer : https://devdocs.magento.com/guides/v2.4/config-guide/varnish/use-multiple-varnish-cache.html



Challenges in a varnish cluster

A varnish cluster is more complex to manage

Ensure the varnish vcl files, front end security configuration (WAF, rate limiting, etc) is managed and kept in sync on all edge nodes.

Managing includes monitoring to ensure none of the servers go down. Now, your site can be down in a specific region for example. Typical monitoring tools such as Pingdom would not work. A purposeful monitoring solution is needed.

A varnish cluster costs for additional servers

Since these would be frontend servers, the amount of RAM and their network speed requirement would depend on what traffic they get.

Number of varnish nodes

Increasing the number of nodes in a varnish cluster does not always help in improving site speed. That is because each varnish node has a different hit ratio. A lower hit ratio leads to more users getting the latency and performance penalties combined – due to a varnish MISS. Traffic pattern and latency have to be taken into account to decide on how many nodes to use in a cluster.

Difficult to warm the cache

Given the distributed nature of the cache, warming each cache independently takes more resources on the server side as well as some changes to the way a cache warmer works.

luroConnect : A modern cloud hosting platform

Schedule a call of a free evaluation and demo!

  • Is horizontal scaling manually or with autoscale right for you?
  • Evaluate if a varnish cluster will help your website performance
  • Show managed dev, staging and production environments
  • How we measure application performance every minute with alerts for a slowdown
  • Can your hosting costs be optimized?
  • How improved hosting can lead to better ROI!

On Magento Cloud? We have special offers if you switch your enterprise license to luroConnect managed AWS cloud.


Magento Cloud and Fastly

It is important to discuss the Magento 2 cloud decision to use Fastly as a frontend.

 

Magento 2 cloud pro version architecture is given below. (Reference : https://devdocs.magento.com/cloud/architecture/pro-architecture.html)

The architecture uses Fastly as a Full Page Cache. Varnish is not installed on the Instances in AWS.

Fastly is a CDN that uses varnish. With the Magento 2 Cloud integration, a custom plugin is used on Magento along with a custom vcl file that runs on Fastly.

Fastly has many “POP” locations. As per fastly documentation, there are 20 POP locations in North America, https://www.fastly.com/network-map

mostly in the USA. Each has its own varnish cache. When a page is not in a POP, it fetches content from the origin. A single page may have to be rendered 20 times for each POP location in North America.

Drawbacks of this architecture from a varnish cluster perspective

  • More POPs do not lead to better experience as a higher percentage of MISS on varnish results in a worse experience for more users.
  • As POPs increase the load on Magento infrastructure increases.
  • It is not possible to use a cache warmer to warm the Fastly cache.

What next?

An ideal situation would be a layered varnish configuration – each “satellite” varnish node serving a local user, caching a subset of the “main” varnish node, reducing the penalty of a varnish MISS.

Share your thoughts here or on social media.


Choosing Hosting platform for Magento : Digital Ocean

One in a while you may want to check on the technology and options available for Magento hosting – technologies change, hosting costs reduce, life becomes easier for you and the merchant. In this multi-part series we will explore cloud platforms and their suitability to hosting a production Magento website.

In this article we will explore how seriously you should consider Digital Ocean as a platform for hosting.

Digital Ocean (DO) offers VMs it calls “droplets”. With an excellent blog and a friendly attitude towards developers, DO is a serious hosting contender. Many developers naturally recommend using DO to merchants for hosting Magento. How real is it? Let me explore a few pros and cons.

DO Choice of vCPU and memory

DO offers “shared CPU” and “dedicated CPU”. For eCommerce hosting we always prefer dedicated CPU. Shared CPUs work differently on all cloud providers and quite often when you need it the most, you do not get enough. Each vCPU is a Intel hyper-thread. As of this writing (July 2020), we see use of Intel Xeon Gold 6140 @ 2.30 GHz with DDR4 memory at 2.6 GHz. Our simple memory speed test showed about 1.8GHz effective throughput to memory.

DO disk : Love / hate relationship!

I love the disk speed – you get good SSD performance with no throttling. Even for attached storage. Unlike other cloud platforms, you don’t have to juggle with figuring out how many IOPS you need, once you understand for that platform how IOPs translates to speed.

To test the disk I use a simple but effective method to test the disk speed. I create a file with /dev/zero using dd of 1GB. dd gives me the write speed. Here is the command I use :

dd if=/dev/zero of=$tempFile bs=1M count=1024 conv=sync oflag=direct
tempFile has the path to a temp file in the mounted disk I am testing

For both direct disk as well as mounted block device we see 200-350 Mbps disk speeds across all droplets our customers use. This is the best we have seen in cloud platforms. Physical hardware can give upto 450Mbps speeds.

So, why the hate?

We have seen disk crashes – thankfully on staging servers. So, when it comes to production servers, we always recommend customers to have a DR plan to minimize loss of data when this happens. Our suspicion is that the storage is not in a managed RAID, hence a disk crash is a droplet specific event.

Network

File transfer speed test

: A large file transfer using scp on the internal network is done in about 130 MB/sec – about a 1 Gb/sec speed.

NFS

NFS of block storage performs poorly. We use nfscache so most reads are served from cache. However, if there is a need for a large number of reads or writes, the performance can drop dramatically.

Examples of NFS hurting performance

Magento 1 : As images are created in the frontend, there is a check to see if an image file exists before an image url is included in a page. This invariably results in slowdown.

Magento 1 and 2 : large log files. Magento stores log files in a shared folder. Multiple lines per hit of logs will result in slowdown.

Experience :  We were in the process of taking a Magento 2 website live that had about 2500 errors written to a log file. The issue was related to M1 migration resulting in some attributes not defined in M2. App servers saw 30% CPU in I/O wait and the site came to a crawl. Php access logs showed under 10% CPU utilization.

Non availability of autoscale

DO does not come with an autoscale option (you could get autoscale in their Kubernetes which we are not considering here). This means you may have to keep capacity for a holiday sale for example.

Managing server costs

Server costs even when you shut them down

DO charges for servers that are shut down. If you do not want to be charged fully for a server, storing an image is an option.

Recommendation

We host customers on Digital Ocean if they do not have a need for scaling and are willing to have a DR plan. This generally increases the cost of the overall solution.
Also stores that do not need scaling such as :

  • PWA – Vue-storefront based stores which use nodejs
  • A high hit ratio varnish FPC magento store (depends on many factors, but essentially requirement for scaling multiple app servers is less likely)

 

How Magento can get near 0 downtime deployment

Factor III of the 12 Factor App says “Store config in the environment”.

12 Factor App is what devops lives by – a set of 12 principles written by Adam Wiggins for predictable web app deployments.

Storing configuration in environment, separate from code has the advantages of reliable deployment along with reduced time to deploy. It allows separation of the build stage from the deploy stage, with some deploys being just a change in a softlink to the web root folder.

Historical preview : Magento 1

Magento 1 did not have much of a build process – js and css were not versioned, magnification was “online” first access based as was database upgrade information, configuration was stored in the database.

The most reliable way to go from a dev configuration to a live configuration would require a set of known steps that would work or changes directly to the database.

luroConnect developed its own build and deploy process. In our build step we

  • get source code from git
  • minify css and js files in the skin and js folders using a grunt based process
  • set appropriate file ownership and permissions

During the deploy phase, we

  • Copy app/etc/local.xml from a secure deployment configuration area (our environment)
  • modify the core config data to add a version string in the skin and js URLs
  • access the website once through the index.php to cause the update scripts to run

Deploy process is of course run with the site in maintenance – we prefer to do this at the nginx level. Mostly it is a small blip.

Historical preview – pre Magento 2.2

Early Magento 2 builds were similar – except there was some help from the bin/magento command. Our deploy process did not need to version the static access anymore. Plugin enable / disable was given via config.php. Our deployment environment contained env.php.

However, developers had to manually configure and experiment with some options.

Site bringup required devops to access the admin panel or update the database with custom sql – enabling varnish, setting up CDN with a static URL, etc.

Magento 2.2 and beyond

Magento adopted the direction of the 12 factor app and presented in Magento Live UK 2017 a new set of features that would help in ensuring an ability to split the application configuration and environment configuration. Application configuration was defined in app/etc/config.php which is advised to be in git and hosting environment and secure details are kept in env.php which should not be kept in git.

It is a slightly weak conformance – as commented by 12factor app “This is a huge improvement over using constants which are checked into the code repo, but still has weaknesses: it’s easy to mistakenly check in a config file to the repo; there is a tendency for config files to be scattered about in different places and different formats, making it hard to see and manage all the config in one place. Further, these formats tend to be language- or framework-specific.”

Magento has fixed this in 2 ways

  1. The language specific aspect is addressed to some extent in Magento by allowing to use bin/magento cli to edit env.php for sensitive data. The config:sensitive:set directly writes to env.php. These commands no not require the database, hence, can be set in a pre-deploy step.
  2. Use of scoped environment variable names. These would be set in Nginx configuration or an include file such as fastcgi_params.

However, there is no documented way to set database details – except to manually edit the env.php file.

The app:config:dump command

A great help in maintaining a known configuration of the application (which 12factor app suggests be committed to git). This ensures communication between developer to operations.

The app:config:dump command writes to config.php and env.php. While config.php is suggested to be committed to git, env.php should not be committed to git.

If a value is in config.php, the Magento admin panel does not allow the parameter to be edited. This locking helps with giving stability to the application configuration. It ensures the application is developed and tested with a known configuration.

The figure alongside shows the suggested flow.

Suggested flow for using app:config:dump

Why is Magento deployment yet keeping site in maintenance?

However, we find that even after 2 1/2 years of announcement, the acceptance and understanding of these features is weak. Leaving websites in maintenance mode as code is deployed.

Developers are failing to maintain a discipline to own the configuration or devops to understand the application’s build and deploy process.

There are some practical problems as well. An eCommerce manager would like to have control on the live website on say, when backorders would be allowed storewide. Since this is locked in config.php, this request has to go through developers or devops.

luroConnect near 0 downtime deploy

luroConnect’s Magento 2 build is in a pipeline – such as a bitbucket pipeline. A commit triggers the pipeline that does the following

  • composer install (with the compose cache to speed this process)
  • bin/magento setup:di:compile
  • bin/magento setup:static-content:deploy

The contents are then tarred and sent to the staging and production servers.

Upon deploy the contents are untared, deployment related files like env.php are copied, media and var are softlinked. The web root softlink is changed to point to this new release. The process is slightly more complicated when multiple autoscale instances are running, as running instances are replaced with ones with new code.

If required the bin/magento setup:upgrade command is run and only then is it required to keep the site in maintenance.

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and autoscale to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for Magento.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimized hosting costs

12 factor app and Magento

Adam Wiggins’ 12 factor app (https://12factor.net) is a highly respected standard for web apps. While written with SaaS applications in mind, let us explore and see how Magento and the ecosystem stands up to these factors.

1. Codebase. One codebase tracked in revision control, many deploys.
Magento is in git and hence a typical Magento project should not have a problem with this.
However, if you use vue-storefront, a popular PWA frontend to Magento, this is broken. Vue-storefront has 2 repos of its own in addition to the Magento repo, all becoming one app.
Another violation happens when a plugin vendor gets ssh access to your live server to fix a plugin issue. Plugin vendors have a serious problem integrating their code into multiple source bases without Magento supporting a versioned plugin architecture out-of-the-box.

2. Dependencies. Explicitly declare and isolate dependencies.
With composer Magento solves this problem.
Violation of plugins is a case in point – many plugins are installed not as composer dependencies. Instead they make it to the merchant repo.

Magento uses php and typical websites are deployed using php-fpm. One may argue that the php-fpm plugins that Magento depends on are not explicitly declared. Leading to the application not working exactly in 2 environments. Another case in point is dependency on php version.

3. Config. Store config in the environment.
12 factor app requires environment variables to be used. Magento has split application and environment configuration between config.php and env.php.
Here is what 12 factor says.
“Another approach to config is the use of config files which are not checked into revision control, such as config/database.yml in Rails. This is a huge improvement over using constants which are checked into the code repo, but still has weaknesses: it’s easy to mistakenly check in a config file to the repo; there is a tendency for config files to be scattered about in different places and different formats, making it hard to see and manage all the config in one place. Further, these formats tend to be language- or framework-specific.”

However, Magento has worked towards this. Specifically, with bin/magento config:set and bin/magento config:sensitive:set commands are a useful way for hosting providers to be 12 factor compliant.

luroConnect has always stored hosting configuration settings separately from the release. Upon deployment of code, the contents of deployment folder are copied. Sometimes they have settings for the application. These include hosting specific as well as sensitive settings. We are moving to using config:set and config:sensitive:set for versions of Magento that support it. We will also move towards storing sensitive variables in secure key stores.

4. Backing services. Treat backing services as attached resources.
“Resources can be attached to and detached from deploys at will.”

While Magento is very good at storing key connections outside the application and database, violations exist in 3rdparty plugins. To “ease” the deployment most store credentials and connectivity details in the database. Another issue is with SMTP plugins, instead of depending on magento’s default use of localhost and let postfix configuration manage the actual email sending, developers see the convenience of storing this information in the database.

Check out this post on SMTP and postfix configurations.

5. Build, release, run. Separate build and run stages.
Magento has been improving the code deployment process. The setup upgrade is the only command that, if needed, requires the site under maintenance.

6. Processes. Execute the app as one or more stateless processes.
Twelve-factor processes are stateless and share-nothing. Any data that needs to persist must be stored in a stateful backing service, typically a database.

Magento is very good on this count if used with nginx and php-fpm.

7. Port binding. Export services as port binding.
“PHP apps might run as a module inside Apache HTTPD” is flagged as a violation if the apache is also used as a webserver.
nginx + php-fpm gives the best isolation and performance of any stack. Php processes can be independently controlled in a server running php-fpm while nginx can be used for routing and handling web requests, terminating SSL, etc.

8. Concurrency. Scale out via a process model.
Magento is very good at this. Aided by php-fpm process model that complies with the 12 factor app, it is possible to build a cluster to handle only checkout urls for example, with routing handled by an application load balancer such as nginx.

9. Disposability. Maximize robustness with fast startup and graceful shutdown.
While Magento and php are good at this, some notes are in order.
A reload of php-fpm by default will kill all php processes even though they may be executing a request. Ensuring no new traffic is coming to the php-fpm, and waiting for draining by checking the status for number of active processes (with a timeout ofcourse) will ensure gracefulness in shutdown.
In order to ensure robustness against sudden death of the php-fpm process, it is best to keep the queue length (listen.backlog) to a small number. Turns out managing the queue to scaleout helps in application performance as well.

10. Dev/prod parity. Keep development, staging, and production as similar as possible.
The 12 factor app describes 3 gaps – time, personnel and tools. Based on our experience, the personnel gap is eliminated by automation. A commit trigger based automated CI/CD pipeline with an automated deploy to staging and production ensures there is no personnel gap.

A development environment with write access to git can be created with a similar infrastructure to help developers debug issues.

11. Logs. Treat logs as event streams.
Magento allows creation of multiple log files. Modern logging such as monolog allows more control of what is and what isn’t logged. Logs are also generated by nginx, php-fpm and other services used.
Streaming logs for querying and analysis is typically done by your hosting provider.

luroConnect uses fluentd to capture logs. Logs are sent to our Insight service, which analyzes data per minute, hour or day.

12. Admin processes. Run admin/management tasks as one-off processes.
Magento supports cron and rabbitmq based processes. In addition, setup upgrade is also used to change the state of the database during deployment.
However, suggested access to developer for “run arbitrary code or inspect the app’s models against the live database” is not recommended by luroConnect due to security and the risks of the application stability with the state being altered arbitrarily.

Magento emails & SMTP

Emails are a very critical portion of an eCommerce website. There are 2 types of emails that are used – transactional and promotional.

Transactional emails are emails the eCommerce system sends for regular customer activity – such as order confirmation email. It may also include abandoned cart email, wish list reminders, password change, etc. If a website visitor action causes an email to go out, it is a transactional email.

In this article we will discuss the best way to configure transactional email sending in Magento. This article applies to both Magento 1 and Magento 2. The focus is on what happens on production systems.

The SMTP Plugin

Traditionally Magento agencies and developers install a “SMTP Plugin”. A simple search in Magento marketplace for SMTP reveals many options. Some are general plugins that allow you to connect to any SMTP provider, others are from vendors such as SendGrid.

Advantages of such plugins are

  • Easy setup
  • Developer testing is easy
  • Websites on shared hosting have to use such a plugin

However, for growing websites, this is a particular problem

  • Outgoing ports from the application server may not be open
  • Many SMTP providers have restrictions such as rate limit on email sending or number of connections. Some SMTP plugins may not handle this condition well – just reporting the error vs trying to send again.
  • They may not be able to alert when email sending is down
  • When upstream providers’ SMTP service is down or slow, cron will slow down as each email times out on sending. (Magento sends emails in cron).

Using postfix

By default, Magento sends emails to the local system where php is running. Linux systems have many options for receiving such requests and sending out the emails. A popular software is postfix. While postfix can directly send emails to the recipients’ email systems, we are interested in the “relay host” mode of postfix. In this mode, postfix acts to relay emails sent to it from Magento to the upstream provider – such as SendGrid.

Advantages of postfix

  • Postfix has an inbuilt queue. When the upstream server’s rate limit is reached, a “defer” status is returned. Postfix understands this and it will queue deferred emails and keep retrying. Emails are not lost.
  • Magento cron will deliver to local postfix in a matter of milliseconds, freeing up cron to perform other activities.
  • Connections can be pooled so if many emails are in queue they will be pooled together and sent using a single connection. Some SMTP servers, notably gmail, have a rate limit on the number of connections.
  • Log files that can be checked by system admins. Log files can also be sent to a log collection tools. Advanced commands help checking the queue, even trimming it using conditions such as email subject.
  • Flexible configuration allows many features – on a staging server for example, only whitelisted recipients can receive emails, preventing accidental trigger of email to customers. Or emails to known spammers can be filtered by email id or domain.

Installation and configuration

Installing postfix is easy – the package is part of any linux system. For RHEL based system like CentOS :

sudo yum -y install postfix
sudo yum -y install cyrus-sasl-plain cyrus-sasl-md5

Configuring postfix to act as a relayhost.

Click on the tab below to see configuration information for the specific SMTP service being used. Commands below should be run as root (or sudo). Ownership of files created, especially the password files is assumed to be root.

  1. Sign in to SendGrid and go to Settings > API Keys.
  2. Create an API key.
  3. Select the permissions for the key. At a minimum, the key must have Mail sendpermissions to send email.
  4. Click Save to create the key. SendGrid generates a new key. This is the only copy of the key, so make sure that you copy the key to clipboard and use if for the next step.
  5. Create the hashmap from the key
    echo [smtp.sendgrid.net]:2525 apikey:{{{PASTE}}} >> /etc/postfix/sasl_passwd
    postmap /etc/postfix/sasl_passwd
    rm /etc/postfix/sasl_passwd
    chmod 600 /etc/postfix/sasl_passwd.db
  6. Add to postfix configuration file main.cf
    cat >> /etc/postfix/main.cf <<EOF
    smtpd_tls_security_level = may
    # enable SASL authentication
    smtp_sasl_auth_enable = yes
    # tell Postfix where the credentials are stored
    smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
    smtp_sasl_security_options = noanonymous
    header_size_limit = 4096000
    relayhost = [smtp.sendgrid.net]:587
    smtp_tls_CAfile =/etc/pki/tls/certs/ca-bundle.crt
    EOF
  7. service postfix restart
  1. Signup for maligun account.
  2. Get credentials for maligun. Mailgun has an option for SMTP username and password under the Domains section of the mailgun portal.
  3. Create the hashmap from the key
    echo [smtp.mailgun.org]:2525 {{user}}:{{password}} >> /etc/postfix/sasl_passwd
    postmap /etc/postfix/sasl_passwd
    rm /etc/postfix/sasl_passwd
    chmod 600 /etc/postfix/sasl_passwd.db
  4. Add to postfix configuration file main.cf
    cat >> /etc/postfix/main.cf <<EOF
    relayhost = [smtp.mailgun.org]:2525
    smtp_tls_security_level = encrypt
    smtp_sasl_auth_enable = yes
    smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
    smtp_sasl_security_options = noanonymous
    smtp_tls_CAfile =/etc/pki/tls/certs/ca-bundle.crt
    EOF
  5. service postfix restart

Amazon SES is the most cost effective SMTP provider to our knowledge. However, it is also the most difficult to setup as it requires you to verify sender emails and sandbox mode (which we will not consider here).

  1. Login to your aws account
  2. Select “Simple Email Service” from the services
  3. Select Domains from the left menu
  4. Click “verify a new domain”
  5. Enter domain and click on verify
    SES will generate the TXT records you need to add to your DNS

    SES will also generate the DKIM CNAME record to be added to your DNS
  6. Ensure domain verification green status
  7. In the SES main screen, select “Email Addresses” from the left menu
  8. Enter the sender email address. Sender email address are set in Magento. Depending on your Magento configuration different emails may need to be configured. Each email needs a mailbox as a confirmation email is sent to it.
  9. Refer https://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-credentials.html to create SMTP credentials. Note username and password.
  10. Depending on the region your SMTP endpoint (relayhost setting in the main.cf below) will be different. Refer https://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-connect.html
    Note : The region of SES is unimportant in that there are no restrictions from where the email is sent. Your Magento instance can be anywhere – not even on AWS.
  11. Save password
    echo [email-smtp.us-west-2.amazonaws.com]:587 USERNAME:PASSWORD >> /etc/postfix/sasl_passwd
    postmap /etc/postfix/sasl_passwd
    rm /etc/postfix/sasl_passwd
    chmod 600 /etc/postfix/sasl_passwd.db
  12. Update main.cf
    cat >> /etc/postfix/main.cf <<EOF
    relayhost = [email-smtp.us-west-2.amazonaws.com]:587
    smtp_sasl_auth_enable = yes
    smtp_sasl_security_options = noanonymous
    smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
    smtp_use_tls = yes
    smtp_tls_security_level = encrypt
    smtp_tls_note_starttls_offer = yes
    smtp_tls_CAfile =/etc/pki/tls/certs/ca-bundle.crt
    EOF

Advanced configurations

Postfix relay gateway

In a more mature environment, each php server may not be able to send on SMTP ports. Even in the presense of a NAT gateway, we would recommend a main postfix gateway to ease logging and prevent loss of emails when an instance is lost in autoscaling. With postfix we can create a relay – php servers are configured to send email to a designated email gateway server which in turn relays to the SMTP upstream server.

Sending promotional emails

Promotional emails should ideally not originate from your website – instead they should be from your marketing tools. However, Magento does support sending promotional emails. To improve deliverability of crucial transactional emails, we recommend using a different sender email domain for promotional emails. If this is configured in Magento, postfix can handle sending promotional emails from a upstream service different from the transactional service. Assuming username / passwords have been created in the SMTP service for the promotional domain, the following configuration allows selecting the relayhost to use based on the sending domain.

cat << /etc/postfix/main.cf >>EOF
smtp_sender_dependent_authentication = yes
sender_dependent_relayhost_maps = hash:/etc/postfix/relayhost_map
EOF

cat << /etc/postfix/relayhost_map >>EOF
user@promodomain  [smtp.xxx.com]:{{port}}
EOF

Add the new domains passwords as the first line in /etc/postfix/sasl_passwd

user@promodomain  {{user}}:{{password}}

luroConnect and Magento emails

The configurations described above are abridged versions of what is used in luroConnect. We use the advanced configuration with a designated server allowed to communicate on port 587 to the outside. We also use secure keys to ensure passwords are not saved in chef cookbooks / configurations. Instead they are stored in a secure key store.

Interested in knowing about the advanced architecture of luroConnect?

Fill the form below and we will contact you.


My company owns the Magento site
Yes, I am a developer
I represent a Magento Agency

Reactions to this post on linkedin

Magento 1 end of life

June 2020 is a few weeks away and many stores are not ready to move away from Magento 1. But wait, you don’t have to update – now or in the long run.

COVID-19 Update : We find many customers have had to delay their Magento 2 launch in these uncertain times. We also know many of them did not have plans to keep Magento 1 uptodate. In fact, we know agencies that have stopped support for Magento 1.

Starting at a low cost of $200 per month with no long term contract. It includes reviewing your current hosting for security, moving your website to the latest Magento 1.9 and latest php supported as well as adding additional security measures to your website. It also includes help signing you up for Mage One or Open Mage projects for support beyond Magento 1 EOL, if required.

Signup now (no credit card required) and we will be in touch with you.

What does end-of-life for Magento 1 mean?

Magento 1 End-of-Life does not mean your website will stop working. It means Adobe will stop giving fixes for Magento 1, even security patches. As php version in use goes out-of-life, no upgrades will be given by Adobe.

However, being an open source platform, your Magento 1 website will not stop working. The code and license do not restrict you from running the website.

Stay on Magento 1 for short or even long term

That is a valid option and many customers are choosing this. Makes sense if

  • You have a lot of investment in the customizations which may be difficult to replicate anywhere
  • You have a stable money generating store and any change looks like a risk
  • Are in the process of migration, but the migration may take some time

What are the options to stay on Magento 1?

  • Use paid support plan from Mage one (https://mage-one.com).
    luroConnect is a partner and we will apply the patches for you as they are released.
  • Use open source Magento 1 fork (https://github.com/OpenMage/magento-lts) with support from the community.

What are the risks?

  • Support from either of the above reduces over time as many websites move out of Magento 1
  • Developer support may reduce as most developers move to Magento 2
  • Plugin vendors have already stopped support or are stopping support.

Magento 1 and PCI

Many merchants received email from or read their post and advice move to another platform after June 2020.

It refers to PCI / DSS Requirement 6 – excerpted here with highlighting for relevance of discussion.

Your Magento 1 store software has many vendors, including Adobe / Magento for the core, but also plugin vendors. Since it is open source, you are free to modify the core and take the responsibility and other requirements may apply.

By switching to an alternative vendor for the Magento 1 core – such as Mage-One or Open Mage, in our non-legal opinion, you are not on Magento 1 any more and do not have Adobe as your vendor. If a plugin vendor does not give security patches to your Magento 1 plugin any more, it is important to take over the plugin code responsibility as a separate contract.

PCI does not have a vendor approval process. However, your vendor may need to justify satisfying some other requirements for safe and secure coding practices.

However, by not recognizing your core application, you may need to talk to PayPal as a merchant to get PCI approval. This would include scans.

luroConnect Support for Magento 1 past EOL

We have built add-on package for Magento 1 EOL support. We have a 4-point plan to support you.

From EUR 29

Mage-One Patches for post EOL Magento 1 support. Will satisfy PCI vendor requirement.

mage-one logo

From EUR 99

Sansec eComscan examines a store for malware, vulnerabilities and unauthorized accounts. Written by well known security expert Willien De Groot, it scans files, databases and 3rd party components of Magento.

From USD 50*

Inbuilt into our Nginx, with M1 rules, protects from OWASP Top 10, with the ability of virtual patching.

From USD 50*

Staging environment to ensure patches are tested before taken live.

* This is in addition to our fees for WAF and staging environments if not included in your support plan. Customer pays for hosting costs of staging server.

How we protect you

  1. File system security to prevent 0-day or new unknown vulnerabilities. Strict file and folder permissions prevent uploads to folders that execute code
  2. Support for Magento 1 Nginx rules not allowing execution of php from skin and  js or php from media. This rule will prevent many malicious code to fail as they depend the ability to upload malicious code and execute.
  3. WAF – Web Application Firewall – with strict Magento 1 rules. This prevents SQL Injection and cross site scripting related attacks from being allowed.
  4. Virtual patching – block URLs that are known to have vulnerabilities. For example, we do not allow saving of the “miscellaneous” header and footer section from being written from the admin login.
  5. Partnership with mage-one to get the latest patches and keep your site uptodate.
  6. Admin login protection via dual password. The first is a basic http challenge. This reduces password guess of the admin URL as 2 passwords have to be guessed.
  7. Password guess prevention by restricting how many failed attempts are allowed in a day from the same IP – implemented at the application server level without changing Magento code.
  8. Staging environment to test patches from open mage or mage one or any other source you may have. Also support php version upgrade first on staging before upgrading production.
  9. Protect source code by using secure deploy process
  10. Secure backup With a proven restore strategy
  11. Support for our secure deploy process that ensures 0 downtime during code deploy and not have git folder in the hosting folder. An ability to rollback by switching to any previous deployed version is an added advantage.
  12. System components upgrade – including php. As versions of php approach their security end-of-life and support for higher versions appear in patches, php version will also be upgraded.
  13. Partnership with Sansec for their eCommscan scanning product.

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.

Hosting help moving to Magento 2

When moving to Magento 2, to reduce the downtime during the move, luroConnect has plans for you.

  • Staging server support plans.
  • Magento 2 transition plan with minimum downtime. Our care even includes URL rewrite rules to ensure SEO value is not lost during transition.

Meet Magento India 2020 – panel discussion

I was on a panel at Meet Magento India 2020 #MM20IN moderated by Brent Peterson of Wagento.

It was an honor to be part of a elite panel – Miguel Ignacio Balparda of Nexess / Liquid Web, Shehzad Karkhanawala of Webscale Networks, Arun of ServerGuy. The discussion was lively and well received.

While the time was too short to address all the questions, Brent was kind enough to circulate the questions ahead of time.

Here would be my responses if there was enough time!

  1. What are some of the latest trends you are seeing in the hosting world
    Integrations into application – Magento Cloud as well as pwa vendors are giving hosting included packages.
    Plugins or apps are going SaaS.
    Underlying tech with Kubernetes is changing the hosting world.
    Magento hosting has to look like SaaS to the merchant – with the flexibility.
  2. Hosting as we knew it five years ago has now become a commodity. There are some interesting approaches to hosting in the market today. In your opinion, what defines and differentiates a great hosting provider for e-commerce?
    If you look at progression of economic value – commodities, goods, services, experience. A quick example being you could use ingredients to make a cake (commodities), you could buy a mix to bake a cake (goods), you could buy a cake from a cake shop (services) or you could have your birthday party organized of which cake is a part (experience). Hosting has reached the experience level with SaaS. We have to make that happen with Magento hosting.
  3. You all provide managed hosting services. What is different about managed as compared to regular hosting?
    Managing is offering hosting at a higher level of abstraction.
    Traditionally companies like Rackspace provided a server or a managed server and charged something like $100 per server as managed or support fee. In reality you paid for a system admin.
    When you give a full application level support, you are abstracting away everything.
    While many managed hosting services talk about 15 minute response time to tickets, we measure ourselves differently – we want no tickets. If a customer has to call and tell us their site is slow or god forbid down, we have lost.
    A managed hosting provider sits on the merchant’s side between the developer and the hosting service.On one hand, we ensure the hosting bill is optimized.
    On the other, we ensure we host a 12-factor app – keeping the developers honest about the quality of their delivery. We partner with many developers.
  4. What is the relationship between the customers and the hosting company in a managed service agreement?
    I will give you an analogy – a few years ago I built a house in Bangalore. I hired an architect and a contractor. I made the contractor report to the architect and paid the architect an additional 6.5% as fee to manage the contractor who managed the workers and material. 
    We are like the architect in this example – On the side of the merchant, making things happen on the hosting.
  5. What’s the difference between public and private cloud?
    everything and nothing!
    Private cloud has power, uptime, connectivity and flexibility issues.
    If the private data center addresses some of these issues, there is no difference from managed hosting perspective.
  6. All around the globe, there are big events like Cyber Monday and Black Friday. How do you help your customers prepare for this type of event?  Auto-scaling
    Autoscaling is an obvious, but there is more to the preparation.
    Autoscaling has a latency in scaling.
    We like to understand marketing strategy – if they are sending SMS – which generates a far higher peak than almost any other marketing activity – what time and volume – so instances can be launched We encourage staggering ofcourse but it is difficult to make marketing understand that. We would prep autoscale to have more instances.
    Reduce as many moving parts as possible.
    We ensure code freeze is well before the event.
    Avoid : Product uploads, inventory sync if possible
    Indexing state
    Cache state – ensure all caches are pre-loaded / primed
  7. What are the biggest differences between a hosting environment for Magento 1 and Magento 2?
    • Varnish.
    • core performance (or performance on a miss) is tough to get in M2.
    • Rabbitmq – but we have not seen mature use of this technology as yet.
    • Build & Release process.
  8. What’s your plan for the Magento 1 EOL in June?
    We are ready to support the hosting. Security is a key concern in the longer run, but we have to live with that – react as and when we know what happens. We have merchants that have not changed their code for years. Yet on old M1, php 5.6, etc. which we will not support.
    Others have invested a lot in M1 to move with feature parity. Features on the admin panel that their packaging staff depend on or rules to decide how to ship based on warehouses.
    (During the panel discussion participants talked about hardened php. That applies to php 5.x which means the merchants are running obsolete versions for a while. I think they should at least upgrade to latest Magento and php 7.2 )
  9. Some merchants are just not ready to move to M2 due to cost and timeline pressures.  Is there anything developers and their hosting provider partners can do to help clients continue to operate after June 2020?
    Absolutely. The numbers are so large that I feel a community effort to continue, if it happens, will be much appreciated. I think the hosting community is ready to support – many of us yet support php 5.6 for example as merchants have not recognized the need to move or have unsupported plugins that have not upgraded.
  10. Security has become the most critical piece of an e-commerce merchant’s infrastructure strategy. Cyber-attacks are growing in number, intensity, and sophistication. What can a hosting provider do to protect storefronts from threats?The biggest threat is from 0-day threats and not patching ontime.
    0-day threats are hitherto undocumented or unaddressed threat.
    In medicine, when a virus hits, there are 2 strategies – handling the virus directly coming up with vaccines, etc. The other is health related advice – such as wash your hands. Crucial things that sound simple, but are instrumental in preventing someone from being infected across multiple types of viruses.I call these fundamental truths. They address a class of problems.I was an engineer when first viruses came to infect 286 computers. My employer then decided to develop expertise in virus combat. We came up with a simple set of rules that prevented many forms of viruses then. How you booted mattered then for example. What you ran on your computer mattered. I wrote some code that my customer sold to Compaq and ended up protecting millions of computers.We try to make such fundamental truths in hosting. Some are taken into standards like cross site scripting. Brent – when you became a Magento Master a few years ago, you mentioned 777 permissions as a challenge. Magento came up with what they called 2 user hosting – a web server user and a command line user. Others we have developed.We try to take a class of attacks and eliminate them.Ofcourse when a known threat exists, we try to ensure that threat is addressed directly.
  11. What’s your take on the current state of Magento Cloud?
    We have typically  3rd party opinion but from what limited 1st hand experience we have, we find that to be having many restrictions. Autoscale, tuning caches like adding more RAM to varnish, are difficult. But, Magento Cloud is there and will improve. Adobe got SaaS wrong the first time, but persevered and fixed it to be one of the most profitable companies of our time.
    I would really hope Magento stops seeing Open Source as competition. The two depend on each other.