hosting

Enterprise grade security for your Magento website

At a time when data theft and leaks are more real than ever, don’t be the next one to explain the reason for a security breach. With threats growing in number, type and modality of attack every day, multiple layers of security are needed, so if one layer were to be breached, another would yet protect.

Introduction

Today, security of a Magento web server is a major requirement for the success of the business that runs on it. With threats growing in number, type and modality of attack every day, multiple layers of security are needed, so if one layer were to be breached, another would yet protect.
With attackers using automated scanners or bots, being small or infamous is no reason to feel secure. Attackers will probe for vulnerabilities and break in before deciding if you had anything of worth.

In this article we discuss various aspects of security and how luroConnect helps in getting your website an enterprise grade security.

Securing the Magento Application

Magento patch updates

Magento is a well maintained software. Magento routinely releases patches even for the community edition. Keeping patches uptodate is important. When magento releases a patch, while it helps you keep your site secure, it also announces vulnerabilities of an unpatched system to the world.
A major hurdle in patch application is when the Magento core files have been modified by developers. Developers do this in answer to high time and cost pressures or for not knowing better.

luroConnect service always patches customers at the earliest — if you have not opted out of the service.

Purchasing plugins from dependable vendors

While Magento is good at giving updates using patches, plugin vendors are not so motivated. Plugins are paid for once and quite often updates require one to be on a maintenance plan. So, while selecting a plugin vendor, keeping the vendors track record would be good idea in the future.

There are many areas where vendors fail to think about security and the following are but a few examples.

Magmi is popular extension for product upload. However, it is known to have security issues. The developer acknowledges this and has advised not using the web interface. In our research and experience, most Magmi users use their UI, leaving security holes open. luroConnect makes magmi installation more secure by providing htaccess based second login, securing the magmi installation with permissions that disallow updates to this from the UI as well as disallowing php execution from Magento’s media folders.

Some extensions require ability to write to folders and files that are unreasonable. With default access this seems to work but opens security holes. In our opinion rather than store plugin configurations in the database (core_config_data), plugins use files, sometimes generating php code. An example of a plugin (Aitoc_AdvancedPermissions) requires ability to change the etc/modules/Aitoc_AltPermissions.xml file.

Some extensions use encrypted code — typically requiring ion decrypter. Apart from being legally on the fence, the requirement of installing a decrypter increases the risk of encrypted malware, making it difficult to detect the existence of malware, even with disk scans.

PhpMyAdmin is another popular extension that makes a site unsecure. PhpMyAdmin uses the database username and password. If accessed from a unsecured url, the credentials of the database are transmitted over the internet in plain text form. Even with a secure URL, the database is left to be probed via username / password detection BOT that can try combinations.

Magento’s open architecture has made a marketplace for developers. Custom development is crucial to the Magento’s success. Custom development from vendors who are aware of security is also crucial. Some aspects that we recommend asking developers include awareness in use of form keys, sql injection and avoidance techniques — specifically validating all input parameters before processing and use of Magento’s model architecture vs writing sql.

Securing the Magento admin panel

Using host file entries instead of DNS update for non public sites

DNS entries are the starting point for all BOTs. Windows, Macs and linux allow an entry in the host file that helps the browser resolve a domain name (such as staging.domain.com or phpMyAdmin.domain.com) to a IP address. While not easily doable on a mobile for testing, we encourage use of host file entries to hide the actual domain name used.

2-password Secure admin access

Using htaccess (even in nginx) to protect admin URLs with a double authentication system, with both passwords being different. This thwarts most BOTs as they do not expect a basic authentication.

Changing the admin url, possibly once a year.

Magento allows obfuscation of the admin url. Selecting a non obvious url for admin and changing it as often as practical helps in preventing leakage of the url. In addition using a different admin url for development and staging compared to production will help keeping the URL on a need to know basis.

Managing roles in admin

Magento admin supports role based access. Giving roles on a need to know basis will protect against a threat popularly referred as “malicious insiders”.

Use of captcha in admin and end user login

Magento supports enabling captcha in admin. That ensures BOTs do not access the site and it thwarts possible username / password combination hacks.

Using strong admin passwords and change them often.

This is general advice for any password. Changing them often is also a general advice.

Hiding Administrators role.

Recently admin logins have been used to insert malicious code in header or footer “miscellaneous” html section. That has prompted us to advice to hide the administrator role – i.e. have no user with the permission to change that. When needed the role can be enhanced temporarily to access that setting.

Database containment and Filesystem Security

Limited privileges to write to code directories

System level file, directory (folder) and process permissions are crucial. A different hosting user and web server user with limited permissions to the web server user is crucial to protect code directories. The hosting user can update the code but the web user is not able — preventing any attempt to upload code.

Plugin and code updates only allowed via deployment process

The deployment process may be as simple as git pull (we prefer git stash before git pull) or a CI based mechanism such as Jenkins.

Readonly git access to live site

If git pull is used to update a site, it is important to ensure that the git user used has readonly access. Even using a CI ensure the git user has readonly access.

Configuration file protection against unauthorized changes

Ensuring site specific configuration files such as env.php or local.xml are in their own git and versioned ensures a protected master copy. Monitoring changes to these files and alerting would be a good way to ensure their security.

Only authorized access to code

The webserver should not have permission to write to the code directory. That requires plugin updates via download manager are done on a staging or development server and code moved via git. Ensuring automation of code deployment will ensure only controlled access to the code.

Watch our webinar on performance and scaling Magento

Its free!

Using analogy to vehicular traffic we explain performance and scaling in Magento.
Key takeaways

  • Know how to compare hosting options
  • Importance of good code
  • How to scale
  • Tuning Magento

Security at the edge — before traffic hits your application

Nginx has many features that allow a very decent edge security — including protection against basic DOS attacks. However, one has to understand the terse configuration options and deflecting a DOS attack may require some manual steps during the attack. There are more automated solutions that one can consider for edge security.

Rate limiting to block bots from accessing the site — with whitelist

Nginx supports rate limiting. Rates can be set for a subset of URLs. Whitelisting of IPs is also allowed so for example your own IP need not have the limitation. Rates are configurable and can be specified in hits per seconds or minute and are best tested for a site for a appropriate number, and then monitored. luroConnect / Insight dashboard shows all IPs that were blocked helping in deciding to either block the IP or add it to whitelist.

Automatic blocking of spambots by whitelisting bots

Nginx also supports an ability to block BOTs using regular expressions for the User Agent identify text each BOT gives. Note that User Agents are easily faked, so this is not a very reliable way of detecting BOTs.

Blacklist IPs from site access

Nginx allows blacklisting IPs from accessing a site. A more advanced geoip resitrction can also be availed. luroConnect / Insight dashboard gives top 10 IPs accessing the site on any particular day. This information can be used to decide which IPs need to be blocked. Note that attackers quite often do not use static IPs, so the banned list needs periodic review or release from blacklist.

Preventing php scripts from running in media folder

Media folders will be open to update and write by the web process. Restricting execution of php on these folders will prevent a malicious upload to execute.

Encryption of user data

SSL certificate

Being fully secure (using https over TLS) is becoming a requirement with Google’s ranking and expected indication. However, Magento has secure and unsecure URLs and by default all user data uses a secure URL. With LetsEncrypt project issuing auto renewable SSL certificates for free, there is no reason for a site not be fully secure. luroConnect service includes free LetsEncrypt certificates and auto redirect rules for http to https.

Securing backup

Security is as strong as the weakest link — securing the backup

Backup file transmission encrypted

Securing backup is an essential component of securing data. Using sftp or rsync over ssh one can ensure backup files are transmitted securely.

Backup server shut after backup — ensure the backup is secure

Backup data can be secured at rest. luroConnect goes a step further — backup server is different for each customer and run only when needed. The server is started before a backup starts and shut down at the end of the backup, ensuring its security.

Disk scans

Scanning disks for known vectors regularly is important.

OS Security

Patch updates

Securing the server is ensured by patching the server with latest security updates and ensuring only required ports are open ensures security.

Enable logs and monitor them

Logs for outgoing emails possibly with subject line, mysql, web server, Magento logs should be enabled and monitored either automatically and continuously or manual and periodically. For example a spam email hack into the site may have left a malicious script that will send out emails. These emails and subject lines will be in the mail log. A high count of emails for example can be a reason to suspect a compromised system.

Conclusion – Securing Magento requires many steps

With new and sophisticated attacks on the rise, having multiple layers is important — a layer may occasionally break but the harm may not come as another layer will kick in. Nothing less than an entire set of enterprise grade security will be needed to secure a Magento site so it is always up for business. luroConnect managed hosting gives you all this and more.

For a study of recent Web security breaches, read our article “Analysis of Recent Website Security Breaches

luroConnect is a managed hosting service for Magento, wherever you are hosted. Multi layered security is one of the aspects.

6 tips to speedup Magento websites

Introduction

Website speed is very crucial to conversions. There are many studies that indicate how website slowdown of even one second can cause drop in conversions. And websites that have taken this learning have seen conversions grow. Take Pier 1 for example in this article.

Our tips are oriented towards what you need to do on the server side to improve performance especially during high traffic times, when server response times tend to be inconsistent. These tips will speedup Magento websites.

Tip # 1 : Restrict Bots

When times are a busy, you need to be mindful of who you allow into your store. Restricting BOTs is easy to implement by checking the ‘user agent” field. With lowering of server load this will speedup Magento or help reduce resources.

Tip # 2 : Rate limit hits from the same IP

This is one more reduce tip that may work in your situation. Hits from the same IP – until your target audience is a group that is behind a firewall – will restrict simple robotic attempts to crowd your site – especially if you are running a time bound flash sale.

Bonus Tip : Use CDN to serve static content

During peak traffic, CDNs help in offloading not (just) servers but actually network bottlenecks of your servers. This causes a speedup in Magento by reducing the load on the entire system.

Tip # 3 : Use php7 instead of php5.x

php7 performance is almost 50% higher than php5. We have seen this in processing CPU intensive Magento hits. But, Magento 1.x does not officially support php7 and 3rd party patches. We like the one from inchoo. Magento 2 should be on php7.

Tip # 4 : Consider adding separate servers or pools for your checkout flow (or admin access)

While site performance is important, checkout flow performance is even more important as it is on the business end of your sales funnel. Consider adding a separate checkout servers or pools. They are akin to High Occupancy (HOV) or diamond lanes in traffic. This helps to speedup Magento’s checkout process.

The same concept can be applied to restrict number of resources you reserve for admin access.

Tip # 5 : Manage Magento Indexing

Leaving Magento Indexes to “Update on Save” can get your newly uploaded products available for shopping faster, but can slow down the site! If business permits, move indexing to slightly low volume times of the day (or night). This setting leads directly to speedup Magento front end.

Tip # 6 : Monitor the performance of your caches

A Magento site has many caches and each one helps in reducing the server load and improve response times. But you need to monitor for hit to miss ratios and out of memory conditions. This achieves a speedup in Magento as more cache hits mean faster response. The caches one should look at are php opcache, Magento block cache, FPC (Full Page Cache) and mysql cache.

Conclusion

By reducing, redirecting and monitoring server, the overall performance of a website can be improved.

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.

Watch our webinar on performance and scaling in Magento

Using an analogy to vehicular traffic we explain performance and scaling in Magento.
Key takeaways

  • Know how to compare hosting options
  • Importance of good code
  • How to scale
  • Tuning Magento

A non technical guide to scaling Magento (or any other website)

Performance & Scaling of a Magento web site are often confused. As a store owner who may not be technical a close analogy with real life will help in talking to your hosting providers and other experts.

It is no coincidence that hits to a website is called as traffic. We take this analogy further, to explain what factors matter to performance and scaling a website.

Website performance is like a car – higher performance cars drive faster and can cover a distance in a shorter time. Similarly, a higher performant website will serve a page fast. This is often measured as page load speed. A critical component of page load that the server is responsible for, is server response time. Like measuring performance of a car, measuring the page load speed is done in test mode with little or no traffic. Sometimes the performance is measured at a random time without looking at other traffic to the site. That is like test driving a car through traffic.

Scaling is like building highways and roads for the cars to move on. Highways are resources – CPU, memory, network that the hits to a website will utilize. The task of a Magento scaling expert is to architect a system – servers and sizes, services to run on each server, connectivity of the servers and access from internet, etc.

Hits to a website is like traffic of random cars on the highway. Each vehicle seems to have a mind of its own, joining and leaving the highway. Each visitor to the website will take their own journey visiting different pages.

Some Observations

Observation 1 : Like a car cannot drive at its highest speed possible at all times due to traffic, a website too cannot perform at its best best all the time. Understanding the factors that make the website perform at its optimal level all the time would be the task of both the developer and the server architect.

Observation  2: Like in traffic we have vehicles of different performance, in a website all URLs do not perform equally. A category page may not perform the same way as product detail page for example.

Observation 3: Better throughput will be achieved with the same resources, if the vehicle performance is improved – some bottlenecks can be avoided if the vehicles moved faster. Similarly, a better performing website is likely to scale better.

Observation  4: Like in traffic in order to scale one has to find the bottleneck in the highway that is causing the current slowdown, fix it and then look for the next bottleneck. This is a change in the hosting infrastructure and architecture, different from the website performance.

Observation  5: A traffic designers job is to ensure maximum number of vehicles can pass the highway at the best speed for each vehicle. A hosting designers job is to ensure maximum traffic is handled in a way that each hit is best served.

What lessons can we learn from traffic management

Lesson 1: To better manage traffic highway system has to be designed that is scalable. Mostly by bottleneck analysis we can derive what needs to be done. For example, is database a bottleneck, is file system access a bottleneck, etc.

Lesson 2: When traffic increases, possibly beyond the capacity of the highway, traffic management has to account for one more variable – starvation. The amount of time a vehicle has to wait at a metered light to enter the highway. The longer the wait, more frustration from the drivers who will find a better route to their destination.

Lesson : On a highway lanes are drawn. A better hosting will make lanes. The way most hosting providers take traffic is analogous to not having lanes with the hope that the maximun throughput will be achieved by letting hits contend for resources. The operating system stands to decide what process gets to use resources.

What are the recommended steps to achieve scaling?

As a first step to server side scaling, we move the database layer out to another instance or server. The main reason is that it is better to allocate resources in a single server when the workload is similar.

In our multi part series we take you through achieving scaling. The series is aimed at a store owner who need not be technical but is ultimately responsible to take a decision on the store. Until now you had to depend on an expert. However, there are no clear answers and the expert is making judgement calls based on most likely their prior experience. As a matter of fact no 2 webstores so results of efforts vary. This series will make you better informed.

We start by looking at a popular form of scaling – using FPC or Full Page Cache and other types of caches.

In order to help with scale, another important aspect is code quality specifically related to scaling. Scaling is difficult to achieve reliably if there is any externally dependent blocking service executed as part of the hit. Examples include sending email directly to a recipient or a external service, sending information from the server to an external service. All such processing should be done with some form of a queue handled either by an different process such as a cron job. Until Magento 1.9.2.4, the default email sending was inline for example, slowing the order success page being shown.

Autoscaling adds and removes servers (and hence resources) – something traffic managers cannot do with highways. This gives website scaling an advantage to be more elastic.

Why not cache it

Magento Caches – Scaling Magento Part 2

Introduction

Caches are an integral part of a strategy for performance and scaling a Magento website. Managing caches is a core function of infrastructure management. A Cache is the term used to describe a mechanism to store calculated values such as a query or HTML so a subsequent request does not need to recalculate.

While Part 1 looked at FPC specifically, in this article we review all other caches needed for a Magento store. Some of these are commonly found to be referred in best practices, but a deeper understanding will help put them in perspective of Magento.

In general, for any cache the factors to consider are

  • Effectiveness – measured in terms of impact on page speed. Caches should be highly effective.
  • Invalidation flexibility – is an inbuilt automatic mechanism available? Is the mechanism too aggressive? A High score means the mechanism is very flexible.
  • Performance of miss – if a miss were to occur what would be the performance in terms of page speed. A low score represents bad performance.
  • Cost of refill –how much would it cost to refill the cache completely to be back in its most effective state?
  • Hit ratio achievable – In a real world environment if hit ratio were to be measured periodically what can be expected. A high would be over 99% hits over a reasonable period such as a day.
  • Memory required : How much memory is needed on the server side to cache the content.
Cache Type Effectiveness Invaliation Flexibility Peformance of miss cost to refill Hit ratio achievable Memory required
Browser High High Low Moderate High N/A
CDN High Low Moderate Moderate High N/A
FPC High Low Low High Moderate High
HTML Block Cache Moderate High Moderate Moderate High Low
Mysql cache High Low Low High High Moderate

Browser cache

  • Easy to setup – at the hosting level. Static resources like css, js and images should have caching enabled.
  • There is no need for invalidation as the browser checks for each cached resource if a newer version is available.
  • The ability of browsers to find changed resources means cacheable resources should have a very long expiry – say over a year.
  • Browser’s request for detecting change has a performance impact based on number of resources.
  • A key question to be answered is how merging css and js files impact browser caching performance. If merging is enabled, each page type (home, category, product, CMS) are likely to have different css and js files. Not merging allows the browser to cache individual files and reuse them across pages. However, in HTTP/1 browsers limit the number of connections to each domain. So, the advice is to not merge css and js files if using HTTP/2 or splitting domains for skin and js if using HTTP/1.

CDN

  • CDN networks cache at edge servers closer to users – reducing round trip latency from browser to server.
  • CDN also offloads the server from serving static cacheable resources, improving network performance of the server as well as freeing up server CPU to serve dynamic content.
  • CDNs may also take the load of SSL validation however, caution is needed here as the traffic between CDN and server may be unsecure making the site vulnerable to some type of attacks.
  • CDNs are notorious for invalidations – some charge for APIs, others take a few minutes before the invalidations are effective across all edge points. When evaluating a CDN, this is a key factor that is not evaluated.
  • Having many edge locations may not be a good thing – as each edge records the first access to a resource as a miss
  • While a single miss is easily retrieved from the server or backup store, a full invalidation requires multiple GBs to be transferred to make the cache effective again
  • CDNs when full give great performance benefit on page load times
  • Modern CDNs like section.io can also do FPC (html) caching using a distributed varnish cache architecture.

FPC

We have reviewed aspects of FPC in part 1 here.

Recap

  • FPC caches full HTML pages – except for variable content
  • Excellent for caching dynamic content
  • Requires very high amount of RAM on the server side
  • Depending on the quality of code, FPC invalidation or miss can have impact on resource utilization
  • Best implemented with autoscale – so servers are added automatically when cache is invalidated

HTML Block Cache

  • Also caches HTML but cached at the block level. Magento uses HTML blocks for building a page.
  • Since blocks may be shared across pages, these blocks do not have a high impact on invalidation as they will be regenerated once and used multiple times
  • Can dramatically improve performance if used consistently and correctly.
  • Needs developer help as many blocks are not cached by default. To cache a block, one needs a unique key that correctly identifies its variation. Check this technical blog info.
  • Invalidation can be either via a key or time (TTL). If using a key, developer needs to write appropriate event callbacks to detect change.
  • Examples of major speed up include home page blocks where latest products are shown. Depending on frequency of store updated, a 10 minute to 1 hour TTL on the block will result in dramatic improvement of home page speed.

We can analyze your site for free

Schedule a call

Do you know how your website performs and want an expert to look at it?

  • We will analyze your site using public information.
  • We will run a synthetic test from the internet.
  • We will ask you to give us a 1 day web server log file.
  • We will also try to identify what steps if any you should take to improve your sites performance goals.

mysql cache

Mysql can store result queries in memory. The amount of memory is specified in my.cnf as a combination of query_cache_limit (max memory for a single query result) and query_cache_size (max memory all cached queries).

  • mysql automatically invalidates a cached query if any table that was used in the query changes.
  • to access the cache mysql uses a lock on a table thereby reducing the effectiveness of the cache
  • Many mysql articles recommend smaller cache values due to the lock problem but it is best to test the size of the cache for your situation. It is best to monitor “low mem prunes” and “table locks wait”. The first gives the number of times a query was cached by removing another and the latter gives the number of times table lock was not immediately available. Both should be as low as possible.

Our experience with mysql cache suggests when front caches are empty, they are useful. The cache has a negative effect on add to cart and checkout.

Mysql caches also have negative side effects when a large catalog is loaded. However, we recommend to keep indexing to be on schedule, disable mysql cache before indexing starts and enable when indexing ends.

There are other caches too

  • Magento has caches other than block. These should always be enabled on a production server.
  • Php opcache
    php code is read and converted to “opcodes” which are then interpreted. An opcode cache stores the opcodes reducing one step each time. Opecode cache should have sufficient RAM and keys (equal to number of php files).
    Using opcache.php the status should be checked regularly.
  • Operating system cache
    linux has an excellent file system cache – whenever additional RAM is available, linux will store opened files in RAM. This is important if for example you do not have a CDN. Files will actually be retrieved most likely from RAM rather than disk.

Transactional Email Deliverability of your Magento Store

The internet started with email and email continues to be a very important means of communication for a Magento site. Emails that are sent directly in relation to an activity on the website such as a registration or purchase – are called transactional emails. Transactional emails occupy a different place in the email marketing category and are governed by less strict rules worldwide.

Importance of Transactional Email for Magento stores

Deliverability of transactional emails is a key to customer satisfaction and loyalty. If a customer requesting a password reset does not get an email in time in the the inbox would result in possibly loosing the customer.

Why is Transaction Email Deliverability a problem?

If email is fundamental to internet why is email deliverability a issue?
In order to protect email infrastructure from spammers, many services created spam lists – IP addresses that have previously been used to spam and are blacklisted. There is no single authority with such lists, leading to the deliverability problem. The IP you get assigned by your cloud provider may not be the clean in all the lists and it is too difficult to find and much more difficult to get cleared. Transactional email providers come to the rescue – their business is to increase deliverbility.

What can be classified by a transactional email?

Newsletters, even opted in, do not classify as transactional email. If you do not send newsletters through Magento, all emails that go out will be transactional.
However, you maybe crossing the line if you send out upsell / crosssell in your email order confirmation for example.

Third party providers

There are many providers and it is a very competitive market a search on google for transactional email will get you many results and comparisons.
Here are a few recently updated comparisons

How to get started with transactional email for Magento?

  • Check if you have an existing subscription to a transactional email service – indirectly. For example, if you are hosted on softlayer, you may get sendgrid credits. If you use Mailchimp to send newsletters, you may have mandrill credits.
  • Signup for the service – most of them have a free tier
  • We think having a Magento plugin is not a requirement if you are self hosted on a VPC or better. Read on, we think using the SMTP service is better option than a plugin or code integration.

Before you install the Magento plugin, read this!

  1. Plugins add a drag to the system – like it or not, each plugin you add, contributes to a slowdown of Magento due to the architecture. Many plugin authors are guilty of passing in additional features into the plugin.
  2. Plugins for transactional emails are “inline” i.e. the email is sent while the purchaser is waiting for a confirmation. That is a dependency on an external system. Occasionally the service may have slowed down and that delay will be added to the wait for the customer.
  3. Local email systems are automatically configured to retry in case of upstream infrastructure failure. If configured at the system level, the email is sent only to the local system, form where it goes into a queue which the systems email service will relay. If for some reason the remote email service is not responding, the queue will remain active and a retry will be attempted after sometime.
  4. Do not select the service based on the availability of a Magento plugin – that is the least important part of the evaluation

How to setup

All providers use TLS for SMTP communication on port 587. It will be required to open port 587 in the firewall to ensure emails be sent out.

Note : Some cloud services notably Google Cloud Platform does not allow communication on ports 25 or 587. For such services you need to use a transactional email service provider that allows SMTP communication over a non standard port.

Use the guide below to get your username and password and then use the steps to setup postfix

For Mandrill

Username : mandrill username
Password : Get Key (Dashboard->Get API Keys->NewAPI Key)
Domain : smtp.mandrillapp.com

For Amazon SES

Username & Password : https://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-credentials.html
Domain (as per region, this is for US West) : [email-smtp.us-west-2.amazonaws.com]:
Domain verification : http://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-domain-procedure.html

For Sendgrid :

Get this certificate and store in /etc/postfix/ssl
wget https://certs.godaddy.com/repository/gd_bundle-g2-g1.crt
username : sendgrid account username
password : sendgrid account password

postfix setup

    1. Ensure SASL authentication package like cyrus is installed.
    2. Ensure you have a FQDN (Fully Qualified Domain Name). The command hostname -f should report a host.domain type of name. It is preferred you use the domain you are sending from
    3. Ensure postfix is installed (and sendmail is not)
    4. Edit /etc/postfix/sasl_passwd and enter SMTP_DOMAIN, username and password as per the transactional email platform.
    5. chmod 600 /etc/postfix/sasl_passwd
    6. psotmap /etc/postfix/sasl_passswd
    7. edit /etc/postfix/main.cf and add the following to the bottom of the file
# enable SASL authentication
smtp_sasl_auth_enable = yes
# tell Postfix where the credentials are stored
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
# use STARTTLS for encryption
relayhost =<refer platform info>
## For mandrill
smtp_use_tls = no
## For sendgrid
smtpd_tls_security_level = may
smtp_tls_CAfile = /etc/postfix/ssl/gd_bundle-g2-g1.crt
## For Anazon SES
smtp_use_tls = yes
smtp_tls_security_level = encrypt
smtp_tls_note_starttls_offer = yes
  1. restart the postfix service
  2. test by sending an email and watching the result in /var/log/maillog

How to evaluate a hosting service for Magento?

Introduction

With so many choices in hosting service it is difficult to decide what to use for hosting of a production Magento server. The first question most commonly asked is – should one use a physical (or bare metal) server or a virtual machine (or cloud server)? Many people think the obvious answer is Virtual Machines. After all this is the way the world is thinking and all cannot be wrong. But, let us take a closer look for Magento hosting. Magento is typified by 2 factors – high CPU utilization for php interpretation and mysql performance limited by both CPU and disk writes typically for operations such as reindexing and high order volumes.
 
In this article we talk about these factors and how to help evaluate your preferred platform.

CPU speed

Most cloud service providers like AWS, Azure or Softlayer do not define what you get when you ask for say 2GHz CPU – do you get 100% of the power of the CPU or is the CPU shared?
Hypervisor technology used to create Virtual Machines, easily allows one to overcommit CPU. Overcommit means that the number of CPUs on a physical hardware can be lesser than the number of CPUs in all the VMs running on that server. Emperical studies such as here(https://www.datadoghq.com/the-top-5-ways-to-improve-your-aws-ec2-performance/) have proven that overcommit does happen.
 
This means that test results and live site performance are subject to current usage of your neighbours.
In addition, VMs need hypervisors to run beneath the VM – this adds to overhead as well as latency.
 
In a study by Forrester a case was made for using bare metal infrastructure, now that some leading cloud providers have made it easy to spin a new bare metal server.
 
On a physical or bare metal server, processor power, disk space, memory and other system resources are not shared with noisy neighbours so there is high correlation between test results and live performance.

Disk speed

High speed disks vs network access disks. A SSD or a 15K SAS for example can give the boost you need when upload products and reindex or you get many orders such as during a holiday season.

  • Unthrottled performance. When you buy guaranteed IOPs for example, what happens when you exceed the limit? What if you have a burst need that exceeds the provisioned IOPs?
  • Use a locally attached SSD where available vs a network storage. Local storage will be faster than network storage by orders of magnitude.
  • Consider RAID configurations for better performance

Here is some raw disk performance metrics1 we got when testing some popular hosting providers

Provider	Softlayer  Softlayer  Softlayer	Azure	Azure   AWS  Ukfast    DO
Type		Physical   Physical   VM	VM-D2V2	VM-D2V2 EC2  Physical  VM
Disk		Magnetic   SSD	      Local	Default	Blob    EBS  SSD       Default
unbufferred 	144	   451	      200	20.6	65      57   130       150
bufferred 	155	   602	      217	1000	65      263  1500      300

Notes

  • All data in MB/sec reported by the linux dd command/li>
  • Unbuffered dd if=/dev/zero of=/tmp/test bs=256M count=4 oflag=dsync
  • bufferred dd if=/dev/zero of=/tmp/test bs=256M count=4
  • Digital Ocean performance varied in a wide range 55 MB/s to 150MB/s

This simple test shows relative disk performance on various platforms without a RAID configuration.
(Refer Roman’s wiki)

Which hosting service is preferred?

  • Physical (or bare metal) servers give the best “performance” of Magento production hosting. The key reason is that they scale in a predictable way when traffic peaks.
  • You can scale horizontally with cloud servers – many hosting providers now give the option to mix VMs and bare metal on the same subnet. >We prefer such vendors.
  • Not all VMs and servers are made equal – test before you commit.

Nginx as a load balancer for Magento

Introduction

During seasonal peaks or as traffic grows, there will be a need to add multiple app servers to your Magento store. A load balancer for Magento becomes essential. We have found that using nginx as a load balancer gives acceptable performance. We have not found many instances where we would recommend a hardware load balancer. Recent tests by nginx confirms this.

We would recommend a different load balancer only for additional features such as autoscaling.

Nginx as a load balancer offers many advantages including

  • uneven upsream servers as nginx can assign weights to each load balancer
  • self healing – takes a upstream server out of a cluster if it stops responding
  • path based load balancing
  • combination of path based and weight based load balancing
  • php upstream servers
  • SSL/TLS termination

This article assumes the process of adding a new app server to a Magento cluster is well understood. Here the focus is on the nginx configuration.

Load Balancer for Magento : Basic architecture

(more…)

Nginx or Apache : Best server for Magento

Introduction

Apache server has been for years been the default http server linux hosts use. However, recently there have been many newer “lighter” http servers. This blog article focuses on Magento hosting. Magento is a php based web eCommerce framework. Nginx requires php-fpm to process php requests. So, this comparison is really apache vs nginx + php-fpm. Apache offers MPM (Multi-Processing Module) configurations pre-fork, worker and event. In this discussion we will use the “event” MPM.
This discussion is very popular. Examples include this. We focus on Magento here.

Key Differences between apache and nginx

There are some differences architecturally that make nginx look slightly better for Magento hosting.
(more…)