Search Results - nginx

Optimal Secure Magento Hosting

Managing the TCO (Total Cost of Ownership) of your Magento website

Our customers’ cloud hosting bill reduced by upto 30%. While your actual saving will vary with the state of your current hosting, we can ensure your hosting bill is optimized – you pay for what you need. Click here to know how we optimize your cloud bill.

  •  Optimal
    • Best response times for each hit – server response is a critical factor to overall page load time.
    • Planned use of memory and other resources.
    • Cost effective solution – neither a cheap solution that may not work nor an over engineered expensive one that may never get used.
  • Scalable
    • A well defined path to scalability, depending on the business need for the next 3, 6 or 12 months or holiday period or even flash sales.
    • A need to scale up as well as scale down is required.
  • Secure – a security breach can be detrimental to your business, but you don’t have to spend a fortune to remain secure.
    • Multi layered security for your customers’ data and the infrastructure.
    • Serve valid human traffic and keep BOTs out.
    • Reasonable protection against DOS attacks.
    • Web Application Firewall to keep application hacks like SQL Injection at bay
    • Backup and disaster recovery plan that is tested and works
luroConnect Insight performance graph

Renting a server for hosting is now a commodity. Most vendors have very similar offerings. A multi vendor strategy that as far as possible avoids vendor lockin is needed. Managed hosting from a provider with multi vendor capability will help you keep your processes and choices clear.

Can this be achieved?

Hosting a Magento website does not have to be either an ignored problem nor should it be rocket science (or maybe magic even). Firm scientific principles can be used to ensure a website is well hosted and has alerts when the system goes out of capacity.

Essential Components

nginx + php-fpm
  • load balance as you scale or as per traffic pattern.
  • rate limit from a single IP to protect against DOS attacks.
  • restrict bad IPs from accessing the sites.
  • allow or keep away BOTS based on their User Agent signature.
  • Secure configuration disables php execution from non code directories
  • Secure hosting ensures web application cannot change code files
varnish
  • Magento 2 supports varnish out-of-the-box
  • Varnish is front page caching technology that stores pre-generated pages in memory and serves in milliseconds
redis
  • Used for storing Magento cache and sessions in memory
  • sessions rejection based on rate preventing Magento lock
mysql
  • oracle mysql is improving but Percona and Mariadb perform better even now
  • Mysql for Magento requires configured balance between memory available and cache size.
luroConnect Insight
  • Cloud tool to analyze log file data from live site
  • Dashboard to show crucial parameters from the site
  • Alerting when site slows or gives error
  • Alerts from analyzing actual hits on the site
  • See Top 10 IPs, BOTS to help decide what to block or allow
  • Rate limit blocking to help find good from bad

Optional Components

Web Application Firewall (WAF)

Using Open Source ModSecurity custom built for nginx, along with custom rules for Magento we enable a reasonable level of security directly on the edge server as part of the stack. Preventing popular SQL injection and Cross Site Scripting using the OWASP ruleset (or the commercial Trustwave ruleset)

PWA hosting with vue-storefront or Magento PWA Studio

Progressive Web Apps (PWAs) are the new wave of headless eCommerce on Magento. vue-storefront is a popular option. Magento 2.3.4 ships with PWA Studio. Both require use of nodejs technology.

luroConnect supports nodejs based technology. With our advanced nginx routing rules, the application developers can decide what URLs should be served from Magento directly or from nodejs.

Secure Image Upload & Optimization

When a site needs frequent uploads of products, upload of images requires insecure access to Magento. Our solution allows safe upload to a “pod” from where images are transferred transparently and automatically to the desired folder.

Similarly our image optimization will optimize Magento generated images, either lossless or to a level of optimization acceptable, keeping the original images intact. This technology is compatible to all modern CDNs.

Also needed
  • CDN

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.
Click here to schedule the call

Video - Magento scaling and performance

Its free!

Using analogy to vehicular traffic we explain performance and scaling in Magento.

  • Know how to compare hosting options
  • Importance of good code
  • How to scale
  • Tuning Magento
Watch video now!
November 21, 2017 By Comments Off on Optimal Secure Magento Hosting Read more...

Thankyou for submitting your site for the scale test.

Your test will be performed as quickly as we can.

  1. Please take time to reply to the email we have just sent to you. This will help us with verification.
  2. If your email address matches the domain, the test will performed immediately and an email will be sent to you.
  3. If the email address and the domain do not match, we will contact you for authorization.

What will this test tell me?

The test will help you understand how your site performs under load. If your expected peak traffic is below the point where the site slows down, you are fine.
If on the other hand, our test indicates your site does not perform well at peak, you might want to improve your hosting.

Note : This test does not perform a page load test. It only tests the server response. Server response is also called as TTFB or Time to First Byte.

Do I need to add more hardware in order to scale?

Not necessarily. If your site not optimally hosted you can tweak more performance of the current hosting. Some low hanging fruits.

  • Use redis for cache.
  • Review memory utilization and php processes. You can free more memory and give it to mysql or redis by reducing the number of php processes to not more than 2x the CPUs.
  • Use nginx and php-fpm instead of apache to get better control of php processes.
  • Consider improving use of block caches in your site.
  • Consider using FPC (Full Page Cache) for your site.

Watch our webinar on performance and scaling in Magento

Its free!

Using analogy to vehicular traffic we explain performance and scaling in Magento.
Key takeaways

  • Know how to compare hosting options
  • Importance of good code
  • How to scale
  • Tuning Magento
Watch video now!

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.
Click here to schedule the call
November 14, 2017 By Comments Off on Thankyou for submitting your site for the scale test. Read more...

Scaling Magento – Video reveals the magic of hosting a scalable Magento website

Using a powerful analogy of vehicles and highway, this video explains how to scale Magento. A version of this presentation was made at Meet Magento India 2018.

What you will learn in this video?

  • What does the performance of a Magento website depend on?
  • Selecting hardware
  • Good code
  • Page Load Times
  • Scaling
  • Tuning
  • How to know how much the current hosting will scale to?

What you will learn in this video?

  • What does the performance of a Magento website depend on?
  • Selecting hardware
  • Good code
  • Page Load Times
  • Scaling
  • Tuning
  • How to know how much the current hosting will scale to?

We can analyze your site for free

Schedule a call

Before the call we will analyze your site using public information. We will ask you to give us a 1 day apache or nginx log file of your site. Our half hour consultation will try to identify what steps if any you should take to improve your sites performance goals.

Click here to schedule the call
October 22, 2017 By Comments Off on Scaling Magento – Video reveals the magic of hosting a scalable Magento website Read more...

A Guide to Affordable Hosting a eCommerce website

Two options to get better hosted without hurting the bank!

We are wilh eCommerce stores on a budget. But, getting a shared hosting is the wrong decision! Instead, get a Virtual Machine!

We have found these two providers very reliable. They have invested in their data centers as well as tools and APIs that allow you to grow before you have to move to a different provider.

And you can start at the cost of a shared hosting plan!

Option 1 : Digital Ocean
Signup at http://bit.ly/lc_do

Option 2 : linode
Sign up at http://bit.ly/lclinode

Why not shared hosting for serious website?

  • You do not know what you get!
    It is like living with strangers.
  • When hits increase, the performance will degrade and you will not know.
    It is a like a friend who runs away when you want them most.
  • Security may be weak.
    phpmyadmin, cpanel, ftp with passwords are useful but risky.
  • It is not cheap!
    You can get a better quality hosting on your own server.
  • At best it is a short term solution.
    There is no path to scale.

Lessons from “The Founder”
for web server infrastructure

Read the article here

We can analyze your site for free

Schedule a call

Before the call, we will analyze your site using public information. We will ask you to give us a 1 day apache or nginx log file of your site. Our half hour consultation will try to identify what steps if any you should take to improve your sites performance goals.

If you are not live, we will talk to you about what hosting options are available to you and how we can help you.

Click here to schedule the call
September 24, 2017 By Comments Off on A Guide to Affordable Hosting a eCommerce website Read more...

Case Study : How a customer achieved site stability

Introduction

Magento response is non linear to traffic increase – i.e. when traffic increases the response time does not go down linearly – instead response drops dramatically and erratically when a saturation point is reached. In this case study we show how a customer faced routine downtime with their Magento site and what we did to get them to host appropriately.

The problem

When hosting Magento for the first time, a big question remains is what size host is appropriate to the requirement. A do-it-yourself person would google for answers, talk to hosting providers, etc. The result is an unscientific assessment and hence wrong size. We routinely get customers such as the one featured in this case study and we help them scale right.

The customer was hosted with US based Arvixe (https://www.arvixe.com/). Arvixe is known for its low priced servers and 24×7 support. With the help of Arvixe engineers, the customer was setup on a bare metal server. As is done in the hosting industry, a WHM (https://cpanel.com/products/) based hosting was made available to the customer and Arvixe engineers help setup the Magento website. Arvixe enginners also helped them setup MX records to point them to the server – a mail server was also enabled and the customers work emails were received and sent from here. In order to ease the customers developers access a phpmyadmin was setup to give access to the database which was running on the same server.

Everything worked with the exception of one problem known to the customer – routine downtime. The Arvixe 24×7 support was very responsive in restarting services and the customer gained expertise in WHM to do some restarts themselves.

During our conversation with the customer we realized they knew that they were outgrowing their environment. We also realized that there were fundamental differences in our approach to hosting a production Magento website and Arvixe’s. But the customer wanted more analysis before they would accept our solution.

Our Solution

We spent 1-2 weeks observing the problems the customer was having with their server. During this time we analyzed their apache log files to see what were the rates of hits per minute they were getting. We graphed it in excel to see the results. We even added a log parameter to show the response time for each hit. We then analysed a downtime incidence – instead of restarting services, we took the error log files, studied the system resource utilization using linux “top”command. We also made more observations of downtimes and came up with different types of downtimes.

Some observations

  1. Database size.
    Here are the top 10 tables sorted by size.

    +--------------------------------+------------+--------+--------+------------+
 | TABLE_NAME                     | table_rows | data   | idx    | Size in MB |
 +--------------------------------+------------+--------+--------+------------+
 | j2t_autoadd_salesquote         |      21712 | 507.55 |   1.86 |     509.41 |
 | core_url_rewrite               |    1517105 | 328.56 | 590.02 |     918.58 |
 | catalog_category_product_index |    2528808 | 233.08 | 241.34 |     474.42 |
 | catalog_product_index_eav_idx  |    1692500 | 214.53 | 363.36 |     577.89 |
 | catalog_product_entity_varchar |    3338042 | 196.70 | 271.59 |     468.30 |
 | catalog_product_index_eav      |    1724290 | 172.58 | 260.41 |     432.98 |
 | log_url_info                   |    1032207 | 169.22 |   0.00 |     169.22 |
 | catalogsearch_fulltext         |     195208 | 120.45 |  81.57 |     202.02 |
 | catalog_product_entity_text    |     505555 | 118.63 |  38.09 |     156.72 |
 | log_visitor_info               |     556272 | 111.16 |   0.00 |     111.16 |
 +--------------------------------+------------+--------+--------+------------+

    Some tables in Magento are truncatable. Some tables like core_url_rewrite required to be rebuilt periodically.

  2. Database cache was not tuned.
    The SQL query used by the menu was not cached. Each hit ran the query.
    The sql query cache tuning was set too low a number. It was as per general recommendations but it hurt this Magento site.
  3. While there were 8 cores CPU, apache was configured to run 200 processes. When there was a spurt in hits, all hits responded badly and mysql could crash under this load as it ran out memory. Each of the apache process would grow to 500MB, increasing the memory requirement on load.
  4. They used email to communicate internally – large design files were routine transferred among employees as attachments. Whenever such an email was sent to many employees – who were all configured on outlook to download emails – the main site slowed down.
  5. There were coding errors that would routinely return an error on a page.
  6. WHM made it too easy for change in configuration settings, many experimental and some leading to a rebuild of the entire software stack, leading to inadvertent downtime of the website.

Our solution included

  • Host on 2 servers – one for database the other for application. Servers were sized as per observed load conditions. The database server had a very fast disk to reduce IO Waits. This would be with our standard open source stack with tuning. It included
    • redis (https://redis.io/) for Magento caching and Magento session storage,
    • nginx + php-fpm,
    • Percona mysql database with percona tools,
    • fully secure with 2 level authentication for admin and
    • no password access.
    • Backup with a disaster recovery plan.
    • With the help of a plugin we implemented a search using Sphinx (http://sphinxsearch.com/)
  • Moved out the email to a office365 with an automated process using imapsync (https://imapsync.lamiral.info/) to move the emails to the new setup.
  • Not give WHM or phpmyadmin access due to potential security concerns.

Our proposal asked them to spend more on hosting and support in return for stability and scalability.

Conclusion

Here is what the customer had to say about us 6 months later.

“We started with shifting our servers to a new host and moving managed services of our magento based shopping cart … our downtimes have come to nearly zero … People are pleasant … and show genuine concern for our business. We look forward to a long term association with them due to their competence and warm positive outlook.”

That was January of 2014. They continue to be our customer and a great source of referral

We can analyze your site for free

Schedule a call

Before the call we will analyze your site using public information. We will ask you to give us a 1 day apache or nginx log file of your site. Our half hour consultation will try to identify what steps if any you should take to improve your sites performance goals.

Click here to schedule the call
September 12, 2017 By Comments Off on Case Study : How a customer achieved site stability Read more...

Thankyou for your registration.

Welcome to luroConnect

You will receive an email from us shortly. Please click on the link for verification.

Upon verification you will receive instructions to install the agent. If you are uncomfortable with ssh, please email us and we will help.

What data?

The agent will monitor your nginx log file and send data to our cloud service. By default data is stored for a max of 7 days. Your stores’ sales data is not sent to our servers.

Check our privacy policy for details on how we handle your data.

Uninstall

Both manual and automatic instructions are available on your cloud login at www.luroconnect.com

With luroConnect / Insight you can :

  1. Know in real time how your website is performing
  2. Setup alarms to alert you when the website slows
  3. What BOTs visit the website
  4. What IPs are popularly hitting the website
  5. What are the most common and slowest URLs
July 19, 2017 By Comments Off on Thankyou for your registration. Read more...

Full Page Cache (FPC): Scaling Magento Part 1

Introduction

The most important and often ignored factor to scaling is the quality of code. Well written code will scale better. The next most important factor is perhaps caching. There are many types of caches that developers, managers and store owners need to understand. Full Page Cache (FPC) is seen by store owners as a magic solution to speed issues. Understanding the benefits and compromises of a caching mechanism is important to understand scaling.

FPC Options

Magento Enterprise 1.x and Magento Open Source & Commerve 2.x, both have a FPC module inbuilt.

There are many plugins available for Magento Community 1.x. Some hosting providers will help setup a Varnish based FPC with appropriate hole punching.

Magento 2 has two mechanisms for FPC – php based (called FPC) and varnish. Varnish is the preferred option for production due to the architecture and speed of response.

The discussion below applies to all these mechanisms as well as to Magento 1.x and Magento 2.

What is Full Page Cache (FPC)?

FPC is a cache of a full HTML page – except variable content such as a login status or items in cart or stock status of a product or sometimes even price of a product. When a hit is encountered – i.e. the page required is in the cache, FPC will return very fast compared to when there is a miss i.e. the page is not in the cache, which will require re-generating the content. FPC may store the cache in files, but more likely for maximum benefit, it will be stored in memory.

FPC affects resource utilization – memory and CPU. As with all caches, we trade memory for CPU time.

Traditional FPC stores the page in Magento cache and is a part of Magento. Varnish stores the page HTML after it has been generated and is not a part of Magento. It is a separate process.

What FPC is not!

Let us understand FPC better

  • Memory needed to store the entire site in FPC
    Let us say each page is 100KB and you have 10000 pages to cache. That would take about 1GB of RAM. The problem is when the number of pages or page size starts rising above this, the RAM requirement goes up. So, if you now had 20000 pages (result of each option in layered navigation for example), you would need 2GB or if each page was 120KB the 20000 pages would need  4GB. Pages are not just products – they are category pages as well. If layered navigation is added the pages multiply fast as each combination is unique and needs to be stored independently. If you start exceeding the RAM available, you need to decide what to do when you hit the memory limit.
  • Cache warming.
    Cache warming is the process of automatically adding pages to the cache before a real visitor hit comes to the cached page. When a cache is cleared, you may need to warm the cache to make FPC effective early. Cache warming uses a crawler to artificially visit pages of a site. A typical crawler will recursively crawl the site starting from the home page. This sounds logical but here are some things to think through

    • If possible find the most likely pages you need to be in the cache and warm the cache with only those pages. This will give the maximum benefit.
    • If you cannot fit all the pages in memory, the use of crawling to warm the caches becomes a problem – they will recycle pages out of memory at random, not based on the end user popularity of the pages.
    • When the cache is being warmed your resource requirement in terms of CPU will rise as both the crawler and real traffic are being served.
    • If possible crawl the site in parallel – the earlier the pages get cached the more likely a visitor to a page will already be in the cache (scoring a hit).

Performance degradation on FPC full invalidation

The above figure shows the bad response immediately when a FPC that had built to 1.5GB was cleared completely. The top image is from redis usage graph from munin and the one below is AWS cloudwatch latency (time to serve a page) averaged per minute. The latency came down as AWS Autoscale added more instances, costing money.

  • Invalidating the cache :
    Magento automatically invalidates FPC (internal or varnish) by tagging or hashing the content with keys that refer to the type of content. For example, it may generate a tag / hash CATEGORY_123 if the page depends on category 123. Now, when category 123 changes, Magento sends out a invalidate message that says “all pages that have tag / hash CATEGORY_123 should be invalid”. Magento has a elaborate tag convention.
  • FPC and robotic crawlers (BOTS)
    Even if you do not use a crawler for warming, robotic crawlers on the internet (such as google’s indexer Googlebot) will start filling the FPC cache with pages they happen to crawl. It is our advice that a site with FPC should have robots.txt and a front end processor (nginx, WAF) restricting BOTs.
  • CPU and time needed to re-generate a page
    A FPC can fully invalidate (clear) due to a (p)html or css file changing or partially due to a data change such as a product update. A miss from FPC results in the page being regenerated. The CPU requirement for a miss is much higher than a hit. If a crawler is used to warm the cache or if traffic is high, CPU requirement can be quite high as the FPC fills up. Yet, the visitor experience is not good during this period. Using autoscale, this performance degradation can be contained to some extent as additional instances are launched to handle the high CPU requirement.
  • Discipline when using FPC – know when invalidation happens
    It is important to add discipline for code update as it has the worst effect on user experience.

    • Code update should be done at low traffic times.
    • Category changes should be carefully planned at low traffic times.
    • Magento indexing should be set to manual (M1) or on schedule (M2) with a cron running the indexer.

Our recommendation for FPC

  • Do not use a random crawler to warm the FPC cache. Use a page popularity based crawler to warm the cache if necessary.
  • Avoid using a crawler during high traffic – the crawler will compete for system resources with live traffic
  • If possible update code during low traffic times as it causes FPC to invalidate
  • If your site is horizontally scaled, pre-launch instances to your load balancer before invalidating FPC, either explicitly or indirectly, so the latency of starting an instance does not further worsen the user experience

Magento 1.x FPC Plugins

  1. Free Lesti FPC : https://github.com/GordonLesti/Lesti_Fpc. Use this guide to install
  2. Magento connect search results for FPC

Should FPC be a part of scaling strategy?

FPC is concerned with speed. Scale is concerned with the process that helps the site add resources when needed. FPC helps in scalability by reducing the use of resources per hit to the website under certain conditions. It changes the dynamics of when and how many resources will be needed.

FPC has to be considered to be part of scaling strategy – but as one of many parts.

Read part 2 where we discuss other Magento caches.

Read the overview of our Magento scaling series here.

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.
Click here to schedule the call
June 6, 2017 By caches, Magento, performance, Redis Cache, scale, Site Speed 0 Comments Read more...

Securing your Magento site

Executive Summary

Owning a Magento website means you have a resource you own and control on the internet. Keeping it secure in all aspects is a responsibility. You need to ensure

  • your business data is secure.
  • your customer data and privacy is secured.
  • your customer’s computers are protected when accessing your website.

Your privacy policy indicates your desire to keep your customers’ data secure. You would intentionally not violate the policy but a cyber attack on your infrastructure can cause a violation.

It is not required to be a famous to be in the eyes of the attacker – most attackers today prowl the internet looking for vulnerable sites and automatically get to work. New types of attacks are being designed and hence keeping a site protected is a continuous battle.

As a person ultimately responsible for your website, you need to periodically review your security processes and see they are enhanced for new vulnerabilities.

This article has an overview of the aspects of security you need to worry about. In addition it goes in depth as well to make it actionable.

Aspects of security infrastructure for a Magento Website

  • Protection of the web edge infrastructure. Ensure you keep the bad guys out and the good guys in. Using a Web Application Firewall (WAF) with appropriate configuration will help.
  • Server protection. Ensure access to the server restricted, preferrably without a password, limit port access and protect files and folders with permissions that are just needed. Also ensure the server has the latest security patches installed. Need to know access should enforced. Code updates should be automated. A staging site should be used for updates from vendors. Plugins should be acquired from known and reputable vendors.
  • Code protection. Ensure your code is patched with the latest Magento (or any platform and plugins). In addition train developers to use safe practices to not leave holes.
  • User data protection in transit. Ensure https access to the entire site so all access is secured against in transit hacking as users use your website. Ensure all admin access is restricted either with dual password or two factor authentication or IP restricted.
  • Review all access periodically Change passwords and admin URLs regularly.
  • Run vulnerability scanners Vulnerability scanners are available for testing many aspects of security. Blackbox scanners such as Trustwave, Secure Works, help in checking if they can find external vulnerabilities. Static analyzers such as Fortify scan the code to find code level vulnerabilities.

Protecting Web Edge Infrastructure

The key role of this protection is to keep the bad guys out and allow the traffic you want in.

At the minimum configure nginx or apache (your webserver) to

  • rate limit hits from a single IP address
    However, if you get legitimate hits from behind a firewall these may generate false positives. We recommend to log the IPs that get restricted and review. Whitelist the ones that are ok.
  • rate limit admin server access if you cannot IP limit the access to admin
    Many bots are trying passwords to get acsess to the admin panel
  • Have a mechanism in place to block IPs.

As with any protection of this nature, detecting bad is a key. OWASP is an online community which has come up with a core rule set (CRS). A Magento site needs a WAF protection either on the nginx server level or with an external service such as Webscale Networks, Cloudflare, Sucuri.
The OWASP ModSecurity Core Rule Set (CRS) provide protections against the following category of attacks.

  • SQL Injection (SQLi)
  • Cross Site Scripting (XSS)
  • Local File Inclusion (LFI)
  • Remote File Inclusion (RFI)
  • Remote Code Execution (RCE)
  • PHP Code Injection
  • Metadata/Error Leakages
  • Project Honey Pot Blacklist
  • GeoIP Country Blocking, etc
System Admin Note

mod_security is a open source project available for apache and nginx. Installation is not difficult. However, you need to tune the rules to your environment. Some rule tuning may require developer or marketing input.

Protect the admin panel login

  • Change the admin URL periodically. Do not use the same name as in test or development environemnts
  • IP restrict admin access to only your known IPs
  • Put a simple http access in front of the admin URL – so there are 2 levels of usernames and passwords and it makes it non standard for automatic hacks
  • 2 factor authentication to admin panel – may need a plugin
  • Change all admin user passwords frequently
  • Monitor log file for frequent unsuccessful attempts to login

Server Protection

A hacked or broken in server – popularly shown in movies – gives access to your server to the hacker.  Web prowlers are continuously trying to see what ports are open in the server and what access they can get. Being a very traditional form of hacking, it is highly automated. By the very same count, protecting a linux server is not very difficult either. Follow some guidelines and review process.

  • Keep only known ports open and limit which IPs can access these ports
  • Disable any form of password access (like ftp, ssh). Use keys instead.
  • Keep server patched with latest security patches
  • Run only services that you need. For example never run ftp as it gives password based access to the server.
  • Follow very strict rules on file / folder permissions as well as linux groups and users
  • Periodically scan servers for viruses signatures
  • Periodically review access keys
  • Automate routine processes and restrict with keys including code update
  • If a db server is a separate server further restrict access to this server . If using a autoscale architecture, further restrict access to app servers as well.
  • Allow no exceptions – even for a critical fix do not give access to a developer for example.

The following system admin note has technical details for linux system administrators to execute the strategies above

System Admin Note

Depending on the configuration we recommend the following incoming ports to be open :

On the only app server without load balancer
Port of external interface Protocol / usage
80 http, main site open to the world if site not fully secure.
443 https, main site open to the world
22 ssh, possibly open only to specific IP addresses
On the db server
Interface / port Protocol / usage
External / 22 Only if you must to restricted IP addresses
Internal / 6603 Restricted to internal IPs that run app servers
Internal / 22 Ssh access from the other servers
Constellation of app servers, a db server and a nginx load balancer
Host / Interface / port Protocol / usage
Load balancer / external / 80, 443 http and https open to the world
Load balancer / external / 22 ssh, possibly open only to specific IP addresses
Load balancer / internal /2049 Nfs server
Db server / internal / 6603 Mysql, for app server access
App server / internal /1110 Nfs, for file access across all app servers

Note : If using a load balancer device or service, port 22 access should be given to only the main app server, possibly the nfs host.

System Admin Note : ssh configuration for secure access

ssh needs to be configured to a more secure mode. (Refer linux man page)

  • Change default port to non standard port (say) 2020. This will thwart an obvious attempt to breakin. However, an attacker can find the port using a port scan.
  • Disable root ssh access. This is crucial. This means that someone can never login to the server directly as root.
  • Disable password access. Weak passwords are the most common way a breakin is successful. It is common that a server being attacked will have multiple password generators trying to access the server.
  • Enable only key access. With this setting only a private key holder who has a public key stored in the .ssh/authorized_keys file will be allowed access.

/etc/ssh/sshd_config :

...
# set port to non standard 2020
Port 2020
...

# Authentication:
#LoginGraceTime 2m
#PermitRootLogin yes
PermitRootLogin no
…
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
PasswordAuthentication no
…
# GSSAPI options
#GSSAPIAuthentication no
GSSAPIAuthentication yes

Note : First enable key access, exchange keys, ensure ssh to keys works before disabling password

Note : When changing ssh configuration ensure a ssh session to the root server is separately running. This will prevent from being locked out of the server when changing ssh configuration.

System Admin Note : What users are needed

We recommend the following users

  • Login user
    • This user has sudo access (without password)
    • Ssh key exchanged
    • Admin logs in as this user
  • Site user (production)
    • Website will be hosted under this users (/home/production/www NOT /var/www/html)
    • This user may have ssh access to deploy code
    • This user does not have sudo access
    • If using multiple app servers, this users id will be shared across all app servers
  • Web server user (apache)
    • Nginx and php processes will be run as this user
System Admin Note : File and Directory Permission settings
  • Only media/pub and var folders should have 770 permissions
  • All other folders in should have 750
  • All files under var should have 660
  • All other files should have 640
  • production user should be in apache group

Protect your code

  • Ensure Magento code is patched to the latest. Subscribe to Magento updates and setup process to check for patch releases on a monthly basis.
  • When the plugin vendor. Many simple looking community plugins can be weak on security and leave the site prone to attacks.
  • Train developers to use safe practices.
  • Keep the code repository (svn / git) access secure and passwordless for developer access.
  • Ensure external respository provider web access is secure and with 2-factor authentication.
  • Ensure backup servers are protected with access to write only given to automatic backup process

User Data Protection in Transit

Customer data in transit is the protection of data sent and received from your website to their browser. Internet connections to websites are not direct – the data hops from server to server in between. A server compromised on the way can tap into your data without anyone knowing it.

  • Using https for the complete site is an obvious requirement. Using higher security options in https is even better.
  • Emails sent should not contain sensitive data. A recent Magento patch fixed such an issue – passwords were earlier sent in emails.

Review all access periodically

Setup a security task force that reviews and reports on security once a month.
The charter of the task force would be evaluate the following

  • Who has access the the server and do they yet require this access. The access may be via keys and when a employee or contractor leaves, the key should be removed
  • Who has access to development environment. Have required access been withdrawn appropriately.
  • Who has access to version control systems. Have required access been withdrawn appropriately.
  • Admin access Are all admin users yet needing access?
  • Were admin passwords reset at agreed frequency?
  • Magento Patch status
  • OS patch and update status
  • External security scan (if arranged) status
  • Action taken report for the last period on any security issues
  • Backup and restore process review to ensure data and code is appropriately backed up

Conclusion

Protecting a eCommerce asset like a website in these times of automated hacks is a major challenge. Periodic reviews is a must as the attacks and defenses are quickly evolving. Being a small site is no excuse – the attackers, typically robots do not care.

Watch our webinar on performance and scaling in Magento

Its free!

Using analogy to vehicular traffic we explain performance and scaling in Magento.
Key takeaways

  • Know how to compare hosting options
  • Importance of good code
  • How to scale
  • Tuning Magento
Watch video now!

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.
Click here to schedule the call
May 17, 2017 By Linux System Administration, Magento, Nginx, Security 0 Comments Read more...

Host Anywhere – freedom from hosting provider lockin

Host Anywhere – we will support you

Support should have never been a reason to be locked into a hosting provider. If you depend on either the hosting provider or a support service that is platform specific, changing hosting will be difficult. Such lockins invariably prove to be expensive in the long run. For example, is the free service provided by your hosting provider, really free?

luroConnect provides you freedom from that lockin.

What does Host Anywhere really mean?

Host anywhere means we will support you anywhere you are hosted. If you want to change your hosting provider or evaluate alternative hosting providers, you get an expert independent evaluation from us. Moreover, our evaluation is based on the pattern of hits to your site as well as the dynamics of your specific site. We believe one size does not fit all.

We have partnerships with many hosting providers, so by choosing us you are not loosing benefits of a platform. Infact, we help you maximize that potential.

What does lockin mean?

Lockin can either be direct – multi year contracts or indirect by being dependent on the hosting provider for free services that have been packaged into the cost of hosting.

Can you give example of what hosting independence can mean for us?

  • For a strong disaster recovery plan, it is recommended you have backups taken into a different hosting provider. This is part of our bespoke disaster recovery plan we create for you.
  • Evaluate another cloud or hosting provider – without disrupting your live server, a test with some traffic patterns can be evaluated, either for improved performance or lower costs.

We can analyze your site for free

Schedule a call

Before the call we will analyze your site using public information. We will ask you to give us a 1 day apache or nginx log file of your site. Our half hour consultation will try to identify what steps if any you should take to improve your sites performance goals.

Click here to schedule the call
April 23, 2017 By Comments Off on Host Anywhere – freedom from hosting provider lockin Read more...

Full Stack Support

Full Stack Support

Our full stack support includes all components you use on the site – nginx, mysql, php, Magento / WordPress / standard framework, redis, mongodb, memcached, etc.
We also support any architecture – from single server to autoscaling infrastructure.
In combination of our devops support we help with documentation, automation and reconfiguration of architecture as the application grows.

luroConnect Advanced Stack

Magento 1.x, 2.x, community and enterprise

We support all versions of Magento any sized store / architecture.

Magento is powerful but notorious for its infrastructure maintenance overhead. We take this headache away from you. We support all components of your stack – from nginx, mysql and php to sphinx or SOLR search engine, redis caching and even autoscaling. All our services cater to all components.

nginx

We use nginx as the webserver software, validating requests then routing requests to appropriate servers. Checkout our blog articles on use of nginx.

Configuring php based on your CPU count

php configuration for performance requires knowing how many CPUs are available to run php. For example, if you have a single 8 CPU (4 cores with hyperthreading) and you are running the entire stack including mysql, redis, nginx we would use a max of 6 for simultaneous php processes.

Percona mysql

Great performance from mysql is available from Percona. We use features such as one file per table for write speed, direct write to disk for data safety and optimal cache configuration for speeding up page response.

Secure bulk image upload via sftp

If you bulk upload products, you would need a way to bulk upload images. There is no reliable way of giving a write only access to just the upload folder. Our stack supports uploading to a folder whose content is them magically moved to the final destination.

Automatic image optimization

Magento creates right sized images in the product/cache folder. Our stack automagically optimizes the images at the system level, without a plugin. Original image files are not changed. The compression is lossy for jpeg files with a controllable quality allowing for a right balance between image quality and size. The image creation is CDN compatible.

css & js minifer

css and js files are minfied by a “offline” process. Minified files are sometimes erroneous. This is due to the minifier being strict about syntax. For example a css file with a “//” comment. With our minifier system, you can exclude files.

IP Blacklisting

Blacklist IP addresses that are either suspicious or spamy. Magento reports the IP addresses that perform actions like creating a new registration. Alternatively log files can be analyzed to determine the list. luroConnect / Insight reports top IP addresses hitting the site. Once you know what IPs you do not want, add to our blacklist. Our nginx configuration will do the rest.

IP Rate Limiting (Basic DOS protection)

A basic DOS protection is to restrict number of hits from a single IP address. Our configuration file defaults to enabling this protection. Hits from the same IP address are limited to the rate specified. Hits faster than the rate are first queued and then rejected. luroConnect / Insight reports these for review.

Your own IP addresses can be whitelisted to bypass this check.

BOT Protection

Over 60% of traffic is generated from BOTs or programmatic access to the web server. Many of them are crawlers. While some such as google are essential, however, most of them are not. BOTs use resources and cause a caches to be misled with what is important, both leading to slowdown of web sites. Actively managing BOT access is essential.

Allow private access to the site - the world sees a maintenance page.

When you upgrade your code you need to put the site in maintenance. Magento’s default functionality is a switch – either everyone sees a maintenance page or every one sees the live site. Our private mode, implemented as a nginx configuration, has whiltelist IPs that get to see the live site while other IPs see the maintenance page.

Any php standard framework

We support any standard php frameworks – such as cakephp, laravel 4.x and 5.x.

We discuss your architecture and make a bespoke support plan.

External search - SOLR, Sphinx or Elastic Search

While Magento’s search has improved, external search can give features not possible with mysql search. Magento Commerce / Enterprise supports SOLR and ElasticSearch out of the box. However, requires configuration and management.

ModSecurity based Web Application Firewall (WAF)

(Beta) A WAF based on open source ModSecurity is a standard part of our stack. Having a WAF as close to the server hosting the website is the best configuration. It avoids dual https hops for each request. With our WAF built into nginx, you have a superior edge protection.

Redis caching and sessions

Magento defaults to file system for caches and sessions. We use redis based in-memory cache. With separate redis processes for Magento cache, Full Page Cache and Sessions, Magento sites can easily server TTFB of under 1 second.

Magento 2 on varnish

Magento 2 supports Varnish out-of-the-box for Full Page Cache. However, there are 2 problems – some 3rd party plugins do not support Varnish. Also, given that varnish does not support https, it is difficult to configure. Our Magento 2 stack supports varnish as well as redis based Full Page Cache, allowing maximum flexibility in speed and configuration.

We can analyze your site for free

Schedule a call

Before the call we will analyze your site using public information. We will ask you to give us a 1 day apache or nginx log file of your site. Our half hour consultation will try to identify what steps if any you should take to improve your sites performance goals.

Click here to schedule the call
April 23, 2017 By Comments Off on Full Stack Support Read more...