It was 2013. Magento community ver 1.7. We were working with the Chinese mobile phone manufacturer Huawei, developing their Magento website and a flash-sale website that could take 100 orders per minute sustained for 10 minutes. Flash sales by nature have many unpredictable peaks. Our tests indicated Magento could not take that many orders. The bottleneck was the database – or more specifically the locks used in Magento and held for as long as it did, leading to deadlocks or lock wait timeouts. While we optimized the code – like removed order grid update and put into a queue to be done later, we were up on a deadline, so we designed a simple cacheable page that would take a simple form submitting to a csv file and queued to add order to Magento.
This early exercise taught us 2 important principles in scaling Magento – caching and queuing.
We quickly learnt another lesson – we could classify each activity the user did in the flash sale and each had a different need. Login, browsing and checkout. We called this laning – a set of servers with different CPU and memory characteristics and business importance. Once you were selected to be sold a phone, we put you on a checkout server pool.
To circumvent the Magento database issues in orders, we had to split the Magento order database into parts and merge them later. Each part was given max of 100 orders with pre-reserved order ids. When exhausted, the corresponding app server was taken out of the LB. This was our version of sharding – a fourth pillar we built into our scalable architecture.
In early 2017, we launched luroConnect – a business focussed on building technology that ensures performance, scale and security of websites. We modelled our business as a “productized service” – we offer the service but in our domain with the features we offer. Like a product a customer can select us based on features, however, it is a service with a human element involved.
Bootstrapped, we added customers at a pace that we could build our technology – which consisted of
- A scalable architecture – separating edge, app, db, caching, search, etc. Deployed either as VMs or containers inside a VM.
- Security rules for hosting – ensuring appropriate folder security, separation of hosting and runtime users, secure access to logs, exports and imports, hosting rules to prevent access to folders from the browser, securing admin pages, etc.
- luroConnect / Insight – a cloud app that monitors nginx log files and other system parameters such as disk and cache status, a dashboard to show many analytics, an alerting engine that reported slow or error hits, etc.
Our journey continues …
We continue our journey, onboarding customers to the new digital experience. On the way we added features to our stack on customer requirement
- Image optimization – a frantic call from a customer sent us on the journey to solve that problem in a seamless manner for the customer.
- WAF (Web Application Firewall) – a simple Web Application Firewall based on open source mod-security.
This led to our own nginx build and a definition of the “luroConnect Edge”.
Today we have multiple customers onboarded. Our product continues to improve as we listen to our customers. We have almost no churn – indicating customers see continued value in our productized service.
If you think your website is a crucial asset of your business, consider our service. and enhance the digital experience of your visitor. You can start by selecting one of the options below :