PWA

luroConnect Magento CI/CD adds meta information to builds

luroConnect CI/CD

luroConnect managed hosting platform includes CI/CD from any git source.

luroConnect CI/CD is an opinionated, platform specific CI/CD. It currently supports Magento (luma, Hyva, ScandiPWA themes), PWAStudio, Angular, nodejs applications. Nodejs builds include yarn and npm based installation.

Builds are done in docker with environment variables. (Note : Our Magento CI/CD also builds in docker without a database).

Builds once made, are ready to be deployed.

Advantage of opinionated CI/CD

  • Built for the target application stack with options.
  • Improved over time in a platform specific way
  • Magento CI/CD builds support features like magepack bundling, css and js minification, luma, Hyva and ScandiPWA themes.
  • Magento CI/CD support multi website and multi store builds
  • PWAStudio supports multi website and mutli store builds with different default currencies.
  • Is built using resources on your AWS account and created to deploy on your AWS account.

Meta Data information

Builds are identified with git commnit-id (shortened). Meta information includes the branch, the build number (generated from git as a serial count of commits in the branch upto this commit-id), and the git comment for this release.

luroConnect dashboard now displays meta data with the current deployed version.

When deciding which version to deploy, meta data information is displayed as well.

Build counts are useful as they are serial numbers – a larger number indicates a later commit serially. Commit comments are useful to know the PR or merge comment used so one is sure of the deployment.

luroConnect build and deploy dashboard

Magento CI/CD metadata

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and autoscale to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for Magento.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimized hosting costs
Click here to schedule the call
August 18, 2023 By CI CD, hosting, Magento, PWA 0 Comments Read more...

Improving PDAStudio performance : Cache more

PWAStudio makes many “storeConfig” calls that – depending on the website – be made cacheable in varnish as they are unlikely to change. This will reduce cookie blocking – or reduce load on php if the authorization is used.

What is the session blocking problem?

When Magento (php) processes a request, the cookie passed is mapped to a session in Magento. Sessions are stored either in var/session or on production systems, in redis.

Magento will lock the session until the php code processes this request. At the end of the request the session is unlocked.

When the session is locked, any hit with the same cookie, is kept in wait state, waiting for the session lock to be released. If there are more than one hits, all hits wait. These are not kept in a ordered queue – instead they each check the state of the session every x seconds and the first one breaking in gets the session lock.

Session lock is important to process hits that update the session – login, cart operations, checkout operations can use sessions. But, the problem is that, not all hits need this lock, and Magento gives no way for a developer to hint that the hit being processed does not need a lock.

The problem with ajax calls – especially PWAStudio – is even more severe. To display a page, many calls for each component on the page are made. If all of these are to be processed by magento, they will all be queued and a single slow query can bring down the site performance.

A PWAStudio Example

As can be seen in the screen below, a page refresh of a PWAStudio website hosted on Magento Commerce Cloud, The arrows highlight the hits that are marked by Magento as non cacheable.

Analyzing the content returned by the getStoreConfig hit.
https://www.soch.com/graphql?query=query+GetStoreConfigForCarouselEE{storeConfig{store_code+product_url_suffix+magento_wishlist_general_is_enabled+enable_multiple_wishlists+__typename}}&operationName=GetStoreConfigForCarouselEE&variables={}

It is clear that these values are unlikely to be changed and can be cached by TTL alone – with a potential ability to clear varnish with a URL / regular expression to help clear the url if an occasion arises.

Improve caching

The hits marked in Blue on the right have been analyzed by luroConnect and we can suggest to cache in varnish on TTL of 1 day. Note : Varnish cache is already hashed with the store key. This is important as some of the data is store specific.

This requires a vcl change.

Yet recommending Magento Commerce Cloud for hosting?

Think again! Analysis like this is only available from luroConnect. Moreover, we will implement the vcl changes.

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and autoscale to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for Magento.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimized hosting costs
Click here to schedule the call
March 3, 2023 By caches, Magento, PWA, Site Speed, varnish 0 Comments Read more...

Meet Magento India 2023 – Speaker Talk – Improve PWA Performance

On Feb 3, 2023, our CEO Pradip Shah presented a talk titled “Improving Headless / PWA Performance” at Meet Magento India in Mumbai, India.

The premise of the talk was the observation that the Headless / PWA eCommerce world promised lightning fast websites but many have ended up with slow performance, especially as measured by Google’s metrics.

This talk discussed issues and solutions that have been implemented in real eCommerce websites to increase page speed.

There are 2 versions of this for you to download

  1. The pdf version (5MB)
  2. The powerpoint version (23MB) – it has a few videos, and hence recommended to download.

Summary

Section 1 : Legacy HTML page technology is not stagnant – with Hyva and on page optimization from technologies like Nitrogen, page speeed scores of Magento traditional HTML page has improved. However, many traditional websites suffer from slower response speeds with uncached pages. Example : www.kalkifashion.com is a Hyva theme with Nigrogen front end caching and optimizaion.

Section 2 : PWA is not magic – the promise of “lightning fast” website needs work. Basic page speed principles apply and so do some new ones.

Section 3 : It is possible to make websites using Magento REST and Graphql. Each one has different challenges. REST by default does not cache and Graphql uses sessions.

Section 4 : When using graphql, avoid POST graphql for queries – use them only for mutations.

Section 5 : Study your application and you can add caching to Graphql’s that Magento may mark as not caceheable.

Section 6 : Custom graphql queries result in database queries that may need optimization. A step by step method to get the underlying mysql query generated is a useful process. We used that to achieve great results. www.mysoresareeudyog.com has a custom default filter for a listing page – latest.

Section 7 : Magento 2.4.5 allows method to prevent session locking using authentication. But by default, authenticated graphql calls are not cached – an approch similar to one presented earlier will be required to ensure optimal caching is achieved.

Section 8 : Go beyond. Add logic in frontend and cache GraphQL results in the browser. This is the ultimate power of headless / PWA. It may not help with page scores that are measured without browser cache, but user experience will improve.

Section 9 : Bulid can improve page loads. Appropriate sized chunks and using hashes instead of version. Another improvement build can give is use of cookie free domains. A static domain for example for js and css bundles allows the main domain to not go through a caching layer, improving page load speed.

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and autoscale to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for Magento.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimized hosting costs
Click here to schedule the call
March 3, 2023 By Magento, page speed, performance, PWA, Site Speed 0 Comments Read more...

Should I move to shopify from Magento?

The platform battle of Magento vs Shopify rages on. Increasingly merchants are moving to Shopify from Magento due to easy of operations and website management.

For many, where the decision was a no-brainer, the move has already been done.
For others, this question rages on, as shopfiy adds more features from itself and its partners in an easy to manage.

This article is written in 2022 for the “others”.

The “others” I refer to have some of these characteristics

  • A working eCommerce business – one that makes money, increasing the risk of a move
  • Migrated to Magento 2 and hence have invested in the technology
  • Customizations that differentiate the store and would be difficult to achieve in Shopify. These may include custom attributes, use of custom options, personalization technology, etc.
  • Headless frontend
  • Worried about data ownership

What are the pain points in Magento that makes merchants look at shopify

  • Managing the Agency or Magento Agencies are expensive!
  • Website speed scores or Magento website speed scores can be bad.
  • Everything is so hard in Magento compared to Shopify

What options you have to ease the pain and stay on Magento?

  • Managing the Agency.In my conversation with merchants unhappy with Magento, the main reason to consider a move is the performance of the Agency – i.e. they have an agency problem, not a platform problem.
    • Agencies have powerful marketing presentations that appeal to the heart of an eCommerce merchant. But, a merchant should pay attention to the compatibility between of the agency to how the business operates before choosing one. Infact, it may be even desirable to manage multiple agency relationships.Some factors to consider.
    • If you have an in-house project manager, you need developers – an strong in development. One of our customers has 2 agencies – a cheaper local and a more established one. Depending on the complexity of the project, they decide which agency will be best. They have their own project manager.
    • Some agencies have a delivery manager that acts as a single point of contact. They work with you on your requirement, discuss with the developers and are responsible for the delivery. Ensure they communicate with all teams – including business and SEO.Some agencies offer a full eCommerce service – including SEO to development. These are very suitable for growing eCommerce arm of a omni channel merchant. Such agencies can work on outcome basis – with a profit share for example.
    • A point of conflict between merchants and agencies is the business model – Agencies like an hourly billing model as it closely matches their biggest cost – the manpower used to execute the project. Many merchants may think of a fixed cost model best suits them – with outcomes well defined. Software on the other hand is very difficult to estimate, especially in an agile development world. Merchants wanting a fixed cost model should be more acceptable to cost escalations due to change requests.
    • Use an expert to interface with an agency. If you have a complex project and do not have an inhouse eCommerce manager, it will be a good idea to hire an consultant to interface with the agency. The consultant can act as a project manager communicating the business needs to developers and ensuring the delivery meets the quality goals.
  • The user interface. From a competitive perspective, UI is the most crucial part that can help you differentiate from your competitors. A more personalized website or an emersive experience will attract more visitors to stay and purchase.But, magento websites are slow on page speed scores, especially those developed on the original Magento 2 theming, which is archaic by todays UI technology standards.The Magento UI stack is evolving. Apart from the many headless options, Hyvä Themes has completely rewritten the theme to be fast and modern.Infact, in my opinion, UI should be the main reason to stay on Magento – the ability to give the best user experience. Rather than have a single page speed goal for the website, have a more user experience view. Even google suggests to have targets for faster website and work towards achieving them.
  • Ease of Shopify vs Magento. A big complaint against Magento is its bloat – too many features, not many, easy to use. A customer recently made this comment when they were trying to enable the watermark in Magento. Turns out, while the watermark feature is shown in Magento’s admin dashboard, enabling it will immediately remove all images – until the developer had accounted for this in their development. The feature bloat meant the developer did not really understand why it would not work. While this feature just works out of the box in shopify.

Magento has a community

Magento has over the years grown to have a very vibrant community. Some estimates put it at over 300,000 strong. Mainly tech professionals, it also consists of many areas of eCommerce business.

For some merchants, it is important to know they (or their online business) is not beholden to a single corporation. A Shopify network for example, though very cohesive, is orchestrated by Shopify. Shopfiy does whet these vendors, but there is no assurance of quality given by Shopify on their behalf. Moreover, Shopfiy can decide to compete with one of the features with any partner in that network, making your favourite plugin obsolete.

Magento on the other hand has independent plugin developers, some of who have made a name for themselves. The agency and merchant have the task of ensuring every plugin they use is good in terms of quality and functionality.

Migration to shopify may not be all roses

On the other hand, shopify’s ease of administration can get in the way of large changes. A process has to be used including using staging site for UAT, git and tools like ThemeKit, Beanstalk and/or the shopify github app. Which brings back the agency and project management question.

Getting feature parity with a Magento website may also not be easy. One merchant moved to Shopify and lost some upsell opportunities as they had used custom options in Magento. They commissioned some custom development and realised that the agency would launch a server and store some data there. This added to a vendor lockin (their agency) they did not want.

luroConnect easing the pain of Magento and Headless/PWA

luroConnect is a managed hosting platform for eCommerce – including Magento and Headless eCommerce.

We believe in serving 2 stake holders in an eCommerce website – the developer / agency and the merchant. We have tools and features to help both. To remove the pain of taking developer code to production. luroConnect supports dev, staging and production environments, with an inbuilt CI/CD for all platforms we support, with low downtime deploys.

The luroConnect stack deploys on the cloud of customers’ choice. The production site is backed by a performance dashboard, configurable alerts and 24×7 support.

Our extensive agency network helps us match a merchant to an agency.

Considering moving from Magento to Shopify? Talk to us. We may just give you a reason to stay back!

We can analyze your site for free

Schedule a call

Not happy with your website performance and want an expert to look at it?

  • We will analyze your site using public information.
  • We will ask you to give us a 1 day web server log file.
  • We will try to identify what steps if any you should take to improve your sites performance goals.
Click here to schedule the call
September 24, 2022 By Magento, PWA, SEO, Site Speed 0 Comments Read more...

Magento and graphql : A slowdown saga

PWA, Magento and graphql : A slowdown saga

Note : I write this article as Magento 2.4.5 is released and fixes some major performance issues that have resulted in the dream of headless and PWA go sour for many early adopters of technologies like PWA Studio and Vue Storefront 2.

PWAs and headless came with the promise of fast navigation and have had a rough ride at best, especially with Magento backends. Early PWAs like Vue Storefront 1 were implemented using Magento Rest API with a api middleware in nodejs that used elastissearch to store cached version of magento catalog for faster access. Magento implemented Graphql to avoid the need for the rest api. Use of direct rest api is ill-advised without a middleware as it exposes a Magento admin token. A middleware would sanitize the request possibly by checking a cookie and also limit the range of queries run. Graphql prevented the use of admin tokens and instead relied on frontend cookies for authentication.Magento’s REST API also results in large datasets. Graphql has a powerful, standardized query syntax to theoretically present a path breaking solution to build Magento headless around. However, early implementations did not fully cover the entire magento data model – Magento even advised the use of REST api when graphql was not sufficient. And when real world websites with large builds and catalogs were deployed, performance issues emerged.

Graphql are ajax calls in Magento that use sessions. In a typical hit processed by Magento, a session is opened, the hit is processed and then the session is closed. (A hit here is either a frontend hit or an ajax call).

Sessions and race conditions

Graphql are ajax calls in Magento that use sessions. In a typical hit processed by Magento, a session is opened, the hit is processed and then the session is closed. A hit is either a frontend hit or an ajax call.

Magento can store information in sessions – cart data for example. In order to prevent 2 hits from writing contents to the session simultaneously – called a race condition, sessions are locked for the duration of the processing a hit. (Take for example a checkout flow and let us assume that the address and payment info is written to session. You would want to ensure both get written. If both “raced”, only one piece of information would be written). Magento identifies an internal session by setting a cookie in the visitors browser. Each hit carries the cookie allowing magento to identify the session. Since a session is unqiue for a visitor, so Magento can process process only one hit per visitor at any given time.

With ajax in general and graphql in particular, many simultaneous hits may go to the server. Magento, through the session locking mechanism, queues them up. The hits are not guaranteed to be processed in the order they were received or any order in particular.

A typical session content in redis looks like

hgetall sess_eba681783f7000bc668cc45005aeaca4
1) "lock"
2) "0"
3) "writes"
4) "5"
5) "data"
6) "_session_validator_data|a:4:{s:11:\"remote_addr\";s:14:\"116.58.200.181\";s:8:\"http_via\";s:0:\"\";s:20:\"http_x_forwarded_for\";s:0:\"\";s:15:\"http_user_agent\";s:117:\"Mozilla/5.0 (Linux; Android 10; RMX2030) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Mobile Safari/537.36\";}_session_hosts|a:1:{s:10:\"host.com\";b:1;}default|a:0:{}customer_base|a:2:{s:11:\"customer_id\";N;s:17:\"customer_group_id\";N;}checkout|a:0:{}"
7) "wait"
8) "0"
9) "pid"
10) "10.0.2.197.host.com|3735"
11) "req"
12) "POST host.com/graphql"
Most of these entries are used internally by the redis session module. Entry 6 is the main session, stored in php serialized format.

If each hit takes only a few milliseconds and the number of ajax calls are not high, the overall response time may not perceptibly slow the site.

But, if even one ajax hit takes time, the page load can slow down. If a customer opens multiple tabs or refreshes the page, the slowdown may be more severe, sometimes leading to errors.

Here is an example. We are analyzing server logs based on IP. Lines 1 and 2 indicate the same link in google was clicked twice – possibly in 2 tabs on the browser. As the page loads we see graphql’s being loaded. The last 2 items are an indication of the problem.

The 2nd last hit is the slow hit – it took 10 seconds to return a result as the item was not in cache. The problem in the last hit was that it was waiting for the 2nd last hit to be processed, although in all probability it was in cache. If the number of items in this queue grows above 6 – for example if a visitor opens tabs and accesses urls not in varnish, a 500 error may be returned, possibly resulting in the UI showing a block not fully loaded at all. A refresh typically seems to solve the problem.

Server can run out of php resources

When too many graphql calls wait, server side can run short of php resources. This time you can see 504 and 502 errors. Adding more php resources for the same server risks running out of RAM. Adding more servers does help with the problem, but that is a waste of resources.

Graphql and caching issues

Graphql in Magento needs many caches to perform well. However, upon cache clear the “galloping horses” problem can bring down a busy website.

Upon cache clear the first access to graphql results in the schema being read. The performance for this action has improved substantially since Magento 2.3, but we yet highlight it as a concern. A cache warming of this is preferred.

Queries like filter in a category page can be slow and hence caching and warming some queries is always a good idea.

Enabling varnish cache is also important – varnish does not use sessions.

Recommended way to use Magento graphql

Graphql needs authorization – it needs to know what customer is asking for the information. It needs it so that the results include say for example the correct tiered pricing for this customer. While sessions include this information, they also store state and hence need the protection of a lock.

Magento supports JWT customer tokens for graphql.
https://devdocs.magento.com/guides/v2.4/graphql/mutations/generate-customer-token.html

Use the generateCustomerToken mutation to create a new customer token.
Example :

mutation {
  generateCustomerToken(
    email: "bobloblaw@example.com"
    password: "b0bl0bl@w"
  )
}

Response

{
  "data": {
    "generateCustomerToken": {
      "token": "ar4116zozoagxty1xjn4lj13kim36r6x"
    }
  }
}

Magento suggests NOT enabling cookie and JWT token authentication at the same time to reduce the chances of encountering problems caused by the differences between the two authorization methods. However, disabling cookie was added a feature only in Magento 2.4.5

However, Magento 2.4.5 varnish sample configuration disables varnish caching for logged in user. This can reduce performance.

Conclusion

Magento performance continues to be a troublesome aspect of the eCommerce platform. At luroConnect we look at performance in great depths.

Would you like to switch to a modern hosting platform?

Schedule a call of a free evaluation!

With features like ~0 downtime code deploy and many scaling options to reduce your hosting costs, luroConnect offers you unparalleled hosting environment for eCommerce – specifically Magento and various PWA / Headless technologies.

Schedule a call and we will show you how we can

  • Improve your hosting, possibly with autoscale
  • Have a managed dev, staging and production environment
  • Server performance measured every minute with alerts for a slowdown
  • A multi point health check every day
  • Optimize hosting costs
Click here to schedule the call
September 12, 2022 By Magento, performance, PWA, Site Speed, varnish 0 Comments Read more...

Choosing Hosting platform for Magento : Digital Ocean

One in a while you may want to check on the technology and options available for Magento hosting – technologies change, hosting costs reduce, life becomes easier for you and the merchant. In this multi-part series we will explore cloud platforms and their suitability to hosting a production Magento website.

In this article we will explore how seriously you should consider Digital Ocean as a platform for hosting.

Digital Ocean (DO) offers VMs it calls “droplets”. With an excellent blog and a friendly attitude towards developers, DO is a serious hosting contender. Many developers naturally recommend using DO to merchants for hosting Magento. How real is it? Let me explore a few pros and cons.

DO Choice of vCPU and memory

DO offers “shared CPU” and “dedicated CPU”. For eCommerce hosting we always prefer dedicated CPU. Shared CPUs work differently on all cloud providers and quite often when you need it the most, you do not get enough. Each vCPU is a Intel hyper-thread. As of this writing (July 2020), we see use of Intel Xeon Gold 6140 @ 2.30 GHz with DDR4 memory at 2.6 GHz. Our simple memory speed test showed about 1.8GHz effective throughput to memory.

DO disk : Love / hate relationship!

I love the disk speed – you get good SSD performance with no throttling. Even for attached storage. Unlike other cloud platforms, you don’t have to juggle with figuring out how many IOPS you need, once you understand for that platform how IOPs translates to speed.

To test the disk I use a simple but effective method to test the disk speed. I create a file with /dev/zero using dd of 1GB. dd gives me the write speed. Here is the command I use :

dd if=/dev/zero of=$tempFile bs=1M count=1024 conv=sync oflag=direct
tempFile has the path to a temp file in the mounted disk I am testing

For both direct disk as well as mounted block device we see 200-350 Mbps disk speeds across all droplets our customers use. This is the best we have seen in cloud platforms. Physical hardware can give upto 450Mbps speeds.

So, why the hate?

We have seen disk crashes – thankfully on staging servers. So, when it comes to production servers, we always recommend customers to have a DR plan to minimize loss of data when this happens. Our suspicion is that the storage is not in a managed RAID, hence a disk crash is a droplet specific event.

Network

File transfer speed test

: A large file transfer using scp on the internal network is done in about 130 MB/sec – about a 1 Gb/sec speed.

NFS

NFS of block storage performs poorly. We use nfscache so most reads are served from cache. However, if there is a need for a large number of reads or writes, the performance can drop dramatically.

Examples of NFS hurting performance

Magento 1 : As images are created in the frontend, there is a check to see if an image file exists before an image url is included in a page. This invariably results in slowdown.

Magento 1 and 2 : large log files. Magento stores log files in a shared folder. Multiple lines per hit of logs will result in slowdown.

Experience :  We were in the process of taking a Magento 2 website live that had about 2500 errors written to a log file. The issue was related to M1 migration resulting in some attributes not defined in M2. App servers saw 30% CPU in I/O wait and the site came to a crawl. Php access logs showed under 10% CPU utilization.

Non availability of autoscale

DO does not come with an autoscale option (you could get autoscale in their Kubernetes which we are not considering here). This means you may have to keep capacity for a holiday sale for example.

Managing server costs

Server costs even when you shut them down

DO charges for servers that are shut down. If you do not want to be charged fully for a server, storing an image is an option.

Recommendation

We host customers on Digital Ocean if they do not have a need for scaling and are willing to have a DR plan. This generally increases the cost of the overall solution.
Also stores that do not need scaling such as :

  • PWA – Vue-storefront based stores which use nodejs
  • A high hit ratio varnish FPC magento store (depends on many factors, but essentially requirement for scaling multiple app servers is less likely)

 

July 31, 2020 By hosting, Magento, performance, PWA, scale, Site Speed 0 Comments Read more...

Deploying a Magento PWA project

Why PWA might be the future of headless eCommerce

Progressive Web Apps (PWAs) are designed to address the mobile revenue gap indicated below.

In most markets, online retail has a higher proportional audience reach among mobile users than desktop. However, mobile sales numbers are much lower. There can be many reasons for this, some in the realm of technology.

At one time it was thought mobile engagement was best achieved using an app. This was based on data for mobile users in general, but was possibly skewed towards gaming and social media. An app has advantages of being able to deliver notifications, use mobile features the way a typically website cannot, be installed as an icon resulting in easier access.

On the flip side, there is data suggesting mobile users were reluctant to install apps due to issues like memory limitations. Other disadvantages include a need for an OK from Apple to be on the app store, lack of easy-to-test infrastructure, the rather slow process of distributing updates – for example some users may not have auto update on.

Using service worker technology, widely supported by browsers today, the PWA (or progressive web application) will give some of the benefits of an app without the cost of consuming memory, requiring an OK from Apple and being as easy to deploy new updates as it is to update a website. PWAs are also as easy to test as a website. A PWA will install on a mobile as an icon – much like an app, service workers allow push notifications as well local storage, allowing for some offline capabilities.

From a technology perspective PWAs pose a completely different problem – the relative newness of technology means developers are limited, as are systems to reliably host & deploy. The cost of development will come down as more developers and websites adopt the technology.

Hosting related issues with PWA for Magento

From a hosting and code deployment perspective. Vue-storefront for example, replicates the entire catalog in elasticsearch and uses 2 nodejs processes to run the frontend of the store. PWA Studio is expected to be Magento (read php) native, yet the reference implementation of its Upward Specifcation Is in nodejs. Both developer and production environments pose challenges.

Developer Environment

It is no more a localhost WAMP stack that you can deploy and get a development environment for PWA setup. A single project will require setup of various components (vue-storefront, vue-storefront-api, Magento, graphql, elasticsearch, redis, rabbitmq, etc).

The developer environment will affect the learning curve of the many new developers starting with these new technologies as well as the productivity of experienced developers.

Here are some challenges

  1. Launch a development environment for a new project
  2. Setup a developer environment for an existing project
  3. An ability to change and test any component easily – for example if a js file was modified that affects the UI, what component(s) need be redeployed? Can this process be automated?

It is too early for us to start work on solutions for developers – since we do not do project work ourselves. However, we are working with our partners and we have an eye on releasing a “developer stack”. Contact us if you would be interested.

Production Environment

We recently took a PWA website live. Developed by our partner Codilar, it was a first for us. Some of the challenges faced and lessons learnt are summarized below.

  1. Setting up a production environment.
    Since there were so many components, not natively supported by Magento, many configuration files had to be manually modified.

    1. Vue-storefront (the UI end that replaces varnish in a classic Magento 2) needs to communicate to Redis for Full Page Cache and the vue-storefont-api
    2. Vue-storefront-api communicates with elasticsearch and Magento 2 backend via a rest API. Ideally vue-storefront should replicate the entire catalog through a indexing process into elasticsearch, but that is not fully operational yet.
    3. Magento 2 has its own redis cache and redis session. Magento 2 FPC is not used. Magento 2.3 uses RabbitMQ in addition to connection to its database.
      Here is our architecture for the deployment. We used Virtual machines as shown. We did not use a containerised architecture. The reasons will possibly be a different blog post.
  1. Starting nodejs processes automatically. Vue-storefront uses pm2 for process management. However, developer information and documentation is written using yarn to run the pm2 processes with log files being stored in ~/.pm2. In order for better control from a system administration perspective, we installed pm2 at the global level, generated systemd files (using pm2 startup) and modified them to suit the environment. We can now use “service vue-storefront start/stop/restart”.
  2. Monitoring all the components.
    Log files for each component are taken to a central log processing server using CNCF project fluentd.
    A key challenge is observability of failures. A Magento 2 API failure is not obvious. An error return code from vue-storefront needs to be traced to vue-storefront-api to Magento. Correlating the actual hit that caused a non-fatal Magento error is another challenge.
  3. How to deploy new code with minimum or even 0 downtime
    For Magento 2, until a database change is required (via a bin/magento setup:upgrade and/or indexing), we have a process to make a deployable package, giving an opportunity to deploy with 0-downtime. Check out our bitbucket pipeline presentation.
  4. How can one deploy a vue-storefront based PWA?
    The project we migrated ran on 2 git repos – one for Magento, the other for vue. Upon deploy we need to find the files that changed since the last release and decide if the change is in Magento, vue-storefront or vue-storefront-api and decide the build steps appropriately. Presently since the repos are different, we have 2 separate builds running on the production servers. A pipeline based deploy is our next step.

Note: We think a monorepo for both Magento and vue is essential in the long run due to possibility of versioning incompatibilities.

Conclusion

This is yet early work-in-progress and we hope to update our process and keep updating this article as we go.

August 12, 2019 By caches, devops, Magento, PWA 0 Comments Read more...

PWA in Magento – how do your images look?

Progressive Web Apps (PWA) in Magento opens a new opportunity to improve image quality on eCommerce websites. Use of good quality images has been known for a long time to be the a key factor in digital experience and hence conversions. In recent years many factors have changed that requires a relook at how images are served. These factors include :

  • Shoppers increasingly use mobile devices for eCommerce
  • The vast improvement in the ability of mobile devices to show high quality images
  • Improvement in the mobile internet connectivity speeds with 4G and above
  • Improvement in compression formats with introduction of webp
  • Advancement in responsive UI technology ranging from HTML picture tag and srcset attribute to PWAs that increase the application awareness of the ability of the mobile device

Through PWAs in particular, the app knows enough of the device to decide what pixel ratio as well as width of the image to get, thereby improving displayed image quality to the best possible.

PWA in Magento is now live. To see a curated list of modern eCommerce sites see https://headless.page/.

Analyzing www.hartsofstur.com for image quality

I picked a magento site and analysed for image quality. The site is the beautifully made. The site looks like a standard magento website – except that on click, the page does not seem to refresh (though the address bar changes URL) – only the content does. That is nicely done.

This site is a PWA created using Deity Falcon – a headless open source library built with ReactJS, NodeJS and GraphQL.

The Category Page

When you navigate to a category listing page from the menu, the category view uses API to get details including images.

The API URL for a category looks like this https://www.hartsofstur.com/api/catalog/products/57. (A real URL has layered navigation and other filters appended).

Such a URL returns category information as json. Let us review a snippet below :

See the “thumnail_url” (3 lines below the highlight) in the json is a standard Magento image URL from the cache. The image comes from this API for each element. The thumbnail URL is not device aware as no hint of the device was passed to the category API.

The website seems to return the same image URL (image size 600×600) for any sized device as shown below.

600×600 image is resized to 165×165 in html

600×600 image is resized to 250×250 in html

Device Pixel Ratios


Using higher Device Pixel Ratios, todays mobile phones give a very high quality image display experience to users. HTML tags have been enhanced to allow their usage. However, many eCommerce sites do not use such features.

No CDN

The other notable hosting of this website is that it does not use a CDN. Use of CDN is every more important for PWAs.

Developing PWA for Magento?

Talk to us about how our hosting stack which includes a image generator, can improve the quality of images you display.

Here is how our stack compares with default Magento for image generation.

  Magento luroConnect
When is an image generated? Pre 2.3, synchronous with frontend

2.3, synchronous with product save

 Asynchronous – queued on product save
Webp support No support Supported – needs img tag change
High resolution 2x, 3x, Retina display Possible with img tag change – but makes synchronous problem worse Possible with img tag change – asynchronously generated
Other image optimizations Not possible In the roadmap
such as progressive jpeg

Request for demo

During the demo we will show you how the luroConnect stack can help your website.

  • Improve the quality of images displayed on your website.
  • Block any malacious hits to the site
  • Divert admin traffic to another server
  • Show what a Disaster Recovery Plan can do
  • See site performance on a dashboard
  • Review your current server size and suggest improvements to gain performance and/or reduce costs of hosting (a $100 value)
March 14, 2019 By Image, Magento, PWA 0 Comments Read more...