Monthly Archives - August 2019

Free as in freedom, not free beer!

Introduction

The open source community has had this saying for long, though there are many, including myself, who do not understand what this difference means.

With recent changes to open source license agreements, this difference has come to the fore. For example, changes in the open source licensing of redis and mongodb has restricted how AWS and other cloud providers can conduct their business. Directly relevant to eCommerce merchants is the effect to open source Magento since Adobe’s acquisition of Magento and the path that has been followed

Magento Open Source vs Enterprise

Adobe’s business reason to have Magento is (in my opinion) to complete their offering. Adobe has seen a huge, public and successful transition from their traditional business of one-time purchase of packaged software to a subscription and even a SaaS model. With the acquisition of Magento and the Commerce Cloud licensing model, Adobe clearly thinks Magento should be packaged with hosting – hence the word Cloud in their offering and their pricing of the Enterprise license that includes cloud hosting. This transition is seen in SAP’s hybris commerce offering that includes hosting to make SAP Commerce Cloud. Unlike SAP hybris though, Magento is open source.

If you accept the Adobe Magento Commerce Cloud offering, you submit to the fixed set of features that are offered by their cloud or subscribe to a SaaS service for integration – though sometimes even that requires qualification or may not be possible.

For example, if you want PWA, you are limited and have to wait for the PWA Studio. If you want an improved search interface, you are limited to their choice. Similarly, for CDN, image optimization and Web Application Firewall, you are limited to Fastly, Adobe’s choice in the matter.

Or, perhaps, you use a plugin that is connected to a SaaS service.

When a Magento website is self-hosted though, the choice was to install a plugin or enhance the code that may require a service which has to be hosted. In the examples above, you may want to use vue-storefront or use one of many systems for search or use ImageMagick as an image optimization solution.

From Free Beer to Freedom!

The earlier licensing model of Magento pushed the decision to a “board level” – companies like ours always take the supported version. Mostly the open source version of Magento was attractive to those who were attracted by the “free beer”.

However, now the decision is that of freedom – since the paid version comes with restrictions.

If you feel guilty of being a taker of open source, you can sponsor community commits back to the open source Magento. The community participation is not negligible. Matt Asay of Adobe suggests it may be as high as 50% in this article.

(Indeed there are community participants who think Adobe is gaining from open source contributions, but that is a different blog article).

Shameless plug!

Full stack managed hosting support from luroConnect, gives you the benefit of supported opensource and the flexibility to build your own solution around it. The entire stack is based open source – from linux to Magento – including nginx, ModSecurity, redis, elasticsearch, sphinx and ofcourse mysql. (We relunctantly also allow ioncube encrypted plugins as well). Coupled with a release process from your git. Hosted on any cloud or open hosting providers – in the age of Uber you don’t need to own your data centers. Our multi-layered security approach and proactive monitoring comes standard. With additional features like a disaster recovery plan, image optimization, peep-hole maintenance and a dashboard to monitor and control key tasks such as code deployment or indexing, we bring peace-of-mind to Magento hosting. Check out our pricing and you can connect with us.

Magento Open Source vs Commerce Cloud

Magento Commerce Cloud does offer additional features. See alongside (zoom for a larger view) for a comparison taken from magento.com. Some key features like WYSIWYG editor will never be released in open source.

However, not everyone needs all of the features and some of these features are available from other plugin vendors or custom development from the many certified and non-certified agencies.

Infact, even if you are a Commerce Cloud customer, you will need customizations and potentially more plugins and even 3rdparty SaaS services to have a fully working store.

Conclusion

Magento Open Source is now a very viable option for all stores – brands and high volume stores included. With many options to customize and integrate, you have the freedom to make your own best of breed solution and not be restricted by the Adobe environment. With Managed hosting service you can get optimized and scalable websites.

August 30, 2019 By hosting, Image, Magento, scale 0 Comments Read more...

Deploying a Magento PWA project

Why PWA might be the future of headless eCommerce

Progressive Web Apps (PWAs) are designed to address the mobile revenue gap indicated below.

In most markets, online retail has a higher proportional audience reach among mobile users than desktop. However, mobile sales numbers are much lower. There can be many reasons for this, some in the realm of technology.

At one time it was thought mobile engagement was best achieved using an app. This was based on data for mobile users in general, but was possibly skewed towards gaming and social media. An app has advantages of being able to deliver notifications, use mobile features the way a typically website cannot, be installed as an icon resulting in easier access.

On the flip side, there is data suggesting mobile users were reluctant to install apps due to issues like memory limitations. Other disadvantages include a need for an OK from Apple to be on the app store, lack of easy-to-test infrastructure, the rather slow process of distributing updates – for example some users may not have auto update on.

Using service worker technology, widely supported by browsers today, the PWA (or progressive web application) will give some of the benefits of an app without the cost of consuming memory, requiring an OK from Apple and being as easy to deploy new updates as it is to update a website. PWAs are also as easy to test as a website. A PWA will install on a mobile as an icon – much like an app, service workers allow push notifications as well local storage, allowing for some offline capabilities.

From a technology perspective PWAs pose a completely different problem – the relative newness of technology means developers are limited, as are systems to reliably host & deploy. The cost of development will come down as more developers and websites adopt the technology.

Hosting related issues with PWA for Magento

From a hosting and code deployment perspective. Vue-storefront for example, replicates the entire catalog in elasticsearch and uses 2 nodejs processes to run the frontend of the store. PWA Studio is expected to be Magento (read php) native, yet the reference implementation of its Upward Specifcation Is in nodejs. Both developer and production environments pose challenges.

Developer Environment

It is no more a localhost WAMP stack that you can deploy and get a development environment for PWA setup. A single project will require setup of various components (vue-storefront, vue-storefront-api, Magento, graphql, elasticsearch, redis, rabbitmq, etc).

The developer environment will affect the learning curve of the many new developers starting with these new technologies as well as the productivity of experienced developers.

Here are some challenges

  1. Launch a development environment for a new project
  2. Setup a developer environment for an existing project
  3. An ability to change and test any component easily – for example if a js file was modified that affects the UI, what component(s) need be redeployed? Can this process be automated?

It is too early for us to start work on solutions for developers – since we do not do project work ourselves. However, we are working with our partners and we have an eye on releasing a “developer stack”. Contact us if you would be interested.

Production Environment

We recently took a PWA website live. Developed by our partner Codilar, it was a first for us. Some of the challenges faced and lessons learnt are summarized below.

  1. Setting up a production environment.
    Since there were so many components, not natively supported by Magento, many configuration files had to be manually modified.

    1. Vue-storefront (the UI end that replaces varnish in a classic Magento 2) needs to communicate to Redis for Full Page Cache and the vue-storefont-api
    2. Vue-storefront-api communicates with elasticsearch and Magento 2 backend via a rest API. Ideally vue-storefront should replicate the entire catalog through a indexing process into elasticsearch, but that is not fully operational yet.
    3. Magento 2 has its own redis cache and redis session. Magento 2 FPC is not used. Magento 2.3 uses RabbitMQ in addition to connection to its database.
      Here is our architecture for the deployment. We used Virtual machines as shown. We did not use a containerised architecture. The reasons will possibly be a different blog post.
  1. Starting nodejs processes automatically. Vue-storefront uses pm2 for process management. However, developer information and documentation is written using yarn to run the pm2 processes with log files being stored in ~/.pm2. In order for better control from a system administration perspective, we installed pm2 at the global level, generated systemd files (using pm2 startup) and modified them to suit the environment. We can now use “service vue-storefront start/stop/restart”.
  2. Monitoring all the components.
    Log files for each component are taken to a central log processing server using CNCF project fluentd.
    A key challenge is observability of failures. A Magento 2 API failure is not obvious. An error return code from vue-storefront needs to be traced to vue-storefront-api to Magento. Correlating the actual hit that caused a non-fatal Magento error is another challenge.
  3. How to deploy new code with minimum or even 0 downtime
    For Magento 2, until a database change is required (via a bin/magento setup:upgrade and/or indexing), we have a process to make a deployable package, giving an opportunity to deploy with 0-downtime. Check out our bitbucket pipeline presentation.
  4. How can one deploy a vue-storefront based PWA?
    The project we migrated ran on 2 git repos – one for Magento, the other for vue. Upon deploy we need to find the files that changed since the last release and decide if the change is in Magento, vue-storefront or vue-storefront-api and decide the build steps appropriately. Presently since the repos are different, we have 2 separate builds running on the production servers. A pipeline based deploy is our next step.

Note: We think a monorepo for both Magento and vue is essential in the long run due to possibility of versioning incompatibilities.

Conclusion

This is yet early work-in-progress and we hope to update our process and keep updating this article as we go.

August 12, 2019 By caches, devops, Magento, PWA 0 Comments Read more...