FAQs-Algo based personalization

GENERAL QUESTIONS

1)How does the algorithm work?
Please refer this

2)How does it work for new customers?
For completely new customers (first time visitors ever), the algo utilizes its second layer – i.e, gives contextual popular products. Given the customer is from Istanbul and it is June, the algo starts by showing the most selling products in Istanbul in June.
As the customer keeps clicking, the algo learns in real-time, and keeps improving the subsequent page

3)How does the algorithm work for non-signed up customers?
The key customer id for the algo is the cookieID. So, even if the customer is not signed-up, we are able to track the customer through the cookieID
BTW, it is our cookieID.

4)We keep changing our products every so often. How does it take care of new products?Depending on how often you change your products, the algo allocates X% of choices to be coming from new products only – this is essentially to capture data. So, say 80% of your recommendations would be coming from our personalization algo, while 20% of the recommendations would be coming from the new products to capture data.
As new products keep getting old (having enough data that effective personalization algos can work on them), they keep moving to the personalization bucket (80% bucket in the above example).

5)What is the data you use for the algorithm?
We use 3 sets of data for the algo:
User interactions at a cookieID level (for users on website/m-site), customerID level (for all signed-up users), and deviceID level (for users on app) :
Eyeballs
Clicks
Purchases
Add to carts
Add to wishlists
Searches
Filters
Product catalog:
All product info that is displayed on your PDP page
User’s context:
Location from IP
Device details

6)What data would you be collecting from my website/app?
We collect 3 sets of data. Please see answer to question 5

7)Would you need our historical data?
It helps, but is not worth the trouble. We are happy to start afresh with the data that we collect starting when you sign-me-up

8)How will the algorithm help my business?The algorithm will deliver the most personalized and relevant products to each visitor, thereby personalizing the experience of each visitor, more info can be found here.

9)On my home page/ PDP page, I have some category widgets. Can those be personalized as well?
Yes

10)How is it different from other recommendation engines in the market?
Three differences:
Our algos are better, giving you better results. This is primarily because of the fact that we are able to read “Eyeball data” which helps us in differentiating “product seen but not clicked” and “product not seen, and hence not clicked”. You will see this after the integration
We are able to personalize each page (including the PLP which is the most important page). We are able to create new personalization features for your website. This includes Boutique pages, Popups, etc. Further, we will be also able to personalize your emails, notifications, searches and advertisements as we keep developing these features.
Our integration is by far the easiest – 15 minutes of your time with us, vs. 2 months with others.

INTEGRATIONS:

1)How can the algorithm be incorporated to my E-commerce website?
Install our JS onto your website, and leave everything to us

2)How about app? It does not have JS
You are right. App integration is slightly more complicated. So, we start with website, and then, post showing the success, we move to the more complicated app integration in Phase 2

3)How long would integration with the app take?
Integration on the app would take about 40 hours of tech effort, which is usually split in 1-2 client sprints
Additionally, in app, because it is an SDK-based integration, hence going live would also be linked to a new app version release

4)How do you get my data?
My JS is able to read it automatically

5)How do you ensure that the look and feel of my website is maintained?
We copy the CSS files in your website, and replicate the interventions with those CSS files – ensuring that the look and feel of our interventions is EXACTLY the same as yours

6)How do you get my product feed?
Our JS reads your PLP and PDP pages every 2 hours, and gets the product data from there. We might also request you for your product feed, if it is easily available with you.

7)How do you come to know of my new products and products getting out of stock?
Our JS reads your PLP and PDP pages every 4 hours, and gets the product data from there. As you keep adding new products, or your existing products keep getting out of stock, the product gets to know it.
However, ideally, we would be taking this from the feed that you would be giving to us.

8)Will it make my website slower?
No. All our installations are asynchronous. Hence, they CANNOT make your website slower
Our API is super fast – our SLA is a response time of 200 ms for 99% of the hits, which is the fastest in the industry.

9)What if your API fails?
We have a built in fall back – If our API does not respond in 200 ms, the intervention collapses, and your website behaves as if nothing happened – just like it behaves now

10)Where is your server located? What about network latency?
Our server is located in Mumbai on AWS. We use Cloudfront with 1 hour caching, to ensure that we are able to respond within 200 ms anywhere in the world.
If, for some reason, we are not able to meet your latency requirements, we are happy to co-locate the server to shave off another 40-60 ms.

11)How safe is my data? What are the privacy policies?
Your data is as safe or more safe than it is in your website. We use Amazon Web Services to store data. For more information on data security, please visit: https://aws.amazon.com/compliance/data-privacy-faq
We can sign an NDA with you to ensure that your data is safe with us.

12)We use caching on our website. Will it be a hindrance?
We handle it. Essentially, we create a hole in your cached HTML, and show our recommendations in that hole.
The rest of your page remains cached.

PROCESS:

1)How will the integration process work?
Day 0: Install our JS onto your website. If you have access to your product feed, give access to us in any format you have it
Day 1: We integrate the data ingestion and product ingestion, and start getting the interaction data
Day 2 and Day 3: We integrate personalization in your website
Day 4: We make the personalization “limited live”. It becomes available at the following link: www.mywebsite.com/?boxxDemo=true
Day 4, Day 5, Day 6: We, and you do multiple rounds of testing on the limited live, iteratively solving any issues
Day 7, Day 8: We make it live and monitor continuously

2)Will we see the personalization before you make it live?
Absolutely. It will be made “limited live” at the following link: www.mywebsite.com/?boxxDemo=true
Only once you approve it, will we go live for your customers

3)Can you make it live for a limited number of customers to start with?
Yes, we can. We usually start with 1% of customers, and slowly grow to 100% as you keep getting confidence with us.

4)Can we test the recommendations visually before you make them live?
Yes, you can. However, we strongly suggest that you let the data speak for itself, rather than bringing human element the the decision making. The algos know what they are doing - let them fail, fall, and learn. With every recommendation that the customer does not click on, the algo improves.

5)How long does it take before the algos are really good?
The algo ideally needs an average of 100 clicks per product. So, if you have 10,000 products, the algo needs 1Mn PDP views. If you have 1 million visits a month, with 3 PDP views per visit, it means that it would take 10 days.
The algo can start with an average of 20 clicks per product. So, if you have 10,000 products, the algo can start converging after 200K PDP views.

6)Can we test this on staging first?
Yes. But it does NOT serve any purpose:
For the UI testing, we will give you access on limited live at this link: www.mywebsite.com/?boxxDemo=true for you to test anyway
For the accuracy of recommendations, the staging website does not have any data - so the algo does not get any data to train on, and hence testing on staging does not help

POST LIVE PROCESS:

1)What happens after we go live?
Week 1, Week 2: The algo tries a large number of models for different customers in different places. Based on the volume you get, 10 to 30 models are tested. By the end of this time, we would deliver the results panel. While we get ourselves accustomed to the panel, it is too early to read into the results
Week 3, Week 4: The algo learns the optimal mix of models in different scenarios. By end of this, we would be able to start measuring the impact of personalization. Now is a good time to start following the results panel
Week 5, Week 6: During this time, based on the results, we should be moving to longer-term planning of the engagement

2)Do you have a panel from where I can manage my engagement?
We will give you access to a basic panel, through which:
You will be able to switch on and switch off the different “interventions”
You will be able to see the results:
The complete conversion pipeline for each “intervention”
Incremental impact with personalization in an A-B model

3)What do you want from us?
Tech:
Putting the JS on the website
Access to your product feed if you have it easily available
Creation of a blank page: www.mywebsite.com/boutique/…
Biz:
Once we give you access to www.mywebsite.com/?boxxDemo=true, we would require you to test it, and give a sign-off to go live
If you make any big changes to your UI/UX, do let us know before that
Commitment:
Commitment for a longer-term engagement if we show an increase in your key metrics in first 4-6 weeks

4)How about my m-site?
It works exactly same as website. No problem

5)How about my app?
App requires a deeper integration. We will do it in Phase 2

6)So, what exactly are the different phases?
Phase 1:
Personalization on your website and m-site using JS-plugin based quick integration
Phase 2:
Personalization using deeper API-based integration on your website and -site
Personalization on your apps
Possibly (we are still working on this) - personalization of your digital marketing communication - emails, notifications and advertisements

7)Is personalization different on App versus web for the same user? Is so how different and why?
Integration:
Integration for website or m-site is done through our JS plugin, and is super easy for the client
Integration on app needs deeper integration, and hence takes more time for the client
Go live:
Go-live on website is much easier - we can go live whenever we decide to
Go-live on app usually needs an app release
UX and Interventions:
The UX in website and app are somewhat different, and may need different interventions. Example, if a person is about to leave on website, then we can show a retention popup. On app, if a person leaves, we can send a “come back” notification.
User behavior and corresponding algo differences:
User behavior on website and app are different. Hence, the mixer algo could converge to different weights in website, m-site and app
The algo also takes as an input the user device and OS for deciding the choices.Hence, the choices shown may be different in the two

RESULTS:

1)What can we expect in the results?
Month 1: Increase in CTR’s by 20% or higher (upto 50%)
Month 2: Increase in at least one of the following metrics by 5% or higher (upto 15%):
Page views per visitor
ATC per visitor
Purchases per visitor
Month 3: Increase in all of the following metrics by 5% or higher (upto 15%):
Page views per visitor
ATC per visitor
Purchases per visitor
Month 6: Measurable improvement in the following metrics:
Bounce rate
Repeat visitors
Organic traffic
Month 6 - Month 12 (If we decide to go for personalized digital marketing): Measurable improvement in the following metrics:
Email CTOR’s
Notification Open Rates
Advertisement CTR’s
Advertisement revenue per ad dollar spent

MISCELLANEOUS

1)What is the size of the JS?
-40kb

2)What is a flicker effect? How we can avoid it?

-Flicker is a visible shift in UI elements on a webpage after the page is loaded due to insertions/replacements of elements. (Common on Js loaded elements on slow network)

3)Backend integration - How does this work?

-We will provide generic SDKs, API(data injestion, recommendation)

4)Rich relevance and Dynamic yield - Does not do show recommendations on listing page?

-They dont.

5)Stress Test Capability do we have?

-Yes

6)What is customer id? Is it same as unique id?

-Customer id is set by client for the user on that site while unique id is a UUID/GUID stored in cookie by boxx to identify internally.

7)How does customer mapping work in case of personalization?
Default: any user on different browsers will have different boxx token ids
Case 1: logged in user on any browser will have one customer id alloted by Client and one boxx token id
Case 2: non-logged in users will be initiated with empty customer id will be identified by boxx token id, incase of later log in initiated after new cookie allocation , cookies are mapped back to the customer id.

8)How product feed data would be passed? - How many time we can call in what duration?

-Its configurable - Partially answered in Q17

9)In what all manner can we accept Product feed? - Most recommended? SFTP Additional process - Not efficient - Partially answered in Q16

-We accept Product XML feed (as per industry standards)

10)Anytime change in category - We need the latest product feed?

-Yes

11)In case of filters - How product listing will / will not show recommendations

• In case of "Sort by" filtering : Recommendations wont work.

• In case of "Shop by" filtering: Recommendations will work if attributes are present in feed but not hardcoded.

12)What is personal boutique page? How does it work?

-Personal boutique is a curated personalised product listing page based on your previous interactions (eg:view, atc, purchase).