A shoppable video platform for AiBUY

Solution

In the course of four years of intensive collaboration, we implemented the following solutions and processes:

• ML-based product recognition in images and videos
• The AiBUY Overlay management portal
• A microservices-based architecture
• DevOps setup
• Agile process optimization for multiple distributed teams

Shoppable video and image overlays

AiBUY Overlay is our customer’s proprietary web platform allowing users to create videos for all online channels, showcasing products and services paired with calls to action and interactions for frictionless buying.

The solution utilizes machine learning technologies. Based on the item detected in an image or a video to be overlaid, the system powered by computer vision software scans images found in marketing affiliate networks, automatically analyzes them, and provides the closest matching products from the database. The automatic recognition results could be moderated by administrators, and the underlying neural network could be retrained to fit changing business needs.

Our team complemented the solution with the following features:

• Customized overlays for the AiBUY platform, YouTube, images, mobile apps, and HTML content
• Integration with leading ecommerce stores, Shopify and Magento, and the Stripe payment system
• Integration of third-party video players using Video.js
• Setup and integration of Salesforce Commerce Cloud

During our collaboration, we made sure to take into account the users’ feedback regarding possible improvements and such metrics as views count per video, conversions per video, smart store impressions, the number of unique users, views by country, etc. As a result, the AiBUY Overlay platform offers the following features:

• In-media checkout: integration with Shopify, Magento, and Stripe allows paying for items right in the Overlay.
• Link-out: users can visit a third-party website to get additional information or make a purchase.
• Lead generation forms: a convenient communication tool that leads potential customers right to the Overlay.
• Hot spots: admins can highlight specific products or information at certain timestamps in the video.
• Cue points: admins can set cue points at a specific time for viewers to receive messages and calls to action during a video playback.
• Analytics: the video and smart store analytics system lets admins get insight into interactions with buyers and sales and trends metrics.
• Linked smart stores: related stores get triggered at the video’s end or as an ad in the product reel.

Architecture redesign

When we started out, the AiBUY Overlay was operating as a frontend in JavaScript, the management portal and the admin area as a middle tier in PHP, and the monolith Service Bus as a backend in Java. This caused a few issues:

• All services were grouped around the monolithic Service Bus and had uncontrolled access to various internal and external data sources and services.
• To change a certain part of the business logic, AiBUY engineers would have to update several services each in different languages.
• The services used outdated versions of libraries and languages.
• Without a cloud environment, the services were deployed on the company’s own hardware in a local data center.
• There was no centralized logging and collection of performance metrics because of the high expenses for external services and the difficult maintenance for internal ones.

In the course of the architecture redesign, we:

• Introduced microservices
• Migrated the solution to the cloud
• Integrated third-party services
• Isolated the logic from the monolith to microservices and integrated the frontend with the related services to avoid logic duplication and begin legacy code migration
• Refactored the Java code preserving the main business logic, eliminating three years of the accumulated technical debt in the process
• Further minimized the technical debt by covering every iteration with unit and integration tests and rewriting parts of the used monolith apps
• Created a centralized repository of configuration and sensitive data
• Implemented a new communication approach for connecting different system parts using RabbitMQ
• Standardized development approaches

Moving to AWS

Itransition’s team provided AWS consulting services and replaced external services, such as CDN and video transcoding, with AWS to simplify the infrastructure and reduce support and usage costs. The services no longer communicated directly with each other but instead used API Gateway/Load Balancer API and other infrastructure perks to enable seamless updates and horizontal scaling if necessary. As a result, the implemented infrastructure was no longer monolith as we transferred the entire backend to microservices.

The microservices communicated via HTTP, where a synchronous response was needed. To cover asynchronous messages, we enabled communication on top of AWS SQS/SNS using AWS Lambda. Only the Management Portal and AiBUY Overlay were left publicly available and distributed via a CDN.

By the end of the redesign, the project ecosystem included 12 environments running independently and containing more than 30 individual microservices, serverless lambdas, frontend single-page applications, and PHP services each.

Platform customizations for AiBUY clients

As part of ongoing partnership, Itransition helps AiBUY deliver custom features for their clients. Here are the examples of the most notable customizations:

Integration for a podcast mobile app 

For an AiBUY client with a podcast application, our team connected AiBUY Overlay with their mobile solution to enable listeners to buy items mentioned in a podcast.

For this, we configured the Overlay platform, removing video content modules, and created product purchase and payment flows for the podcast application. To enable users to see items mentioned in a podcast, we implemented an overlay widget that opens AiBUY Overlay via a public API. As the podcast listener clicks on cue points for products or additional information, the app displays the corresponding item or redirects them to the information page.  

Integration for a mobile AR app 

We integrated AiBUY Overlay with a mobile AR application aimed at making online experiences more interactive. Our team customized the AiBUY Overlay’s look and feel and implemented a payment widget to allow users to purchase items in-app. 

As a result, the client’s application can access the user’s mobile device camera and overlay AR elements that users can purchase through the AiBUY widget.

Integration for a workout videos website 

Itransition provided another of AiBUY’s clients the feature for integrating sporting goods ads into workout videos on their website. 

To achieve this, we implemented a custom overlay compatible with the functionality of JW Player the company uses as a video player. We also integrated AiBUY Overlay with JW Player API, modified AiBUY look and feel, and added the option of adding an ad video link manually. 

NY fashion week content streaming

Another integration was implemented for New York Fashion Week as we connected AiBUY Overlay with the client’s custom video player and added two essential features.

The first feature was intended for the event live streaming and enabled the client to create a product set in advance and then activate and deactivate relevant items during live streams of runway shows.

The second feature automatically generates slide outs from live events that are displayed during the video playback. It enabled the client to repeatedly activate and deactivate products during the show in on-demand videos without creating new overlays manually.

Results

During this three-year-long partnership, Itransition was fully responsible for the shoppable video platform development, coming up with the most effective solutions to AiBUY’s new ideas.

During our collaboration, we have achieved the following results:

• We deployed machine learning features providing automated product recognition among 1.5 million images.
• We fully redesigned the platform’s architecture from a monolithic to microservices-based one and performed an AWS migration to the cloud.
• We integrated the solution with third-party payment systems, video platforms, Magento, and Shopify.
• We sped up release deployment from 40 hours to 30 minutes.
• We achieved 2.5x higher productivity with better resource utilization and lower downtime.
• We introduced frequent by-feature releases, with the feature delivery costs cut by 30%.