Development of In-house Bot Management Solutions and their Pitfalls | Download the White Paper

Development of In-house Bot Management Solutions and their Pitfalls | Download the White Paper

ConsumerAffairs Enhances BI Data Quality With ShieldSquare

ConsumerAffairs is a popular US-based online marketplace that provides listings for over 3000 brands across more than 300 product and service categories. The website features over a million posted reviews to help users navigate through major life events. It also provides buyers guides and consumer news, and is affiliated with a large number of retailers and service providers who are listed on the portal.

Business Problem

As ConsumerAffairs grew its user base, it was beleaguered by bot traffic, which skewed overall traffic data that was used to derive business intelligence. This meant that their Business Intelligence (BI) team was not getting good, usable data to glean insights into their overall traffic. Differentiating between genuine and bot traffic was a time-consuming and frustrating effort for the marketplace as well as its affiliates.

Their BI team then evaluated a leading bot management solution, but found that it returned a high level of False Positives, which meant that some users were being misidentified as bots (and consequently presented with CAPTCHAs to prove they were human). In addition, the solution they first evaluated required DNS redirection, which could increase the site’s latency and slow it down for users. The lack of transparency in the traffic data provided by that solution also proved to be a major hindrance.

ConsumerAffairs then decided to build their own bot management solution, using a User Agent-based string-matching technique to filter bots, as well as logic that flagged any visitor that navigated across several web pages in seconds — something that no human could possibly accomplish. The in-house solution proved to be less than ideal, as sophisticated bot detection and management was not their core competency. Moreover, the complexity of developing and maintaining a solution that could detect increasingly-sophisticated bots led to the company’s decision to abandon their in-house effort and adopt a specialized bot management solution.

ShieldSquare Solution

ConsumerAffairs decided to evaluate ShieldSquare’s performance and detection accuracy by using our Bot Intelligence Feed to analyze and identify bot traffic. Genuine users visiting the site looking for products or services would typically navigate across information pages for at least two or three products or services before clicking on a link that took them to an affiliated retailer or service provider. However, bots in general would directly visit the page from which they would be redirected to an affiliate.

Their BI team leveraged ShieldSquare’s feed to cross-reference the IP addresses and signatures of these bots with the data received from their affiliates to ensure that only real consumers were being taken into account.

ShieldSquare proved to be a satisfactory solution for ConsumerAffairs, as their BI team was able to get data of high quality and integrity. This is turn helped them get better insights into key performance indicators to fine tune their marketing strategy.

Benefits After ShieldSquare Rollout

1. Better understanding of overall traffic to drive marketing strategy

2. Ability to accurately bill affiliates for referrals

3. Minimized latency for better user experience

4. Control over infrastructure without DNS rerouting

AT A GLANCE

Customer:   ConsumerAffairs
Industry:   SaaS & Online Marketplace
Problem faced:
  • Bots distorted overall traffic data and hindered marketing strategies
Why ShieldSquare?
  • Better data quality and integrity were imperative to derive business insights
  • The need for a deeper understanding of overall traffic
  • In-house solution was unsatisfactory

Bot traffic to our marketplace significantly skewed our BI data, which hindered our marketing strategy. With ShieldSquare, we were able to get high quality traffic data which gave us a deeper understanding of user behavior and navigation to better inform our overall product strategy.


Ron Timoshenko
Director of Architecture,
ConsumerAffairs

Step Up and Take Action

Powered by Think201