Njuskalo Stops Content Scraping and Spam Leads with Radware Bot Manager
Njuskalo, Croatia's largest and most popular marketplace, connects hundreds of thousands of buyers and sellers every day. The platform enables trading between individuals as well as businesses. Njuskalo is the 3rd most visited website in Croatia. The site’s popularity makes it a prime target for competitors, aggregators, and fraudsters.
Hundreds of thousands of users visit Njuskalo every day to buy and sell real estate properties, goods and chattels. Along with these regular users, thousands of bots also crawl Njuskalo. The company is targeted by competitors and fraudsters due to its massive popularity across user segments and age groups.
Before the deployment of Radware Bot Manager, advertisements and user information on the Njuskalo portal were scraped by competitors using highly sophisticated human-like bots. Competitors used bots to programmatically fill lead generation forms, resulting in large volumes of spam leads and entries. The significant presence of automated traffic on the website and mobile app degraded performance, impacted user experience, and skewed traffic analytics.
Njuskalo’s technology team was challenged by the persistence of bots on their website and mobile app. The team tried to restrain bot traffic using manual log examination, but the technique proved to be ineffective, unscalable, and time-consuming.
Radware Bot Manager Solution
The in-house solution was too cumbersome hence Njuskalo evaluated several bot mitigation solutions and decided that Radware Bot Manager was the best solution for their requirements.
Radware Bot Manager helped Njuskalo stop competitors and aggregators from scraping content and user information. Dashboard provided them with granular bot traffic analytics and clear insights into the most-attacked sections of their website.
Competitors and aggregators executed large-scale distributed attacks that were often ‘low and slow’ and went unnoticed by their in-house solution and WAF. The bots were distributed across thousands of IP addresses, and prevailing methods including manual log examination were inadequate in mitigating bots with human-like usage patterns.
Radware Bot Manager's proprietary Intent-based Deep Behavior Analysis (IDBA) technology performs behavioral analysis at a higher level of abstraction of ‘intent,’ unlike commonly-used techniques that perform shallow ‘interaction’-based behavior analysis. IDBA helped in understanding the specific intent behind automated attacks (i.e., content and user information scraping) on Njuskalo’s internet properties. Leveraging collective bot intelligence and device fingerprinting technologies, Radware Bot Manager accurately detected and filtered bad bots across Njuskalo’s web and mobile platforms.
In addition to these machine learning models, Radware Bot Manager also used challenge-response authentication mechanisms such as CAPTCHAs to ensure zero false positives for Njuskalo. CAPTCHAs were shown to visitors with a higher risk score, and the responses to these challenges helped us build a closed-loop feedback system to minimize false positives down to negligible values.
11% Drop in Ads on Competitor X
Four weeks after the deployment of Radware Bot Manager, a competitor to Njuskalo (‘Competitor X’ in figure #1) registered an 11% drop in ads, while during the same period, Njuskalo observed growth in their ad volumes. It was earlier observed that Competitor X used to scrape Njuskalo advertisements and repost them on their marketplace. Within the first few weeks of Radware Bot Manager deployment, Njuskalo recorded a 4% growth in ads.
After a month of implementation, Competitor X saw a 32% drop in one of the ad categories, whereas Njuskalo recorded 5% growth (shown in figure #2).
Benefits After Radware Bot Manager Roll-Out
- Njuskalo strengthened its market position with 4% growth in the total number of ads across all verticals within weeks of implementation.
- Content and user information scraping and spam entries were stopped.
- Spam leads were reduced to a negligible value.
- Eliminated bad bots with zero false positives. There were a few claims of false positives; however, after a detailed investigation, the Njuskalo team confirmed that those were also bots.
AT A GLANCE
- Scraping of advertisements
- Scraping of advertisers’ details through form spam
- Spam leads
- Website performance degradation
- Real-time protection against content scraping and spam leads.
- Intent-based deep behavior analysis to understand the intent behind sophisticated automated attacks.
- Comprehensive analytics that detail the most targeted web pages and categories.
- Track record of ensuring zero false positives across large web applications with millions of page visits every month.
“Together with Radware Bot Manager, we made our website a much safer environment for our users and their data, further branding Njuskalo.hr as a place to go for buying and selling online.”
Boris Nađ, Technical Operations Manager