Njuskalo Stops Content Scraping and Spam Leads with ShieldSquare
Njuskalo, Croatia's largest and most popular marketplace, connects hundreds of thousands of buyers and sellers every day. The platform enables trading between individuals as well as businesses. Njuskalo is the 3rd most visited website in Croatia. The site’s popularity makes it a prime target for competitors, aggregators, and fraudsters.
Hundreds of thousands of users visit Njuskalo every day to buy and sell real estate properties, goods and chattels. Along with these regular users, thousands of bots also crawl Njuskalo. The company is targeted by competitors and fraudsters due to its massive popularity across user segments and age groups.
Before the deployment of ShieldSquare’s bot mitigation solution, advertisements and user information on the Njuskalo portal were scraped by competitors using highly sophisticated human-like bots. Competitors used bots to programmatically fill lead generation forms, resulting in large volumes of spam leads and entries. The significant presence of automated traffic on the website and mobile app degraded performance, impacted user experience, and skewed traffic analytics.
Njuskalo’s technology team was challenged by the persistence of bots on their website and mobile app. The team tried to restrain bot traffic using manual log examination, but the technique proved to be ineffective, unscalable, and time-consuming.
The in-house solution was too cumbersome hence Njuskalo evaluated several bot mitigation solutions and decided that ShieldSquare was the best solution for their requirements.
ShieldSquare helped Njuskalo stop competitors and aggregators from scraping content and user information. Dashboard provided them with granular bot traffic analytics and clear insights into the most-attacked sections of their website.
Competitors and aggregators executed large-scale distributed attacks that were often ‘low and slow’ and went unnoticed by their in-house solution and WAF. The bots were distributed across thousands of IP addresses, and prevailing methods including manual log examination were inadequate in mitigating bots with human-like usage patterns.
ShieldSquare’s proprietary Intent-based Deep Behavior Analysis (IDBA) technology performs behavioral analysis at a higher level of abstraction of ‘intent,’ unlike commonly-used techniques that perform shallow ‘interaction’-based behavior analysis. IDBA helped in understanding the specific intent behind automated attacks (i.e., content and user information scraping) on Njuskalo’s internet properties. Leveraging collective bot intelligence and device fingerprinting technologies, ShieldSquare accurately detected and filtered bad bots across Njuskalo’s web and mobile platforms.
In addition to these machine learning models, ShieldSquare also used challenge-response authentication mechanisms such as CAPTCHAs to ensure zero false positives for Njuskalo. CAPTCHAs were shown to visitors with a higher risk score, and the responses to these challenges helped us build a closed-loop feedback system to minimize false positives down to negligible values.
11% Drop in Ads on Competitor X
Four weeks after the deployment of ShieldSquare, a competitor to Njuskalo (‘Competitor X’ in figure #1) registered an 11% drop in ads, while during the same period, Njuskalo observed growth in their ad volumes. It was earlier observed that Competitor X used to scrape Njuskalo advertisements and repost them on their marketplace. Within the first few weeks of ShieldSquare’s deployment, Njuskalo recorded a 4% growth in ads.
After a month of implementation, Competitor X saw a 32% drop in one of the ad categories, whereas Njuskalo recorded 5% growth (shown in figure #2).
Benefits After ShieldSquare Roll-Out
- Njuskalo strengthened its market position with 4% growth in the total number of ads across all verticals within weeks of implementation.
- Content and user information scraping and spam entries were stopped.
- Spam leads were reduced to a negligible value.
- Eliminated bad bots with zero false positives. There were a few claims of false positives; however, after a detailed investigation, the Njuskalo team confirmed that those were also bots.
AT A GLANCE
- Scraping of advertisements
- Scraping of advertisers’ details through form spam
- Spam leads
- Website performance degradation
- Real-time protection against content scraping and spam leads.
- Intent-based deep behavior analysis to understand the intent behind sophisticated automated attacks.
- Comprehensive analytics that detail the most targeted web pages and categories.
- Track record of ensuring zero false positives across large web applications with millions of page visits every month.
“Together with ShieldSquare, we made our website a much safer environment for our users and their data, further branding Njuskalo.hr as a place to go for buying and selling online.”
Boris Nađ, Technical Operations Manager