Price Scraping is a method by which the pricing data, along with other information such as product SKUs (Stock Keeping Units), are illegally or unethically copied from e-commerce and travel websites or mobile applications using scraper bots. These bots, usually deployed by competitors and fraudsters, programmatically crawl your product pages to obtain information about your pricing strategies and extract product catalog information and other valuable content from your site. Bots can automatically scrape pricing information from multiple sites, compare them, and undercut competitors to draw more customers.
The Impact of Price Scraping
Undercutting Your Pricing Strategy
Your e-commerce business has its own pricing strategy depending on the markets you cater to and the products you sell. Along with other factors that contribute to the overall customer experience, your pricing strategies influence your consumers’ buying decisions, customer acquisition, and ultimately your revenue. However, when competitors and unauthorized parties scrape your prices and other content such as inventory and product information, they gain an unethical advantage over you and weaken your competitive advantage.
Bots affect your business by not only scraping prices but also by skewing your web analytics with their traffic volumes. They crawl through vast numbers of product pages, artificially boosting page-view statistics, and negatively influence your decision making and business strategy by making it appear as though there is escalating consumer interest in your product lineup.
ShieldSquare Bot Manager leverages domain-specific machine learning techniques to identify visitors’ intent and block scraper bots from affecting business KPIs.
During discount sales and peak shopping periods, scraper bots may add hundreds of products to shopping carts, and abandon them later. These bot-created shopping carts never convert into sales, and only end up reducing your inventory levels and make popular products unavailable to real buyers, who may get frustrated and proceed to shop at one of your competitors.
Our bot detection engine uses semi-supervised machine learning, intent analysis and collective bot intelligence to block price scraping attempts.
As scraper bots crawl your website or mobile app, the load on your servers dramatically increases, affecting website usability and page load times. Genuine buyers end up having a poor experience when exploring products or trying to make a purchase. Another aspect of concern is that your search engine rankings can be downgraded as a result of website slowdowns and outages.
ShieldSquare analyzes a range of visitor characteristics, not just mouse movements and keystrokes, to detect advanced human-like bots that carry out scraping attacks.
How Can Price Scraping be Prevented?
IP blocking and in-house manual bot detection techniques may work for a while, but usually end up being a futile exercise as scrapers change their tactics. Stopping bots requires a dedicated in-house team keeping vigil around the clock. However, this is easier said than done ─ the team should be equipped to detect bot patterns, behaviors, bot sources and devices, and then make an informed decision to block scraper bots. The challenge here is that it is very difficult to ensure zero false positives and ensure that real users are never blocked. When scrapers realize that their bots are being blocked, they devise newer, more sophisticated methods to bypass in-house blocking mechanisms, making it a never-ending cat and mouse game that simply will not work as bots get more sophisticated.
While WAFs are well suited to protect your website from SQL injection, DDoS attacks, cross-site scripting (XSS) and other Web application vulnerabilities, they are not equipped to stop advanced human-like bots that can change IP addresses, devices, and attack methods.
An Analysis of Real-time Sophisticated Attacks On E-commerce Businesses
The Ultimate Guide to Bot Management
Combating Web Scraping in Online Businesses
Learn How a Flight Booking Portal Secured its Booking Engine from Malicious Bots
ShieldSquare Protects Your Business From Price Scraping Attacks
*1 Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
*2 The Forrester New Wave™ is copyrighted by Forrester Research, Inc. Forrester and Forrester Wave™ are trademarks of Forrester Research, Inc. The Forrester New Wave™ is a graphical representation of Forrester's call on a market. Forrester does not endorse any vendor, product, or service depicted in the Forrester New Wave™. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.