How Our Scout Bot Works
Welcome! If you've landed on this page, it means you've likely encountered our automated scout-bot, a crucial part of the eShop Scout experience.
Our scout-bot diligently scans the websites of our partnered stores around the clock. Its mission is simple: to find the best special deals, incredible bargains, exciting rollbacks, and exclusive special offers as they become available.
All of its discoveries are then exclusively displayed on our main website, eShopScout.com, providing our users with a curated selection of the best online savings.
Why Our Bot Is Different
Respectful Crawling
Follows robots.txt protocols and respects website resources with optimized crawl rates.
Lightning Fast
Identifies new deals within minutes of publication, ensuring you never miss out.
Smart Filtering
Advanced algorithms filter out irrelevant content, focusing only on genuine deals.
Our Purpose & Your Privacy
The eShop Scout Bot is designed to be a passive and respectful visitor. It operates with the sole purpose of identifying public deal information, much like a human browsing a store's promotions page.
It does not collect any personal data, track user activity, or interact with private information. Its operations are strictly limited to publicly available product and pricing data to fulfill its role in finding deals for our users.
We adhere to best practices for web crawling and respect all standard `robots.txt` protocols. If you are a store owner and have questions about our bot's activity or wish to discuss partnership opportunities, please feel free to contact us.
Frequently Asked Questions
- IP Address: 199.188.200.149
- ASN: AS22612 Namecheap, Inc.
- Current User Agent: eShopScout-Bot/2.2 (+https://eshopscout.com/scout-bot)
- OPT-Out Request: We respect every request from a merchant who explicitly chooses to OPT-Out of our scanning activities.
- Blocked: Our Scout Bot might be blocked by the stores robots.txt rules or by the websites anti-bot detection systems.
- JavaScript Dependence: Some websites dynamically render content by using advanced JavaScript frameworks such as React.js, Vue.js, Angular, etc. Our Scout Bot currently does not use a headless browser and cannot view content rendered by JavaScript at this time. If a website is exclusively dependent on JavaScript to render HTML, the Scout Bot will return a 'Failed_to_Fetch' error.
- No Qualified Items: Some websites simply don't have qualifying products that the Scout Bot is looking for.
- Link Congestion & Crawl Limitations: We place a hard limit to how many pages the Scout Bot is allowed to crawl with respect to each website. If a website has a bunch of noisy links or uses dynamic URLs, the Scout Bot could potentially waste time following those links and stopping short before actually finding a product page. We try to minimize this wasteful use of resources by using advanced filters so that the Scout Bot only follows relevant links that are associated with Sales pages, but its not always perfect.
- Website Structure Inconsistencies: Sometimes our Scout Bot may encounter a website with unusual structure, bad structure, confusing or complex structure, or which uses semantically incorrect html elements. The Scout Bot expects to find products within normal, typical, semantically correct html patterns. When there is a deviation from normal patterns, it may confuse our Scout Bot which might result in a false positive or return no products at all. When there is a false positive, you might see irrelevant content, inaccurate or missing information, or even broken product links. Scout Bot does attempt to account for different variations, but it cannot possibly be successful in every case.
- Deprioritization: The Scout Bot does not just indiscriminately scan stores. Our bot uses an intelligent scoring system which allows it determine the priority and reputation of a merchant based on previous scan results. The scanning frequently is determined by this scoring system. If a store consistently fails to scan or fails to produce relevant products, Scout Bot will deprioritize that merchant and eventually stop scanning that store entirely. This approach helps us save bandwidth, keeps us from wasting resources, and from running useless time consuming network request.
Ready to Explore the Deals?
Discover the amazing deals our bot has found from across the web
Visit eShopScout.com NowNo registration required • Updated 24/7 • Free to use