Topic
Automated extraction of data from websites, often used to gather content or pricing at scale; platforms frequently restrict it via technical or legal means.
Apple's undocumented App Store review feed returns well-formed pages with zero entries inside its 10-page range, a DEV Community post reports. A scraper that breaks on the first empty page can report no reviews for an app that has them.
Reality
A KAYAK scraper's first run quoted higher fares than a run three minutes later on 18 of 24 itineraries, its author reports. The poll returns the cheapest fare among sellers who have answered so far, so a count-based stop gives a high price.
Reality
Amadeus shut its free Self-Service API, Hotel Search included, on 17 July, so small developers lack a free official hotel price feed. A dev.to post by a scraper vendor says what remains is reading Google Hotels' public page; selling rooms still takes a supplier contract.
Reality
Booking.com's priceTotal omitted 9-16% of the stay price across 24 search rows, including $175.84 on a New York Hyatt, a scraper author reported. Sheets sorted on that field understate cost until they add the excluded-charges field. Rates differ by city, so cross-city rankings tilt.
Reality
Four of ten rows in a Mercado Libre search were sponsored ad-counter links that hold the product ID only in a query parameter, a scraper author reported. Path readers get no ID from them, and the page's /up/ links defeated the author's own router.
Reality
Tom Watson, a recruitment-firm founder, is accused of running an X account with almost 50,000 followers that doxxed executives who hire minority staff. Companies that name one executive as the public face of diverse hiring now have that person's safety to plan for.
Reality
Cloudflare blocks AI agent and training traffic by default on ad-carrying pages of domains added since September 15, 2026. Search crawlers stay allowed, so a browsing assistant can be refused a page that a search indexer reads freely.
Reality
OpenAI's ChatGPT chief wrote that its products "are largely substitutive, period," in internal documents made public September 17 in the Times lawsuit. That helps publishers argue over AI search, where Fast Company says the money is, but the labs still set the price.
Reality
Reddit will end RSS feeds on Nov. 13 and public API access in March 2027, citing scraping. Moderators and bot builders get migration paths into Reddit's own tools, but people who follow subreddits in a feed reader get no replacement.
Perspective Coverage
8 publishersReality
Google is routing many Search result clicks through a google.com/goto URL, with broad US adoption by late August after tests began in July. Rank trackers, scrapers and referrer-based reports built on a direct link from result to page now have a two-step path to resolve.
Reality
Twenty-one Shopify stores supplied 3,173 of the 4,112 sale items a homelab shopping agent pulled from 85 fashion sources in one run. What each store returned depended on whether its own front end already served structured prices and per-size stock.
Reality
A dev.to guide to GPT-5.6 pricing shows batch halving both token rates and a cheap-first cascade saving money until 9 in 10 calls escalate. Batch is opt-in and caching fails silently on short prefixes, so the default request often pays list price.
Reality
DataDome measured malicious bot activity up 124% in the year to June 2026 against 13.2% growth for humans, and its own detection test found most popular sites challenged none of ten simulated bots.
Reality
Automated traffic is now aimed at the screens where customers sign in and pay, and DataDome's year of request data shows how concentrated that pressure has become. Filtering it means deciding which humans get let in too.
Reality
Seven variations of the old ytInitialPlayerResponse scrape returned zero bytes from a clean residential IP on 2026-09-21. Getting text back now costs either a maintained library or a headless browser per request.
Reality
A developer building an index across 2,127 companies found that guessing a job-board slug from a company name lands on a live board belonging to someone else about as often as it lands on the right one.
Reality
Smash Ventures led the Series B. Firecrawl announced it alongside Alexandria, a cloud service that puts crawled pages, third-party feeds and its own datasets of paper abstracts and code behind one API.
Reality
DataDome says malicious automated traffic grew 124% in the year to June. Its June test of 21,491 popular sites found that claiming a trusted crawler's name is usually enough to get through the door.
Reality
LinkedIn sued last October over a network of bogus accounts it said numbered in the millions. A California judge has now finalized a consent judgment requiring the two firms to stop scraping, stop selling, and destroy what they took.
Reality
The same Camoufox code cleared Mercado Libre's proof-of-work challenge six times in twelve at 4096 MB and nine times in ten at 8192 MB, with the low-memory failures reporting an error that looks exactly like a target hanging up.
Reality
Build · September 5, 2026 · One report1 publisher