Solutions

Proxies for price and stock monitoring

Price monitoring means asking the same pages the same questions again and again. Doing it from one address gets noticed fast; spreading the load keeps your data flowing.

Price Monitoring
Residential ProxyFrom TRY 350 / month

The challenge

What makes this job hard

  • Repetitive request patterns

    Regular checks of the same URLs are easy for defences to spot and block.

  • Personalised pricing

    Prices can depend on country, device or browsing history, so results from one setup may not generalise.

  • Volume

    Large catalogs across many retailers add up to millions of requests a month.

The approach

How the right proxies help

  1. Use cheap volume where possible

    Datacenter IPs handle tolerant retail sites at the lowest cost per request.

  2. Use residential for protected sites

    Rotating residential IPs get through where datacenter ranges are filtered.

  3. Target by country

    Select the country whose price you want to record to compare markets consistently.

In depth

The details that decide the outcome

Tiering SKUs and pricing a single observation

Checking every product at the same frequency wastes money on items that rarely change and under-samples the ones that move. Split the catalog into tiers: hot items (best sellers, direct competitors, anything in a promotion) hourly, warm items daily, cold items weekly. For an illustrative 10,000-SKU catalog with 500 hot, 2,500 warm and 7,000 cold SKUs, that is 15,500 fetches a day, about 465,000 a month. Checking all 10,000 hourly would be 7.2 million a month, more than fifteen times as many, with little extra information to show for it.

Now price those 465,000 fetches two ways. On an IPv4 Datacenter Proxy pack of 100 IPs (400 TRY a month) the load is about 11 requests a minute across 100 addresses, roughly 0.1 per address, at about 0.0009 TRY per fetch. On the Residential Proxy pool at an illustrative 100 KB per page, the same fetches are about 46.5 GB, which fits the 50 GB plan (2,500 TRY) at about 0.0054 TRY per fetch. Datacenter is roughly six times cheaper, so use it wherever a retailer accepts it and pay for residential only where it does not.

Spikes need a different plan type. During a seasonal sale you might check the 500 hot SKUs every 10 minutes for 24 hours: 72,000 fetches, about 7.2 GB at 100 KB each, which a 10 GB Residential Proxy plan (650 TRY) covers. The daily unlimited rotating residential plan costs 5,000 TRY, equal to about 167 GB at the 300 GB plan’s 30 TRY per GB, so it only pays off for bursts more than twenty times larger than that. Ask sales about fair-use terms for unlimited plans, which are not published here.

TierSKUs (example)CadenceFetches per month
Hot500Hourly360,000
Warm2,500Daily75,000
Cold7,000Weekly30,000
Total10,000Mixed465,000

Getting the same price twice: normalization

A price monitor is only as good as its comparability. Fix the observation conditions: one exit country per market, one language, a clean session without cookies, one device profile and a timestamp in UTC. Then decide which price you mean: the list price, the current sale price, the per-unit price, or the price including tax and delivery. Retailers present these differently by country, and mixing them produces phantom price cuts. Capture the variant too, because size, color and pack count often carry different prices under one product URL.

Prefer structured sources to scraping the visible layout. Many product pages embed schema.org structured data with the offered price, currency and availability, and many sites load prices from a JSON endpoint that the page itself calls; both are lighter than a full render and less fragile when the design changes. Use only endpoints the page calls publicly, within the site’s terms. If you must render with a headless browser, expect several times the bytes of the HTML alone, so recompute your GB plan size from measured bytes rather than from the HTML size.

FieldWhy it driftsWhat to store
Currency and taxPrices are shown inclusive or exclusive of tax depending on the marketAmount, ISO currency code and a tax-inclusive flag
Sale versus list priceStrike-through prices change on their own scheduleBoth values plus the discount label
VariantSize, color or pack count changes the priceVariant ID with the price, not just the page URL
AvailabilityIn stock, low stock, backorder and out of stock look alike in textAn enumerated status, never inferred from a missing price
Delivery costFree above a threshold or varying by regionDelivery fee and threshold when displayed

Validating, alerting and acting on a change

Separate three events that look alike: a real price change, a parse failure and a block. A missing price field is not a zero price and not an out-of-stock label; record it as unknown and count it in your parse-failure metric. Validate every value against rules before it reaches a dashboard. A currency that differs from the market’s, a price that drops to zero or moves by more than, say, 50 percent in one step, and a page far smaller than the retailer’s usual are all signals to hold the value back for review.

Confirm before alerting. When a value crosses the change threshold you set, re-fetch it from a different IP, and for large moves from a second location a few minutes later; alert only if the observations agree. This one rule removes many false alerts caused by A/B tests, cached edge nodes and momentary errors. Keep raw snapshots, HTML or JSON, for a retention period you choose, so that when someone disputes an alert you can show what the page returned. Store only public product data, never personal information.

Playbook

A practical implementation plan

Four steps from a first test to stable, monitored operation.

  1. Build the SKU map and tiers

    Match each of your products to the competitor URLs and variants, and assign hot, warm or cold cadence by sales impact and price volatility.

  2. Pilot each retailer on the cheapest IP type

    Fetch 500 pages from datacenter IPs, measure the valid-price rate and block rate, and move a retailer to residential only if it misses your threshold.

  3. Normalize and validate

    Store amount, currency, tax flag, variant, availability, exit country and UTC time, and apply plausibility rules before any value reaches a report.

  4. Schedule, alert and review

    Spread checks with jitter and per-domain limits, confirm changes with a second fetch, and review coverage and false-alert rate every week.

Worked scenarios

How this looks in practice

Illustrative examples — the numbers are assumptions to show the reasoning, not measured results.

Small retailer, three competitors

A shop tracks 150 SKUs on three competitor sites twice a day: 900 fetches a day, 27,000 a month. The 30-IP IPv4 Datacenter Proxy pack (250 TRY a month) carries this at about 0.6 requests a minute in total, roughly 0.009 TRY per fetch. The pilot decides whether any of the three sites needs residential instead.

Four-country price comparison

A brand records its own product page in four countries, 60 products daily: 240 fetches and about 36 MB a day at an illustrative 150 KB each, so roughly 1.1 GB a month. A 5 GB Residential Proxy plan (350 TRY) covers it with margin, and every record stores the currency and tax flag so comparisons stay valid.

JavaScript-heavy storefront

A team first renders 3,000 product pages daily in a headless browser at an illustrative 1.5 MB each, which is 4.5 GB a day and 135 GB a month, so a 200 GB plan (7,000 TRY). After finding the JSON endpoint the page itself calls publicly, at about 8 KB per response, the same job is roughly 0.7 GB a month and fits a 5 GB plan (350 TRY).

What to measure

Track these from the first day so problems show up as numbers, not surprises.

  • Coverage, the share of SKUs with a valid observation inside their tier’s window (hourly, daily or weekly).
  • Parse-success rate and block rate per retailer, tracked separately so a layout change is not mistaken for a block.
  • Cost per 1,000 valid observations, combining plan price, retries and wasted bytes.
  • False-alert rate, meaning the share of price-change alerts that a confirming re-fetch did not reproduce.
  • Observation lag, the time from a retailer’s real change to your recorded change, estimated from sampled events.

Common mistakes

Patterns that waste budget or get access blocked.

  • Treating a missing price as zero or as out of stock, which creates phantom price cuts and false availability alerts.
  • Comparing amounts across currencies, tax treatments or variants without normalizing them first.
  • Checking every SKU at the same frequency, overspending on stable items while undersampling volatile ones.
  • Scheduling every job at the top of the hour, a regular and easily recognized pattern that also stacks load; add jitter.
  • Skipping raw snapshots, which leaves no evidence to explain a disputed alert or to repair a parser bug retroactively.

Practical tips

  • Randomise check times and keep requests per minute modest.
  • Fetch only the data you need, not full-page assets.
  • Store the IP country with each price point.
  • Re-check surprising price drops from a second location before acting.

Use proxies responsibly

Monitor publicly displayed information, honour site terms and rate limits, and do not overload small shops. Do not use scraped data in ways that breach competition or data-protection rules.

Datacenter or residential for price tracking?

Try datacenter first for cost; switch to residential for retailers that block datacenter ranges.

How do I avoid personalised prices skewing results?

Use clean sessions without cookies, a consistent location and user agent, and compare like with like.

Can I monitor prices in several countries?

Yes, with the residential pool’s country targeting.

Do I need residential proxies for every retailer?

No. Pilot each retailer on datacenter IPs first and measure the share of valid price responses. Many shops accept low-volume checks from datacenter ranges while others filter them, and only the latter need residential. This keeps the cost per observation low.

How do I handle prices that load with JavaScript?

Look first for structured data embedded in the page, or for a JSON endpoint the page itself calls publicly, because both are far lighter than rendering. If you must use a headless browser, block images, fonts and media and measure bytes per page before sizing a GB plan. Always stay within the site’s terms.

How can I tell an A/B test or personalized price from a real change?

Repeat the fetch from a clean session and, for important changes, from a second IP and location a few minutes apart. A real change shows up in every observation, while a test or personalized price tends to appear in only some. Store the exit country and time with each value so the pattern is visible.

Enterprise

Need volume, custom locations or a procurement-friendly setup?

Talk to sales about large IP counts, long contracts and dedicated onboarding. Tell us the target countries and traffic profile and we will propose a fit.