ENFRKOantony langlois
work

Vinbot

automationPythonDiscordWeb Scraping

Deal-sniping bot for Vinted · polls an undocumented private API every 5 s behind rotating free proxies · pings Discord seconds after a listing goes live

Architecture

free tier
External · proxy source
ProxyScrape
api.proxyscrape.com
Free French HTTP proxy list · JSON · no key, no cost, no SLA
api/v2
External · target platform
Vinted private API
vinted.fr/api/v2
Undocumented JSON API behind the web app · /catalog/items · /items/{id}
url_data
Config · saved searches
Watchlist
script.py · url_data
15 saved searches: catalog, brand, price cap, target channel
thread
Background · proxy pool
Proxy refresher
script.py · update_proxies_thread
Daemon thread rebuilds the shared proxy pool every 120 s
Python
HTTP client · patched fork
Patched vinted_scraper
vintedWrapperModified.py
Proxy support added · session cookie auto-fetch · retry on 401 · returns None on failure, never crashes
Python
Core · polling loop
Deal poller
script.py · main()
Every 5 s: run all 15 searches through a random proxy, diff against seen items, hydrate new ones, push
5 s cadencerandom proxy per cycleid + channel dedupefirst-pass baselinedetail hydrationskip-on-failure
VPS
Delivery · sibling service
Discord bot API
separate project · hosted on a VPS
Receives flat POST /article payloads · owns embeds, channels, fallback images
SaaS
Delivery · destination
Discord channels
11 price-bucketed channels
Members get pinged the moment a deal lands in their bucket

Every 5 s the Deal poller draws a random proxy from the pool the Proxy refresher keeps rebuilding off ProxyScrape, then runs all 15 Watchlist searches through the patched vinted_scraper against Vinted's /api/v2/catalog/items. Results are diffed against everything already seen, keyed by item id + channel; each genuinely new listing gets one extra /items/{id} call for seller reputation and photos, then lands as a POST /article on the VPS-hosted Discord bot API, which formats the embed and pings the matching price-bucketed channel.

Deployed: Poller → long-running Python process · local machine · Discord bot API → VPS · always-on Discord connection

Outcomes

  • 5 s · poll cadence across 15 saved searches
  • 120 s · proxy pool auto-refresh, zero manual restarts
  • 0 € · proxy cost: rotating free proxies, no paid pool
  • 15 → 11 · searches routed into price-bucketed channels

Skills demonstrated: private API reverse-engineering · dependency patching (proxies, auth retry) · proxy rotation without paid infra · graceful degradation over crash-fast · identity-based dedupe design · producer / consumer service split

Problems and solutions

Every choice answers one constraint: Vinted has no public API and rate-limits aggressive pollers, but underpriced deals sell in minutes.

▤ Speak the private API, not the HTML

Problem: Underpriced items sell within minutes, so alerts must arrive in seconds. Scraping rendered HTML is slow, brittle against every markup change, and easy for the site to fingerprint.

Reverse-engineered the JSON API the Vinted web app itself calls: replay a real session cookie and browser User-Agent against /api/v2/catalog/items, exactly like the frontend does.

  • Rejected browser automation (Selenium): far heavier, slower per poll, and breaks on every redesign
  • Trade-off accepted: an undocumented API can change without notice
  • Bought 5 s polling of 15 searches with tiny JSON payloads instead of full pages

▒ Rotating free proxies with a background refresher

Problem: Polling every 5 s from one home IP gets rate-limited or banned fast, and free proxies die constantly: a stale list would stall the bot mid-run.

Each cycle picks a random proxy from a shared pool that a daemon thread rebuilds from ProxyScrape every 120 s, filtered to French exits to match the store.

  • Rejected paid residential proxies: recurring cost is overkill at hobby scale
  • Trade-off accepted: free proxies are slow and flaky, so every failure must be survivable
  • Bought zero infra cost and no manual intervention when proxies die

▓ Patch the library, do not rewrite the client

Problem: The off-the-shelf vinted_scraper had no proxy support and raised RuntimeError on any non-200 response, killing the whole loop on one flaky proxy.

Modified two files of the library: threaded proxies down into requests, auto-refetch the session cookie on 401 and retry, and return None on failure so the caller skips a bad cycle instead of dying.

  • Rejected a from-scratch requests client: it would re-implement cookie and User-Agent handling the library already does well
  • Rejected try/except at every call site: that scatters error policy across the codebase
  • Bought a poller that survives proxy death and cookie expiry without a restart

▞ Dedupe by item id + channel, in memory

Problem: Newest-first result pages overlap between polls; re-pinging the same listing spams channels and trains members to ignore alerts.

First pass only baselines what already exists. After that, every result is wrapped with its target channel and compared by a custom equality on item id + channel; only unseen pairs alert.

  • Channel is part of the key on purpose: one listing can legitimately alert in both the 50 € and 150 € buckets
  • Rejected a database: a plain in-memory list is enough for one long-running process
  • Trade-off accepted: a restart silently re-baselines, no persistence across runs

░ Two-stage fetch: cheap search, then hydrate

Problem: Search results lack the seller reputation and photos a useful embed needs, but fetching full details for every result would multiply API calls and ban risk.

Poll with cheap 5-items-per-page searches; only listings confirmed new by the dedupe get a second /items/{id} call for full details before posting.

  • Rejected always-hydrating: steady state stays at 15 requests per cycle regardless of result volume
  • New-item bursts are rare, so the extra call is the exception, not the rule
  • Bought richer alerts (stars, review count, country) without raising the baseline request rate

▚ Split the poller from Discord delivery

Problem: Mixing scraping and Discord formatting in one process couples an unstable, ban-prone scraper to the bot's uptime and channel logic.

The poller only produces: it POSTs a flat JSON payload over POST /article to a bot service hosted on a VPS that owns embeds, channel routing, and fallback images for photo-less listings.

  • The VPS keeps the Discord connection up 24/7: a scraper crash or restart never drops the bot
  • Rejected embedding discord.py in the poller: one process, one responsibility
  • Trade-off accepted: two deployments to run and a network hop between them