VouDeOfertas
Multi-marketplace price monitoring platform (Amazon, Mercado Livre, Shopee) that surfaces real price history to separate genuine deals from artificial discounts.
Problem
Marketplaces like Amazon, Mercado Livre, and Shopee publish deals without trustworthy historical context. The product challenge was exposing real price history to separate genuine promotions from artificial discounts — and, on the engineering side, unifying scrapers across heterogeneous sources without corrupting the catalog when retries fail.
Solution & impact
A full-stack Turborepo monorepo with batch ingestion, content hashing, and BullMQ queues to keep the pipeline idempotent. Collection is HTTP-first (Undici + Cheerio), with Playwright only when a page blocks or needs rendering. Persistence in MySQL with Redis caching and a Next.js/React frontend for analytical history. The engineering outcome is a flow resilient to blocking and retries — no invented business conversion metrics.
Architecture & trade-offs
Central trade-off: prefer cheap HTTP collection and reserve Playwright for exceptions — higher throughput and lower browser cost, at the price of two collection paths. Batch + hashing prevents duplicate products or prices on retry failures. Redis covers hot history reads; MySQL stays the source of truth. The monorepo simplifies contracts across scraper, API, and UI, with the cost of Turborepo boundary discipline.
Operations & observability
Docker and GitHub Actions for reproducible deploy; healthchecks on services. Per-connector telemetry is a conscious gap — retrospective points to block rate and selector drift metrics as the next investment.
Technical challenges
Marketplace heterogeneity (markup, anti-bot, SPAs), retries that must not duplicate entities, and keeping the pipeline operable with Docker and GitHub Actions without over-engineering observability at this stage.
What I'd do differently
Today I would extract per-marketplace adapters with clearer parsing contracts and invest earlier in per-source failure metrics (block rate, selector drift). HTTP-first + fallback remains the right shape; what I would pull forward is connector telemetry to react faster when a site changes.
Technical context
- Multi-marketplaceSourcesUnified pipeline across heterogeneous scrapers
- HTTP-firstCollectionPlaywright only for exceptions
Documented decisions (ADR)
- ADR 001: HTTP-first scraping
Playwright reserved for exceptions — more throughput and lower cost on the common path.
Stack
- Next.js
- React
- TypeScript
- Node.js
- Express
- MySQL
- Redis
- BullMQ
- Undici
- Cheerio
- Playwright
- Docker
- GitHub Actions