WheyScore Backend Node

Problem

Monitoring prices, generating reports, and firing notifications requires reliable background routines — without manual intervention when a job fails or a marketplace changes layout.

Solution & impact

Node.js backend with MySQL, node-cron for tracking and report routines, Winston for structured logs, and webhooks for failure alerts. Docker for reproducible deploy. Product-oriented automation with pragmatic observability, without enterprise APM.

Architecture & trade-offs

Trade-off: node-cron inside the process instead of an external queue (BullMQ/Redis) — sufficient for current volume, with limited retries and isolation. Clear separation between collection jobs, MySQL persistence, and webhook notifications. Winston centralizes logs for post-deploy troubleshooting.

Operations & observability

HTTP healthchecks, Winston structured logs, and webhooks for job failures. No APM dashboard — alerts and logs are the first line of defense. Rollback via Docker container redeploy.

Technical challenges

Ensuring scheduled jobs do not duplicate work after restart, keeping webhooks reliable when external APIs fail, and balancing operational simplicity with future per-job metrics needs.

What I'd do differently

At higher volume I would move critical jobs to a queue with explicit retries. Today cron + webhooks fits; I would add per-job failure counters in logs earlier to reduce diagnosis time.

Documented decisions (ADR)

• ADR 003: node-cron vs external queue

  Cron sufficient for current volume; external queue would add infra without proportional gain.

Stack

• Node.js
• MySQL
• node-cron
• Winston
• Webhooks
• Docker