From 25ab6cd448cbce2871a4f6599cc2b0569eb7f750 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Cau=C3=AA=20Faleiros?= Date: Thu, 11 Jun 2026 11:22:22 -0300 Subject: [PATCH] docs: refresh project context --- CONTEXT.md | 43 +++++++++++++++++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 8 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index d503712..3679808 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -30,9 +30,12 @@ This project (often referred to as "Nexstar Graphs" or simply "Graphs") is a rea ## 3. Architecture & Design Decisions * **Decoupled Client-Server:** The frontend is a statically built SPA served by Nginx, communicating with an isolated Node.js API. * **Idempotent Database Operations:** Due to potential retry/spam from n8n webhooks, database insertions strictly use `INSERT ... ON CONFLICT DO UPDATE SET` (UPSERTs). The backend dynamically generates fallback IDs using composite keys (`Name_Date_Value`) to prevent historical data squashing when explicit `ID_Pedido` fields are missing. -* **"Waiting Room" Debounce Pattern:** To prevent webhook spam during massive inventory updates, the `/api/stock` route intercepts payloads with large deltas (`>= 100`). It parks them in an in-memory dictionary grouped by Base Product Name, accumulates the numbers over a 30-minute setTimeout, and then executes a single aggregated webhook to N8N targeting the Top 100 buyers. +* **Persistent Campaign Queue:** Stock increases are no longer held in memory. Positive stock deltas are inserted into `stock_campaign_queue`, grouped by normalized base product, and processed later by a scheduled n8n workflow. This survives restarts and lets operators inspect/retry campaign work. +* **Scheduled Campaign Processing:** n8n calls `POST /api/internal/process-stock-campaigns` on fixed schedules, currently intended for 12:00 and 18:00 BRT. The endpoint claims ready product groups where accumulated delta is at least 100, combines all ready products into one campaign payload, and sends it to `N8N_WHATSAPP_TRIGGER_URL`. +* **All-Time Top Buyers:** Campaign targeting uses the top 100 buyers across all order history. There is no date filter in this query. Customers are grouped by `cliente_fone` and ordered by total spend (`SUM(quantidade * valor_unitario)`). * **Smart Polling vs. SSE:** Real-time UI updates are handled via client-side polling (`setInterval`) rather than Server-Sent Events (SSE) to bypass persistent connection drops caused by production reverse proxies. -* **Client-Side Analytics:** The backend returns raw data arrays; sorting, mapping, deduplication (e.g., grouping unique orders by time), and chart metric generation are processed dynamically in the React `useMemo` hooks to reduce server load. +* **Backend Analytics Foundation:** Raw `/api/data` still exists for legacy pages, but dashboard/product/client aggregate endpoints now exist under `/api/analytics/*`. Dashboard already uses the backend dashboard aggregation endpoint so it does not wait on the full raw order download. +* **Route Code Splitting:** Frontend routes are lazy-loaded with `React.lazy`/`Suspense` so the first JS payload stays smaller. Recharts-heavy page chunks are loaded on demand. ## 4. Directory Structure ```text @@ -40,7 +43,12 @@ This project (often referred to as "Nexstar Graphs" or simply "Graphs") is a rea ├── .gitea/workflows/ # CI/CD pipeline definitions ├── backend/ # Node.js Express API │ ├── Dockerfile # Backend container definition -│ ├── index.js # Core API logic, DB initialization, and Webhook handlers +│ ├── index.js # Backend entry point +│ ├── db.js # PostgreSQL pool and startup schema/migration SQL +│ ├── routes/ # Express route modules +│ ├── services/ # Business logic and campaign processing +│ ├── mappers/ # Payload normalization helpers +│ ├── test/ # Node test runner tests │ └── package.json ├── public/ # Static assets (Favicons) ├── src/ # React Frontend Application @@ -55,10 +63,16 @@ This project (often referred to as "Nexstar Graphs" or simply "Graphs") is a rea ``` ## 5. Core Business Rules & Domain Entities -* **Order Entity (`orders` table):** Tracks `cliente_nome`, `data_pedido`, `valor_pedido`, `produto_id`, `quantidade`, `valor_unitario`, `pedido_id`, and `cliente_fone`. +* **Order Entity (`orders` table):** Tracks `cliente_nome`, `data_pedido`, normalized `data_pedido_date`, `valor_pedido`, `produto_id`, `quantidade`, `valor_unitario`, `pedido_id`, and `cliente_fone`. * **Stock Entity (`stock` table):** Tracks `produto_id`, `nome`, `saldo` (absolute current inventory), and `delta_estoque`. The database treats the ERP's `saldo` as the Absolute Truth (overwriting existing values rather than performing math) to prevent desynchronization. -* **WhatsApp Marketing Integration:** The system actively extracts phone numbers from incoming n8n payloads (checking `Fone_Cliente`, `fone`, or `celular`). Numbers are exposed in the UI for direct "Click-to-Chat" links and exported to CSV files for bulk marketing. -* **Filter Persistence:** User preferences for Date Ranges, Sort options, and Auto-Refresh intervals are rigidly persisted to `localStorage` to survive page reloads. The "Hoje" (Today) date preset explicitly extends to `23:59:59.999` to ensure incoming real-time webhooks remain visible on the current day's graph. +* **Campaign Queue Entity (`stock_campaign_queue` table):** Tracks queued product restock deltas by `base_product_name`, product ID/name, delta, status (`pending`, `processing`, `sent`, `failed`, `skipped`), attempts, errors, and timestamps. +* **WhatsApp Campaign Payload:** The backend sends one n8n webhook payload per scheduled campaign run. The payload includes `baseProduct`, `productsText`, `total_delta`, `sizes`, `products`, and `customers`. +* **Campaign Product Display Names:** Product display text is customer-facing and may differ from internal grouping. Current aliases: `BASE LISA CAMISETA ...` -> `Camiseta Premium ...`, `BASE LISA OVER SIZE ...` -> `Camiseta Premium Over Size ...`, and `BASE LISA MOLETOM CANGURU ...` -> `Moletom Canguru Premium ...`. +* **Campaign Product List Format:** Because Meta templates collapsed/ignored newlines inside one parameter, `productsText` is a single-line separator list: `Produto 1 • Produto 2 • Produto 3`. There is no leading bullet before the first product. +* **Base Product Normalization:** Apparel size suffixes like `TAMANHO - P`, `- M`, `- G`, `- GG`, and `- M/G/GG` are stripped for campaign grouping. `ETIQUETA...` products are special-cased and preserve their full variant name, e.g. `ETIQUETA BRANCA TAMANHO GG`. +* **Campaign Observability:** The frontend has a `Campanhas` page backed by `/api/campaigns`, `/api/campaigns/preview`, `/api/campaigns/process`, and `/api/campaigns/retry`. +* **WhatsApp Marketing Integration:** The system extracts phone numbers from incoming n8n payloads (checking `Fone_Cliente`, `fone`, or `celular`). Numbers are exposed in the UI for direct "Click-to-Chat" links and exported to CSV files. +* **Filter Persistence:** User preferences for Date Ranges, Sort options, and Auto-Refresh intervals are persisted to `localStorage` to survive page reloads. ## 6. CI/CD & Deployment * **Gitea Actions:** A workflow located in `.gitea/workflows/deploy.yml` triggers on pushes to the `main` branch. @@ -90,10 +104,23 @@ This project (often referred to as "Nexstar Graphs" or simply "Graphs") is a rea npm run build ``` +**Backend Tests:** +```bash +cd backend +npm test +``` + +**Persistent Local Stack:** +```bash +docker compose up -d --build +``` +The local Docker services use `restart: unless-stopped`, so containers should come back after laptop restart if Docker starts. + ## 8. Coding Standards & AI Directives * **Strict Type Safety:** Use explicit TypeScript interfaces (defined in `types.ts`). Avoid `any` where possible. Do not bypass type checks with `// @ts-ignore`. * **Idiomatic React:** Use functional components and hooks (`useState`, `useEffect`, `useMemo`). Complex data transformations (like merging arrays into chart-ready datasets) MUST be wrapped in `useMemo` to prevent unnecessary re-renders. * **Tailwind Architecture:** All styling must be handled via Tailwind CSS utility classes. Avoid custom CSS files unless defining global font families or root variables in `index.css`. * **Robust Data Handling:** Always implement graceful fallbacks for missing data. Never assume an API payload will contain all keys. (e.g., `item.id || item.ID_Pedido || ''`). -* **Database Migrations:** There is no ORM (like Prisma or Sequelize). Table schemas and indexes are managed via raw SQL statements inside the `initDB()` function in `backend/index.js` using `IF NOT EXISTS` clauses for safe startup execution. -* **API Security:** All backend modifications exposing or altering data MUST use the `verifyToken` middleware for frontend requests or `authenticateAPIKey` for external n8n webhooks. \ No newline at end of file +* **Database Migrations:** There is no ORM (like Prisma or Sequelize). Table schemas, indexes, and lightweight data repairs are managed via raw SQL statements inside `initDB()` in `backend/db.js` using `IF NOT EXISTS` and safe startup execution patterns. +* **API Security:** All backend modifications exposing or altering data MUST use the `verifyToken` middleware for frontend requests or `authenticateAPIKey` for external n8n webhooks. +* **Build Discipline:** After frontend/backend behavior changes, run `npm run lint`, `npm run build`, and relevant backend tests. The user prefers builds after changes to catch issues before deployment.