149 lines
12 KiB
Markdown
149 lines
12 KiB
Markdown
# Context
|
|
|
|
## 1. Project Overview
|
|
This project (often referred to as "Nexstar Graphs" or simply "Graphs") is a real-time sales and inventory dashboard. Its primary purpose is to ingest live webhook payloads from an external ERP (Tiny ERP) via n8n, securely store that data, and provide a visually rich, responsive dashboard for business analytics. It tracks total sales, product performance, customer behavior, and live inventory levels, while also providing tools for WhatsApp marketing campaigns.
|
|
|
|
## 2. Tech Stack & Tooling
|
|
|
|
**Frontend:**
|
|
* **Library:** React 19.2.5
|
|
* **Language:** TypeScript 6.0.2
|
|
* **Build Tool:** Vite 8.0.10
|
|
* **Styling:** Tailwind CSS 4.2.4
|
|
* **Icons:** Lucide React 1.14.0
|
|
* **Charts:** Recharts 3.8.1
|
|
* **Routing:** React Router DOM 7.14.2
|
|
|
|
**Backend:**
|
|
* **Environment:** Node.js
|
|
* **Framework:** Express 5.2.1
|
|
* **Database:** PostgreSQL (via `pg` 8.20.0)
|
|
* **Authentication:** JWT (`jsonwebtoken` 9.0.3)
|
|
* **CORS & Middleware:** `cors`, `body-parser`
|
|
|
|
**Infrastructure & CI/CD:**
|
|
* **Containerization:** Docker & Docker Compose
|
|
* **Proxy:** Nginx
|
|
* **CI/CD:** Gitea Actions (deploy.yml)
|
|
* **Automation:** n8n (External trigger source)
|
|
|
|
## 3. Architecture & Design Decisions
|
|
* **Decoupled Client-Server:** The frontend is a statically built SPA served by Nginx, communicating with an isolated Node.js API.
|
|
* **Idempotent Database Operations:** Due to potential retry/spam from n8n webhooks, database insertions strictly use `INSERT ... ON CONFLICT DO UPDATE SET` (UPSERTs). The backend dynamically generates fallback IDs using composite keys (`Name_Date_Value`) to prevent historical data squashing when explicit `ID_Pedido` fields are missing.
|
|
* **Persistent Campaign Queue:** Stock increases are no longer held in memory. Positive stock deltas are inserted into `stock_campaign_queue`, grouped by normalized base product, and processed later by a scheduled n8n workflow. This survives restarts and lets operators inspect/retry campaign work.
|
|
* **Scheduled Campaign Processing:** n8n calls `POST /api/internal/process-stock-campaigns` on fixed schedules, currently intended for 12:00 and 18:00 BRT. The endpoint claims ready product groups where accumulated delta is at least 100, combines all ready products into one campaign payload, and sends it to `N8N_WHATSAPP_TRIGGER_URL`.
|
|
* **All-Time Top Buyers:** Campaign targeting uses the top 100 buyers across all order history. There is no date filter in this query. Customers are grouped by `cliente_fone` and ordered by total spend (`SUM(quantidade * valor_unitario)`).
|
|
* **Smart Polling vs. SSE:** Real-time UI updates are handled via client-side polling (`setInterval`) rather than Server-Sent Events (SSE) to bypass persistent connection drops caused by production reverse proxies.
|
|
* **Backend Analytics Foundation:** Raw `/api/data` still exists for legacy pages, but dashboard/product/client aggregate endpoints now exist under `/api/analytics/*`. Dashboard already uses the backend dashboard aggregation endpoint so it does not wait on the full raw order download.
|
|
* **Route Code Splitting:** Frontend routes are lazy-loaded with `React.lazy`/`Suspense` so the first JS payload stays smaller. Recharts-heavy page chunks are loaded on demand.
|
|
|
|
## 4. Directory Structure
|
|
```text
|
|
/
|
|
├── .gitea/workflows/ # CI/CD pipeline definitions
|
|
├── backend/ # Node.js Express API
|
|
│ ├── Dockerfile # Backend container definition
|
|
│ ├── index.js # Backend entry point
|
|
│ ├── db.js # PostgreSQL pool and startup schema/migration SQL
|
|
│ ├── routes/ # Express route modules
|
|
│ ├── services/ # Business logic and campaign processing
|
|
│ ├── mappers/ # Payload normalization helpers
|
|
│ ├── test/ # Node test runner tests
|
|
│ └── package.json
|
|
├── public/ # Static assets (Favicons)
|
|
├── src/ # React Frontend Application
|
|
│ ├── components/ # Reusable UI elements (Layout, DateRangePicker)
|
|
│ ├── pages/ # Route-level views (Dashboard, Products, Clients)
|
|
│ ├── dataService.ts # Centralized API fetch logic and JWT handling
|
|
│ ├── types.ts # Shared TypeScript interfaces
|
|
│ └── main.tsx # React entry point
|
|
├── docker-compose.yml # Local orchestration and environment variable mapping
|
|
├── nginx.conf # Production web server routing
|
|
└── vite.config.ts # Frontend build configuration
|
|
```
|
|
|
|
## 5. Core Business Rules & Domain Entities
|
|
* **Order Entity (`orders` table):** Tracks `cliente_nome`, `data_pedido`, normalized `data_pedido_date`, `valor_pedido`, `produto_id`, `quantidade`, `valor_unitario`, `pedido_id`, and `cliente_fone`.
|
|
* **Stock Entity (`stock` table):** Tracks `produto_id`, `nome`, `saldo` (absolute current inventory), and `delta_estoque`. The database treats the ERP's `saldo` as the Absolute Truth (overwriting existing values rather than performing math) to prevent desynchronization.
|
|
* **Campaign Queue Entity (`stock_campaign_queue` table):** Tracks queued product restock deltas by `base_product_name`, product ID/name, delta, status (`pending`, `processing`, `sent`, `failed`, `skipped`), attempts, errors, and timestamps.
|
|
* **WhatsApp Campaign Payload:** The backend sends one n8n webhook payload per scheduled campaign run. The payload includes `baseProduct`, `productsText`, `total_delta`, `sizes`, `products`, and `customers`.
|
|
* **Campaign Product Display Names:** Product display text is customer-facing and may differ from internal grouping. Current aliases: `BASE LISA CAMISETA ...` -> `Camiseta Premium ...`, `BASE LISA OVER SIZE ...` -> `Camiseta Premium Over Size ...`, and `BASE LISA MOLETOM CANGURU ...` -> `Moletom Canguru Premium ...`.
|
|
* **Campaign Product List Format:** Because Meta templates collapsed/ignored newlines inside one parameter, `productsText` is a single-line separator list: `Produto 1 • Produto 2 • Produto 3`. There is no leading bullet before the first product.
|
|
* **Base Product Normalization:** Apparel size suffixes like `TAMANHO - P`, `- M`, `- G`, `- GG`, and `- M/G/GG` are stripped for campaign grouping. `ETIQUETA...` products are special-cased and preserve their full variant name, e.g. `ETIQUETA BRANCA TAMANHO GG`.
|
|
* **Campaign Observability:** The frontend has a `Campanhas` page backed by `/api/campaigns`, `/api/campaigns/preview`, `/api/campaigns/process`, and `/api/campaigns/retry`.
|
|
* **WhatsApp Marketing Integration:** The system extracts phone numbers from incoming n8n payloads (checking `Fone_Cliente`, `fone`, or `celular`). Numbers are exposed in the UI for direct "Click-to-Chat" links and exported to CSV files.
|
|
* **Filter Persistence:** User preferences for Date Ranges, Sort options, and Auto-Refresh intervals are persisted to `localStorage` to survive page reloads.
|
|
* **RFV Segmentation:** The Portuguese UI calls customer segmentation **RFV** (`Recência`, `Frequência`, `Valor`). Internal code/API names may still use `rfm` to avoid route/type churn. The visible app labels should use RFV.
|
|
* **RFV Date Semantics:** Date filters use local calendar-day boundaries and send stable `YYYY-MM-DD` query params. Backend analytics treats `start` and `end` as inclusive `data_pedido_date` bounds.
|
|
* **RFV Tag Aggregation Rule:** RFV is not recalculated from only the selected period. For a selected period, the backend first calculates each buyer's RFV tag using history **before the period starts**. It then aggregates only the purchases inside the selected period by that prior tag. Example: if a client was `Champions` before today and buys today, `Champions` gets +1 client and that period revenue. If an `Em Risco` client does not buy today, `Em Risco` does not get counted for today's filter.
|
|
* **RFV Display Rule:** The RFV table's `Última Compra` date is the purchase date inside the selected period. Date-only strings must be parsed as local dates in the frontend to avoid UTC shifting (`2026-06-15` must render as `15/06/2026`, not `14/06/2026`). The small recency text below it displays `Hoje`, `Ontem`, or `X dias` relative to the selected range end.
|
|
|
|
## 6. CI/CD & Deployment
|
|
* **Gitea Actions:** A workflow located in `.gitea/workflows/deploy.yml` triggers on pushes to the `main` branch.
|
|
* **Docker Registry:** The pipeline builds the `frontend` and `backend` Docker images and pushes them directly to `gitea.blyzer.com.br/blyzer/`.
|
|
* **Production Deployment:** Updates are deployed manually via Portainer by pulling the `latest` image tags from the Gitea registry and redeploying the stack.
|
|
* **Environment Variables:** Security secrets (`API_KEY`, `JWT_SECRET`, `POSTGRES_PASSWORD`, `N8N_WHATSAPP_TRIGGER_URL`) are injected via the Portainer stack configuration and passed into containers via `docker-compose.yml`.
|
|
|
|
## 7. Environment Setup & Scripts
|
|
|
|
**Running Locally:**
|
|
1. Start the database:
|
|
```bash
|
|
docker compose up -d db
|
|
```
|
|
2. Start the Backend (from `/backend`):
|
|
```bash
|
|
npm install
|
|
npm start
|
|
```
|
|
3. Start the Frontend (from project root):
|
|
```bash
|
|
npm install
|
|
npm run dev # For HMR development
|
|
npm run preview # For production build testing
|
|
```
|
|
|
|
**Building the Frontend:**
|
|
```bash
|
|
npm run build
|
|
```
|
|
|
|
**Backend Tests:**
|
|
```bash
|
|
cd backend
|
|
npm test
|
|
```
|
|
|
|
**Frontend Date/RFV Helper Test:**
|
|
```bash
|
|
node --experimental-strip-types --test src/dateRanges.test.ts
|
|
```
|
|
|
|
**Persistent Local Stack:**
|
|
```bash
|
|
docker compose up -d --build
|
|
```
|
|
The local Docker services use `restart: unless-stopped`, so containers should come back after laptop restart if Docker starts.
|
|
|
|
## 8. Coding Standards & AI Directives
|
|
* **Strict Type Safety:** Use explicit TypeScript interfaces (defined in `types.ts`). Avoid `any` where possible. Do not bypass type checks with `// @ts-ignore`.
|
|
* **Idiomatic React:** Use functional components and hooks (`useState`, `useEffect`, `useMemo`). Complex data transformations (like merging arrays into chart-ready datasets) MUST be wrapped in `useMemo` to prevent unnecessary re-renders.
|
|
* **Tailwind Architecture:** All styling must be handled via Tailwind CSS utility classes. Avoid custom CSS files unless defining global font families or root variables in `index.css`.
|
|
* **Robust Data Handling:** Always implement graceful fallbacks for missing data. Never assume an API payload will contain all keys. (e.g., `item.id || item.ID_Pedido || ''`).
|
|
* **Database Migrations:** There is no ORM (like Prisma or Sequelize). Table schemas, indexes, and lightweight data repairs are managed via raw SQL statements inside `initDB()` in `backend/db.js` using `IF NOT EXISTS` and safe startup execution patterns.
|
|
* **API Security:** All backend modifications exposing or altering data MUST use the `verifyToken` middleware for frontend requests or `authenticateAPIKey` for external n8n webhooks.
|
|
* **Build Discipline:** After frontend/backend behavior changes, run `npm run lint`, `npm run build`, and relevant backend tests. The user prefers builds after changes to catch issues before deployment.
|
|
|
|
## 9. Recent RFV Work
|
|
Recent commits related to RFV/date behavior:
|
|
* `73033ca Aggregate RFM period buyers by prior tag` - backend RFV aggregation now groups period buyers by their prior tag.
|
|
* `ae73a54 Fix RFM period date display` - frontend date-only display and period recency fixed.
|
|
* `9a58b2a Improve RFM recency labels` - period recency labels show `Hoje`/`Ontem`/`X dias`.
|
|
* `f811d3c Rename visible RFM labels to RFV` - visible UI wording changed from RFM to RFV.
|
|
|
|
Files most relevant to RFV:
|
|
* `backend/services/analyticsService.js` - `getRfmAnalytics`, date filters, RFV score/tag logic.
|
|
* `backend/test/analyticsService.test.js` - unit coverage for date filters and RFV aggregation.
|
|
* `src/pages/Rfm.tsx` - RFV page UI, matrix/table/export labels, period recency display.
|
|
* `src/components/DateRangePicker.tsx` and `src/dateRanges.ts` - local calendar date ranges and ISO query param formatting.
|