graphs/CONTEXT.md

# Context

## 1. Project Overview
This project (often referred to as "Nexstar Graphs" or simply "Graphs") is a real-time sales and inventory dashboard. Its primary purpose is to ingest live webhook payloads from an external ERP (Tiny ERP) via n8n, securely store that data, and provide a visually rich, responsive dashboard for business analytics. It tracks total sales, product performance, customer behavior, and live inventory levels, while also providing tools for WhatsApp marketing campaigns.

## 2. Tech Stack & Tooling

**Frontend:**
*   **Library:** React 19.2.5
*   **Language:** TypeScript 6.0.2
*   **Build Tool:** Vite 8.0.10
*   **Styling:** Tailwind CSS 4.2.4
*   **Icons:** Lucide React 1.14.0
*   **Charts:** Recharts 3.8.1
*   **Routing:** React Router DOM 7.14.2

**Backend:**
*   **Environment:** Node.js
*   **Framework:** Express 5.2.1
*   **Database:** PostgreSQL (via `pg` 8.20.0)
*   **Authentication:** JWT (`jsonwebtoken` 9.0.3)
*   **CORS & Middleware:** `cors`, `body-parser`

**Infrastructure & CI/CD:**
*   **Containerization:** Docker & Docker Compose
*   **Proxy:** Nginx
*   **CI/CD:** Gitea Actions (deploy.yml)
*   **Automation:** n8n (External trigger source)

## 3. Architecture & Design Decisions
*   **Decoupled Client-Server:** The frontend is a statically built SPA served by Nginx, communicating with an isolated Node.js API.
*   **Idempotent Database Operations:** Due to potential retry/spam from n8n webhooks, database insertions strictly use `INSERT ... ON CONFLICT DO UPDATE SET` (UPSERTs). The backend dynamically generates fallback IDs using composite keys (`Name_Date_Value`) to prevent historical data squashing when explicit `ID_Pedido` fields are missing.
*   **Persistent Campaign Queue:** Stock increases are no longer held in memory. Positive stock deltas are inserted into `stock_campaign_queue`, grouped by normalized base product, and processed later by a scheduled n8n workflow. This survives restarts and lets operators inspect/retry campaign work.
*   **Scheduled Campaign Processing:** n8n calls `POST /api/internal/process-stock-campaigns` on fixed schedules, currently intended for 12:00 and 18:00 BRT. The endpoint claims ready product groups where accumulated delta is at least 100, combines all ready products into one campaign payload, and sends it to `N8N_WHATSAPP_TRIGGER_URL`.
*   **All-Time Top Buyers:** Campaign targeting uses the top 100 buyers across all order history. There is no date filter in this query. Customers are grouped by `cliente_fone` and ordered by total spend (`SUM(quantidade * valor_unitario)`).
*   **Smart Polling vs. SSE:** Real-time UI updates are handled via client-side polling (`setInterval`) rather than Server-Sent Events (SSE) to bypass persistent connection drops caused by production reverse proxies.
*   **Backend Analytics Foundation:** Raw `/api/data` still exists for legacy pages, but dashboard/product/client aggregate endpoints now exist under `/api/analytics/*`. Dashboard already uses the backend dashboard aggregation endpoint so it does not wait on the full raw order download.
*   **Route Code Splitting:** Frontend routes are lazy-loaded with `React.lazy`/`Suspense` so the first JS payload stays smaller. Recharts-heavy page chunks are loaded on demand.
*   **Analytics Client Cache:** Frontend analytics requests use a small shared cache/deduping layer in `src/dataService.ts`. Navigating between Dashboard, RFV, Products, and Clients should reuse fresh responses instead of refetching every route mount unless query params change or the cache expires.

## 4. Directory Structure
```text
/
├── .gitea/workflows/      # CI/CD pipeline definitions
├── backend/               # Node.js Express API
│   ├── Dockerfile         # Backend container definition
│   ├── index.js           # Backend entry point
│   ├── db.js              # PostgreSQL pool and startup schema/migration SQL
│   ├── routes/            # Express route modules
│   ├── services/          # Business logic and campaign processing
│   ├── mappers/           # Payload normalization helpers
│   ├── test/              # Node test runner tests
│   └── package.json       
├── public/                # Static assets (Favicons)
├── src/                   # React Frontend Application
│   ├── components/        # Reusable UI elements (Layout, DateRangePicker)
│   ├── pages/             # Route-level views (Dashboard, Products, Clients)
│   ├── dataService.ts     # Centralized API fetch logic and JWT handling
│   ├── types.ts           # Shared TypeScript interfaces
│   └── main.tsx           # React entry point
├── docker-compose.yml     # Local orchestration and environment variable mapping
├── nginx.conf             # Production web server routing
└── vite.config.ts         # Frontend build configuration
```

## 5. Core Business Rules & Domain Entities
*   **Order Entity (`orders` table):** Tracks `cliente_nome`, `data_pedido`, normalized `data_pedido_date`, `valor_pedido`, `produto_id`, `quantidade`, `valor_unitario`, `pedido_id`, and `cliente_fone`. Tiny ERP metadata is also persisted when present: `cliente_nome_fantasia`, `id_vendedor`, `nome_vendedor`, `marketplace`, `canal_venda`, and `numero_ecommerce`.
*   **Order Metadata Preservation:** Order upserts preserve existing non-empty phone and Tiny metadata values when incoming backfill rows send `NULL` or empty strings. Core mutable order/item fields still overwrite normally.
*   **Client Identity Rule:** Analytics groups clients by a stable normalized identity so adding a phone number later does not split an existing buyer into a duplicate client. When a phone is added after historical purchases without phone, the old and new rows should resolve to the same client profile and history.
*   **Stock Entity (`stock` table):** Tracks `produto_id`, `nome`, `saldo` (absolute current inventory), and `delta_estoque`. The database treats the ERP's `saldo` as the Absolute Truth (overwriting existing values rather than performing math) to prevent desynchronization.
*   **Campaign Queue Entity (`stock_campaign_queue` table):** Tracks queued product restock deltas by `base_product_name`, product ID/name, delta, status (`pending`, `processing`, `sent`, `failed`, `skipped`), attempts, errors, and timestamps.
*   **WhatsApp Campaign Payload:** The backend sends one n8n webhook payload per scheduled campaign run. The payload includes `baseProduct`, `productsText`, `total_delta`, `sizes`, `products`, and `customers`.
*   **Campaign Product Display Names:** Product display text is customer-facing and may differ from internal grouping. Current aliases: `BASE LISA CAMISETA ...` -> `Camiseta Premium ...`, `BASE LISA OVER SIZE ...` -> `Camiseta Premium Over Size ...`, and `BASE LISA MOLETOM CANGURU ...` -> `Moletom Canguru Premium ...`.
*   **Campaign Product List Format:** Because Meta templates collapsed/ignored newlines inside one parameter, `productsText` is a single-line separator list: `Produto 1 • Produto 2 • Produto 3`. There is no leading bullet before the first product.
*   **Base Product Normalization:** Apparel size suffixes like `TAMANHO - P`, `- M`, `- G`, `- GG`, and `- M/G/GG` are stripped for campaign grouping. `ETIQUETA...` products are special-cased and preserve their full variant name, e.g. `ETIQUETA BRANCA TAMANHO GG`.
*   **Campaign Observability:** The frontend has a `Campanhas` page backed by `/api/campaigns`, `/api/campaigns/preview`, `/api/campaigns/process`, and `/api/campaigns/retry`.
*   **WhatsApp Marketing Integration:** The system extracts phone numbers from incoming n8n payloads (checking `Fone_Cliente`, `fone`, or `celular`). Numbers are exposed in the UI for direct "Click-to-Chat" links and exported to CSV files.
*   **Filter Persistence:** User preferences for Date Ranges, Sort options, and Auto-Refresh intervals are persisted to `localStorage` to survive page reloads.
*   **RFV Segmentation:** The Portuguese UI calls customer segmentation **RFV** (`Recência`, `Frequência`, `Valor`). Internal code/API names may still use `rfm` to avoid route/type churn. The visible app labels should use RFV.
*   **RFV Lifecycle Thresholds:** The active lifecycle is monthly, not yearly. Recency score 3 means last purchase in `0-7` days, score 2 means `8-15` days, score 1 starts at `16+` days, and clients with no purchase for `30+` days are forced into `Perdido`.
*   **RFV Date Semantics:** Date filters use local calendar-day boundaries and send stable `YYYY-MM-DD` query params. Backend analytics treats `start` and `end` as inclusive `data_pedido_date` bounds.
*   **RFV Tag Aggregation Rule:** RFV is not recalculated from only the selected period. For a selected period, the backend first calculates each buyer's RFV tag using history **before the period starts**. It then aggregates only the purchases inside the selected period by that prior tag. Example: if a client was `Champions` before today and buys today, `Champions` gets +1 client and that period revenue. If an `Em Risco` client does not buy today, `Em Risco` does not get counted for today's filter.
*   **RFV Display Rule:** The RFV table's `Última Compra` date is the purchase date inside the selected period. Date-only strings must be parsed as local dates in the frontend to avoid UTC shifting (`2026-06-15` must render as `15/06/2026`, not `14/06/2026`). The small recency text below it displays `Hoje`, `Ontem`, or `X dias` relative to the selected range end.
*   **Client Page Filters:** The main Clients page has one filter button plus client search. The filter panel contains date shortcuts/custom dates, sorting, marketplace, sales channel, seller, and client type. Marketplace/channel/seller options come from distinct non-empty order metadata across all orders so options remain visible regardless of the current date window.
*   **Seller Metadata Display:** Seller labels are cleaned for UI display. All-caps seller names are title-cased, trailing Tiny IDs like `#977226210` are stripped from labels, and leading fantasy-name prefixes like `[3]` are removed for display. IDs may still be used internally as stable filter values.
*   **Seller Dashboard Charts:** The main Dashboard includes seller performance charts above the existing charts, using backend aggregates for revenue and order count by seller.

## 6. CI/CD & Deployment
*   **Gitea Actions:** A workflow located in `.gitea/workflows/deploy.yml` triggers on pushes to the `main` branch.
*   **Docker Registry:** The pipeline builds the `frontend` and `backend` Docker images and pushes them directly to `gitea.blyzer.com.br/blyzer/`.
*   **Production Deployment:** Updates are deployed manually via Portainer by pulling the `latest` image tags from the Gitea registry and redeploying the stack.
*   **Environment Variables:** Security secrets (`API_KEY`, `JWT_SECRET`, `POSTGRES_PASSWORD`, `N8N_WHATSAPP_TRIGGER_URL`) are injected via the Portainer stack configuration and passed into containers via `docker-compose.yml`.

## 7. Environment Setup & Scripts

**Running Locally:**
1. Start the database:
   ```bash
   docker compose up -d db
   ```
2. Start the Backend (from `/backend`):
   ```bash
   npm install
   npm start
   ```
3. Start the Frontend (from project root):
   ```bash
   npm install
   npm run dev      # For HMR development
   npm run preview  # For production build testing
   ```

**Building the Frontend:**
```bash
npm run build
```

**Backend Tests:**
```bash
cd backend
npm test
```

**Frontend Date/RFV Helper Test:**
```bash
node --experimental-strip-types --test src/dateRanges.test.ts
```

**Persistent Local Stack:**
```bash
docker compose up -d --build
```
The local Docker services use `restart: unless-stopped`, so containers should come back after laptop restart if Docker starts.

## 8. Coding Standards & AI Directives
*   **Strict Type Safety:** Use explicit TypeScript interfaces (defined in `types.ts`). Avoid `any` where possible. Do not bypass type checks with `// @ts-ignore`.
*   **Idiomatic React:** Use functional components and hooks (`useState`, `useEffect`, `useMemo`). Complex data transformations (like merging arrays into chart-ready datasets) MUST be wrapped in `useMemo` to prevent unnecessary re-renders.
*   **Tailwind Architecture:** All styling must be handled via Tailwind CSS utility classes. Avoid custom CSS files unless defining global font families or root variables in `index.css`.
*   **Robust Data Handling:** Always implement graceful fallbacks for missing data. Never assume an API payload will contain all keys. (e.g., `item.id || item.ID_Pedido || ''`).
*   **Database Migrations:** There is no ORM (like Prisma or Sequelize). Table schemas, indexes, and lightweight data repairs are managed via raw SQL statements inside `initDB()` in `backend/db.js` using `IF NOT EXISTS` and safe startup execution patterns.
*   **API Security:** All backend modifications exposing or altering data MUST use the `verifyToken` middleware for frontend requests or `authenticateAPIKey` for external n8n webhooks.
*   **Build Discipline:** After frontend/backend behavior changes, run `npm run lint`, `npm run build`, and relevant backend tests. The user prefers builds after changes to catch issues before deployment.

## 9. Recent RFV Work
Recent commits related to RFV/date and client metadata behavior:
*   `4131bf9 Update RFV lifecycle to monthly windows` - RFV recency/lost thresholds changed to `0-7`, `8-15`, `16-29`, and `30+` lost.
*   `ef27cff Add seller performance dashboard charts` - dashboard now includes seller revenue/order charts.
*   `ca9615a Clean seller metadata display names` - seller/fantasy labels are normalized for display.
*   `b129307 Fix seller filter options query` - seller filter options are selected reliably from order metadata.
*   `b40f6fa Restore seller dropdown filter` - seller filter uses a dropdown option list again.
*   `aeb801c Allow searching sellers in client filters` - seller filter UI supports searching.
*   `b7a2059 Show all client metadata filter options` - client metadata filter options are global, not date-scoped.
*   `06faada Refresh client filter options reliably` - filter option fetching was made more robust.
*   `7d60073 Smooth analytics page reloads` - analytics fetches now reuse cached/deduped results between pages.
*   `335d231 Fix client identity when phone is added` - client grouping prevents phone-added duplicate profiles.
*   `73033ca Aggregate RFM period buyers by prior tag` - backend RFV aggregation now groups period buyers by their prior tag.
*   `ae73a54 Fix RFM period date display` - frontend date-only display and period recency fixed.
*   `9a58b2a Improve RFM recency labels` - period recency labels show `Hoje`/`Ontem`/`X dias`.
*   `f811d3c Rename visible RFM labels to RFV` - visible UI wording changed from RFM to RFV.

Files most relevant to RFV:
*   `backend/services/analyticsService.js` - `getRfmAnalytics`, date filters, RFV score/tag logic.
*   `backend/test/analyticsService.test.js` - unit coverage for date filters and RFV aggregation.
*   `src/pages/Rfm.tsx` - RFV page UI, matrix/table/export labels, period recency display.
*   `src/components/DateRangePicker.tsx` and `src/dateRanges.ts` - local calendar date ranges and ISO query param formatting.
*   `src/dataService.ts` - analytics fetch/cache helpers and client filter option requests.
*   `src/displayFormatters.ts` - shared seller/fantasy/client metadata display cleanup.