first commit
This commit is contained in:
4
.dockerignore
Normal file
4
.dockerignore
Normal file
@@ -0,0 +1,4 @@
|
||||
node_modules
|
||||
.env
|
||||
.git
|
||||
npm-debug.log*
|
||||
24
.env.example
Normal file
24
.env.example
Normal file
@@ -0,0 +1,24 @@
|
||||
TINY_API_TOKEN=
|
||||
DATABASE_URL=postgres://postgres:postgres@postgres:5432/graphs
|
||||
|
||||
# Tiny API v2 defaults are inferred and can be overridden if Tiny changes names.
|
||||
TINY_API_BASE_URL=https://api.tiny.com.br/api2
|
||||
TINY_LIST_ENDPOINT=ordens.producao.pesquisa.php
|
||||
TINY_DETAIL_ENDPOINT=ordem.producao.obter.php
|
||||
TINY_HTTP_METHOD=POST
|
||||
TINY_START_DATE_PARAM=dataInicial
|
||||
TINY_END_DATE_PARAM=dataFinal
|
||||
TINY_DATE_FORMAT=DD/MM/YYYY
|
||||
TINY_FETCH_DETAILS=false
|
||||
TINY_REQUEST_DELAY_MS=5000
|
||||
TINY_BLOCK_RETRY_MS=60000
|
||||
TINY_MAX_RETRIES=3
|
||||
|
||||
# backfill: use SYNC_START_DATE through SYNC_END_DATE, or today when SYNC_END_DATE is empty.
|
||||
# recent: use today plus the prior RECENT_SYNC_DAYS - 1 days, unless explicit dates are provided.
|
||||
SYNC_MODE=backfill
|
||||
SYNC_TIME_ZONE=America/Sao_Paulo
|
||||
RECENT_SYNC_DAYS=2
|
||||
SYNC_START_DATE=
|
||||
SYNC_END_DATE=
|
||||
DRY_RUN=false
|
||||
4
.gitignore
vendored
Normal file
4
.gitignore
vendored
Normal file
@@ -0,0 +1,4 @@
|
||||
node_modules/
|
||||
.env
|
||||
npm-debug.log*
|
||||
coverage/
|
||||
10
Dockerfile
Normal file
10
Dockerfile
Normal file
@@ -0,0 +1,10 @@
|
||||
FROM node:22-alpine
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY package*.json ./
|
||||
RUN npm ci --omit=dev
|
||||
|
||||
COPY src ./src
|
||||
|
||||
CMD ["npm", "run", "sync"]
|
||||
134
README.md
Normal file
134
README.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# production-orders-sync
|
||||
|
||||
Standalone worker that syncs Tiny ERP production orders directly into the existing Graphs Postgres tables used by `GET /api/production-orders`.
|
||||
|
||||
This is intentionally separate from `api-tiny-n8n`; it does not call n8n and does not depend on the Tiny -> n8n bridge.
|
||||
|
||||
## Sync Design
|
||||
|
||||
Tiny/Olist API 2.0 public webhook docs do not list an official production-order webhook/event. The documented webhook/event surfaces found are for stock updates, product sending, tracking code sending, invoice sending, product price sending, sales order status updates, and freight quote lookup.
|
||||
|
||||
The production-order API page found is an action endpoint, `gerar.ordem.producao.pedido.php`, which generates production orders from a sales order. That is not a live production-order webhook.
|
||||
|
||||
Docs checked:
|
||||
|
||||
- https://tiny.com.br/api-docs/api2-webhooks
|
||||
- https://tiny.com.br/api-docs/api2-webhooks-tiny
|
||||
- https://tiny.com.br/api-docs/api2-webhooks-atualizacao-situacao-pedido
|
||||
- https://tiny.com.br/api-docs/api2-pedidos-gerar-ordem-producao
|
||||
|
||||
For that reason, this worker is designed around:
|
||||
|
||||
- backfill mode for historical date ranges
|
||||
- recent polling mode for periodic syncs, usually every 15 or 30 minutes from an external scheduler
|
||||
- idempotent upserts by `tiny_id`, so repeated polling updates existing rows without duplicates
|
||||
|
||||
An optional future optimization is for a sales-order webhook in the existing middleware to trigger a recent production-order sync. Polling should still remain the source of truth because production-order status can change later without a sales-order webhook.
|
||||
|
||||
## What It Writes
|
||||
|
||||
The worker upserts `production_orders` by `tiny_id`, stores the full Tiny production-order object in `tiny_payload`, updates `updated_at` on every conflict update, and replaces `production_order_markers` for each synced order.
|
||||
|
||||
It preserves the existing frontend/backend contract:
|
||||
|
||||
- `GET /api/production-orders?start=YYYY-MM-DD&end=YYYY-MM-DD&search=...`
|
||||
- Response shape: `{ orders, counts }`
|
||||
- Status values: `open`, `in_progress`, `finished`, `canceled`
|
||||
|
||||
## Configure
|
||||
|
||||
Copy `.env.example` to `.env` and fill in:
|
||||
|
||||
```sh
|
||||
TINY_API_TOKEN=...
|
||||
DATABASE_URL=postgres://...
|
||||
SYNC_MODE=backfill
|
||||
SYNC_START_DATE=2026-07-02
|
||||
SYNC_END_DATE=2026-07-02
|
||||
DRY_RUN=true
|
||||
TINY_REQUEST_DELAY_MS=5000
|
||||
```
|
||||
|
||||
Tiny production-order endpoint names vary by API/account documentation. I could not confirm public documentation for these exact production-order endpoint names, so treat the defaults as inferred and account-specific until dry-run logs prove the payload shape:
|
||||
|
||||
```sh
|
||||
TINY_LIST_ENDPOINT=ordens.producao.pesquisa.php
|
||||
TINY_DETAIL_ENDPOINT=ordem.producao.obter.php
|
||||
TINY_FETCH_DETAILS=false
|
||||
```
|
||||
|
||||
If Tiny returns only summary rows from the list endpoint, set `TINY_FETCH_DETAILS=true` after confirming the detail endpoint for the account.
|
||||
|
||||
The worker keeps `SYNC_START_DATE` and `SYNC_END_DATE` in `YYYY-MM-DD` format. Tiny request dates default to `DD/MM/YYYY`; override with `TINY_DATE_FORMAT=YYYY-MM-DD` if the endpoint expects ISO dates.
|
||||
|
||||
Backfill behavior:
|
||||
|
||||
- `SYNC_MODE=backfill`
|
||||
- `SYNC_START_DATE` is required.
|
||||
- `SYNC_END_DATE` is optional; when empty, the worker syncs through today in `SYNC_TIME_ZONE`, default `America/Sao_Paulo`.
|
||||
|
||||
Recent polling behavior:
|
||||
|
||||
- `SYNC_MODE=recent`
|
||||
- `RECENT_SYNC_DAYS=2` syncs today plus yesterday.
|
||||
- Run the same manual command from cron, Swarm, Kubernetes, or another scheduler every 15 or 30 minutes.
|
||||
- Upserts by `tiny_id` keep repeated runs idempotent.
|
||||
|
||||
This worker does not include its own scheduler yet; deployment should run it manually first, then schedule the same command externally.
|
||||
|
||||
## Manual Run
|
||||
|
||||
```sh
|
||||
npm install
|
||||
npm run sync
|
||||
```
|
||||
|
||||
For a no-write validation run:
|
||||
|
||||
```sh
|
||||
DRY_RUN=true \
|
||||
SYNC_MODE=backfill \
|
||||
SYNC_START_DATE=2026-07-02 \
|
||||
SYNC_END_DATE=2026-07-02 \
|
||||
TINY_REQUEST_DELAY_MS=5000 \
|
||||
npm run sync
|
||||
```
|
||||
|
||||
Inspect the dry-run logs before enabling writes. Dry-run logs include the Tiny response shape, discovered array paths, the first extracted raw order preview, and the exact `production_orders` / `production_order_markers` values that would be written.
|
||||
|
||||
## Docker
|
||||
|
||||
Build and run the worker once:
|
||||
|
||||
```sh
|
||||
docker compose run --rm production-orders-sync
|
||||
```
|
||||
|
||||
The included `postgres` service is only a local example. In normal deployment, set `DATABASE_URL` to the existing Graphs Postgres database.
|
||||
|
||||
## Logs
|
||||
|
||||
The worker logs:
|
||||
|
||||
- start/end config without secrets
|
||||
- `[Production Orders Sync]` prefix for easy separation from `api-tiny-n8n`
|
||||
- current page, endpoint, date range, and Tiny-formatted request dates
|
||||
- orders found per page
|
||||
- mapped rows and dry-run rows that would be upserted
|
||||
- rows upserted
|
||||
- skipped/invalid rows with raw payload preview
|
||||
- Tiny block/rate-limit messages and retries
|
||||
|
||||
## Tiny Field Mapping
|
||||
|
||||
The mapper accepts several common Tiny/PT-BR field variants. Required fields are:
|
||||
|
||||
- Tiny production order ID, mapped to `tiny_id`
|
||||
- product description, mapped to `product_description`
|
||||
|
||||
Status normalization accepts:
|
||||
|
||||
- `em_aberto`, `em aberto`, `aberta` -> `open`
|
||||
- `andamento`, `em andamento` -> `in_progress`
|
||||
- `finalizada`, `finalizado` -> `finished`
|
||||
- `cancelada`, `cancelado`, `cancelled` -> `canceled`
|
||||
27
docker-compose.yml
Normal file
27
docker-compose.yml
Normal file
@@ -0,0 +1,27 @@
|
||||
services:
|
||||
production-orders-sync:
|
||||
build: .
|
||||
container_name: production-orders-sync
|
||||
env_file:
|
||||
- .env
|
||||
environment:
|
||||
TINY_REQUEST_DELAY_MS: ${TINY_REQUEST_DELAY_MS:-5000}
|
||||
DRY_RUN: ${DRY_RUN:-false}
|
||||
restart: "no"
|
||||
|
||||
# Example only. In production, point DATABASE_URL at the existing Graphs Postgres.
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
profiles:
|
||||
- local-db
|
||||
environment:
|
||||
POSTGRES_DB: graphs
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: postgres
|
||||
ports:
|
||||
- "5432:5432"
|
||||
volumes:
|
||||
- production-orders-sync-postgres:/var/lib/postgresql/data
|
||||
|
||||
volumes:
|
||||
production-orders-sync-postgres:
|
||||
175
package-lock.json
generated
Normal file
175
package-lock.json
generated
Normal file
@@ -0,0 +1,175 @@
|
||||
{
|
||||
"name": "production-orders-sync",
|
||||
"version": "1.0.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "production-orders-sync",
|
||||
"version": "1.0.0",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"dotenv": "^17.4.2",
|
||||
"pg": "^8.20.0"
|
||||
}
|
||||
},
|
||||
"node_modules/dotenv": {
|
||||
"version": "17.4.2",
|
||||
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-17.4.2.tgz",
|
||||
"integrity": "sha512-nI4U3TottKAcAD9LLud4Cb7b2QztQMUEfHbvhTH09bqXTxnSie8WnjPALV/WMCrJZ6UV/qHJ6L03OqO3LcdYZw==",
|
||||
"license": "BSD-2-Clause",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://dotenvx.com"
|
||||
}
|
||||
},
|
||||
"node_modules/pg": {
|
||||
"version": "8.22.0",
|
||||
"resolved": "https://registry.npmjs.org/pg/-/pg-8.22.0.tgz",
|
||||
"integrity": "sha512-8wih1vVIBMxoUM2oB4soJsD9tDnDpLv4OXBJ+EJzFsvycD+lfyIreC2gGHq78f8jbLLt+bvlPTFdFZfJkOuzAA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"pg-connection-string": "^2.14.0",
|
||||
"pg-pool": "^3.14.0",
|
||||
"pg-protocol": "^1.15.0",
|
||||
"pg-types": "2.2.0",
|
||||
"pgpass": "1.0.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 16.0.0"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"pg-cloudflare": "^1.4.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"pg-native": ">=3.0.1"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"pg-native": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/pg-cloudflare": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",
|
||||
"integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",
|
||||
"license": "MIT",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/pg-connection-string": {
|
||||
"version": "2.14.0",
|
||||
"resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",
|
||||
"integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/pg-int8": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
|
||||
"integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">=4.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/pg-pool": {
|
||||
"version": "3.14.0",
|
||||
"resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
|
||||
"integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"pg": ">=8.0"
|
||||
}
|
||||
},
|
||||
"node_modules/pg-protocol": {
|
||||
"version": "1.15.0",
|
||||
"resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.15.0.tgz",
|
||||
"integrity": "sha512-cq9sECI5s0+uPUXjbz8ioyPJni6RzsRib0US67i5IoTZKw8fNeYlVE7u8F4dG7vEJJtc5wdD1K189lCCUwqWTQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/pg-types": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
|
||||
"integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"pg-int8": "1.0.1",
|
||||
"postgres-array": "~2.0.0",
|
||||
"postgres-bytea": "~1.0.0",
|
||||
"postgres-date": "~1.0.4",
|
||||
"postgres-interval": "^1.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=4"
|
||||
}
|
||||
},
|
||||
"node_modules/pgpass": {
|
||||
"version": "1.0.5",
|
||||
"resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
|
||||
"integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"split2": "^4.1.0"
|
||||
}
|
||||
},
|
||||
"node_modules/postgres-array": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
|
||||
"integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=4"
|
||||
}
|
||||
},
|
||||
"node_modules/postgres-bytea": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
|
||||
"integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/postgres-date": {
|
||||
"version": "1.0.7",
|
||||
"resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
|
||||
"integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/postgres-interval": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
|
||||
"integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"xtend": "^4.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/split2": {
|
||||
"version": "4.2.0",
|
||||
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
|
||||
"integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">= 10.x"
|
||||
}
|
||||
},
|
||||
"node_modules/xtend": {
|
||||
"version": "4.0.2",
|
||||
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
|
||||
"integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.4"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
24
package.json
Normal file
24
package.json
Normal file
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "production-orders-sync",
|
||||
"version": "1.0.0",
|
||||
"description": "Standalone worker that syncs Tiny ERP production orders into the Graphs Postgres database.",
|
||||
"main": "src/index.js",
|
||||
"type": "commonjs",
|
||||
"scripts": {
|
||||
"sync": "node src/index.js",
|
||||
"start": "node src/index.js",
|
||||
"test": "node --test",
|
||||
"check": "node --check src/index.js && node --check src/config.js && node --check src/db.js && node --check src/logger.js && node --check src/mapper.js && node --check src/sync.js && node --check src/tinyClient.js"
|
||||
},
|
||||
"keywords": [
|
||||
"tiny-erp",
|
||||
"production-orders",
|
||||
"postgres"
|
||||
],
|
||||
"author": "",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"dotenv": "^17.4.2",
|
||||
"pg": "^8.20.0"
|
||||
}
|
||||
}
|
||||
154
src/config.js
Normal file
154
src/config.js
Normal file
@@ -0,0 +1,154 @@
|
||||
require('dotenv').config({ quiet: true });
|
||||
|
||||
const parseBoolean = (value, fallback = false) => {
|
||||
if (value === undefined || value === null || value === '') return fallback;
|
||||
return ['1', 'true', 'yes', 'y', 'sim'].includes(String(value).trim().toLowerCase());
|
||||
};
|
||||
|
||||
const parseInteger = (value, fallback) => {
|
||||
const parsed = Number.parseInt(value, 10);
|
||||
return Number.isFinite(parsed) ? parsed : fallback;
|
||||
};
|
||||
|
||||
const parseDate = (value, name) => {
|
||||
if (!value) return null;
|
||||
if (!/^\d{4}-\d{2}-\d{2}$/.test(value)) {
|
||||
throw new Error(`${name} must use YYYY-MM-DD format`);
|
||||
}
|
||||
const date = new Date(`${value}T00:00:00Z`);
|
||||
if (Number.isNaN(date.getTime())) {
|
||||
throw new Error(`${name} is not a valid date`);
|
||||
}
|
||||
return value;
|
||||
};
|
||||
|
||||
const parseEnum = (value, allowedValues, fallback, name) => {
|
||||
const normalized = String(value || fallback).trim().toUpperCase();
|
||||
if (!allowedValues.includes(normalized)) {
|
||||
throw new Error(`${name} must be one of: ${allowedValues.join(', ')}`);
|
||||
}
|
||||
return normalized;
|
||||
};
|
||||
|
||||
const toIsoDateInTimeZone = (date = new Date(), timeZone = 'America/Sao_Paulo') => {
|
||||
const parts = new Intl.DateTimeFormat('en-US', {
|
||||
timeZone,
|
||||
year: 'numeric',
|
||||
month: '2-digit',
|
||||
day: '2-digit'
|
||||
}).formatToParts(date);
|
||||
const values = Object.fromEntries(parts.map((part) => [part.type, part.value]));
|
||||
return `${values.year}-${values.month}-${values.day}`;
|
||||
};
|
||||
|
||||
const addDays = (isoDate, days) => {
|
||||
const [year, month, day] = isoDate.split('-').map(Number);
|
||||
const date = new Date(Date.UTC(year, month - 1, day + days));
|
||||
return date.toISOString().slice(0, 10);
|
||||
};
|
||||
|
||||
const buildConfig = (env = process.env) => ({
|
||||
tinyApiToken: env.TINY_API_TOKEN || '',
|
||||
databaseUrl: env.DATABASE_URL || '',
|
||||
tinyApiBaseUrl: env.TINY_API_BASE_URL || 'https://api.tiny.com.br/api2',
|
||||
tinyListEndpoint: env.TINY_LIST_ENDPOINT || 'ordens.producao.pesquisa.php',
|
||||
tinyDetailEndpoint: env.TINY_DETAIL_ENDPOINT || 'ordem.producao.obter.php',
|
||||
tinyHttpMethod: (env.TINY_HTTP_METHOD || 'POST').toUpperCase(),
|
||||
tinyStartDateParam: env.TINY_START_DATE_PARAM || 'dataInicial',
|
||||
tinyEndDateParam: env.TINY_END_DATE_PARAM || 'dataFinal',
|
||||
tinyDateFormat: parseEnum(env.TINY_DATE_FORMAT, ['YYYY-MM-DD', 'DD/MM/YYYY'], 'DD/MM/YYYY', 'TINY_DATE_FORMAT'),
|
||||
tinyFetchDetails: parseBoolean(env.TINY_FETCH_DETAILS, false),
|
||||
tinyRequestDelayMs: parseInteger(env.TINY_REQUEST_DELAY_MS, 5000),
|
||||
tinyBlockRetryMs: parseInteger(env.TINY_BLOCK_RETRY_MS, 60000),
|
||||
tinyMaxRetries: parseInteger(env.TINY_MAX_RETRIES, 3),
|
||||
syncMode: parseEnum(env.SYNC_MODE, ['BACKFILL', 'RECENT'], 'BACKFILL', 'SYNC_MODE').toLowerCase(),
|
||||
syncTimeZone: env.SYNC_TIME_ZONE || 'America/Sao_Paulo',
|
||||
recentSyncDays: parseInteger(env.RECENT_SYNC_DAYS, 2),
|
||||
syncStartDate: parseDate(env.SYNC_START_DATE, 'SYNC_START_DATE'),
|
||||
syncEndDate: parseDate(env.SYNC_END_DATE, 'SYNC_END_DATE'),
|
||||
dryRun: parseBoolean(env.DRY_RUN, false)
|
||||
});
|
||||
|
||||
const config = buildConfig();
|
||||
|
||||
const validateConfig = (nextConfig = config) => {
|
||||
if (!nextConfig.tinyApiToken) {
|
||||
throw new Error('TINY_API_TOKEN is required');
|
||||
}
|
||||
if (!nextConfig.dryRun && !nextConfig.databaseUrl) {
|
||||
throw new Error('DATABASE_URL is required unless DRY_RUN=true');
|
||||
}
|
||||
if (!['GET', 'POST'].includes(nextConfig.tinyHttpMethod)) {
|
||||
throw new Error('TINY_HTTP_METHOD must be GET or POST');
|
||||
}
|
||||
if (nextConfig.tinyRequestDelayMs < 0) {
|
||||
throw new Error('TINY_REQUEST_DELAY_MS must be zero or greater');
|
||||
}
|
||||
if (nextConfig.tinyBlockRetryMs < 0) {
|
||||
throw new Error('TINY_BLOCK_RETRY_MS must be zero or greater');
|
||||
}
|
||||
if (nextConfig.tinyMaxRetries < 0) {
|
||||
throw new Error('TINY_MAX_RETRIES must be zero or greater');
|
||||
}
|
||||
if (nextConfig.recentSyncDays < 1) {
|
||||
throw new Error('RECENT_SYNC_DAYS must be 1 or greater');
|
||||
}
|
||||
if (nextConfig.syncMode === 'backfill' && !nextConfig.syncStartDate) {
|
||||
throw new Error('SYNC_START_DATE is required when SYNC_MODE=backfill');
|
||||
}
|
||||
if (nextConfig.syncStartDate && nextConfig.syncEndDate && nextConfig.syncStartDate > nextConfig.syncEndDate) {
|
||||
throw new Error('SYNC_START_DATE cannot be after SYNC_END_DATE');
|
||||
}
|
||||
};
|
||||
|
||||
const resolveSyncDateRange = (nextConfig = config, now = new Date()) => {
|
||||
const today = toIsoDateInTimeZone(now, nextConfig.syncTimeZone);
|
||||
const endDate = nextConfig.syncEndDate || today;
|
||||
|
||||
if (nextConfig.syncMode === 'recent') {
|
||||
const startDate = nextConfig.syncStartDate || addDays(endDate, -(nextConfig.recentSyncDays - 1));
|
||||
return {
|
||||
mode: nextConfig.syncMode,
|
||||
startDate,
|
||||
endDate,
|
||||
source: nextConfig.syncStartDate ? 'explicit recent range' : `recent ${nextConfig.recentSyncDays} day window`
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
mode: nextConfig.syncMode,
|
||||
startDate: nextConfig.syncStartDate,
|
||||
endDate,
|
||||
source: nextConfig.syncEndDate ? 'explicit backfill range' : 'backfill through today'
|
||||
};
|
||||
};
|
||||
|
||||
const publicConfig = (nextConfig = config) => ({
|
||||
tinyApiBaseUrl: nextConfig.tinyApiBaseUrl,
|
||||
tinyListEndpoint: nextConfig.tinyListEndpoint,
|
||||
tinyDetailEndpoint: nextConfig.tinyDetailEndpoint,
|
||||
tinyHttpMethod: nextConfig.tinyHttpMethod,
|
||||
tinyStartDateParam: nextConfig.tinyStartDateParam,
|
||||
tinyEndDateParam: nextConfig.tinyEndDateParam,
|
||||
tinyDateFormat: nextConfig.tinyDateFormat,
|
||||
tinyFetchDetails: nextConfig.tinyFetchDetails,
|
||||
tinyRequestDelayMs: nextConfig.tinyRequestDelayMs,
|
||||
tinyBlockRetryMs: nextConfig.tinyBlockRetryMs,
|
||||
tinyMaxRetries: nextConfig.tinyMaxRetries,
|
||||
syncMode: nextConfig.syncMode,
|
||||
syncTimeZone: nextConfig.syncTimeZone,
|
||||
recentSyncDays: nextConfig.recentSyncDays,
|
||||
syncStartDate: nextConfig.syncStartDate,
|
||||
syncEndDate: nextConfig.syncEndDate,
|
||||
dryRun: nextConfig.dryRun,
|
||||
hasDatabaseUrl: Boolean(nextConfig.databaseUrl),
|
||||
hasTinyApiToken: Boolean(nextConfig.tinyApiToken)
|
||||
});
|
||||
|
||||
module.exports = {
|
||||
config,
|
||||
buildConfig,
|
||||
validateConfig,
|
||||
resolveSyncDateRange,
|
||||
publicConfig
|
||||
};
|
||||
95
src/db.js
Normal file
95
src/db.js
Normal file
@@ -0,0 +1,95 @@
|
||||
const { Pool } = require('pg');
|
||||
|
||||
const UPSERT_ORDER_SQL = `
|
||||
INSERT INTO production_orders (
|
||||
tiny_id,
|
||||
number,
|
||||
status,
|
||||
order_reference,
|
||||
issue_date,
|
||||
expected_date,
|
||||
product_sku,
|
||||
product_description,
|
||||
quantity,
|
||||
unit,
|
||||
integration_status,
|
||||
tiny_payload,
|
||||
updated_at
|
||||
)
|
||||
VALUES ($1, $2, $3, $4, $5::date, $6::date, $7, $8, $9, $10, $11, $12::jsonb, CURRENT_TIMESTAMP)
|
||||
ON CONFLICT (tiny_id) DO UPDATE SET
|
||||
number = EXCLUDED.number,
|
||||
status = EXCLUDED.status,
|
||||
order_reference = EXCLUDED.order_reference,
|
||||
issue_date = EXCLUDED.issue_date,
|
||||
expected_date = EXCLUDED.expected_date,
|
||||
product_sku = EXCLUDED.product_sku,
|
||||
product_description = EXCLUDED.product_description,
|
||||
quantity = EXCLUDED.quantity,
|
||||
unit = EXCLUDED.unit,
|
||||
integration_status = EXCLUDED.integration_status,
|
||||
tiny_payload = EXCLUDED.tiny_payload,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
RETURNING id;
|
||||
`;
|
||||
|
||||
class ProductionOrderRepository {
|
||||
constructor(databaseUrl) {
|
||||
this.pool = new Pool({ connectionString: databaseUrl });
|
||||
}
|
||||
|
||||
async connect() {
|
||||
await this.pool.query(`SET TIME ZONE 'America/Sao_Paulo';`);
|
||||
}
|
||||
|
||||
async close() {
|
||||
await this.pool.end();
|
||||
}
|
||||
|
||||
async upsertOrder(order, markers) {
|
||||
const client = await this.pool.connect();
|
||||
try {
|
||||
await client.query('BEGIN');
|
||||
const result = await client.query(UPSERT_ORDER_SQL, [
|
||||
order.tinyId,
|
||||
order.number,
|
||||
order.status,
|
||||
order.orderReference,
|
||||
order.issueDate,
|
||||
order.expectedDate,
|
||||
order.productSku,
|
||||
order.productDescription,
|
||||
order.quantity,
|
||||
order.unit,
|
||||
order.integrationStatus,
|
||||
JSON.stringify(order.tinyPayload)
|
||||
]);
|
||||
|
||||
const productionOrderId = result.rows[0].id;
|
||||
await client.query('DELETE FROM production_order_markers WHERE production_order_id = $1', [productionOrderId]);
|
||||
|
||||
for (const marker of markers) {
|
||||
await client.query(
|
||||
`
|
||||
INSERT INTO production_order_markers (production_order_id, label, color)
|
||||
VALUES ($1, $2, $3)
|
||||
ON CONFLICT (production_order_id, label) DO UPDATE SET color = EXCLUDED.color;
|
||||
`,
|
||||
[productionOrderId, marker.label, marker.color || null]
|
||||
);
|
||||
}
|
||||
|
||||
await client.query('COMMIT');
|
||||
return productionOrderId;
|
||||
} catch (error) {
|
||||
await client.query('ROLLBACK');
|
||||
throw error;
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
ProductionOrderRepository
|
||||
};
|
||||
16
src/index.js
Normal file
16
src/index.js
Normal file
@@ -0,0 +1,16 @@
|
||||
const { config, validateConfig } = require('./config');
|
||||
const logger = require('./logger');
|
||||
const { runSync } = require('./sync');
|
||||
|
||||
const main = async () => {
|
||||
validateConfig();
|
||||
await runSync(config);
|
||||
};
|
||||
|
||||
main().catch((error) => {
|
||||
logger.error('production-orders-sync failed', {
|
||||
message: error.message,
|
||||
stack: error.stack
|
||||
});
|
||||
process.exitCode = 1;
|
||||
});
|
||||
26
src/logger.js
Normal file
26
src/logger.js
Normal file
@@ -0,0 +1,26 @@
|
||||
const LOG_PREFIX = '[Production Orders Sync]';
|
||||
|
||||
const serializeMeta = (meta) => {
|
||||
if (!meta || Object.keys(meta).length === 0) return '';
|
||||
return ` ${JSON.stringify(meta)}`;
|
||||
};
|
||||
|
||||
const log = (level, message, meta = {}) => {
|
||||
const timestamp = new Date().toISOString();
|
||||
const output = `${timestamp} ${level.toUpperCase()} ${LOG_PREFIX} ${message}${serializeMeta(meta)}`;
|
||||
if (level === 'error') {
|
||||
console.error(output);
|
||||
return;
|
||||
}
|
||||
if (level === 'warn') {
|
||||
console.warn(output);
|
||||
return;
|
||||
}
|
||||
console.log(output);
|
||||
};
|
||||
|
||||
module.exports = {
|
||||
info: (message, meta) => log('info', message, meta),
|
||||
warn: (message, meta) => log('warn', message, meta),
|
||||
error: (message, meta) => log('error', message, meta)
|
||||
};
|
||||
179
src/mapper.js
Normal file
179
src/mapper.js
Normal file
@@ -0,0 +1,179 @@
|
||||
const STATUS_LABELS = {
|
||||
open: 'Em aberto',
|
||||
in_progress: 'Em andamento',
|
||||
finished: 'Finalizada',
|
||||
canceled: 'Cancelada'
|
||||
};
|
||||
|
||||
const normalizeText = (value) => String(value || '')
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.normalize('NFD')
|
||||
.replace(/[\u0300-\u036f]/g, '');
|
||||
|
||||
const normalizeStatus = (status) => {
|
||||
const normalized = normalizeText(status).replace(/\s+/g, ' ');
|
||||
if (['open', 'em_aberto', 'em aberto', 'aberta'].includes(normalized)) return 'open';
|
||||
if (['in_progress', 'andamento', 'em andamento'].includes(normalized)) return 'in_progress';
|
||||
if (['finished', 'finalizada', 'finalizado'].includes(normalized)) return 'finished';
|
||||
if (['canceled', 'cancelada', 'cancelado', 'cancelled'].includes(normalized)) return 'canceled';
|
||||
return 'open';
|
||||
};
|
||||
|
||||
const firstDefined = (...values) => values.find((value) => value !== undefined && value !== null && value !== '');
|
||||
|
||||
const getPath = (source, path) => path.split('.').reduce((value, key) => {
|
||||
if (value === undefined || value === null) return undefined;
|
||||
return value[key];
|
||||
}, source);
|
||||
|
||||
const pick = (source, paths) => {
|
||||
for (const path of paths) {
|
||||
const value = getPath(source, path);
|
||||
if (value !== undefined && value !== null && value !== '') return value;
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
const parseDate = (value) => {
|
||||
if (!value) return null;
|
||||
if (value instanceof Date && !Number.isNaN(value.getTime())) return value.toISOString().slice(0, 10);
|
||||
const text = String(value).trim();
|
||||
const isoMatch = text.match(/^(\d{4})[-/](\d{1,2})[-/](\d{1,2})/);
|
||||
if (isoMatch) {
|
||||
const [, year, month, day] = isoMatch;
|
||||
return `${year}-${month.padStart(2, '0')}-${day.padStart(2, '0')}`;
|
||||
}
|
||||
const brMatch = text.match(/^(\d{1,2})[-/](\d{1,2})[-/](\d{4})/);
|
||||
if (brMatch) {
|
||||
const [, day, month, year] = brMatch;
|
||||
return `${year}-${month.padStart(2, '0')}-${day.padStart(2, '0')}`;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
const parseNumber = (value) => {
|
||||
if (value === undefined || value === null || value === '') return 0;
|
||||
if (typeof value === 'number') return Number.isFinite(value) ? value : 0;
|
||||
const normalized = String(value).trim().replace(/\./g, '').replace(',', '.');
|
||||
const parsed = Number.parseFloat(normalized);
|
||||
return Number.isFinite(parsed) ? parsed : 0;
|
||||
};
|
||||
|
||||
const unwrapTinyRecord = (record) => {
|
||||
if (!record || typeof record !== 'object') return record;
|
||||
return record.ordem_producao || record.ordemProducao || record.ordem || record.production_order || record;
|
||||
};
|
||||
|
||||
const normalizeMarker = (marker) => {
|
||||
if (!marker) return null;
|
||||
if (typeof marker === 'string') {
|
||||
const label = marker.trim();
|
||||
return label ? { label, color: null } : null;
|
||||
}
|
||||
const label = firstDefined(marker.label, marker.nome, marker.name, marker.descricao, marker.description);
|
||||
if (!label) return null;
|
||||
return {
|
||||
label: String(label).trim(),
|
||||
color: firstDefined(marker.color, marker.cor, marker.colour, null)
|
||||
};
|
||||
};
|
||||
|
||||
const extractMarkers = (raw) => {
|
||||
const candidates = [
|
||||
pick(raw, ['marcadores', 'markers', 'tags', 'etiquetas']),
|
||||
pick(raw, ['marcador', 'marker', 'tag', 'etiqueta'])
|
||||
].filter(Boolean);
|
||||
|
||||
const markers = [];
|
||||
for (const candidate of candidates) {
|
||||
const list = Array.isArray(candidate) ? candidate : [candidate];
|
||||
for (const entry of list) {
|
||||
const normalized = normalizeMarker(
|
||||
entry?.marcador || entry?.marker || entry?.tag || entry?.etiqueta || entry
|
||||
);
|
||||
if (normalized) markers.push(normalized);
|
||||
}
|
||||
}
|
||||
|
||||
return [...new Map(markers.map((marker) => [marker.label, marker])).values()];
|
||||
};
|
||||
|
||||
const buildOrderReference = (raw) => {
|
||||
const direct = pick(raw, [
|
||||
'order_reference',
|
||||
'referencia_pedido',
|
||||
'pedido_venda',
|
||||
'pedidoVenda',
|
||||
'numero_pedido',
|
||||
'numeroPedido',
|
||||
'pedido.numero',
|
||||
'pedido.id'
|
||||
]);
|
||||
if (direct) return String(direct);
|
||||
|
||||
const linkedOrders = pick(raw, ['pedidos', 'pedidos_venda', 'pedidosVenda']);
|
||||
if (!Array.isArray(linkedOrders)) return '';
|
||||
return linkedOrders
|
||||
.map((item) => pick(unwrapTinyRecord(item), ['numero', 'id', 'codigo']))
|
||||
.filter(Boolean)
|
||||
.join(', ');
|
||||
};
|
||||
|
||||
const mapProductionOrder = (record) => {
|
||||
const raw = unwrapTinyRecord(record);
|
||||
if (!raw || typeof raw !== 'object') {
|
||||
return { order: null, markers: [], reason: 'record is not an object' };
|
||||
}
|
||||
|
||||
const tinyId = pick(raw, ['id', 'id_ordem_producao', 'idOrdemProducao', 'codigo', 'tiny_id']);
|
||||
const productDescription = pick(raw, [
|
||||
'product_description',
|
||||
'descricao_produto',
|
||||
'produto_descricao',
|
||||
'descricao',
|
||||
'produto.descricao',
|
||||
'produto.nome',
|
||||
'nome_produto'
|
||||
]);
|
||||
|
||||
if (!tinyId) {
|
||||
return { order: null, markers: [], reason: 'missing Tiny production order id', raw };
|
||||
}
|
||||
if (!productDescription) {
|
||||
return { order: null, markers: [], reason: 'missing product description', raw };
|
||||
}
|
||||
|
||||
const status = normalizeStatus(pick(raw, ['status', 'situacao', 'situacao_descricao', 'estado']));
|
||||
const order = {
|
||||
tinyId: String(tinyId),
|
||||
number: String(firstDefined(pick(raw, ['number', 'numero', 'codigo_ordem', 'numero_ordem']), tinyId)),
|
||||
status,
|
||||
statusLabel: STATUS_LABELS[status] || status,
|
||||
orderReference: buildOrderReference(raw),
|
||||
issueDate: parseDate(pick(raw, ['issue_date', 'data_emissao', 'dataEmissao', 'data_criacao', 'data', 'created_at'])),
|
||||
expectedDate: parseDate(pick(raw, ['expected_date', 'data_prevista', 'dataPrevista', 'data_previsao', 'previsao', 'data_entrega'])),
|
||||
productSku: String(firstDefined(pick(raw, ['product_sku', 'codigo_produto', 'produto_codigo', 'sku', 'produto.codigo', 'produto.sku']), '')),
|
||||
productDescription: String(productDescription),
|
||||
quantity: parseNumber(pick(raw, ['quantity', 'quantidade', 'qtde', 'produto.quantidade'])),
|
||||
unit: String(firstDefined(pick(raw, ['unit', 'unidade', 'produto.unidade']), 'UN')),
|
||||
integrationStatus: String(firstDefined(pick(raw, ['integration_status', 'situacao_integracao', 'integracao_status', 'status_integracao']), '')),
|
||||
tinyPayload: raw
|
||||
};
|
||||
|
||||
return {
|
||||
order,
|
||||
markers: extractMarkers(raw),
|
||||
reason: null,
|
||||
raw
|
||||
};
|
||||
};
|
||||
|
||||
module.exports = {
|
||||
STATUS_LABELS,
|
||||
normalizeStatus,
|
||||
parseDate,
|
||||
parseNumber,
|
||||
unwrapTinyRecord,
|
||||
mapProductionOrder
|
||||
};
|
||||
163
src/sync.js
Normal file
163
src/sync.js
Normal file
@@ -0,0 +1,163 @@
|
||||
const logger = require('./logger');
|
||||
const { publicConfig, resolveSyncDateRange } = require('./config');
|
||||
const { mapProductionOrder } = require('./mapper');
|
||||
const { TinyClient, findArrayPaths, getTopLevelShape } = require('./tinyClient');
|
||||
const { ProductionOrderRepository } = require('./db');
|
||||
|
||||
const safeRawPreview = (raw, maxKeys = 25) => {
|
||||
if (!raw || typeof raw !== 'object') return raw;
|
||||
const keys = Object.keys(raw).slice(0, maxKeys);
|
||||
return keys.reduce((preview, key) => {
|
||||
preview[key] = raw[key];
|
||||
return preview;
|
||||
}, {});
|
||||
};
|
||||
|
||||
const orderWritePreview = (order, markers) => ({
|
||||
production_orders: {
|
||||
tiny_id: order.tinyId,
|
||||
number: order.number,
|
||||
status: order.status,
|
||||
order_reference: order.orderReference,
|
||||
issue_date: order.issueDate,
|
||||
expected_date: order.expectedDate,
|
||||
product_sku: order.productSku,
|
||||
product_description: order.productDescription,
|
||||
quantity: order.quantity,
|
||||
unit: order.unit,
|
||||
integration_status: order.integrationStatus,
|
||||
tiny_payload_keys: order.tinyPayload && typeof order.tinyPayload === 'object' ? Object.keys(order.tinyPayload) : []
|
||||
},
|
||||
production_order_markers: markers
|
||||
});
|
||||
|
||||
const runSync = async (config) => {
|
||||
const syncDateRange = resolveSyncDateRange(config);
|
||||
logger.info('Start config', {
|
||||
...publicConfig(config),
|
||||
resolvedDateRange: syncDateRange
|
||||
});
|
||||
|
||||
const tinyClient = new TinyClient(config);
|
||||
const repository = config.dryRun ? null : new ProductionOrderRepository(config.databaseUrl);
|
||||
|
||||
const stats = {
|
||||
pagesFetched: 0,
|
||||
ordersFound: 0,
|
||||
rowsMapped: 0,
|
||||
rowsWouldUpsert: 0,
|
||||
rowsUpserted: 0,
|
||||
skippedInvalidRows: 0,
|
||||
detailErrors: 0
|
||||
};
|
||||
|
||||
if (repository) await repository.connect();
|
||||
|
||||
try {
|
||||
let page = 1;
|
||||
let totalPages = 1;
|
||||
|
||||
do {
|
||||
const pageResult = await tinyClient.listProductionOrders(page, {
|
||||
startDate: syncDateRange.startDate,
|
||||
endDate: syncDateRange.endDate
|
||||
});
|
||||
|
||||
stats.pagesFetched += 1;
|
||||
totalPages = pageResult.totalPages;
|
||||
logger.info('Tiny production orders page fetched', {
|
||||
page,
|
||||
totalPages,
|
||||
dateRange: {
|
||||
startDate: syncDateRange.startDate,
|
||||
endDate: syncDateRange.endDate,
|
||||
mode: syncDateRange.mode,
|
||||
source: syncDateRange.source
|
||||
},
|
||||
endpoint: config.tinyListEndpoint,
|
||||
ordersFound: pageResult.orders.length
|
||||
});
|
||||
|
||||
if (config.dryRun) {
|
||||
logger.info('DRY_RUN Tiny payload shape', {
|
||||
page,
|
||||
endpoint: config.tinyListEndpoint,
|
||||
shape: getTopLevelShape(pageResult.raw),
|
||||
arrayPaths: findArrayPaths(pageResult.raw)
|
||||
});
|
||||
logger.info('DRY_RUN extracted order preview', {
|
||||
page,
|
||||
extractedOrders: pageResult.orders.length,
|
||||
firstOrderRawPreview: safeRawPreview(pageResult.orders[0] || null)
|
||||
});
|
||||
}
|
||||
|
||||
for (const summaryRecord of pageResult.orders) {
|
||||
stats.ordersFound += 1;
|
||||
let record = summaryRecord;
|
||||
|
||||
if (config.tinyFetchDetails) {
|
||||
const mappedSummary = mapProductionOrder(summaryRecord);
|
||||
const tinyId = mappedSummary.order?.tinyId;
|
||||
if (tinyId) {
|
||||
try {
|
||||
record = await tinyClient.getProductionOrderDetail(tinyId);
|
||||
} catch (error) {
|
||||
stats.detailErrors += 1;
|
||||
logger.warn('Tiny detail fetch failed, using list summary payload', {
|
||||
tinyId,
|
||||
error: error.message
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const mapped = mapProductionOrder(record);
|
||||
if (!mapped.order) {
|
||||
stats.skippedInvalidRows += 1;
|
||||
logger.warn('Skipped invalid Tiny production order row', {
|
||||
reason: mapped.reason,
|
||||
rawPreview: safeRawPreview(mapped.raw || record)
|
||||
});
|
||||
continue;
|
||||
}
|
||||
stats.rowsMapped += 1;
|
||||
|
||||
if (config.dryRun) {
|
||||
stats.rowsWouldUpsert += 1;
|
||||
logger.info('DRY_RUN would upsert production order', {
|
||||
tinyId: mapped.order.tinyId,
|
||||
number: mapped.order.number,
|
||||
status: mapped.order.status,
|
||||
markers: mapped.markers.length,
|
||||
writePreview: orderWritePreview(mapped.order, mapped.markers),
|
||||
rawPreview: safeRawPreview(mapped.raw)
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
await repository.upsertOrder(mapped.order, mapped.markers);
|
||||
stats.rowsUpserted += 1;
|
||||
}
|
||||
|
||||
logger.info('Page sync complete', {
|
||||
page,
|
||||
rowsMappedSoFar: stats.rowsMapped,
|
||||
rowsWouldUpsertSoFar: stats.rowsWouldUpsert,
|
||||
rowsUpsertedSoFar: stats.rowsUpserted,
|
||||
skippedInvalidRowsSoFar: stats.skippedInvalidRows
|
||||
});
|
||||
|
||||
page += 1;
|
||||
} while (page <= totalPages);
|
||||
|
||||
logger.info('End summary', stats);
|
||||
return stats;
|
||||
} finally {
|
||||
if (repository) await repository.close();
|
||||
}
|
||||
};
|
||||
|
||||
module.exports = {
|
||||
runSync
|
||||
};
|
||||
222
src/tinyClient.js
Normal file
222
src/tinyClient.js
Normal file
@@ -0,0 +1,222 @@
|
||||
const logger = require('./logger');
|
||||
const { unwrapTinyRecord } = require('./mapper');
|
||||
|
||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
const isBlockMessage = (message) => {
|
||||
const text = String(message || '').toLowerCase();
|
||||
return [
|
||||
'limite',
|
||||
'bloque',
|
||||
'rate',
|
||||
'muitas requisi',
|
||||
'too many',
|
||||
'temporariamente indisponivel',
|
||||
'temporariamente indisponível'
|
||||
].some((term) => text.includes(term));
|
||||
};
|
||||
|
||||
const joinUrl = (baseUrl, endpoint) => `${baseUrl.replace(/\/+$/, '')}/${String(endpoint).replace(/^\/+/, '')}`;
|
||||
|
||||
const formatTinyDate = (date, format) => {
|
||||
if (!date || format === 'YYYY-MM-DD') return date;
|
||||
const match = String(date).match(/^(\d{4})-(\d{2})-(\d{2})$/);
|
||||
if (!match) return date;
|
||||
const [, year, month, day] = match;
|
||||
return `${day}/${month}/${year}`;
|
||||
};
|
||||
|
||||
const describeValue = (value, depth = 0) => {
|
||||
if (value === null) return { type: 'null' };
|
||||
if (Array.isArray(value)) {
|
||||
return {
|
||||
type: 'array',
|
||||
length: value.length,
|
||||
sample: value.length > 0 && depth < 2 ? describeValue(value[0], depth + 1) : undefined
|
||||
};
|
||||
}
|
||||
if (typeof value === 'object') {
|
||||
const keys = Object.keys(value);
|
||||
return {
|
||||
type: 'object',
|
||||
keys,
|
||||
sample: depth < 2
|
||||
? keys.slice(0, 10).reduce((shape, key) => {
|
||||
shape[key] = describeValue(value[key], depth + 1);
|
||||
return shape;
|
||||
}, {})
|
||||
: undefined
|
||||
};
|
||||
}
|
||||
return { type: typeof value, sample: String(value).slice(0, 120) };
|
||||
};
|
||||
|
||||
const getTopLevelShape = (payload) => describeValue(payload);
|
||||
|
||||
const findArrayPaths = (value, path = [], depth = 0) => {
|
||||
if (depth > 4 || value === null || value === undefined) return [];
|
||||
if (Array.isArray(value)) {
|
||||
return [{
|
||||
path: path.join('.') || '<root>',
|
||||
length: value.length,
|
||||
sampleKeys: value[0] && typeof value[0] === 'object' ? Object.keys(unwrapTinyRecord(value[0])).slice(0, 30) : []
|
||||
}];
|
||||
}
|
||||
if (typeof value !== 'object') return [];
|
||||
return Object.entries(value).flatMap(([key, child]) => findArrayPaths(child, [...path, key], depth + 1));
|
||||
};
|
||||
|
||||
class TinyClient {
|
||||
constructor(config) {
|
||||
this.config = config;
|
||||
this.lastRequestAt = 0;
|
||||
}
|
||||
|
||||
async pace() {
|
||||
const elapsed = Date.now() - this.lastRequestAt;
|
||||
const waitMs = Math.max(this.config.tinyRequestDelayMs - elapsed, 0);
|
||||
if (waitMs > 0) await sleep(waitMs);
|
||||
this.lastRequestAt = Date.now();
|
||||
}
|
||||
|
||||
async request(endpoint, params, context) {
|
||||
const body = new URLSearchParams({
|
||||
token: this.config.tinyApiToken,
|
||||
formato: 'json',
|
||||
...params
|
||||
});
|
||||
|
||||
for (let attempt = 1; attempt <= this.config.tinyMaxRetries + 1; attempt += 1) {
|
||||
await this.pace();
|
||||
|
||||
const method = this.config.tinyHttpMethod;
|
||||
const url = new URL(joinUrl(this.config.tinyApiBaseUrl, endpoint));
|
||||
const fetchOptions = { method };
|
||||
if (method === 'GET') {
|
||||
for (const [key, value] of body.entries()) url.searchParams.set(key, value);
|
||||
} else {
|
||||
fetchOptions.headers = { 'Content-Type': 'application/x-www-form-urlencoded' };
|
||||
fetchOptions.body = body.toString();
|
||||
}
|
||||
|
||||
let response;
|
||||
let payload;
|
||||
const requestedUrl = method === 'GET' ? url.toString().replace(this.config.tinyApiToken, '<redacted>') : url.toString();
|
||||
try {
|
||||
logger.info('Tiny request', {
|
||||
...context,
|
||||
endpoint,
|
||||
method,
|
||||
attempt,
|
||||
url: requestedUrl,
|
||||
params: Object.fromEntries([...body.entries()].map(([key, value]) => [key, key === 'token' ? '<redacted>' : value]))
|
||||
});
|
||||
response = await fetch(url, fetchOptions);
|
||||
const text = await response.text();
|
||||
payload = text ? JSON.parse(text) : {};
|
||||
} catch (error) {
|
||||
if (attempt > this.config.tinyMaxRetries) throw error;
|
||||
logger.warn('Tiny request failed, retrying', { ...context, attempt, error: error.message });
|
||||
await sleep(this.config.tinyBlockRetryMs);
|
||||
continue;
|
||||
}
|
||||
|
||||
const message = payload?.retorno?.erros?.[0]?.erro || payload?.retorno?.erro || payload?.erro || payload?.message;
|
||||
if (response.status === 429 || isBlockMessage(message)) {
|
||||
logger.warn('Tiny block/rate limit message', {
|
||||
...context,
|
||||
attempt,
|
||||
statusCode: response.status,
|
||||
message: message || 'HTTP 429'
|
||||
});
|
||||
if (attempt > this.config.tinyMaxRetries) {
|
||||
throw new Error(`Tiny rate/block limit persisted: ${message || response.status}`);
|
||||
}
|
||||
await sleep(this.config.tinyBlockRetryMs);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Tiny HTTP ${response.status}: ${message || response.statusText}`);
|
||||
}
|
||||
|
||||
const status = payload?.retorno?.status;
|
||||
if (status && String(status).toUpperCase() === 'ERRO') {
|
||||
throw new Error(`Tiny API error: ${message || 'unknown error'}`);
|
||||
}
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
throw new Error('Tiny request exhausted retries');
|
||||
}
|
||||
|
||||
extractOrders(payload) {
|
||||
const retorno = payload?.retorno || payload;
|
||||
const candidates = [
|
||||
retorno?.ordens_producao,
|
||||
retorno?.ordensProducao,
|
||||
retorno?.ordens,
|
||||
retorno?.production_orders,
|
||||
retorno?.registros,
|
||||
retorno?.itens,
|
||||
Array.isArray(retorno) ? retorno : null
|
||||
];
|
||||
const list = candidates.find((candidate) => Array.isArray(candidate)) || [];
|
||||
return list.map(unwrapTinyRecord).filter(Boolean);
|
||||
}
|
||||
|
||||
getTotalPages(payload) {
|
||||
const retorno = payload?.retorno || {};
|
||||
const total = Number.parseInt(retorno.numero_paginas || retorno.total_paginas || retorno.pages || '1', 10);
|
||||
return Number.isFinite(total) && total > 0 ? total : 1;
|
||||
}
|
||||
|
||||
async listProductionOrders(page, dateRange) {
|
||||
const params = { pagina: String(page) };
|
||||
if (dateRange.startDate) params[this.config.tinyStartDateParam] = formatTinyDate(dateRange.startDate, this.config.tinyDateFormat);
|
||||
if (dateRange.endDate) params[this.config.tinyEndDateParam] = formatTinyDate(dateRange.endDate, this.config.tinyDateFormat);
|
||||
|
||||
logger.info('Fetching Tiny production orders page/date', {
|
||||
page,
|
||||
startDate: dateRange.startDate || null,
|
||||
endDate: dateRange.endDate || null,
|
||||
tinyStartDate: params[this.config.tinyStartDateParam] || null,
|
||||
tinyEndDate: params[this.config.tinyEndDateParam] || null,
|
||||
tinyDateFormat: this.config.tinyDateFormat,
|
||||
endpoint: this.config.tinyListEndpoint
|
||||
});
|
||||
|
||||
const payload = await this.request(this.config.tinyListEndpoint, params, { operation: 'list', page });
|
||||
return {
|
||||
orders: this.extractOrders(payload),
|
||||
totalPages: this.getTotalPages(payload),
|
||||
raw: payload
|
||||
};
|
||||
}
|
||||
|
||||
async getProductionOrderDetail(tinyId) {
|
||||
const payload = await this.request(
|
||||
this.config.tinyDetailEndpoint,
|
||||
{ id: String(tinyId) },
|
||||
{ operation: 'detail', tinyId: String(tinyId) }
|
||||
);
|
||||
const retorno = payload?.retorno || payload;
|
||||
return unwrapTinyRecord(
|
||||
retorno?.ordem_producao ||
|
||||
retorno?.ordemProducao ||
|
||||
retorno?.ordem ||
|
||||
retorno?.production_order ||
|
||||
retorno
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
TinyClient,
|
||||
sleep,
|
||||
isBlockMessage,
|
||||
formatTinyDate,
|
||||
getTopLevelShape,
|
||||
findArrayPaths
|
||||
};
|
||||
91
test/config.test.js
Normal file
91
test/config.test.js
Normal file
@@ -0,0 +1,91 @@
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const { buildConfig, resolveSyncDateRange, validateConfig } = require('../src/config');
|
||||
|
||||
test('uses safe Tiny request pacing and DD/MM/YYYY request date format by default', () => {
|
||||
const config = buildConfig({
|
||||
TINY_API_TOKEN: 'test-token',
|
||||
DRY_RUN: 'true',
|
||||
SYNC_START_DATE: '2026-07-02'
|
||||
});
|
||||
|
||||
validateConfig(config);
|
||||
assert.equal(config.tinyRequestDelayMs, 5000);
|
||||
assert.equal(config.tinyDateFormat, 'DD/MM/YYYY');
|
||||
});
|
||||
|
||||
test('validates env date format before syncing', () => {
|
||||
assert.throws(
|
||||
() => buildConfig({
|
||||
TINY_API_TOKEN: 'test-token',
|
||||
DRY_RUN: 'true',
|
||||
SYNC_START_DATE: '02/07/2026'
|
||||
}),
|
||||
/SYNC_START_DATE must use YYYY-MM-DD format/
|
||||
);
|
||||
});
|
||||
|
||||
test('accepts ISO Tiny request date format override', () => {
|
||||
const config = buildConfig({
|
||||
TINY_API_TOKEN: 'test-token',
|
||||
DRY_RUN: 'true',
|
||||
SYNC_START_DATE: '2026-07-02',
|
||||
TINY_DATE_FORMAT: 'YYYY-MM-DD'
|
||||
});
|
||||
|
||||
validateConfig(config);
|
||||
assert.equal(config.tinyDateFormat, 'YYYY-MM-DD');
|
||||
});
|
||||
|
||||
test('requires DATABASE_URL only for real DB writes', () => {
|
||||
assert.doesNotThrow(() => validateConfig(buildConfig({
|
||||
TINY_API_TOKEN: 'test-token',
|
||||
DRY_RUN: 'true',
|
||||
SYNC_START_DATE: '2026-07-02'
|
||||
})));
|
||||
|
||||
assert.throws(
|
||||
() => validateConfig(buildConfig({
|
||||
TINY_API_TOKEN: 'test-token',
|
||||
DRY_RUN: 'false',
|
||||
SYNC_START_DATE: '2026-07-02'
|
||||
})),
|
||||
/DATABASE_URL is required unless DRY_RUN=true/
|
||||
);
|
||||
});
|
||||
|
||||
test('backfill mode syncs through today when SYNC_END_DATE is empty', () => {
|
||||
const config = buildConfig({
|
||||
TINY_API_TOKEN: 'test-token',
|
||||
DRY_RUN: 'true',
|
||||
SYNC_MODE: 'backfill',
|
||||
SYNC_START_DATE: '2026-06-01',
|
||||
SYNC_TIME_ZONE: 'UTC'
|
||||
});
|
||||
|
||||
validateConfig(config);
|
||||
assert.deepEqual(resolveSyncDateRange(config, new Date('2026-07-02T12:00:00Z')), {
|
||||
mode: 'backfill',
|
||||
startDate: '2026-06-01',
|
||||
endDate: '2026-07-02',
|
||||
source: 'backfill through today'
|
||||
});
|
||||
});
|
||||
|
||||
test('recent mode defaults to today plus yesterday', () => {
|
||||
const config = buildConfig({
|
||||
TINY_API_TOKEN: 'test-token',
|
||||
DRY_RUN: 'true',
|
||||
SYNC_MODE: 'recent',
|
||||
RECENT_SYNC_DAYS: '2',
|
||||
SYNC_TIME_ZONE: 'UTC'
|
||||
});
|
||||
|
||||
validateConfig(config);
|
||||
assert.deepEqual(resolveSyncDateRange(config, new Date('2026-07-02T12:00:00Z')), {
|
||||
mode: 'recent',
|
||||
startDate: '2026-07-01',
|
||||
endDate: '2026-07-02',
|
||||
source: 'recent 2 day window'
|
||||
});
|
||||
});
|
||||
61
test/mapper.test.js
Normal file
61
test/mapper.test.js
Normal file
@@ -0,0 +1,61 @@
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const { mapProductionOrder, normalizeStatus, parseDate, parseNumber } = require('../src/mapper');
|
||||
const { formatTinyDate } = require('../src/tinyClient');
|
||||
|
||||
test('normalizes accepted production order statuses', () => {
|
||||
assert.equal(normalizeStatus('em_aberto'), 'open');
|
||||
assert.equal(normalizeStatus('Em aberto'), 'open');
|
||||
assert.equal(normalizeStatus('Aberta'), 'open');
|
||||
assert.equal(normalizeStatus('andamento'), 'in_progress');
|
||||
assert.equal(normalizeStatus('Em andamento'), 'in_progress');
|
||||
assert.equal(normalizeStatus('Finalizada'), 'finished');
|
||||
assert.equal(normalizeStatus('finalizado'), 'finished');
|
||||
assert.equal(normalizeStatus('Cancelada'), 'canceled');
|
||||
assert.equal(normalizeStatus('cancelled'), 'canceled');
|
||||
assert.equal(normalizeStatus('status desconhecido'), 'open');
|
||||
});
|
||||
|
||||
test('parses Tiny date and numeric variants', () => {
|
||||
assert.equal(parseDate('02/07/2026'), '2026-07-02');
|
||||
assert.equal(parseDate('2026-07-02'), '2026-07-02');
|
||||
assert.equal(parseNumber('1.234,56'), 1234.56);
|
||||
});
|
||||
|
||||
test('formats request dates for Tiny without changing env date semantics', () => {
|
||||
assert.equal(formatTinyDate('2026-07-02', 'DD/MM/YYYY'), '02/07/2026');
|
||||
assert.equal(formatTinyDate('2026-07-02', 'YYYY-MM-DD'), '2026-07-02');
|
||||
assert.equal(formatTinyDate(null, 'DD/MM/YYYY'), null);
|
||||
});
|
||||
|
||||
test('maps Tiny production order payload defensively', () => {
|
||||
const mapped = mapProductionOrder({
|
||||
ordem_producao: {
|
||||
id: 123,
|
||||
numero: 'OP-77',
|
||||
situacao: 'Em andamento',
|
||||
data_emissao: '02/07/2026',
|
||||
data_prevista: '2026-07-10',
|
||||
produto: {
|
||||
codigo: 'SKU-1',
|
||||
descricao: 'Produto teste',
|
||||
unidade: 'UN'
|
||||
},
|
||||
quantidade: '2,5',
|
||||
pedido_venda: 'PV-9',
|
||||
marcadores: [{ marcador: { nome: 'Urgente', cor: '#ff0000' } }]
|
||||
}
|
||||
});
|
||||
|
||||
assert.equal(mapped.order.tinyId, '123');
|
||||
assert.equal(mapped.order.number, 'OP-77');
|
||||
assert.equal(mapped.order.status, 'in_progress');
|
||||
assert.equal(mapped.order.statusLabel, 'Em andamento');
|
||||
assert.equal(mapped.order.orderReference, 'PV-9');
|
||||
assert.equal(mapped.order.issueDate, '2026-07-02');
|
||||
assert.equal(mapped.order.expectedDate, '2026-07-10');
|
||||
assert.equal(mapped.order.productSku, 'SKU-1');
|
||||
assert.equal(mapped.order.productDescription, 'Produto teste');
|
||||
assert.equal(mapped.order.quantity, 2.5);
|
||||
assert.deepEqual(mapped.markers, [{ label: 'Urgente', color: '#ff0000' }]);
|
||||
});
|
||||
Reference in New Issue
Block a user