diff --git a/README.md b/README.md index 306cb87..2c2e0b4 100644 --- a/README.md +++ b/README.md @@ -1,76 +1,72 @@ -# eBay MPN Matcher +# eBay Deep Scan Dashboard -Multi-step pipeline: eBay search → exact MPN match → AI match → price comparison. +A professional eBay scanning and price monitoring tool. It automates searching for products across international eBay marketplaces, filters for the best deals based on target prices, and uses AI to verify product authenticity and seller reputation. -## Setup +## 🚀 Key Features -```bash -npm install -cp .env.example .env -# Fill in EBAY_CLIENT_ID, EBAY_CLIENT_SECRET, OPENAI_API_KEY -``` +- **Profile Management**: Create and manage different scan profiles (e.g., RAM, SSD, CPUs) with individual target prices and price ratios. +- **International Search**: Automatically scans eBay US, UK, AU, CA, and DE marketplaces. +- **Smart Filtering**: + - Skips **Group Products** (multi-variation listings) that lack deterministic logic. + - Filters out **defective/broken** items and listings from **China (CN)**. + - Advanced **Quantity Extraction**: Intelligently parses quantities (e.g., `4x256GB`, `Lot of 10`) while avoiding technical specs (e.g., `NVMe x4`, `PCIe x16`). +- **AI-Powered Analysis**: Uses **OpenAI GPT-4o-mini** to analyze listings for "fake" red flags, seller credibility, and technical spec matches. +- **Real-time Dashboard**: + - **Progress Bar**: Monitor scan progress in real-time. + - **Enriched Details**: View high-res images, full descriptions, and calculated profit margins. + - **Usage Tracking**: Summarizes total "Quote Requests" and AI tokens consumed at the end of each process. +- **Persistence**: Powered by SQLite for reliable data storage of profiles, keywords, and found items. -## Files required +## 🛠 Setup & Installation -| File | Description | -|------|-------------| -| `keywords.txt` | One search keyword per line | -| `prices.xlsx` | Col A: partnumber, Col B: price (USD) | -| `system_prompt.txt` | AI system prompt for this product profile | +1. **Install Dependencies**: + ```bash + npm install + ``` -## Run +2. **Configure Environment Variables**: + Copy `.env.example` to `.env` and fill in your credentials: + ```bash + cp .env.example .env + ``` + Required keys: + - `EBAY_CLIENT_ID` & `EBAY_CLIENT_SECRET`: From your eBay Developer account. + - `OPENAI_API_KEY`: From OpenAI. -```bash -node index.js -``` +3. **Database Initialization**: + The database (`data/ebay_items.db`) is automatically initialized on the first run. -Output saved to `./output/`: -- `results_.xlsx` — full results with PASS/FAIL -- `results_.json` — same data as JSON -- `run_.log` — full run log with timing -- `ai_cache.json` — cached AI matches (reused on next run) +## 🏃 Getting Started -## Flow +1. **Start the Server**: + ```bash + npm start + # or + node server.js + ``` -``` -keywords.txt - │ - ▼ Step 1: eBay search (all keywords, dedup by URL item ID) - │ - ▼ Step 2: Exact MPN match (regex + normalized) - │ ├─ matched → list A - │ └─ unmatched → Step 3 - │ - ▼ Step 3: AI match (gpt-4o-mini, batch 40 items) - │ ├─ GOOD_MATCH / VARIANT_MISMATCH → list B - │ ├─ INSUFFICIENT_DATA → fetch detail → retry AI - │ └─ UNRELATED_PRODUCT → skip - │ - ▼ Step 4: Merge A+B → compare price → PASS/FAIL → export -``` +2. **Access the Web UI**: + Open your browser and navigate to: `http://localhost:4000` -## Match methods in output +3. **Usage Workflow**: + - **Create a Profile**: Go to "Manage Profiles" to add a new category. + - **Add Keywords**: Use the "Manage Keywords" button to add part numbers and search terms for a profile (supports bulk import). + - **Run Scan**: Click "Start Scan" to begin searching eBay. The progress bar will show real-time status. + - **Review Items**: Review items that passed the price filter (PASS status). Click on an item title to see AI analysis, high-res images, and available quantity. -| Method | Description | -|--------|-------------| -| `exact` | Regex found MPN in title/specs | -| `ai_exact` | AI matched with GOOD_MATCH | -| `ai_variant` | AI matched with VARIANT_MISMATCH | -| `ai_detail` | AI matched after fetching detail page | -| `ai_cached` | From previous run cache | +## 📂 Project Structure -## Config (.env) +- `server.js`: Express API and web server. +- `scanner.js`: Core scanning engine (eBay API integration, filtering, and quantity logic). +- `db.js`: Database layer (SQLite using `better-sqlite3`). +- `ai.js`: AI analysis layer (OpenAI integration). +- `public/`: Frontend dashboard (HTML/CSS/JS). -| Key | Default | Description | -|-----|---------|-------------| -| `AI_BATCH_SIZE` | 40 | Listings per AI request | -| `AI_CONFIDENCE_THRESHOLD` | 50 | Min confidence to count as PASS | -| `OPENAI_MODEL` | gpt-4o-mini | OpenAI model | -| `EBAY_MARKETPLACE` | EBAY_US | eBay marketplace ID | +## 📊 Monitoring -## Adding a new product profile +At the end of every scan, the system provides a **Usage Summary** in the logs: +- **Total Quote Requests**: Number of items successfully enriched with details. +- **Total AI Tokens**: Total consumption of OpenAI tokens. -1. Create new `keywords_.txt` -2. Create new `system_prompt_.txt` explaining the product domain -3. Update `.env` to point to new files -4. Run — AI cache is per-itemId so it auto-separates +--- +*Developed for professional eBay asset recovery and price monitoring.*