update readme
This commit is contained in:
Joseph
parent
ec9e80d36f
commit
95becfd0ee
116
README.md
116
README.md
|
|
@ -1,76 +1,72 @@
|
||||||
# eBay MPN Matcher
|
# eBay Deep Scan Dashboard
|
||||||
|
|
||||||
Multi-step pipeline: eBay search → exact MPN match → AI match → price comparison.
|
A professional eBay scanning and price monitoring tool. It automates searching for products across international eBay marketplaces, filters for the best deals based on target prices, and uses AI to verify product authenticity and seller reputation.
|
||||||
|
|
||||||
## Setup
|
## 🚀 Key Features
|
||||||
|
|
||||||
|
- **Profile Management**: Create and manage different scan profiles (e.g., RAM, SSD, CPUs) with individual target prices and price ratios.
|
||||||
|
- **International Search**: Automatically scans eBay US, UK, AU, CA, and DE marketplaces.
|
||||||
|
- **Smart Filtering**:
|
||||||
|
- Skips **Group Products** (multi-variation listings) that lack deterministic logic.
|
||||||
|
- Filters out **defective/broken** items and listings from **China (CN)**.
|
||||||
|
- Advanced **Quantity Extraction**: Intelligently parses quantities (e.g., `4x256GB`, `Lot of 10`) while avoiding technical specs (e.g., `NVMe x4`, `PCIe x16`).
|
||||||
|
- **AI-Powered Analysis**: Uses **OpenAI GPT-4o-mini** to analyze listings for "fake" red flags, seller credibility, and technical spec matches.
|
||||||
|
- **Real-time Dashboard**:
|
||||||
|
- **Progress Bar**: Monitor scan progress in real-time.
|
||||||
|
- **Enriched Details**: View high-res images, full descriptions, and calculated profit margins.
|
||||||
|
- **Usage Tracking**: Summarizes total "Quote Requests" and AI tokens consumed at the end of each process.
|
||||||
|
- **Persistence**: Powered by SQLite for reliable data storage of profiles, keywords, and found items.
|
||||||
|
|
||||||
|
## 🛠 Setup & Installation
|
||||||
|
|
||||||
|
1. **Install Dependencies**:
|
||||||
```bash
|
```bash
|
||||||
npm install
|
npm install
|
||||||
cp .env.example .env
|
|
||||||
# Fill in EBAY_CLIENT_ID, EBAY_CLIENT_SECRET, OPENAI_API_KEY
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Files required
|
2. **Configure Environment Variables**:
|
||||||
|
Copy `.env.example` to `.env` and fill in your credentials:
|
||||||
| File | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `keywords.txt` | One search keyword per line |
|
|
||||||
| `prices.xlsx` | Col A: partnumber, Col B: price (USD) |
|
|
||||||
| `system_prompt.txt` | AI system prompt for this product profile |
|
|
||||||
|
|
||||||
## Run
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
node index.js
|
cp .env.example .env
|
||||||
|
```
|
||||||
|
Required keys:
|
||||||
|
- `EBAY_CLIENT_ID` & `EBAY_CLIENT_SECRET`: From your eBay Developer account.
|
||||||
|
- `OPENAI_API_KEY`: From OpenAI.
|
||||||
|
|
||||||
|
3. **Database Initialization**:
|
||||||
|
The database (`data/ebay_items.db`) is automatically initialized on the first run.
|
||||||
|
|
||||||
|
## 🏃 Getting Started
|
||||||
|
|
||||||
|
1. **Start the Server**:
|
||||||
|
```bash
|
||||||
|
npm start
|
||||||
|
# or
|
||||||
|
node server.js
|
||||||
```
|
```
|
||||||
|
|
||||||
Output saved to `./output/`:
|
2. **Access the Web UI**:
|
||||||
- `results_<runId>.xlsx` — full results with PASS/FAIL
|
Open your browser and navigate to: `http://localhost:4000`
|
||||||
- `results_<runId>.json` — same data as JSON
|
|
||||||
- `run_<runId>.log` — full run log with timing
|
|
||||||
- `ai_cache.json` — cached AI matches (reused on next run)
|
|
||||||
|
|
||||||
## Flow
|
3. **Usage Workflow**:
|
||||||
|
- **Create a Profile**: Go to "Manage Profiles" to add a new category.
|
||||||
|
- **Add Keywords**: Use the "Manage Keywords" button to add part numbers and search terms for a profile (supports bulk import).
|
||||||
|
- **Run Scan**: Click "Start Scan" to begin searching eBay. The progress bar will show real-time status.
|
||||||
|
- **Review Items**: Review items that passed the price filter (PASS status). Click on an item title to see AI analysis, high-res images, and available quantity.
|
||||||
|
|
||||||
```
|
## 📂 Project Structure
|
||||||
keywords.txt
|
|
||||||
│
|
|
||||||
▼ Step 1: eBay search (all keywords, dedup by URL item ID)
|
|
||||||
│
|
|
||||||
▼ Step 2: Exact MPN match (regex + normalized)
|
|
||||||
│ ├─ matched → list A
|
|
||||||
│ └─ unmatched → Step 3
|
|
||||||
│
|
|
||||||
▼ Step 3: AI match (gpt-4o-mini, batch 40 items)
|
|
||||||
│ ├─ GOOD_MATCH / VARIANT_MISMATCH → list B
|
|
||||||
│ ├─ INSUFFICIENT_DATA → fetch detail → retry AI
|
|
||||||
│ └─ UNRELATED_PRODUCT → skip
|
|
||||||
│
|
|
||||||
▼ Step 4: Merge A+B → compare price → PASS/FAIL → export
|
|
||||||
```
|
|
||||||
|
|
||||||
## Match methods in output
|
- `server.js`: Express API and web server.
|
||||||
|
- `scanner.js`: Core scanning engine (eBay API integration, filtering, and quantity logic).
|
||||||
|
- `db.js`: Database layer (SQLite using `better-sqlite3`).
|
||||||
|
- `ai.js`: AI analysis layer (OpenAI integration).
|
||||||
|
- `public/`: Frontend dashboard (HTML/CSS/JS).
|
||||||
|
|
||||||
| Method | Description |
|
## 📊 Monitoring
|
||||||
|--------|-------------|
|
|
||||||
| `exact` | Regex found MPN in title/specs |
|
|
||||||
| `ai_exact` | AI matched with GOOD_MATCH |
|
|
||||||
| `ai_variant` | AI matched with VARIANT_MISMATCH |
|
|
||||||
| `ai_detail` | AI matched after fetching detail page |
|
|
||||||
| `ai_cached` | From previous run cache |
|
|
||||||
|
|
||||||
## Config (.env)
|
At the end of every scan, the system provides a **Usage Summary** in the logs:
|
||||||
|
- **Total Quote Requests**: Number of items successfully enriched with details.
|
||||||
|
- **Total AI Tokens**: Total consumption of OpenAI tokens.
|
||||||
|
|
||||||
| Key | Default | Description |
|
---
|
||||||
|-----|---------|-------------|
|
*Developed for professional eBay asset recovery and price monitoring.*
|
||||||
| `AI_BATCH_SIZE` | 40 | Listings per AI request |
|
|
||||||
| `AI_CONFIDENCE_THRESHOLD` | 50 | Min confidence to count as PASS |
|
|
||||||
| `OPENAI_MODEL` | gpt-4o-mini | OpenAI model |
|
|
||||||
| `EBAY_MARKETPLACE` | EBAY_US | eBay marketplace ID |
|
|
||||||
|
|
||||||
## Adding a new product profile
|
|
||||||
|
|
||||||
1. Create new `keywords_<profile>.txt`
|
|
||||||
2. Create new `system_prompt_<profile>.txt` explaining the product domain
|
|
||||||
3. Update `.env` to point to new files
|
|
||||||
4. Run — AI cache is per-itemId so it auto-separates
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue