PipeCheck
An ops-grade CSV ingestion pipeline with deduplication, idempotency, and real-time monitoring dashboard
🚀 Live Demo
Try the PipeCheck CSV validation tool now!
Coming Soon
Features
- Robust CSV Processing: Parse, validate, normalize, and store CSV data with comprehensive error handling
- Deduplication: Automatic detection of duplicate rows using SHA-256 hashing
- Idempotency: Re-uploading the same file won't create duplicate records
- Real-time Dashboard: Monitor processing status, view statistics, and track errors
- Structured Logging: JSON-formatted logs with run IDs for easy debugging
- Error Tracking: Detailed error logs with row indices and failure codes
- SQLite/PostgreSQL Support: Start with SQLite, easily migrate to Postgres
Coming Soon
The full PipeCheck application is currently under development
View on GitHub