Engine · Available

Document

Metadata, pdf-parse, OCR ports, chunks, and search.

@keystone/engine-document

Purpose

Document identity, ingest, and chunk search separate from any one product’s upload UI. PDFs extract via pdf-parse; scanned pages use OCR ports (Textract or Tesseract). Costly OCR meters through Billing.

In a product

Ingest a contract PDF → extract text → chunk → search later; turn on OCR for scans and Billing records the pages.

How it works

  1. Create a document with name, mime type, and optional storage pointers.
  2. ingestBuffer runs processing → extract (pdf-parse or plain text) → optional OCR → ~1000-character chunks → ready or failed.
  3. Chunk put/list/delete and substring search keep retrieval scoped to the organization.
  4. Processing and chunk events publish for Automation and observability.

Capabilities

  • Document CRUD with status lifecycle
  • Ingest with pdf-parse and optional OCR
  • Chunk upsert, list, delete, and substring search
  • Meters document.extract and OCR kinds via Billing
  • Events: document.processing.* and document.chunks.upserted
  • API under /v1/documents (CRUD, chunks, ingest, search)

Related

Billing · AI · Event · Automation

All enginesBuild on Keystone