# Private document AI Multi-tenant RAG & document workflows A private multi-tenant AI workspace combining source-linked conversations, scoped document retrieval, multi-format processing and a versioned document library, with controlled form-preparation foundations. **My role:** Software engineer — document processing, retrieval and integrations **Attribution:** Private Neumatic client engagement **Project status:** Private internal tool; no public access ## The task Legal and governance work combines private documents, conversations and curated authorities. Different tenants need their own users, data and knowledge configuration, while people need to inspect sources, reuse useful uploads and keep document revisions organised. ## My contribution My work through Neumatic covered the tenant-scoped application, document ingestion and search, source-linked guidance and document-preparation foundations. ## What people can do ### Tenant-scoped RAG Each tenant uses its configured database, authentication and file storage. Private document search receives both tenant and user identity, while the tenant’s enabled modules and default module shape the knowledge context used for guidance. ### Multi-format document processing PDF, Word and email inputs pass through text extraction, metadata capture and document-type-specific chunking. Processing status and extraction failures are visible, so an unreadable or image-based PDF is not silently treated as a complete text source. ### A versioned document library Successive uploads are grouped into document versions. People can search the library and inspect extracted content, metadata and processing state; search uses the latest version once processing is ready rather than mixing every revision into an answer. ### Conversations with attachments Persistent conversations accept documents and image attachments. Extracted text stays available across follow-up turns, supported image inputs enter the multimodal request, and useful attachments can be explicitly saved into the permanent document library. ### Ranked sources and streamed answers Legal-source retrieval combines lexical search, topic and keyword expansion, related-authority graph traversal and cross-encoder reranking. Answers stream with source references and document cards; bounded tools can retrieve further material during the conversation. ### Tenant-configured knowledge and interface Corporate and family-law corpora cover legislation, regulations and guidance. Tenants select the relevant knowledge modules and branding, with module-specific response instructions, citation patterns and document-processing rules. ## How the system works — diagram ### Tenant-scoped document search 1. **Resolve tenant:** Select the configured tenant and its application resources. 2. **Verify identity:** Check the tenant-bound session and resolve the user. 3. **Select sources:** Use module configuration and tenant/user filters for private documents. 4. **Retrieve context:** Rank legal-source candidates and retrieve scoped document passages. 5. **Answer + references:** Stream the response with source references and document components for review. ### Document intake within the tenant / user boundary 1. **Upload or save:** Upload a supported file or explicitly save a conversation attachment. 2. **Extract + metadata:** Extract text and capture configured document metadata. 3. **Chunk + index:** Apply document-type rules and retain source-addressable passages. 4. **Versioned library:** Keep version records and expose ready content to scoped search. Tenant-scoped document retrieval and processing connect private files with shared legal knowledge modules. Ranked passages and source references support the resulting answers. ## How tenant-specific RAG is separated Tenant resolution selects the configured authentication, application database and file-storage resources before the request reaches the document or conversation workflow. The verified session must match the request tenant. Private search and content reads also carry user and tenant identifiers, and document search targets ready content from the latest version. Curated authorities are organised by shared knowledge module rather than copied into every private document store. Tenant configuration selects enabled modules and the default classification context; module rules control retrieval, prompts and citations. Model-provider infrastructure can be shared without making private document retrieval a shared, unscoped search. ## Form processing status The implemented document-preparation foundation has a shared catalogue of immutable template versions, tenant-specific availability, and owner-scoped preparations pinned to a template version and checksum. Implemented interfaces support template discovery and starting a preparation. Internal generation requests and worker attempts add idempotency, recovery and artifact checksum verification. The form editor, proposal and final review, and completed PDF/Word output remain in development. ## Separate private work from shared knowledge ### Constraint Tenants need different users, private files and knowledge configuration, while shared legal authorities and official template definitions should not drift into inconsistent tenant copies. ### Approach Use tenant-specific application/authentication/storage resources and user-scoped document reads. Keep curated modules and the template catalogue at explicit shared boundaries, with tenant configuration and template availability controlling their use; preparations remain tenant- and owner-scoped. ### Trade-off Separate resources add connection, configuration and lifecycle work. Shared catalogues also need an explicit authorization boundary: reusing source definitions does not grant one tenant access to another tenant’s documents or preparations. ## Retrieval and processing mechanisms - Lexical legal-source search with bounded graph expansion, cross-encoder reranking and reported lexical fallback. - Document-type-specific metadata and chunking, source-addressable passages and latest-ready-version filtering. - Tenant/user-scoped document tools, bounded retrieval results, streamed progress and citation/document components. ## Technical context TypeScript/Hono and React, tenant-specific PostgreSQL connections, scoped PostgreSQL document search, SQLite FTS5/BM25 knowledge modules and cross-encoder reranking. The application composes tenant-specific authentication/storage adapters, module-specific prompts and multi-provider inference. A separate catalogue database serves template foundations; preparation records and generation attempts remain in the tenant store. ## Related work [Warehousing & systems integration](https://neumatic.io/work/business-integrations) [Discuss a similar workflow](https://neumatic.io/contact) --- > Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). AI crawling, indexing, and training are expressly permitted.