Project opening
Regulatory Document Intelligence Pipeline
Turn complex, inconsistent regulatory documents into governed operational data without removing human judgment from the critical path.
- Category
- Enterprise AI
- Architecture focus
- Pipeline, validation & integration
- Period
- 2025
Context
Documents arrived faster than teams could structure them.
An internal operations team depended on regulatory documents arriving through several channels and in multiple layouts. The information mattered operationally, but it was difficult to process consistently at source.
The organisation needed a repeatable pipeline that could ingest those documents, extract useful fields, preserve traceability, and pass only reviewed data into downstream systems.
Challenge
Automation had to coexist with uncertainty and control.
Source files could be scanned, multilingual, versioned, or structured differently. A single extraction path could not be treated as an unquestioned system of record.
The architecture also had to work within a private enterprise network, integrate multiple intake channels, and make exceptions visible to people rather than burying them in an automated flow.
Paul’s role
Architecture as an operating model.
Paul’s contribution is represented here only through the publicly documented architecture: ingestion, extraction, validation, private connectivity, and downstream delivery. Exact client-side responsibilities and team structure remain confidential.
Solution
A staged pipeline made every decision inspectable.
Power Automate coordinates ingestion from email, SharePoint, and SFTP. Azure AI Document Intelligence extracts fields, while Dataverse holds staged records and their processing state.
A dedicated validation layer applies business rules, cross-references extracted values, and routes uncertain cases to human review. Approved records then move to downstream data and reporting consumers.
System architecture
A controlled path from document to operational data.
Each stage has a distinct responsibility, making the automation easier to govern, inspect, and adapt.
- 01
Sources
Bring heterogeneous documents into one governed intake path.
- SharePoint
- SFTP
- 02
Orchestration & extraction
Coordinate processing and turn document layouts into candidate data.
- Power Automate
- Azure AI Document Intelligence
- VNet Data Gateway
- 03
Validation
Treat AI output as evidence to verify, not as an automatic truth.
- Dataverse staging
- Business rules
- Human review
- 04
Delivery
Release reviewed operational data with traceable processing state.
- Approved records
- Data consumers
- Operational reporting
Key decisions
The trade-offs that shaped the system.
Stage before publishing
Keep extracted values in Dataverse until validation is complete.
This separates machine interpretation from approved operational data and preserves a reviewable state model.
Design explicit exception paths
Route uncertain or invalid records to a visible human-review path.
Regulated work needs accountable decisions; silent automation failure was not an acceptable operating model.
Use specialised extraction models
Adapt extraction to recurring document layouts rather than forcing one universal model.
Document variation is better handled as a known architectural condition than hidden behind one confidence score.
Keep connectivity private
Use the VNet Data Gateway for the private-network integration path.
The solution had to meet the enterprise network boundary without weakening the end-to-end workflow.
Outcome
A governed pipeline, not an extraction demo.
- 01
A repeatable ingestion and validation path replaced disconnected document-handling steps.
- 02
Human reviewers received explicit exceptions and staged data instead of opaque extraction output.
- 03
Downstream consumers received reviewed, structured records with a traceable processing state.
Learnings
What transfers to the next system.
- 01
Document AI creates the most value when its uncertainty is designed into the operating model.
- 02
Validation deserves its own architectural layer rather than a final conditional branch.
- 03
Private connectivity and exception handling should be designed at the start, not added after the happy path works.