Azure Document Intelligence

Use Azure Document Intelligence for cloud-based document conversion with advanced OCR and layout analysis

Azure Document Intelligence (formerly Form Recognizer) provides cloud-based document conversion with advanced OCR, layout analysis, and formula extraction capabilities.

Overview

The Document Intelligence converter uses the prebuilt-layout model to extract Markdown directly from documents with:

• High-resolution OCR
• Table detection and extraction
• Formula recognition (LaTeX output)
• Font style preservation
• Multi-column layout handling

Document Intelligence is particularly useful for scanned documents, images, and complex PDFs where offline conversion may struggle.

Prerequisites

1. Install Optional Dependencies

   pip install markitdown[az-doc-intel]
   

   This installs:

   • azure-ai-documentintelligence
   • azure-identity
2. Create Azure Resource

   1. Sign in to Azure Portal
   2. Create a Document Intelligence resource
   3. Note your Endpoint URL (e.g., https://YOUR_ENDPOINT.cognitiveservices.azure.com/)
   4. Get your API Key from the “Keys and Endpoint” section
3. Configure Authentication

   Set your API key as an environment variable:

   export AZURE_API_KEY="your-api-key-here"
   

   Or use Azure credential mechanisms (see Authentication below)

Usage

Command Line

Use the -d / --use-docintel flag with -e / --endpoint:

markitdown -d -e https://YOUR_ENDPOINT.cognitiveservices.azure.com/ document.pdf

Save to file:

markitdown -d -e https://YOUR_ENDPOINT.cognitiveservices.azure.com/ document.pdf -o output.md

Document Intelligence requires:

• A file path (stdin is not supported)
• A valid endpoint URL
• Valid authentication credentials

Python API

Basic Usage

from markitdown import MarkItDown

# Using AZURE_API_KEY environment variable
md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/"
)

result = md.convert("document.pdf")
print(result.markdown)

Authentication

Document Intelligence supports multiple authentication methods:

Environment Variable (Default)

Set AZURE_API_KEY:

export AZURE_API_KEY="your-api-key"

md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/"
)
# Automatically uses AZURE_API_KEY

API Key Credential

Explicitly provide the key:

from azure.core.credentials import AzureKeyCredential

md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/",
    docintel_credential=AzureKeyCredential("your-api-key")
)

Azure Identity (Recommended for Production)

Use DefaultAzureCredential for automatic credential detection:

from azure.identity import DefaultAzureCredential

md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/",
    docintel_credential=DefaultAzureCredential()
)

This supports:

• Environment variables (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET)
• Managed Identity (when running in Azure)
• Azure CLI authentication
• Visual Studio Code authentication

Service Principal

from azure.identity import ClientSecretCredential

credential = ClientSecretCredential(
    tenant_id="your-tenant-id",
    client_id="your-client-id",
    client_secret="your-client-secret"
)

md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/",
    docintel_credential=credential
)

Configuration Options

API Version

Specify the Document Intelligence API version:

md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/",
    docintel_api_version="2024-07-31-preview"  # Default
)

File Types

Control which file types use Document Intelligence:

from markitdown import MarkItDown
from markitdown.converters._doc_intel_converter import DocumentIntelligenceFileType

# Only use Document Intelligence for PDFs and images
md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/",
    docintel_file_types=[
        DocumentIntelligenceFileType.PDF,
        DocumentIntelligenceFileType.JPEG,
        DocumentIntelligenceFileType.PNG,
    ]
)

Available file types:

• DOCX - Word documents
• PPTX - PowerPoint presentations
• XLSX - Excel spreadsheets
• HTML - HTML files
• PDF - PDF documents (with OCR)
• JPEG - JPEG images (with OCR)
• PNG - PNG images (with OCR)
• BMP - BMP images (with OCR)
• TIFF - TIFF images (with OCR)

By default, Document Intelligence is used for: DOCX, PPTX, XLSX, PDF, JPEG, PNG, BMP, and TIFF files.

Supported Features

OCR File Types

Document Intelligence provides high-resolution OCR for:

PDF Documents

md = MarkItDown(docintel_endpoint="...")
result = md.convert("scanned.pdf")

Analysis Features

For OCR-supported file types, Document Intelligence enables:

• High-Resolution OCR: Enhanced text extraction from images
• Formula Extraction: Mathematical formulas converted to LaTeX
• Font Style Detection: Preserves bold, italic, and other styles

# These features are automatically enabled for PDF, JPEG, PNG, BMP, TIFF
md = MarkItDown(docintel_endpoint="...")
result = md.convert("math_document.pdf")
# Output includes LaTeX formulas like: $\frac{a}{b}$

Non-OCR File Types

For DOCX, PPTX, XLSX, and HTML files, Document Intelligence performs layout analysis without OCR features.

Error Handling

from markitdown import MarkItDown, MissingDependencyException
from azure.core.exceptions import HttpResponseError

md = MarkItDown(docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/")

try:
    result = md.convert("document.pdf")
except MissingDependencyException as e:
    print("Install dependencies: pip install markitdown[az-doc-intel]")
except HttpResponseError as e:
    if e.status_code == 401:
        print("Authentication failed. Check your API key or credentials.")
    elif e.status_code == 403:
        print("Access forbidden. Check your endpoint URL and permissions.")
    else:
        print(f"Azure error: {e.message}")
except Exception as e:
    print(f"Conversion failed: {e}")

Cost Considerations

Document Intelligence is a paid Azure service. Each document conversion consumes API credits.

Cost factors:

• Pages processed: Charged per page
• Features used: OCR, formulas, and style analysis may incur additional costs
• Model type: prebuilt-layout model pricing

See Azure Document Intelligence pricing for details.

Minimize Costs

# Only use Document Intelligence for specific file types
md = MarkItDown(
    docintel_endpoint="...",
    docintel_file_types=[
        DocumentIntelligenceFileType.PDF,  # Only PDFs
    ]
)

# Other file types will use offline converters
result1 = md.convert("document.pdf")  # Uses Document Intelligence
result2 = md.convert("document.docx")  # Uses offline converter

Converter Priority

When Document Intelligence is enabled, it takes priority over built-in converters for supported file types:

md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/"
)

# Document Intelligence converter is registered at the top of the stack
# and will be tried first for supported file types

To explicitly control converter priority:

from markitdown import MarkItDown, PRIORITY_SPECIFIC_FILE_FORMAT
from markitdown.converters import DocumentIntelligenceConverter

md = MarkItDown(enable_builtins=False)
md.enable_builtins()  # Register built-in converters

# Register Document Intelligence with custom priority
md.register_converter(
    DocumentIntelligenceConverter(
        endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/"
    ),
    priority=-1.0  # Higher priority than built-ins (0.0)
)

Examples

Scanned PDF with Tables

from markitdown import MarkItDown

md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/"
)

result = md.convert("scanned_invoice.pdf")
print(result.markdown)
# Output includes detected tables in Markdown format

Batch Processing

from markitdown import MarkItDown
from pathlib import Path

md = MarkItDown(
    docintel_endpoint="https://YOUR_ENDPOINT.cognitiveservices.azure.com/"
)

input_dir = Path("documents")
output_dir = Path("markdown")
output_dir.mkdir(exist_ok=True)

for pdf_file in input_dir.glob("*.pdf"):
    result = md.convert(str(pdf_file))
    output_file = output_dir / f"{pdf_file.stem}.md"
    output_file.write_text(result.markdown)
    print(f"Converted: {pdf_file.name}")

With Docker

docker run \
  -e AZURE_API_KEY="your-api-key" \
  -v $(pwd):/data \
  markitdown \
  -d -e https://YOUR_ENDPOINT.cognitiveservices.azure.com/ \
  /data/document.pdf -o /data/output.md

Troubleshooting

Missing Dependencies

MissingDependencyException: DocumentIntelligenceConverter requires [az-doc-intel]

Solution:

pip install markitdown[az-doc-intel]

Authentication Errors

HttpResponseError: (401) Unauthorized

Check:

• Endpoint URL is correct
• AZURE_API_KEY is set or credential is provided
• API key is valid and not expired

Endpoint Not Found

Document Intelligence Endpoint is required when using Document Intelligence.

Solution:

markitdown -d -e https://YOUR_ENDPOINT.cognitiveservices.azure.com/ file.pdf

Stdin Not Supported

Filename is required when using Document Intelligence.

Document Intelligence requires a file path:

# Wrong:
cat file.pdf | markitdown -d -e ...

# Correct:
markitdown -d -e https://YOUR_ENDPOINT.cognitiveservices.azure.com/ file.pdf

Comparison: Document Intelligence vs Offline

Feature	Document Intelligence	Offline Converters
Cost	Paid (per page)	Free
OCR Quality	High-resolution	Limited/None
Table Detection	Advanced	Basic
Formula Support	LaTeX output	Limited
Complex Layouts	Excellent	Good
Internet Required	Yes	No
Speed	Network latency	Fast
Privacy	Cloud-based	Local only

Use Document Intelligence for:

• Scanned documents
• Complex layouts with tables
• Documents with formulas
• High-quality OCR requirements

Use offline converters for:

• Cost-sensitive applications
• Privacy-critical documents
• Simple, well-structured documents
• Offline environments