Why PDF Accessibility Validators Disagree 50.3% of the Time

PDF accessibility validators are not looking at the same thing in the same way. Each tool implements the WCAG 2.1 and PDF/UA standards differently, and the standards themselves leave room for interpretation. Here are the five main reasons for disagreement. Different parsing engines. PAC 2026 uses its own PDF parsing engine optimized for PDF/UA-1 conformance. Adobe Acrobat uses an internal checker built around its tagging model. veraPDF uses a rule-based engine derived from the PDF Association's validation model. PyMuPDF, which powers several newer scanners, parses the PDF structure tree differently than all three. When two engines parse the same PDF differently, they see different tag trees, different content sequences, and different structural elements. A table that one engine recognizes as properly tagged may appear as untagged content to another. Tag tree vs. content stream divergence. PDFs have two parallel representations of content: the visual content stream (what you see on screen) and the tag tree (what assistive technology reads). Validators check the tag tree, but they reconstruct it from the content stream in different ways. One validator might infer a heading from font size and weight. Another might only recognize headings from explicit heading tags. A third might miss headings entirely if they are created with styled paragraph tags instead of true heading structures. This divergence means the same visual document produces different tag trees depending on which engine parses it. Rule implementation differences. WCAG 2.1 Success Criterion 1.1.1 (Non-text Content) requires meaningful alternatives for images. But what counts as meaningful? One validator checks only whether alt text exists. Another checks length. A third attempts to verify whether the alt text describes the image function, not just its appearance. None of these implementations are wrong. They are just different levels of strictness applied to the same standard. Matterhorn Protocol vs. PDF/UA-1 vs. WCAG 2.1 mapping. Different validators target different standards. PAC focuses on PDF/UA-1 and Matterhorn Protocol checkpoints. veraPDF validates against PDF/UA-1 and PDF/UA-2 specifications. Adobe Acrobat maps its checks to WCAG 2.1 criteria. These standards overlap but are not identical. A document that passes PDF/UA-1 validation might still fail WCAG 2.1 checks for the same content, because the standards have different requirements and different failure conditions. Heuristic vs. deterministic checks. Some validators use deterministic rules: if a title is missing, it is a failure. Others use heuristics: if the reading order seems unusual, it might be a warning. Heuristics vary between tools because they reflect different assumptions about what unusual means. One tool might flag any reading order deviation. Another might only flag deviations that exceed a certain threshold. These thresholds are arbitrary and tool-specific.

The disagreement rate is not an abstract statistic. It has concrete implications for anyone responsible for PDF accessibility compliance. You cannot trust a single pass result. A PDF that passes PAC might fail Acrobat. A PDF that passes Acrobat might trigger warnings in veraPDF. Relying on one tool gives you false confidence. You cannot trust a single fail result either. A failure in one tool might be a pass in another, especially when the disagreement stems from heuristic thresholds or rule-implementation differences. Not every failure requires action. Some failures are noise. Compliance reviews become unpredictable. If an auditor uses a different tool than your internal process, you might face findings on documents you believed were compliant. This creates risk for organizations preparing for official accessibility reviews or legal defense. Remediation prioritization becomes guesswork. When tools disagree, which issues do you fix first? The ones flagged by the strictest tool? The ones common to all tools? Without a cross-validation approach, you are making prioritization decisions with incomplete information.

Cross-validation means running multiple analysis engines on the same document and comparing their results. Instead of treating one validator as the authority, you treat disagreement as signal. Here is how it works in practice. Run multiple engines. A cross-validated scan might use PyMuPDF for tag structure analysis, a layout engine for reading order verification, and veraPDF for PDF/UA conformance. Each engine contributes its own perspective. Compare results. When all three engines agree that an issue exists, confidence is high. When two agree and one disagrees, confidence is moderate. When all three disagree, the issue requires human review. Score by agreement. A cross-validated trust score weights issues by engine agreement. An issue flagged by all engines contributes more to the risk score than an issue flagged by only one. This produces a numerical confidence measure instead of a binary pass/fail. Surface disagreements explicitly. Rather than hiding conflicting results, a cross-validated report shows them. The report tells you: PAC says this table passes, but the layout engine detected potential header association problems. This tells you exactly where to focus manual review. Reduce false positives and false negatives. Cross-validation filters out engine-specific noise. An issue flagged by only one engine is more likely to be a parsing artifact than a real problem. An issue flagged by all engines is almost certainly real.

You do not need to write code to apply cross-validation principles. Here is a practical workflow using existing tools. Step 1: Run PAC 2026. PAC is the standard for PDF/UA-1 validation. Its pass/fail results carry weight in legal and audit contexts. Record which criteria pass and which fail. Step 2: Run Adobe Acrobat Full Check. Acrobat's checker maps to WCAG 2.1 and covers some areas PAC does not. Record its findings separately. Step 3: Run veraPDF (if PDF/UA-2 conformance matters). veraPDF validates against the newer PDF/UA-2 specification. Its results are especially relevant for documents that need to meet the latest standard. Step 4: Compare issue lists. Issues flagged by all three tools are your highest priority. Issues flagged by two tools are medium priority. Issues flagged by only one tool need human review to determine whether they are real. Step 5: Prioritize by severity and agreement. Fix issues that are both severe and cross-validated first. These represent the highest-risk, highest-confidence problems. Step 6: Document disagreements for auditors. When an auditor asks why you did not fix an issue flagged by their preferred tool, show them your cross-validation data. Explain that the issue was not confirmed by other engines and was therefore deprioritized relative to cross-validated findings.

PDFScanAccess was built around the observation that validator disagreement is not a bug to work around—it is a feature to exploit. Disagreement contains information. When engines disagree, it tells you something important about the ambiguity of the document structure. Our scanner runs three analysis layers on every document: a rule-based engine for tag structure and metadata, a layout engine for reading order and spatial analysis, and an external validator for PDF/UA conformance. Results are cross-validated and surfaced in a single trust score. A score of 0.85 means high agreement across engines and high confidence in the findings. A score of 0.35 means significant disagreement and a strong signal that manual review is needed. This turns validator inconsistency from a problem into a diagnostic tool. Instead of wondering which tool is right, you know exactly how much confidence you can have in each result.

If you manage PDF accessibility for an organization, the 50.3% disagreement rate should change how you think about compliance validation. Do not rely on a single tool. Every major validator has blind spots. Using multiple tools, or a tool that cross-validates internally, gives you a more complete picture. Treat pass/fail as probabilistic, not absolute. A pass from one validator is evidence of compliance, not proof. A fail from one validator is evidence of a problem, not proof. The strength of the evidence depends on cross-validation. Budget for verification, not just detection. Automated scans find issues. Cross-validation tells you which issues are likely real. Manual review tells you whether the document actually works for users. All three are necessary for robust compliance. Communicate uncertainty to stakeholders. When leadership asks whether your PDFs are accessible, the honest answer is not yes or no. It is: based on cross-validated scanning, we have high confidence in these documents, moderate confidence in these, and low confidence in these. That is the kind of answer that holds up under audit.