Validity of the Hypergeometric Convergence Test

Sixteen regimes jointly require 14 documentation fields with P < 0.001

The paper defends the Decision Accounting convergence result against three methods objections: the wrong null model, coder dependence, and selection bias. The reported statistic is the intersection size of regime-level requirement vectors, not a reliability score.

• Observed convergence statistic: C = 14 fields required by all 16 regimes
• Candidate pool: M = 47 documentation fields
• Regime sizes: k_i ranges from 8 to 22 required fields
• Tail probability: P(C ≥ 14) < 0.001 by exact enumeration

The paper answers three threats to the convergence test

The reviewer argues that hand-coded prose is not an urn draw, that the same two coders could manufacture overlap, and that the sixteen regimes may have been chosen because they already looked similar.

• Wrong-test objection: hypergeometric draws may not fit coded regulatory text
• Coder-dependence objection: shared coder judgment could inflate C
• Selection objection: regimes might have been selected on field overlap
• The paper treats these as separate threats with separate answers

Each regime is reduced to a fixed binary vector over the same 47-field taxonomy

The test operates after coding. Each regime i has a binary vector v_i of length M, where v_ij = 1 if the regime requires field j and 0 otherwise. The test uses those fixed vectors as inputs.

• The pool is the union of observed requirements plus plausible absent fields
• Appendix F includes rejected candidates: COST, ALTERNATIVES-as-separate, and PRECEDENT
• Example fields include RESPONSIBLE-PARTY, RISK-ASSESSMENT-METHODOLOGY, MITIGATION-PLAN, and APPROVAL-AUTHORITY
• Partial requirements are coded as required if the regime imposes any obligation related to the field

The null is independent random subsets with each regime’s observed size held fixed

The hypergeometric null asks how much overlap would occur if each regime independently selected exactly k_i fields from the shared pool of M fields. It does not claim regulators actually choose fields randomly.

• Each regime selects exactly k_i fields uniformly from M = 47
• Selections are without replacement because a regime cannot require the same field twice
• The probability a given field is required by all regimes is product_i(k_i / M)
• Expected overlap is E = M * product_i(k_i / M)

The test matches the overlap-of-random-subsets problem

The paper’s target question is set overlap: how many fields would all N regimes share by chance, given each regime’s fixed number of required fields. That is the combinatorial setting for the hypergeometric distribution.

• The requirement vector is a set, not a multiset
• The binomial approximation treats field selections as independent within a regime
• Fixed k_i induces negative correlation across fields within each regime
• With M = 47 and k_i from 8 to 22, the binomial approximation can overstate large-overlap probability

Equal field probability makes rejection harder under the paper’s argument

The null treats all 47 candidate fields as equally likely before selection. The paper argues this inflates the chance of spurious overlap compared with a world where some fields are naturally more likely than others.

• Equal prior probability is the same baseline used in Fisher’s exact test and gene set enrichment
• The paper says equal probability maximizes null variance
• The reported P < 0.001 is framed as an upper bound on significance under this assumption
• The conservatism claim concerns the null, not coder accuracy

C = 14 is cross-regime overlap, while alpha = 0.89 is coder reliability

The paper says the methods dispute comes partly from conflating two forms of agreement. Krippendorff’s alpha measures whether coders agree on field attributions. C measures whether regimes share required fields.

• Krippendorff’s alpha is reported as 0.89 across all regimes and fields
• C is reported as 14 fields jointly required by all 16 regimes
• Proposition 1 shows alpha can be 1 when C = 0
• The same proposition shows C can be high when alpha is low

The hypergeometric P-value depends on C, M, and k_i, not on who coded the vectors

1 states that once the regime vectors are fixed, the convergence statistic and its P-value are functions only of the vectors and their sizes. Coder dependence can bias inputs, but it does not enter the conditional null calculation.

• C is the cardinality of the intersection of the fixed regime vectors
• The P-value is computed from C, M, and the observed k_i values
• Shared coder bias is handled by reliability analysis and blind replication
• The test asks a conditional question: given these vectors, how surprising is the overlap

The sixteen regimes were selected as major documentation regimes, not on overlap

The paper’s selection defense is that the sample was purposive but not selected on the dependent variable. The criteria were binding regulatory framework, documentation requirements, and public English source text.

• Financial services: Basel III, DORA, SM&CR, APRA CPS 230
• Aviation, nuclear, food safety, maritime, healthcare, and cross-sectoral frameworks are included
• Post-hoc fitting is rejected because selection and taxonomy construction came before the convergence test
• Four later-coded regimes, EU AI Act, ISSB S1/S2, NIS2, and CSRD, show the same pattern

The paper rejects copying because shared structure appears with different vocabulary

The institutional-isomorphism alternative says regulators may have copied one another. The paper argues the corpus has the opposite signature: the same field appears through different institutional language.

• OCC: "identify the responsible party"
• NRC: "completeness and accuracy attributed to named individuals"
• FAA: "accountable executive"
• DORA: "the management body shall define, approve, oversee"

Disjoint agencies weaken the diffusion explanation

The paper adds an institutional-lineage argument against copying. The relevant agencies and regimes come from different eras, continents, and professional settings, with no formal coordination mechanism for decision documentation requirements.

• OCC was established in 1863
• FDA was established in 1906
• NRC was established in 1946
• FAA was established in 1958
• APRA CPS 230, DORA, the Monaco Memorandum, SM&CR, the WHO Surgical Safety Checklist, and ISO 31000 are described as independently developed

The claim fails if blind coders cannot reproduce the regime vectors

The paper makes reproducibility the empirical test. Independent coders must use the published source texts and Appendix F protocol to reproduce the requirement vectors, then the convergence result must survive the same test.

• Two independent coders have not seen the original coding
• They receive the source texts for all 16 regimes and the M-field taxonomy
• They independently code each regime field by field
• If the replicated vectors cannot support the convergence claim, the empirical claim fails