| tidyMacro reproduces the reference implementations | ||||
| One row per compared object; rows grouped by method, data and reference code | ||||
| Compared object | tidyMacro | Type | Max abs diff | Verdict |
|---|---|---|---|---|
| Short run (Cholesky) · Bloom2009 · Cesa-Bianchi VAR Toolbox 4.0 | ||||
| Point IRF, all 8x8 shocks, h = 0..48 | fVAR + fCholeskyIRF | Deterministic | 4.2 × 10−8 |
Identical (numerical precision)
|
| FEVD, all shocks, h = 0..48 | fFEVDChol | Deterministic | 6.6 × 10−7 |
Identical (numerical precision)
|
| Residual covariance Sigma | fVAR | Deterministic | 8.9 × 10−14 |
Identical (numerical precision)
|
| IRF median | fBootstrapChol (wild, 1000 draws) | Monte Carlo | 3.3 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo68 | fBootstrapChol (wild, 1000 draws) | Monte Carlo | 3.9 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi68 | fBootstrapChol (wild, 1000 draws) | Monte Carlo | 1.1 × 10−1 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo90 | fBootstrapChol (wild, 1000 draws) | Monte Carlo | 6.1 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi90 | fBootstrapChol (wild, 1000 draws) | Monte Carlo | 1.2 × 10−1 |
Same distribution (gap = Monte Carlo noise)
|
| Long run (Blanchard-Quah) · Gali1999 · Cesa-Bianchi VAR Toolbox 4.0 | ||||
| Point IRF (growth rates), both shocks, h = 0..40 | fBQIRF | Deterministic | 4.3 × 10−7 |
Identical (numerical precision)
|
| IRF median | fBootstrapBQ (residual, 10000 draws, cumulate = 1:2) | Monte Carlo | 2.7 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo68 | fBootstrapBQ (residual, 10000 draws, cumulate = 1:2) | Monte Carlo | 5.3 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi68 | fBootstrapBQ (residual, 10000 draws, cumulate = 1:2) | Monte Carlo | 4.1 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo90 | fBootstrapBQ (residual, 10000 draws, cumulate = 1:2) | Monte Carlo | 7.3 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi90 | fBootstrapBQ (residual, 10000 draws, cumulate = 1:2) | Monte Carlo | 4.7 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| External instrument · Kaenzig2021 · Kaenzig 2021 | ||||
| Point IRF (unit oil-price shock), h = 0..48 | fBootstrapIVMBB | Deterministic | 2.7 × 10−9 |
Identical (numerical precision)
|
| IRF lo68 | fBootstrapIVMBB (MBB, block 24, 2000 draws) | Monte Carlo | 1.8 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi68 | fBootstrapIVMBB (MBB, block 24, 2000 draws) | Monte Carlo | 1.1 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo90 | fBootstrapIVMBB (MBB, block 24, 2000 draws) | Monte Carlo | 4.0 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi90 | fBootstrapIVMBB (MBB, block 24, 2000 draws) | Monte Carlo | 2.9 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| MSW plug-in IRF | fMSW | Deterministic | 2.0 × 10−9 |
Identical (numerical precision)
|
| MSW delta-method SE | fMSW | Deterministic | 2.4 × 10−9 |
Identical (numerical precision)
|
| Delta-method 90% lower | fMSW | Deterministic | 4.0 × 10−9 |
Identical (numerical precision)
|
| Delta-method 90% upper | fMSW | Deterministic | 2.8 × 10−9 |
Identical (numerical precision)
|
| Anderson-Rubin 90% lower | fMSW | Deterministic | 3.3 × 10−9 |
Identical (numerical precision)
|
| Anderson-Rubin 90% upper | fMSW | Deterministic | 5.8 × 10−9 |
Identical (numerical precision)
|
| MSW Wald (HAC-robust first-stage F) | fMSW | Deterministic | 4.7 × 10−8 |
Identical (numerical precision)
|
| First stage (zero-filled proxy): F, robust F, R2, adj. R2 | fOLS | Deterministic | 7.7 × 10−8 |
Identical (numerical precision)
|
| First stage (proxy sample): F, robust F, R2, adj. R2 | fOLS | Deterministic | 6.1 × 10−8 |
Identical (numerical precision)
|
| Impact vector s (unit normalisation) | fOLS (2SLS) | Deterministic | 4.0 × 10−10 |
Identical (numerical precision)
|
| Unit-shock IRF from s and Wold, h = 0..48 | fWoldIRF | Deterministic | 2.0 × 10−9 |
Identical (numerical precision)
|
| FEVD of the IV shock, h = 0..48 | fFEVDIV | Deterministic | 5.5 × 10−9 |
Identical (numerical precision)
|
| Sign restrictions · Uhlig2005 · Cesa-Bianchi VAR Toolbox 4.0 | ||||
| IRF median | fSignRestr (1000 accepted draws, inference = 1) | Monte Carlo | 1.4 × 10−1 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo68 | fSignRestr (1000 accepted draws, inference = 1) | Monte Carlo | 1.2 × 10−1 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi68 | fSignRestr (1000 accepted draws, inference = 1) | Monte Carlo | 1.7 × 10−1 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo90 | fSignRestr (1000 accepted draws, inference = 1) | Monte Carlo | 3.0 × 10−1 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi90 | fSignRestr (1000 accepted draws, inference = 1) | Monte Carlo | 2.3 × 10−1 |
Same distribution (gap = Monte Carlo noise)
|
| Sign + narrative · ADRR2018 · Cesa-Bianchi VAR Toolbox 4.0 | ||||
| IRF median | fSignRestr (narrative = Volcker 1979m10 sign + dominance) | Monte Carlo | 5.4 × 10−3 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo68 | fSignRestr (narrative = Volcker 1979m10 sign + dominance) | Monte Carlo | 7.8 × 10−3 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi68 | fSignRestr (narrative = Volcker 1979m10 sign + dominance) | Monte Carlo | 1.7 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF lo90 | fSignRestr (narrative = Volcker 1979m10 sign + dominance) | Monte Carlo | 2.5 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| IRF hi90 | fSignRestr (narrative = Volcker 1979m10 sign + dominance) | Monte Carlo | 2.5 × 10−2 |
Same distribution (gap = Monte Carlo noise)
|
| LP (exogenous shock) · JordaTaylor2025 · Cesa-Bianchi VAR Toolbox 4.0 | ||||
| IRF | fLP | Deterministic | 1.0 × 10−12 |
Identical (numerical precision)
|
| Newey-West SE | fLP | Deterministic | 7.8 × 10−14 |
Identical (numerical precision)
|
| 95% lower | fLP | Deterministic | 8.5 × 10−13 |
Identical (numerical precision)
|
| 95% upper | fLP | Deterministic | 1.2 × 10−12 |
Identical (numerical precision)
|
| LP-IV · JordaTaylor2025IV · Cesa-Bianchi VAR Toolbox 4.0 | ||||
| IRF | fLPIV | Deterministic | 1.3 × 10−14 |
Identical (numerical precision)
|
| HAC SE | fLPIV | Deterministic | 2.9 × 10−15 |
Identical (numerical precision)
|
| 95% lower | fLPIV | Deterministic | 1.8 × 10−14 |
Identical (numerical precision)
|
| 95% upper | fLPIV | Deterministic | 1.2 × 10−14 |
Identical (numerical precision)
|
| First-stage F (raw) | fLPIV | Deterministic | 9.6 × 10−1 |
Differs by df convention only
|
| First-stage F (rescaled to the Toolbox df) | fLPIV | Deterministic | 8.7 × 10−14 |
Identical (numerical precision)
|
| Panel LP · MianSufiVerner2017 · Mian, Sufi and Verner (2017) | ||||
| Estimates (household + firm debt), h = 0..6 | fLPPanel | Deterministic | 4.9 × 10−16 |
Identical (numerical precision)
|
| Time-clustered SE | fLPPanel | Deterministic | 1.0 × 10−16 |
Identical (numerical precision)
|
| 95% CI bounds | fLPPanel | Deterministic | 6.1 × 10−16 |
Identical (numerical precision)
|
| Imbens-Kolesar SE (ssc = 'IK') | fLPPanel | Deterministic | 9.7 × 10−17 |
Identical (numerical precision)
|
| Imbens-Kolesar degrees of freedom | fLPPanel | Deterministic | 5.7 × 10−14 |
Identical (numerical precision)
|
| Imbens-Kolesar 95% CI bounds | fLPPanel | Deterministic | 6.7 × 10−16 |
Identical (numerical precision)
|
| Two-way clustered estimates + SE | fLPPanel(cluster = ~country + year) | Deterministic | 0.0 |
Identical (numerical precision)
|
| Two-way clustered estimates + SE vs fixest (CGM, no ssc) | fLPPanel(cluster = ~country + year) | Deterministic | 7.8 × 10−16 |
Identical (numerical precision)
|
| LP-DiD · DGJT main-text simulation (200 datasets) / BankingDeregulation · Dube-Girardi-Jorda-Taylor 2025 | ||||
| Point estimates, Variance-weighted, 200 simulated panels x event time -5..5 | fLPDID | Deterministic | 9.7 × 10−8 |
Identical (numerical precision)
|
| Point estimates, Reweighted (equal-weight), 200 simulated panels x event time -5..5 | fLPDID | Deterministic | 1.1 × 10−7 |
Identical (numerical precision)
|
| Point estimates, PMD baseline, 200 simulated panels x event time -5..5 | fLPDID | Deterministic | 9.1 × 10−8 |
Identical (numerical precision)
|
| Point estimates, Reweighted + PMD, 200 simulated panels x event time -5..5 | fLPDID | Deterministic | 1.1 × 10−7 |
Identical (numerical precision)
|
| Banking deregulation (specs A, C, D): estimates | fLPDID | Deterministic | 1.5 × 10−16 |
Identical (numerical precision)
|
| Banking deregulation (specs A, C, D): clustered SE | fLPDID | Deterministic | 1.0 × 10−16 |
Identical (numerical precision)
|
| Banking deregulation (specs A, C, D): 95% lower | fLPDID | Deterministic | 2.8 × 10−16 |
Identical (numerical precision)
|
| Banking deregulation (specs A, C, D): 95% upper | fLPDID | Deterministic | 1.4 × 10−16 |
Identical (numerical precision)
|
| Banking deregulation (specs A, C, D): observations | fLPDID | Deterministic | 0.0 |
Identical (numerical precision)
|
| Monte Carlo rows report the largest absolute gap between tidyMacro and reference bands at 1,000 draws (2,000 for the moving-block bootstrap, 10,000 for the long-run bootstrap). Going from 1,000 to 10,000 long-run draws shrank the tidyMacro-Toolbox gap 3.3 to 5.7 times, at least the √10 ≈ 3.2 that pure Monte Carlo error predicts. The LP-IV first-stage F differs only in its degrees of freedom: tidyMacro uses n − kcontrols − kz, the Toolbox n − kz. | ||||
Validation
tidyMacro against the reference MATLAB and Stata codes
Each method in tidyMacro was run on the package datasets and compared with the different set of code suits in MATLAB and STATA.
Two kinds of objects are compared:
- Deterministic objects (point estimates, standard errors, analytic confidence bands, FEVDs) are compared cell by cell. Max abs diff is the largest absolute difference over all variables, shocks and horizons.
- Monte Carlo objects (bootstrap bands, posterior medians and credible bands) cannot agree draw by draw, because the two codes use different random number generators. Their gap is compared with the gap between two runs of the same code under different seeds. Same distribution means tidyMacro and the reference differ no more than each differs from itself.