What this validation evidence means
Coventra validation evidence is a reproducibility record for the current statistical implementation. It compares Coventra's prepared analysis inputs and displayed outputs with direct calls to the relevant R packages using the same dataset and method profile wherever that profile is available.
The goal is not to ask users to trust a proprietary calculation. The goal is to make the calculation path inspectable: raw validation input, direct R package output, Coventra analysis output, selected method profile, package version, and generated visual artifacts are kept together.
A successful validation checkpoint means the tested workflow produced the same numerical result as the direct package call for that dataset, scale, method profile, and package version, within the rounding used for display. It does not mean every possible dataset, design, extraction value, or user-selected method has been proven correct.
This is validation evidence for tested implementations, not a blanket guarantee of statistical validity for every review. Review teams remain responsible for extracted data, selected methods, assumptions, interpretation, and manuscript claims.
How the validation library is structured
The validation library stores each test case as a small reproducibility package. The raw dataset is exported in Coventra's expected input shape, the corresponding direct R package result is saved, the Coventra analysis result is saved, and generated plots are archived separately from the numerical JSON output.
For each dataset, the validation record can include package-native defaults and recommended settings. Package-native defaults mean the installed R package version's own defaults. Recommended settings mean Coventra's explicit, documented settings for that analysis family. They should be reported by model, package, version, effect measure, and interval settings rather than by a broad label.
- Input files preserve the Coventra rows and analysis settings needed to reproduce the comparison.
- Direct R results come from package calls such as meta, netmeta, metafor, bayesmeta, or mada where the analysis family has a direct package analogue.
- Coventra results come from the same statistical analysis service used by saved analysis runs in the app.
- Stored fields include pooled estimates, confidence or credible intervals, tau-squared, I-squared, p-values, and model-specific outputs such as P-scores, posterior summaries, or diagnostic accuracy summaries when applicable.
- Visual outputs are stored in a separate archive so figures can be inspected without embedding base64 images into the numerical result file.
Current coverage snapshot
The June 16, 2026 validation record contains 11 validation datasets. Public built-in datasets are used where available, and demo fixtures are used where the installed package library does not provide a compact public dataset matching Coventra's input schema for that analysis family.
The current validation set covers pairwise binary arm-level synthesis, pairwise continuous synthesis, generic inverse-variance synthesis, single-arm proportion synthesis, single-arm rate synthesis, single-arm raw mean synthesis, contrast-based network meta-analysis, arm-level binary network meta-analysis, Bayesian pairwise random-effects synthesis, survival hazard-ratio synthesis, and diagnostic test accuracy synthesis.
Validation coverage snapshot
| Analysis family | Validation data | Direct R package | Profiles compared |
|---|---|---|---|
| Pairwise continuous mean difference | meta::Fleiss1993cont | meta 8.5-0 | Package-native and recommended |
| Single-arm proportion | meta::Fleiss1993bin aspirin arm | meta 8.5-0 | Package-native and recommended |
| Pairwise binary risk ratio | Metadat/metafor BCG vaccine data | metafor 5.0-1 and meta 8.5-0 comparison context | Package-native and recommended |
| Generic inverse-variance | Metadat/metafor Berkey periodontal estimates | metafor 5.0-1 and meta 8.5-0 comparison context | Package-native and recommended |
| Contrast network meta-analysis | netmeta::Senn2013 | netmeta 3.5-0 | Package-native and recommended |
| Arm-level binary network meta-analysis | netmeta::smokingcessation | netmeta 3.5-0 | Recommended settings |
| Bayesian pairwise random-effects model | bayesmeta::Rubin1981 | bayesmeta 3.5 | Package-native and recommended |
| Diagnostic test accuracy | Compact Reitsma bivariate-model fixture | mada 0.5.12 | Direct bivariate-model comparison |
| Survival HR, single-arm rate, and raw mean | Schema-shaped fixtures | R package calls matching each analysis family | Recommended settings |
Demo fixtures are marked as demo data in the validation record. They are useful for checking integration, scale handling, and output shape, but they are not substitutes for published worked examples or external statistical review.
Package and method provenance
Each validation result records the analysis family, method profile, package name, package version, and selected or resolved settings returned by the statistical service. This matters because package defaults can change between versions, and a result is only reproducible when the package version and method profile are known.
At this validation point, the statistical environment reports R 4.3.3 and includes meta 8.5-0, netmeta 3.5-0, metafor 5.0-1, metamedian 1.2.2, mada 0.5.12, bayesmeta 3.5, rjags 4-17, gemtc 1.1-1, and brglm2 1.1.0. A separate Zenodo environment manifest can be used to cite the analysis environment, but a software-environment citation is not the same thing as peer-reviewed statistical validation.
- Package-default mode means the installed package version decides package-controlled defaults.
- Recommended settings means Coventra sends explicit settings documented in the methodology resources.
- Manual settings means the user or advanced caller supplies exposed method settings explicitly.
- Effect measure and estimand are not left to package defaults; generic and network contrast inputs require an explicit summary measure because the same TE/seTE values can represent different estimands.
Selected numeric agreement examples
The tables below show selected validation comparisons. Small differences are expected rounding differences between direct package output and the rounded values returned for display by Coventra.
These examples are included because they are easy to inspect: a pairwise continuous model, a single-arm proportion model, a Bayesian pairwise model with explicit recommended priors, and a diagnostic test accuracy bivariate model.
Pairwise continuous mean difference: meta::Fleiss1993cont, recommended settings
| Metric | Coventra | Direct R package |
|---|---|---|
| Random-effects pooled MD | -0.7509 | -0.750881 |
| 95 percent CI | -1.7224 to 0.2207 | -1.722439 to 0.220678 |
| tau-squared | 0.2742 | 0.274161 |
| I-squared | 29.33 percent | 29.3302 percent |
| p-value | 0.0984 | 0.098444 |
Single-arm proportion: meta::Fleiss1993bin aspirin arm, recommended settings
| Metric | Coventra | Direct R package |
|---|---|---|
| Random-effects pooled proportion | 0.1045 | 0.104461 |
| 95 percent CI | 0.0808 to 0.1340 | 0.080819 to 0.134011 |
| tau-squared | 0.1301 | 0.130125 |
| I-squared | 96.86 percent | 96.8579 percent |
Bayesian pairwise: bayesmeta::Rubin1981, recommended priors
| Metric | Coventra | Direct R package |
|---|---|---|
| Posterior median mu | 1.4729 | 1.472867 |
| 95 percent credible interval for mu | -2.0614 to 5.0071 | -2.061377 to 5.007102 |
| Posterior median tau | 0.3368 | 0.336815 |
| 95 percent credible interval for tau | 0.0156 to 1.1193 | 0.015649 to 1.119258 |
| Posterior P(mu <= 0) | 0.2070 | 0.207021 |
Diagnostic test accuracy: Reitsma bivariate-model fixture
| Metric | Coventra | Direct mada::reitsma |
|---|---|---|
| Pooled sensitivity | 0.8062 | 0.806201 |
| Sensitivity 95 percent CI | 0.7480 to 0.8536 | 0.748007 to 0.853587 |
| Pooled specificity | 0.8948 | 0.894804 |
| Specificity 95 percent CI | 0.8564 to 0.9239 | 0.856350 to 0.923879 |
| AUC | 0.9070 | 0.907009 |
What remains outside this validation claim
Validation evidence reduces integration risk. It does not remove the need to review the science. The same software can be numerically faithful to R and still produce an inappropriate result if the wrong input values, scale, design assumptions, or method settings are supplied.
For journal use, the review team should preserve the validation record, export the analysis data, report the package versions and settings, and obtain appropriate statistical or methodological review when the design or assumptions are complex.
- The validation library does not prove that PDF extraction values in a user's project are correct.
- It does not prove that the selected effect measure matches the protocol or clinical question.
- It does not prove transitivity, proportional hazards, missing-data assumptions, exchangeability, or absence of selective reporting.
- It does not replace published worked-example validation for every advanced design.
- It does not convert local R-service validation into a guarantee that every production project workflow has been independently reviewed.
How reviewers should use this record
- Check that the analysis family and effect measure selected in the project match the review protocol.
- Confirm the package, package version, method profile, and effective settings saved with the analysis run.
- Inspect the extraction rows and source evidence for the studies contributing to the model.
- Use the validation resources to understand how Coventra's statistical output was compared with direct R package calls.
- Export the data and reproducibility materials when a statistician, methodologist, co-author, journal reviewer, or auditor needs to inspect the result outside the app.
- Describe the method specifically in manuscript text instead of relying on broad claims such as automated meta-analysis.