Methodology

Why Patch Response Time Matters

The cybersecurity industry over-indexes on CVE volume — how many vulnerabilities affect a vendor’s products. But CVE count is a meaningless metric. What matters is how fast vendors patch disclosed vulnerabilities.

Patch response time reflects operational maturity, engineering discipline, and vendor commitment to security. A vendor with fewer CVEs but slower patches is less reliable than a vendor with more CVEs but faster response times.

PatchDelta measures the delta between CVE publication and patch availability for each vendor, providing a quantitative comparison of vendor responsibility.

The Metric: Patch Response Time

Definition

Patch response time is the number of days between:

Example

Why NVD Publication Date?

We use NVD publication date as the baseline for consistency across all vendors. The NVD is maintained by NIST and provides a standardized, vendor-agnostic disclosure date.

In future versions, we will enrich this with the full disclosure timeline (researcher disclosure → CISA advisory → NVD publication → vendor advisory → patch available) to show the complete picture.

Data Sources

Primary: National Vulnerability Database (NVD)

Secondary: Vendor Advisories

For each tracked vendor, we parse their official advisory/PSIRT pages:

We match CVE IDs from the advisory pages to compute the patch available date.

Enrichment: KEV and EPSS

How We Calculate Metrics

Median Patch Response Time

The median is the middle value when all patch response times are sorted. We use median (not mean/average) because:

Rolling 12-Month Window

By default, all metrics reflect the most recent rolling 12 months of CVEs. This:

Severity Segmentation

We calculate separate medians for each CVSS severity level:

Default view: Critical + High combined. Users can filter to any severity level or “All.”

Per-Category & Per-Vendor Metrics

We compute metrics separately for:

Confidence Scores

Every vendor’s data carries a confidence score (0–1, displayed as a percentage) based on:

Factors That Increase Confidence

✓ Vendor publishes structured, machine-readable advisories
✓ Patch availability dates are explicitly timestamped
✓ Advisory data is complete (rarely missing CVEs)
✓ Vendor has high CVE coverage in our system

Factors That Decrease Confidence

✗ Vendor advisories are unstructured or require manual parsing
✗ Patch dates are ambiguous or inferred from product release notes
✗ Advisory data has gaps or inconsistencies
✗ Low CVE coverage in our system

Why This Matters

A vendor with a 60% confidence score and a 14-day median is less reliable than a vendor with an 85% confidence score and a 16-day median. Confidence scores are visible on every vendor card and scorecard — not hidden in fine print. Low transparency is itself a signal.

Currently Unpatched CVEs

CVEs with no patch available are:

This section provides accountability for open vulnerabilities and reflects the vendor’s current security posture, not just historical performance.

Limitations & Edge Cases

Known Limitations

  1. Parser brittleness: Each vendor’s advisory page has a unique structure. If a vendor redesigns their PSIRT site, our parser may break temporarily. We log parsing failures and escalate manually.

  2. Ambiguous patch dates: Some vendors don’t explicitly publish a patch date. We infer from product release notes, creating uncertainty. Low confidence scores reflect this.

  3. Product-category mapping is manual: We manually map products to categories (e.g., Fortinet FortiOS → NGFW). This can lag if a vendor releases a new product. Our admin panel allows quick updates.

  4. CPE matching is imperfect: NVD CPE strings sometimes don’t exactly match vendor product names. We do our best, but some CVEs may be missed or incorrectly attributed.

  5. Disclosure date ambiguity: We use NVD publication date, not researcher disclosure date. For embargoed disclosures, this may understate vendor response time. We document this limitation prominently.

Time Windows & Recency

Transparency & Reproducibility

Open Methodology

This methodology document is the source of truth. All calculations are deterministic and logged.

Future: Open-Source Calculation Logic

We plan to publish the full calculation logic on GitHub, allowing:

Feedback & Corrections

Found an error in our data? Vendor advisory parser broken? Please report it. We take accuracy seriously.

PatchDelta is an independent informational resource. Data is provided as-is and should not be the sole basis for procurement, legal, or security decisions. We make no warranty regarding accuracy, completeness, or fitness for a particular purpose.

Patch response times are computed from publicly available vulnerability disclosure and vendor advisory data. Our methodology has known limitations (documented above). Use this data to inform your vendor evaluation, not to replace it.

Questions?

FAQ

Q: Why exclude unpatched CVEs from the median? A: The median should reflect how fast vendors patch disclosed vulnerabilities. Unpatched CVEs are still open, so there’s no delta to measure. They’re shown separately to hold vendors accountable for delays.

Q: Does a low confidence score mean the data is wrong? A: No, it means we’re less confident in its completeness or precision. Low confidence typically means the vendor’s advisory format is harder to parse or data is sparse. The underlying data is likely correct, but may not be a complete picture of the vendor’s true response time.

Q: Can I compare vendors across product categories? A: Yes, the main leaderboard shows overall metrics across all products. You can also filter by category to see category-specific performance. Vendors may perform differently across categories (e.g., NGFW vs SASE).

Q: How often is the data refreshed? A: Weekly by default. Check the footer for the exact “Last updated” timestamp. We plan to increase refresh frequency to daily in Phase 2.

Q: Why don’t you include [vendor X]? A: We start with the most widely-used vendors (Fortinet, Palo Alto, Cisco, Check Point). We accept nominations for additional vendors. Adding a vendor requires writing a custom advisory parser and ensuring data quality.