DevelopingSecurity

GitHub reports secret scanner cut false positives and scan time with new classifier

GitHub said a two-stage pipeline pairing regex filters with a fine-tuned ModernBERT model raised secret scanning accuracy and cut post-push scan time by 86%.

Why it matters

The disclosure is engineering and vendor-reported, with no independent measurement available in this set of evidence, so its significance is limited. A cheaper, faster secret scanner would mainly affect GitHub's own serving costs and the alert noise developers receive.

Key facts

GitHub said it replaced an LLM-based secret detection pipeline with a two-stage system combining deterministic regex candidate generation and a fine-tuned ModernBERT model for context-based classification.

Reported by Blockchain.News

GitHub reported an 11.5% improvement in repository owner-weighted acceptance for the new secret classifier.

11.5 %Reported by Blockchain.News

GitHub reported an 86% reduction in median post-push scan execution time.

86 %Reported by Blockchain.News

GitHub reported serving cost more than 98% lower than the remote LLM approach.

98 %Reported by Blockchain.News

GitHub said offline evaluation on post-push scanning showed 13% higher precision and 19% higher recall than the prior LLM pipeline on the same public traffic.

Reported by Blockchain.News

The classifier evaluates each candidate secret alongside its file path, secret type and surrounding code or text, surfacing high-confidence findings as alerts and suppressing likely false positives.

Reported by Blockchain.News

GitHub said the pipeline targets strings that resemble credentials but are harmless placeholders, documentation examples, checksums or generated values.

Reported by Blockchain.News

GitHub validated the system in a customer A/B experiment whose outcome is not given in the available reporting.

Reported by Blockchain.News

What happened

GitHub said on Oct. 13, 2026 that it had replaced the LLM-based pipeline in its secret detection stack with a two-stage system that uses deterministic regex filters to generate candidate secrets and a compact fine-tuned ModernBERT model to classify them by context, taking the file path, secret type and surrounding code or text into account. The company said the system addresses strings that resemble credentials but are placeholders, documentation examples, checksums or generated values, and that high-confidence findings become alerts while likely false positives are filtered out before reaching developers. GitHub reported an 11.5% improvement in repository owner-weighted acceptance, an 86% reduction in median post-push scan execution time and more than 98% lower serving cost compared with the remote LLM approach. On offline evaluation of post-push scanning, the company said the system delivered 13% higher precision and 19% higher recall than the prior pipeline on the same public traffic. Blockchain.News is the only outlet covering the post, and the figures come from GitHub's own testing; the write-up does not include the customer A/B experiment results it refers to.

How the story developed

  1. First report

    First report by Blockchain.News

    GitHub says new secret classifier cuts scan time and false positives

    Blockchain.News

  2. Official statement

    GitHub publishes blog post describing new secret scanning classifier

    GitHub described the two-stage regex plus ModernBERT pipeline and its reported gains in acceptance, execution time and serving cost.

    Blockchain.News

  3. Update

    Blockchain.News reports GitHub's secret scanning results

    The outlet attributed the metrics to GitHub and did not present independent verification.

    Blockchain.News

Affected entities

Original sources

PublisherReportRolePublished
Blockchain.News
Crypto media
GitHub says new secret classifier cuts scan time and false positivesOriginal