A vulnerability scan is an automated check that compares your systems against a database of known weaknesses and produces a list of findings. A penetration test is a time-boxed engagement in which a qualified human tester tries to exploit those weaknesses, chain them together and prove what an attacker could actually reach. Scans are broad, frequent and cheap per run; pen tests are deep, periodic and manual. Most organizations need both, and frameworks such as PCI DSS and NYDFS Part 500 require both.
What a vulnerability scan is
A scanner probes hosts, applications or cloud accounts, identifies software versions, open services and configurations, and matches what it finds against known vulnerabilities (CVEs) and misconfiguration checks. NIST SP 800-115, the federal technical guide to security testing, describes vulnerability scanning as a way to identify hosts and their attributes and the vulnerabilities associated with them.
Scans come in two main forms:
- Unauthenticated (external) scans see what an outsider sees: exposed ports, banners, TLS settings and internet-facing software.
- Authenticated (credentialed) scans log in to systems and inspect installed software, patch levels and local settings. They are far more accurate and are now required for internal scans under PCI DSS v4.0.1 (requirement 11.3.1.2).
What scans find well
- Missing patches and outdated software with published CVEs.
- Default or weak configurations, expired certificates and weak protocols.
- New hosts or services that appeared since the last scan.
What scans miss
- Business logic flaws, such as a user changing an ID in a request to see another customer's records.
- Chained attacks, where three "medium" findings combine into full domain compromise.
- Weak or reused credentials, excessive permissions and Active Directory attack paths.
- Context: a scanner cannot tell you whether a finding actually matters in your environment, and it produces false positives that someone has to triage.
What a penetration test is
NIST SP 800-115 describes penetration testing as security testing in which assessors mimic real-world attacks to find ways to circumvent the security features of an application, system or network. A tester usually starts with scanning and reconnaissance, then works manually: validating findings, exploiting them, escalating privileges and moving laterally until they reach an agreed objective or run out of time. Our guide to penetration testing methodology walks through the phases.
Common types include external network, internal network and Active Directory, web application and API, cloud configuration, and social engineering. A pen test is not a red team engagement: a pen test aims to find as many exploitable issues as possible in a defined scope, while a red team pursues one goal and tests whether your team detects it.
What pen tests find well
- Exploitable paths from the internet, or from a phished laptop, to sensitive data.
- Authorization and logic flaws in applications and APIs.
- Whether network segmentation actually isolates what you think it isolates.
- Which scanner findings are real, and which ones matter most.
What pen tests miss
- Anything that changes after the test window. A pen test is a point-in-time snapshot.
- Systems outside the agreed scope.
- Full coverage. Testers go deep on the most promising paths rather than listing every outdated library on every host.
Side-by-side comparison
| Vulnerability scan | Penetration test | |
|---|---|---|
| Goal | Find known weaknesses across many assets | Prove what an attacker could actually achieve |
| Method | Automated tool, signature and configuration checks | Manual testing by a qualified person, supported by tools |
| Breadth vs depth | Broad and shallow | Narrow and deep |
| Typical frequency | Continuous, weekly, monthly or at least quarterly, plus after changes | At least annually, plus after significant changes |
| Who performs it | Internal IT/security staff, a managed security provider, or (for PCI external scans) an Approved Scanning Vendor | A qualified internal or external tester with organizational independence |
| Exploitation | No; identifies and flags | Yes, within agreed rules of engagement |
| False positives | Common; need triage | Rare; findings are validated with evidence |
| Deliverable | Machine-generated list of findings by host with CVE and CVSS scores; for PCI, an ASV scan report | Written report: executive summary, scope and method, validated findings with evidence, attack narrative, remediation guidance, often an attestation letter |
| Typical cost | Lower per run; usually a subscription | Higher per engagement; priced by scope (see penetration testing cost) |
What compliance frameworks actually require
Auditors, regulators and customers rarely accept one as a substitute for the other. Here is what the main frameworks say as of October 2026.
PCI DSS v4.0.1
PCI DSS v4.0.1 is the active version of the standard, available from the PCI Security Standards Council document library. Requirement 11.3 covers scanning and 11.4 covers penetration testing:
- 11.3.1: internal vulnerability scans at least once every three months, with high-risk and critical findings resolved and rescanned, performed by qualified personnel with organizational independence. 11.3.1.2 requires these scans to be authenticated.
- 11.3.2: external vulnerability scans at least once every three months by a PCI SSC Approved Scanning Vendor (ASV), with rescans until you achieve a passing scan.
- 11.3.1.3 and 11.3.2.1: internal and external scans after any significant change.
- 11.4.2 and 11.4.3: internal and external penetration testing at least once every 12 months and after any significant infrastructure or application upgrade or change, by a qualified internal resource or qualified external third party.
- 11.4.5: if you use segmentation to reduce scope, test those controls at least once every 12 months (every six months for service providers under 11.4.6).
NYDFS 23 NYCRR Part 500
Section 500.5 of the NYDFS cybersecurity regulation requires covered entities to have written vulnerability management policies and procedures that include:
- Penetration testing at least annually, from both inside and outside the information systems' boundaries, by a qualified internal or external party.
- Automated scans of information systems, plus manual review of systems not covered by those scans, at a frequency set by the risk assessment and promptly after any material system changes.
- A monitoring process for new vulnerabilities and timely remediation, prioritized by risk.
Our NYDFS Part 500 compliance checklist covers the rest of the regulation.
SOC 2
SOC 2 does not set a fixed cadence for either activity. The Trust Services Criteria point of focus under CC7.1 describes conducting vulnerability scans periodically and after significant changes to the environment, and taking action on what they find. A penetration test is not mandated by the criteria, but auditors often look for one as evidence that controls work, and enterprise customers frequently ask for a recent pen test summary in security questionnaires.
HIPAA
The current HIPAA Security Rule does not specify how often to scan or pen test. It requires an accurate risk analysis and periodic technical and nontechnical evaluation of your safeguards, and scanning and pen testing are common ways to support both. In January 2025, HHS published a proposed update to the Security Rule that would require automated vulnerability scans at least every six months and penetration testing at least every 12 months. As of October 2026 that rule has not been finalized, but it shows where regulators expect healthcare organizations to be.
Cyber insurers and customers
Cyber insurance applications and enterprise security questionnaires commonly ask whether you scan for vulnerabilities, how quickly you patch critical findings and when you last had an independent penetration test. Answer accurately: an attestation you cannot support with reports can create problems when you make a claim or face an audit.
How scans and pen tests fit together
Think of scanning as the smoke detector and the pen test as the fire drill. In a working vulnerability management program, each feeds the other:
- Know your assets. You cannot scan or test what you have not inventoried.
- Scan continuously or on a fixed schedule, authenticated wherever possible, including after major changes.
- Prioritize by risk, not CVSS alone. Use exploitation evidence and asset criticality, as described in our guide to risk-based vulnerability management.
- Remediate and rescan to confirm fixes, and track time to remediate.
- Pen test at least annually and after significant changes, to find what scanning cannot: chained attacks, logic flaws and broken segmentation.
- Feed pen test results back into the program. If testers exploited something the scanner missed, fix the coverage gap, not just the finding.
A pen test against an environment that is never scanned tends to produce a long list of missing patches, which wastes expensive testing time on issues a scanner would have caught. Scanning without ever testing leaves you with a clean dashboard and no evidence that an attacker could not get in anyway. Many organizations run scanning through a managed security provider and bring in independent testers once or twice a year.
Which one do you need right now?
- You have no regular scanning: start there. It is the foundation, and it will make your first pen test more valuable.
- You scan regularly but have never been tested: schedule a pen test, starting with your external perimeter and most important application.
- A customer, auditor or regulator is asking: check exactly what they require. PCI DSS and NYDFS require both, on defined schedules.
- You want to know whether your team would detect an attack: that is a red team question, not a scan or pen test question.
If you are not sure which applies, talk to an expert and we will help you map requirements to a testing plan.
Frequently Asked Questions
Is a vulnerability scan the same as a penetration test?
No. A vulnerability scan is an automated check that lists known weaknesses. A penetration test is a time-boxed engagement where a qualified tester tries to exploit weaknesses, chain them together and show real business impact.
How often should we run vulnerability scans?
At least quarterly, and more often for internet-facing systems, plus after significant changes. PCI DSS requires internal and external scans at least once every three months. Many organizations scan continuously or weekly.
How often do we need a penetration test?
At least once a year and after significant changes for most organizations. PCI DSS and NYDFS Part 500 both require annual penetration testing, and many customers and auditors expect it.
Can a vulnerability scan replace a penetration test for compliance?
Usually not. PCI DSS and NYDFS Part 500 require both, as separate activities. An automated scan relabelled as a pen test will not satisfy an assessor who expects manual testing by a qualified person.
Does HIPAA require penetration testing?
The current HIPAA Security Rule does not name a scan or pen test frequency; it requires risk analysis and periodic evaluation. HHS proposed annual pen testing and six-monthly scans in January 2025, but that rule was not final as of October 2026.
Should the same vendor do our scanning and our pen test?
It can work, but independence matters. PCI DSS requires organizational independence of the tester, and fresh eyes find different things. Many organizations run scanning through their managed security provider and rotate pen test perspectives.
Ready to test what your scanner can't see? Send us your scope and Defensible will return a quote with methodology and deliverables. Request a Pen Test Quote.
