Attackers do not think in terms of dashboards, scorecards, or coloured severity badges. They think in terms of pages, parameters, headers, sessions, forgotten paths, weak configurations, and input handling mistakes. A vulnerability scanner approaches a website in a very similar way, except it does it systematically, at scale, and with rules designed to detect risk safely.

That matters because many teams run scans without really knowing what happens between entering a URL and receiving a report. When you understand the mechanism, the results start to make more sense. You can see why one issue appears as high severity, why another finding needs manual confirmation, why some pages are missed, and why scan depth changes both coverage and runtime. A scanner is not a magic box. It is a process, usually built around three core phases: crawl, probe, and detect.

Once you see those phases clearly, vulnerability scan results stop looking random. They become evidence tied to what the scanner actually reached, what it actually tested, and what it actually observed in the response.

The crawl phase
The probe phase
The detection phase
Authenticated vs unauthenticated scanning
Scan depth and coverage trade-offs
What a vulnerability scanner cannot do
The 3 phases at a glance
FAQ
Conclusion
Related articles

The crawl phase: how a scanner maps your attack surface

The crawl phase is where the scanner tries to understand what exists. Before it can test for vulnerabilities, it has to find pages, forms, parameters, scripts, endpoints, and response patterns worth testing. If the crawl is shallow, the rest of the scan will also be shallow. If the crawl is broad and accurate, the scanner has a much better chance of finding meaningful issues.

At a basic level, a crawler starts with one or more seed URLs and follows what the application exposes. That often includes links in HTML, navigation menus, form actions, query string parameters, redirects, canonical URLs, script references, and sometimes sitemap files. Every discovered page becomes another place to extract more URLs, more inputs, and more clues about the application structure.

Spidering links, forms, endpoints, and parameters

A good scanner is not just counting pages. It is building a map of interaction points. That includes:

Links to reachable pages and sub-paths
Forms and the fields they accept
Query parameters such as ?id=123 or ?search=term
POST endpoints used by forms or APIs
Cookies and headers that affect application state
JavaScript-discovered routes in modern front-end applications

For example, a product page might look simple to a user, but a scanner may record several separate test surfaces from it: the page URL, an item ID parameter, a search box, a newsletter form, a review submission form, and a hidden API call used to fetch related products. Each of those is a different entry point for security testing.

This is one reason attackers and scanners both care so much about mapping. The visible page is only part of the story. The real attack surface is the collection of inputs the application trusts.

How hidden paths are found

Not every interesting path is linked from the homepage. Scanners often try to uncover additional attack surface through several passive and active discovery techniques. Some check obvious resources like robots.txt, sitemap.xml, or framework-specific files that reveal useful paths. Others test common application locations such as admin portals, login routes, staging patterns, backup filenames, API prefixes, or documentation endpoints.

Modern scanners also parse JavaScript to extract hard-coded routes, API endpoints, asset URLs, and parameter names. That matters on JavaScript-heavy sites where the browser builds the interface dynamically and a plain HTML crawl would miss large parts of the application. A single script bundle can expose route names, GraphQL paths, API versions, or internal references that expand what the scanner tests next.

None of this guarantees complete coverage. Some paths require specific user behaviour, complex state changes, or multi-step flows before they appear. But the crawl phase is designed to reduce blind spots as much as possible before payloads are ever sent.

What crawlers can and cannot see

A scanner can only test what it can reach. That sounds obvious, but it explains many misunderstandings about automated scanning. Public pages, exposed forms, accessible endpoints, static assets, and reachable scripts are generally fair game for unauthenticated scanning. Private dashboards, role-restricted settings, internal workflows, and post-login features are not, unless the scanner has a valid way to access them.

Single-page applications can also make crawling harder. If routes depend on client-side rendering, browser storage, API-driven state, or interaction events that are hard to reproduce automatically, some content may stay partially hidden. Heavy rate limiting, anti-bot protections, broken HTML, or unusual navigation patterns can also reduce what the crawler sees.

This is where realistic expectations matter. If the scan only touched 50 URLs, the report reflects what was discovered across those 50 URLs, not the entire application universe. The quality of the crawl sets the boundary for the rest of the assessment.

The probe phase: how payloads are sent and responses analysed

Once the scanner has a map of inputs and endpoints, it starts testing them. This is the probe phase. Here, the scanner sends controlled inputs into parameters, form fields, cookies, headers, and sometimes request bodies to see how the application behaves. The aim is not chaos. The aim is to trigger meaningful, measurable differences in a safe and structured way.

Different vulnerability classes require different probes. A scanner might test whether an input is reflected unsafely in the response, whether a server error appears when special characters are used, whether access control behaves differently for missing or modified identifiers, or whether a response time changes in a way that suggests a blind injection issue. The important point is that scanners do not test everything in the same way. They choose payloads based on context and category.

Where payloads are sent

Payloads are not limited to visible text boxes. A scanner may inject safe test values into:

URL query parameters
Path segments
Form fields
JSON request bodies
HTTP headers
Cookies
Hidden fields and state parameters

That matters because vulnerabilities often appear in places developers do not think of as “user input”. A custom header passed through a reverse proxy, a hidden form value trusted by server logic, or a cookie reflected in a debug page can all become useful test surfaces.

What “safe” probing means

Well-designed scanners do not need to damage a site to identify common weaknesses. Safe probing means using payloads intended to reveal how the application processes input without creating destructive side effects. For example, a scanner testing for reflected XSS may use a marker string that should never appear unescaped in the response. A scanner testing for SQL injection may use boolean or timing-based probes that reveal risky input handling without changing data.

Safe does not mean invisible. Probing still generates requests, consumes server resources, and may trigger logging or rate limiting. But the intent is defensive validation, not operational disruption. This is also why responsible scanners tend to avoid payloads that create state changes, send bulk destructive commands, or intentionally break application data.

When Vulnify is used for website security testing, the same principle matters: the value comes from practical validation and exposure discovery, not reckless interaction. Focused tools such as the Headers Analyzer or SSL Checker are good examples of narrow, low-friction checks where the goal is clear evidence, not noise.

How response differences reveal vulnerabilities

A scanner does not merely send a strange payload and declare success. It compares what comes back. Detection signals often include:

Changes in HTTP status codes
Differences in response body content
Reflected input appearing in unsafe contexts
Unexpected redirects
Error messages or stack traces
Response length changes
Timing differences between similar requests
Header changes that indicate altered server behaviour

For example, if a normal request returns a standard product page, but a modified request causes a SQL error, a long delay, or a content shift that only makes sense under certain database logic, the scanner has a stronger basis for a finding. The important word here is difference. Vulnerability testing is often about measuring how the application reacts to controlled variation.

Code example: simplified XSS probe vs normal request

GET /search?q=blue+shoes HTTP/1.1 Host: example.com User-Agent: Scanner/1.0 Accept: text/html Connection: close

GET /search?q=%3Cscript%3Escan_probe_123%3C%2Fscript%3E HTTP/1.1 Host: example.com User-Agent: Scanner/1.0 Accept: text/html X-Scan-Mode: non-destructive Connection: close

The first request is a normal search. The second sends a controlled test string that should be encoded or neutralised before it reaches the browser. If the application reflects that value directly into HTML without escaping it, the scanner has a reason to investigate further. In practice, mature scanners do more than look for raw reflection. They also examine where the payload appears, such as HTML body context, attribute context, script context, or encoded output, because context decides exploitability.

The detection phase: how findings are identified and scored

After crawling and probing comes the detection phase. This is where a scanner decides whether the evidence is strong enough to produce a finding. Detection logic usually combines signatures, pattern matching, context checks, behavioural analysis, and validation rules to separate ordinary application behaviour from likely security issues.

This is the phase users see most clearly in the final report, but it is really the product of everything before it. If the crawler missed the page or the probe was not appropriate for the input, the detector has nothing useful to work with.

Pattern matching vs behavioural analysis

Some findings are heavily signature-driven. A missing security header, an outdated server banner, or a known file exposure can often be detected by matching a fairly direct pattern in the response. If the response header lacks Strict-Transport-Security, that is a straightforward observation. If the server exposes directory indexing or a known admin path, that can also be close to signature-based detection.

Other findings require behavioural analysis. Blind injection issues, access control inconsistencies, response timing anomalies, or context-sensitive input handling are not always visible from a single string match. The scanner may need multiple requests, control comparisons, or confirmation steps to decide whether a behaviour is suspicious enough to report. That is one reason better scanners usually generate fewer junk findings than simplistic ones. They do not stop at the first unusual response.

False positives: why they happen and how good scanners reduce them

False positives happen when a signal looks like a vulnerability but is not one in practice. Reflection is a good example. Seeing user input appear in the response does not automatically mean exploitable XSS. If the content is safely encoded, placed into a harmless context, or neutralised before execution, the security impact may be minimal or nonexistent. Likewise, a generic server error after a malformed request does not automatically prove SQL injection.

Good scanners reduce false positives by asking follow-up questions through logic, not intuition. Was the payload actually reflected in executable context? Did the timing difference repeat consistently? Did the response change only when a specific condition was introduced? Was the output encoded? Was the endpoint reachable without unusual assumptions? The more validation steps the engine performs, the more confidence the user can have in the finding.

This is also why experienced teams still review evidence manually. Automation narrows the search space. It does not replace judgement. A mature scanner saves time by surfacing the most credible signals first.

Severity scoring: CVSS basics and practical prioritisation

Once a finding is identified, it needs to be prioritised. Most scanners classify issues using severity levels such as critical, high, medium, and low. Under the hood, this often maps to some version of CVSS thinking: how easy the issue is to exploit, whether privileges are required, how much user interaction is needed, and what the potential impact is on confidentiality, integrity, and availability.

CVSS is useful because it gives teams a common language, but it is not the whole story. A medium severity issue on a highly exposed login endpoint may deserve faster action than a technically higher-rated issue on a rarely used feature. Context matters. Public reachability, asset importance, compensating controls, and business impact can all change what should be fixed first.

That is why the best reports are not just lists. They connect findings to affected URLs, technical evidence, likely impact, and remediation priorities. Vulnify’s documentation around scan depth and reporting is useful here because it helps teams connect result quality to scan scope, rather than treating every output as if it came from the same level of coverage.

Code example: what a reflected XSS confirmation looks like in a raw HTTP response

HTTP/1.1 200 OK Content-Type: text/html; charset=UTF-8 Cache-Control: no-store <html> <body> <h1>Search Results</h1> <p>You searched for: <script>scan_probe_123</script></p> </body> </html>

On its own, this response is a strong warning sign because the probe appears unescaped inside the HTML body. In real scanning, the engine would still consider context and may run follow-up checks to reduce false positives. But this kind of raw evidence is the reason XSS findings can be explained clearly in a report. The scanner is not guessing. It is showing what came back.

Authenticated vs unauthenticated scanning: why it changes everything

Unauthenticated scanning shows what the public internet can reach without logging in. That is valuable because attackers often start there. Public pages, login forms, exposed APIs, forgotten paths, JavaScript assets, and configuration mistakes are all part of the real attack surface. But unauthenticated scanning has a hard limit: it cannot see what sits behind access control.

That missing area is often where sensitive business logic lives. Account settings, admin workflows, support portals, order management views, billing pages, role-specific dashboards, internal search tools, and export functions may all sit behind authentication. If the scanner never enters those areas, it cannot test their forms, parameters, permissions, or session behaviours.

Authenticated scanning changes the quality of assessment because it lets the engine operate as a real user session. That introduces new complexity. The scanner must preserve cookies, follow redirects, keep tokens fresh, respect session expiry, handle CSRF protections, and avoid accidentally logging itself out. It may also need multiple roles if you want to compare what ordinary users can access versus what administrators can access.

Even when a platform does not yet expose every possible authenticated workflow publicly, the principle is still essential for understanding scan results. If you want a useful reference for how scan coverage changes with depth and scope, see Vulnify’s scan depths documentation. It helps explain a broader truth about scanning: what the engine can reach determines what it can meaningfully test.

Scan depth and coverage trade-offs

Not every scan should run with maximum depth. Faster scans are useful when you want a quick signal after a deployment, a narrow validation of obvious issues, or a lightweight review of a smaller site. Deeper scans are useful when you need broader path discovery, more payload variation, more confirmation logic, and stronger confidence that exposed attack surface has been meaningfully exercised.

Mechanically, scan depth changes several things at once. A deeper scan may follow more links, explore more nested paths, parse more scripts, test more input combinations, try more payload classes, repeat more checks for confirmation, and apply additional heuristics before producing a final finding. All of that improves coverage, but it also increases time, server load, and result volume.

This is why a quick scan and a deep scan can produce different results against the same target. The difference is not randomness. The deeper scan simply explored more surface and spent more time validating suspicious behaviour. That is also why teams should avoid comparing outputs from different scan depths as if they were identical processes.

Vulnify’s scan depth guidance is useful in this context because it reinforces the practical side of the decision. If you only need a fast view of obvious exposure, a lighter check may be enough. If you want broader assurance, the engine needs more room to crawl, probe, and confirm. And if you do not need a full vulnerability scan at all, focused free checks like the Headers Analyzer or SSL Checker can answer narrower questions much faster.

The key trade-off is simple: more coverage usually means more time, more requests, and more evidence to review. There is no honest way around that. A scanner cannot test what it never reaches, and it cannot validate what it never re-checks.

What a vulnerability scanner cannot do

A scanner is powerful, but it is not omniscient. There are categories of weakness that automated testing will miss, partially understand, or flag only indirectly.

Logic flaws and business logic bugs

Business logic flaws are a classic example. A scanner may find the endpoint that applies discount codes, but it may not understand that a user can apply the same code repeatedly through a race condition and receive unintended credit. It may reach a transfer flow without realising that the approval sequence can be bypassed through a strange but valid series of actions. Those are not simple input-validation bugs. They are application-specific design failures.

Zero-days and issues without known detection logic

Scanners also depend on testing logic, behavioural rules, or signatures that exist today. If a zero-day appears tomorrow and the engine does not yet know how to recognise it, automation will not magically invent full coverage. Good tools evolve quickly, but no scanner is ahead of every new vulnerability at all times.

Human judgement still matters

Some findings need context only a person can provide. Is this admin endpoint actually exposed to the internet in production, or only in a lab snapshot? Is this cookie setting a major risk in your real deployment, or is there a compensating control that changes the priority? Is this information disclosure trivial, or does it meaningfully help an attacker move forward? These are judgement calls.

This is why vulnerability scanning and penetration testing are related, but not identical. A scanner is designed to automate broad, repeatable, high-value checks. A penetration test goes further into chaining issues, exploring unusual application logic, adapting to custom workflows, and exercising human creativity against the system.

The best security programmes use both ideas correctly. Automated scanning gives you repeatability and breadth. Manual testing gives you depth where context, creativity, and business understanding matter most.

The 3 phases at a glance

Phase | What happens | Typical evidence | Main limitation --------|---------------------------------------------------|--------------------------------------|------------------------------- Crawl | Maps pages, forms, paths, scripts, parameters | URL inventory, inputs, endpoints | Can only map what it can reach Probe | Sends safe test inputs and compares responses | Errors, reflections, timing shifts | Poor payload choice reduces signal Detect | Validates patterns and scores likely findings | Confirmed evidence with severity | Still needs human review for context

FAQ

Is vulnerability scanning the same as penetration testing?

No. Vulnerability scanning is automated and repeatable. It is designed to cover a broad set of known weaknesses across exposed attack surface with consistent logic. Penetration testing includes manual reasoning, chained exploitation, custom workflows, and deeper investigation of application-specific behaviour.

A scanner is often the fastest way to establish baseline visibility. A penetration test is the better tool when you need human-led depth, adversarial thinking, and validation of complex business logic.

Can a vulnerability scanner cause damage to my site?

A well-designed scanner is built around non-destructive testing principles, but it still sends large numbers of requests and can affect fragile systems, weak rate limits, or unstable staging environments. The goal is to identify issues safely, not to break production.

Risk depends on scan depth, application quality, and how responsibly the scan engine behaves. That is one reason it is useful to choose the right depth for the task and understand what the scanner is actually doing under the hood.

How long does a vulnerability scan take?

It depends on site size, scan depth, application complexity, and how much of the target is reachable. A small public site with limited paths may finish quickly. A larger application with more forms, more scripts, more endpoints, and deeper confirmation logic will take longer.

The core reason is mechanical: more crawling, more probing, and more validation steps mean more runtime. Faster does not automatically mean worse, but deeper almost always means broader coverage.

What does a vulnerability scanner look for?

It looks for evidence of exploitable weaknesses and misconfigurations across the exposed application surface. That can include injection issues, cross-site scripting, weak headers, SSL/TLS problems, access control anomalies, exposed files, insecure defaults, information disclosure, and other web security risks.

Importantly, the scanner does not just look for names. It looks for behaviour, patterns, and response differences that suggest the application is handling input, configuration, or trust boundaries unsafely.

How accurate are vulnerability scanners?

Accuracy varies by engine quality, target complexity, and scan scope. Better scanners reduce false positives by using confirmation logic, context analysis, and repeated verification instead of relying on a single signal.

Even strong scanners are not perfect. Results should be reviewed as evidence-based leads, especially when a finding depends on behaviour rather than a simple signature. Trustworthy tools help by showing the affected endpoint, the request, the response evidence, and the reasoning behind the severity.

Why do some scans miss issues I later find manually?

Usually because the scanner never reached the relevant path, did not have the right session state, or could not recognise the flaw with available testing logic. Automation is powerful, but it is still bounded by reachability, payload choice, and what the engine knows how to detect.

That is why understanding crawl, probe, and detect is so useful. It tells you where blind spots come from and how to reduce them.

Conclusion

A vulnerability scanner works by mapping what it can reach, testing what it finds with controlled inputs, and turning the resulting evidence into prioritised findings. Once you understand crawl, probe, and detect, scan results stop feeling mysterious and start feeling explainable.

That understanding also makes you better at choosing scan depth, reviewing evidence, and knowing when automation is enough, and when human testing still needs to step in.

Table of Contents

Security Insights Blog

How Does a Vulnerability Scanner Work?