In short and in plain language
A penetration test is a planned attack on your own shop. Experts look for gaps, with permission. A scan, on the other hand, is a program. It only looks for known errors.
A test is most useful for a new shop. Or when you add your own programs. Or when you connect a new way to pay.
Before the test, you need a written order. The hosting company must agree, too. Often the test runs on a copy of the shop.
After that, you get a report. It sorts the errors by danger. Serious errors are fixed right away, small ones later.
At the end, someone checks again that everything is fixed. This is called a retest. We prepare the test and fix the errors as part of maintenance.
A penetration test sounds like something for banks, government agencies or large corporations. For an online shop, the question is more down to earth: when does a commissioned attack on your own system deliver more than the automated vulnerability scan that already runs under many maintenance contracts? The German Federal Office for Information Security (BSI) describes penetration tests as a method for assessing the chances of success of a deliberate attack and for checking how effective the existing security measures are (BSI). A scan finds known gaps in known versions. A penetration test looks for the paths a person with intent and time will find: a price that can be overwritten at checkout, an order that opens with someone else's number, an extension that allows more than it should. This article shows how the two kinds of testing differ, which occasions make a test worthwhile, how to set scope and test environment, how to read the report and how findings move through the maintenance plan to the retest.
Key takeaways
- An automated scan matches versions against known vulnerabilities and also reports false positives (NIST). A penetration test chains weaknesses like an attacker would and checks logic no scanner sees, such as prices at checkout (OWASP).
- A test pays off before a relaunch, after custom extensions, with a new payment or ERP connection, after an incident and when insurers or major customers ask for proof.
- Article 32 GDPR requires a process for regularly testing the security measures (GDPR). It does not mandate a penetration test, but a test is a traceable way to document that review.
- Before the test, you need a written engagement covering target, depth and time frame, plus your hosting provider's consent. Under Section 202a StGB, gaining unauthorized access to secured data by overcoming access protection can mean up to three years in prison (StGB).
- The BSI assigns an immediate response to high findings, a short-term response to medium ones and a medium-term response to low ones (BSI). With an owner and a deadline, every finding goes into the maintenance plan.
- Only the retest shows that a fix works (PCI SSC 2017). According to the BSI guide, the test is repeated every two to three years (BSI), with regular scans running in between.
Scan or penetration test: two different checks
A vulnerability scanner works like a lookup. It identifies the operating system, web server, shop software and extensions along with their version numbers and compares them with the entries in its database. That way it finds outdated software, missing patches and misconfigurations, quickly and as often as needed. In its guide SP 800-115, the US standards institute NIST also describes the downside: although the process itself is highly automated, scanners can have a high false positive rate, meaning they report vulnerabilities that do not exist (NIST). Someone with experience in network and system security should therefore interpret the results before they turn into work.
A penetration test goes one step further. As NIST describes it, testers mimic real-world attacks to find ways of circumventing the security features of an application; most tests look for combinations of vulnerabilities that together grant more access than any single one (NIST). The PCI Security Standards Council, which publishes the security rules of the payment card industry, calls penetration testing essentially a manual endeavor (PCI SSC 2017). Tools help with routine steps, but the tester makes the decisions. NIST notes that manual testing can identify new or obscure vulnerabilities that automated scanners may miss (NIST).
| Aspect | Automated scan | Penetration test |
|---|---|---|
| Goal | find and list known gaps in known versions | find ways around protective measures and document the impact |
| Approach | automated, matching against a vulnerability database | mostly manual, tools only as an aid |
| Typical findings | outdated extension, missing patch, open service | price that can be manipulated, someone else's order visible, escalated permissions |
| False positives | common, need interpretation | less common, because findings are confirmed by an attack attempt |
| Cadence | regularly, automated as part of maintenance | per trigger and at longer intervals |
| Result | list of findings with ratings | report with proof, rating and recommendation |
For a shop, the difference is tangible. OWASP, an open community for application security, describes a shop in its Web Security Testing Guide (WSTG) where an attacker goes back to the cart summary page, injects a lower price within the same valid session and completes the order. According to the guide, a vulnerability scanner cannot detect such business logic flaws; finding them relies on the skills and creativity of the tester (OWASP). Access rights are similar: for the 2025 edition of the OWASP Top 10, some form of broken access control was found in 100 percent of the applications tested, including cases where other people's accounts can be viewed or edited through their identifier (OWASP). In a shop, that means anyone who changes the number in their order URL must not see another customer's invoice. A person with two test accounts can check that in a few minutes; a scanner usually cannot check it at all.
When the test pays off
In its practical guide to IS penetration tests, the BSI summarizes when such a test is usually sought: after an IT system first goes into operation, after significant changes or when an attack has already taken place (BSI). Applied to running a shop, that results in six occasions where the effort usually pays off:
- Relaunch or new shop: Before go-live is a good time, because flaws can still be fixed without real customer data and without live orders.
- Custom extensions: Nobody outside your own team has reviewed custom plugins, checkout changes or customer account features, and no scanner knows their code.
- New payment or data connection: A new payment method, an ERP interface or a marketplace connection opens paths to order and customer data.
- Proof for third parties: Cyber insurers, major customers or payment providers ask for an independent review.
- After an incident: Once the cleanup is done, a test shows whether the entry point is closed and whether there are others. The steps before that are covered in the emergency plan for hacked websites.
- Major platform upgrade: Moving to a new major version changes routes, permissions and interfaces all at once.
Why configuration and permissions deserve the focus is shown by Bitkom's study report Wirtschaftsschutz 2026. Bitkom surveyed 1,003 companies in Germany with at least 10 employees and annual revenue of at least 1 million euros (Bitkom). 58 percent of the companies had suffered damage from cyberattacks in the previous twelve months (Bitkom). Asked which factors contributed to the attackers' success, the affected companies named insufficient detection of security incidents at 59 percent, followed by misconfigurations at 57 percent and insufficient identity and access management at 55 percent (Bitkom). Technical vulnerabilities followed at 50 percent, outdated hardware or software at 43 percent (Bitkom). Outdated software and known misconfigurations already show up in the scan. Permission errors and misconfigurations that only surface in the interplay of accounts, roles and processes, on the other hand, are the findings where a manual test plays to its strengths.
When a scan is enough for now
Not every shop needs a penetration test every year. A standard shop without custom code, with well-maintained extensions from known sources and an outsourced payment page benefits from the basics first: timely security updates, regular scans, a hardened server and secured admin access. The BSI also recommends running a short IS audit (IS-Kurzrevision) before a penetration test, meaning a spot check of the baseline protection (BSI). NIST advises running less labor-intensive tests on a regular basis between penetration tests; a program of regularly scheduled network and vulnerability scanning, interspersed with periodic penetration testing, can help prevent many types of attacks (NIST).
Across the board, systematic testing is not yet routine: in 2024, 34.64 percent of enterprises in the EU with at least 10 persons employed used ICT security tests (Eurostat 2024). For a shop that handles payments, customer accounts and order histories, the question is therefore less whether to test and more how deep: the scan covers the breadth, the penetration test covers the places where custom code and custom processes live.
What the law and the payment industry require
There is no general obligation for online shops to run penetration tests. However, Article 32(1)(d) of the General Data Protection Regulation requires a process for regularly testing, assessing and evaluating the effectiveness of technical and organizational measures (GDPR). The regulation leaves open what that review looks like. For a shop with customer accounts and order data, a documented test with report and retest is one way to show it in a traceable manner; regular scans and logged updates are another, and the two usually complement each other.
The card payment standard PCI DSS is stricter. The PCI SSC information supplement on penetration testing states that under Requirements 11.3.1 and 11.3.2, testing must take place at least annually and after any significant change, such as an infrastructure or application upgrade or modification (PCI SSC 2017). The same overview calls for vulnerability scans at least quarterly (PCI SSC 2017). The requirement numbers refer to the version of the standard in force at the time. Which checks a shop actually has to prove depends on how payment is integrated; the payment provider gives the binding answer. How to monitor scripts on the payment page is covered in the article on script monitoring under PCI DSS.
On top of that come proofs that arise from contracts rather than from the law. Cyber insurers ask about security measures in their risk questionnaire, major customers have their suppliers confirm their security, and companies within the scope of NIS2 face their own risk management obligations. The article on maintenance evidence for cyber insurance covers which documents count; whether a shop falls under NIS2 at all is explained in the article on NIS2 for online shops. A test report does not replace any of these documents, it adds the outside view.
Setting scope and test environment
The most common mistake when commissioning a test is an unclear scope. Based on the threat situation and protection needs, the BSI recommends focusing on the systems that are particularly at risk or business-critical, and looking from an attacker's point of view at where the interfaces are (BSI). In a shop, that usually means checkout and payment callback, customer account and order history, vouchers and discounts, the admin interface, the APIs for app, ERP or marketplace, and all custom extensions. Test depth is part of it, too: the BSI generally recommends a white-box test, where testers receive extensive information about the system, because a black-box test can miss vulnerabilities (BSI). For a shop, that means test accounts with different roles, a list of installed extensions and, where available, access to the code of custom plugins.
Then comes the question of where to test. A staging environment that matches production in versions, extensions and configuration protects live orders and real customer data. The BSI points out, however, that results from a replicated environment can only partly be transferred to the original system (BSI). That is why parity with production matters: same versions, same extensions, same server settings, but anonymized test data. If production is tested after all, a low-traffic window and a fresh, verified backup belong to the plan. NIST reminds us that systems may be damaged or rendered inoperable during a penetration test and that this risk can never be fully eliminated (NIST).
Protective layers in front of the shop need special attention. The BSI recommends examining a website first without a security gateway in front of it (BSI). The PCI SSC explains why: the test is meant to evaluate the susceptibility of the services themselves, not the ability of the protection systems to block attacks (PCI SSC 2017). If a web application firewall fails later or is bypassed, the application should still hold up. In practice, the testers' addresses are allowlisted in the firewall for the test window, rather than letting the test bounce off it and leave an overly favorable impression.
Written engagement, hosting provider and data protection
Without a written engagement, an attack on someone else's systems is not a test but, in the worst case, a criminal offense. Under Section 202a of the German Criminal Code, anyone who, without authorization, overcomes an access protection to gain access to data that is not intended for them and is specially protected against unauthorized access faces up to three years in prison or a fine (StGB). The BSI advises testing only with a written engagement and adds a point that is often overlooked in shop operations: if services are outsourced to a hosting provider, that provider must be included in the contract as well (BSI). If your shop runs with a hosting provider, you therefore get their consent; where their responsibility ends is described in the article hosting provider vs. maintenance. If personal data is affected, the BSI guide calls for involving the data protection officer before the test (BSI). And no maintenance work should fall within the test window, because a test on a system that is being changed loses its significance (BSI).
- Target: domains, subdomains, IP addresses, admin path and interfaces that may be tested
- Depth: attack intensity and explicitly excluded tests, such as load attacks or calls to staff
- Time frame: test window, tester addresses and a reachable emergency contact on both sides
- Test accounts: accounts with different roles and their deletion after the test
- Data: handling of data found, retention period, proof of deletion and confidentiality
- Outcome: report format, debriefing and an agreed retest
One point deserves its own sentence: the IS penetration test as the BSI conducts it contains no social engineering elements (BSI). According to the BSI, however, real attacks would most likely start with social engineering or include it (BSI). A technical test therefore says nothing about whether someone hands out a password over the phone. How to protect access against such calls is covered in the article call from a fake service provider.
The process in five steps
NIST divides a penetration test into four phases: planning, discovery, attack and reporting (NIST). In the planning phase, rules are identified, management approval is finalized and documented and testing goals are set; no actual testing happens in this phase (NIST). The BSI divides its milestone plan into a familiarization phase, a test phase and a reporting phase (BSI). For shop operations, a five-step structure is practical because it explicitly includes preparing the test environment and the retest:
Define the scope
Agree in writing on target, depth, time frame, test accounts and contacts. Involve the hosting provider and the data protection officer, and name exclusions explicitly.
Prepare the test environment
Set up staging with the same versions and anonymized data, or pick a quiet window in production. Verify the backup beforehand and allowlist the tester addresses in the firewall.
Run the test
The testers explore the shop and try to exploit the vulnerabilities they find. Maintenance work pauses during this time, and an emergency contact stays reachable.
Review the report
Reproduce the findings, rate them for your own shop and add them to the maintenance plan with an owner and a deadline. Record false positives with a reason.
Retest
Have the fixed findings checked again, remove test accounts and tools, and document the result with a date.
Agree in writing on target, depth, time frame, test accounts and contacts. Involve the hosting provider and the data protection officer, and name exclusions explicitly.
Set up staging with the same versions and anonymized data, or pick a quiet window in production. Verify the backup beforehand and allowlist the tester addresses in the firewall.
The testers explore the shop and try to exploit the vulnerabilities they find. Maintenance work pauses during this time, and an emergency contact stays reachable.
Reproduce the findings, rate them for your own shop and add them to the maintenance plan with an owner and a deadline. Record false positives with a reason.
Have the fixed findings checked again, remove test accounts and tools, and document the result with a date.
The duration depends on the scope. A test that only covers checkout and customer account of a standard shop is smaller than one that also includes an ERP interface and the admin area. With limited resources, the BSI advises testing first the interfaces where an attack is most likely and covering other areas in later tests (BSI). For many shops, that is the sensible way in: checkout, account and custom extensions in depth rather than the entire server on the surface.
How to read and rate the report
According to the BSI, a good report starts with a description of the test target and the test conditions (BSI). After that come a management summary with the key findings and the vulnerabilities found, with precise descriptions and recommendations for the technical team. When you read the report, first check whether target and time frame match the engagement. A report on a staging address with an outdated extension says little about the live shop.
For rating, the BSI uses four levels: findings with high damage potential call for an immediate response, medium ones for a short-term response, low ones for a medium-term response, and notes for information are addressed in the long term (BSI). High means, for example, that an attacker could take control of or modify the tested system, or read sensitive data. An important addition is that the same vulnerability is rated differently depending on how much protection the processed data needs (BSI). An exposed test page without customer data weighs differently from the same flaw in the customer account.
Many reports also include a CVSS score. The scoring system maintained by the organization FIRST rates scores from 9.0 to 10.0 as critical, 7.0 to 8.9 as high, 4.0 to 6.9 as medium and 0.1 to 3.9 as low (FIRST). The base score describes the vulnerability itself; the specification recommends enriching it with information about the threat situation and your own environment to get a better basis for risk assessment (FIRST). A critical flaw in a feature that is switched off in your shop therefore weighs less than a medium one at checkout. This rating is part of the work that starts after the report and belongs to ongoing CVE management.
- Can the finding be reproduced from the details in the report, or is the proof missing?
- Does it affect the live shop, the test environment or both?
- Which data or functions would be affected: payments, customer accounts, administration?
- Is the cause in the shop software, in an extension, in custom code or in the server configuration?
- Who fixes it, by when, and how can you tell that the finding has been fixed?
Moving findings into the maintenance plan
A report that sits in a folder improves nothing. NIST puts it plainly: the results of penetration testing should be taken seriously, and any vulnerabilities discovered should be mitigated (NIST). In shop operations, this works when every finding lands where planning already happens: in the maintenance plan, with an owner, a deadline and proof. The levels from the BSI guide translate directly:
| Rating (BSI) | Response (BSI) | Handling in shop operations | Proof |
|---|---|---|---|
| high | immediate | emergency path: hotfix, disable the extension or block the feature, even outside the regular window | ticket with timestamp, retest of the finding |
| medium | short term | next maintenance window, tested on staging first | change log, regression test |
| low | medium term | planning of the next updates or the next release | entry in the maintenance plan |
| for information | long term | consider in the next revision | note on the report |
Medium and low findings follow the same discipline as any update: staging first, then regression tests in a scheduled maintenance window. Findings that cannot be fixed right away, for example because an extension has no patch yet, get an interim measure and a documented decision. An accepted risk is legitimate as long as someone carries it consciously and has recorded it with a date. False positives also go on the list with a reason, so the next test does not carry them as open again. High findings also need a path outside the regular window, for example through emergency support.
Describe identical findings once
The retest closes the loop
The PCI SSC describes the retest as a fixed part of the process: once the organization has implemented its fixes, the tester should test again to confirm that the new controls mitigate the original risk (PCI SSC 2017). If remediation drags on for a long time, the same supplement says a new test may be needed so that the results reflect the current state (PCI SSC 2017). Cleanup is part of closing: accounts created for the test are deleted and tools installed by the tester are removed so they cannot be used against the shop later (PCI SSC 2017). The same applies to a test copy built specifically for the test: once it is no longer needed, it is shut down so it does not linger online as a forgotten test site.
And after that? On its topic page, the BSI recommends repeating the tests at regular intervals to gain a good long-term picture of security (BSI). In its practical guide, it names an interval of two to three years, because new vulnerabilities and attack methods regularly become known (BSI). NIST writes that, because of its high cost and potential impact, an annual penetration test of networks and systems may be sufficient (NIST). For a shop, this does not result in a fixed calendar but in a rule: a test is due when one of the triggers occurs, and at the latest after two to three years; in between, the maintenance scans keep running. The BSI also advises against having the same targets tested repeatedly by the same testers, to avoid operational blindness (BSI).
What maintenance contributes to the test
A penetration test is a snapshot. The BSI recommends stating in the contract that the results are valid only at the time of the test (BSI). What happens between two tests is decided by maintenance: it applies security updates promptly, runs regular vulnerability scans, watches for anomalies in monitoring and documents every change. As part of an SLA maintenance contract, we prepare the test on the shop side and then work off the findings in the maintenance plan. The test itself is carried out by specialized partners when needed.
Scans in maintenance
Regular automated vulnerability scans with a prioritized list, so known gaps do not wait until the next test.
Test environment
Staging with the same versions and extensions, anonymized data, verified backup before the test.
Approvals
Test accounts with roles, coordination with the hosting provider, tester addresses allowlisted for the test window.
Findings list
Every finding with rating, owner and deadline in the maintenance plan, false positives with a reason.
Remediation
Patches and fixes on staging first, then live with a regression test in the maintenance window.
Retest
Have fixed findings checked again, delete test accounts, record the result with a date.
If you are not sure whether a test is due in your case, start with an inventory: which custom extensions, which interfaces, which payment integration and when the last review took place. That tells you whether scanning and hardening are enough for now or whether a penetration test brings the greater benefit. The overview of our services and the page on Shopware maintenance show how maintenance, monitoring and emergency support fit together; for an assessment of your shop, reach us via the contact page.
Sources
Related Articles
Week 53 and Year End: Avoiding Date Errors in Your Shop
January 1, 2027 still falls in week 53 of 2026. How to check week formats, time zones, number ranges, holidays and promotions in your shop before New Year.
Fake Service Provider Calls: Protecting Shop Access
Fake hosts and agencies call asking for a password, a code or an MFA reset. Callbacks, a code word and two-person approval keep your shop access protected.
Fake Customer Accounts: Stop Bot Sign-ups in Your Shop
Bots create fake customer accounts in bulk. Learn how to spot bot sign-ups in your shop and stop them with confirmation emails, rate limits and a honeypot.