Running a Phishing Simulation: Tools, Scenarios, and Evaluation

TL;DR

Phishing simulations are controlled tests where simulated phishing emails are sent to your own employees. They measure the click rate, credential submission rate, and reporting rate as a baseline for the awareness program.
The simulation must be carefully planned: coordination with the works council and data protection, defining the target group, selecting realistic scenarios, and technical preparation (whitelisting in the email filter, tracking infrastructure).
Common tools include KnowBe4, Proofpoint Security Awareness, Cofense PhishMe, Hoxhunt, and the open-source solution GoPhish. The choice depends on budget, company size, and desired features.
Evaluation focuses on three metrics: click rate (who clicked the link), compromise rate (who entered data), and reporting rate (who reported the email). The trend across multiple simulations is more meaningful than a single result.
The biggest mistake is publicly shaming individual employees. Phishing simulations are not a tool for punishment but for improvement. Results are anonymized or evaluated at department level, never published by name.

Why Phishing Simulations Are Essential

You can run a hundred training sessions on phishing and still not know whether your employees would recognize a real phishing email. The gap between knowledge and action is particularly wide in information security. In training, everyone recognizes the obvious warning signs: the wrong sender address, the spelling mistakes, the suspicious link. In daily work, under time pressure, between two meetings, with a full inbox, reality looks different.

Phishing simulations bridge this gap. They test actual behavior under realistic conditions, not theoretical knowledge in a training setting. They show how many employees click on a link, how many enter their credentials on a fake page, and how many report the suspicious email.

For your ISMS, phishing simulations provide measurable metrics that demonstrate the progress of your awareness program. ISO 27001 requires employee sensitization for information security in Annex A.6.3. NIS2 demands "cyber hygiene" and training measures in Article 21. Phishing simulations are the evidence that these measures do not just exist on paper but actually work.

Legal and Organizational Prerequisites

Before you send the first phishing email, you must clear several legal and organizational hurdles. A phishing simulation without this preparation can lead to labor law issues, data protection violations, and lasting loss of trust among employees.

Involve the Works Council

If your organization has a works council, they must be informed and involved before the simulation is conducted. Phishing simulations can be considered performance and behavior monitoring under the Works Constitution Act, which triggers the works council's co-determination right.

Involving the works council is not an obstacle but an opportunity. If the works council supports the simulation, employee acceptance increases. Agree with the works council that results will be anonymized or evaluated at department level, that no individual consequences will be derived from the results, and that the simulation will be communicated as a learning measure, not a control instrument.

Clarify Data Protection

Phishing simulations collect personal data: who clicked on which link and when, and who entered credentials. This processing requires a legal basis under Art. 6 DSGVO (GDPR). In most cases, the legitimate interest (Art. 6(1)(f)) is the appropriate basis because the simulation serves to improve information security and thus pursues a legitimate interest of the organization.

Document the processing in the records of processing activities, inform the data protection officer, and ensure that the collected data is used only for the defined purpose. The retention period for individual results should be as short as possible. Aggregated statistics (click rate per department, trend over time) can be retained longer because they contain no personal reference.

Inform Executive Management and IT

Executive management must approve the simulation and ideally actively support it. The IT department must whitelist the simulation emails in the spam filter and email security solution so they actually reach the employees. And the IT helpdesk must be informed because experience shows that employees will call to report suspicious emails — which is exactly the desired behavior.

Designing Scenarios

The quality of a phishing simulation stands and falls with the realism of the scenarios. If the simulated phishing email looks obviously fake, you are not testing employee attentiveness but the quality of your simulation.

Difficulty Levels

Plan the simulations in ascending difficulty levels. The first simulation should be deliberately easier so that the entire workforce does not "fail" and become frustrated. Later simulations increase complexity and test increasingly subtle attack forms.

Easy: Obvious warning signs like foreign sender domains, spelling errors, generic greeting, and a link that obviously does not belong to the claimed organization. Example: "Your Microsoft account has been locked. Click here to unlock it" from support@microsft-security.xyz.

Medium: More plausible sender, correct company logo, personalized greeting, but recognizable inconsistencies upon closer inspection. Example: An email that looks like an internal IT notification requesting a password change via a link pointing to an external domain.

Hard: Highly targeted, referencing real company contexts. Example: An email seemingly from the CEO referencing an attached strategy document to be accessed via a cloud link. Or an email referencing a recent company announcement and linking to a "follow-up article."

Context-Based Scenarios

The most effective scenarios relate to the target group's daily work. For accounting, a fake invoice is plausible. For HR, a fake job application is realistic. For executive management, an email from a supposed tax advisor or attorney is convincing.

Use current occasions: a phishing email about an upcoming company event, a fake parcel delivery notification shortly before Christmas, or a supposed salary adjustment at the start of the year. The better the scenario matches the work context, the more meaningful the simulation.

Technical Implementation of the Scenario

A phishing simulation consists of several technical components. The email itself contains a link to a landing page. This landing page can be a simple informational page (for simulations that only measure the click) or a fake login page (for simulations that also measure credential entry).

If the simulation uses a login page, it must not store actual credentials. The page only registers that an entry was made but does not store the entered password. This detail is mandatory for data protection reasons.

After clicking the link or entering data, the employee is redirected to a learning page that explains it was a simulation, what warning signs were recognizable, and how to recognize similar emails in the future. This "teachable moment" is the most didactically valuable part of the simulation because the employee has just had the concrete experience and is particularly receptive to the learning content.

Tool Comparison

Various commercial and open-source tools are available for conducting phishing simulations.

KnowBe4

KnowBe4 is the market leader in security awareness training and phishing simulation. The platform offers over 15,000 pre-built phishing templates, automated campaign scheduling, comprehensive reporting features, and integration with its own training platform. Particularly useful is the feature where employees who fall for a simulation are automatically directed to a targeted micro-training. Pricing is based on company size and the selected package.

Proofpoint Security Awareness

Proofpoint combines phishing simulation with its proprietary email security platform. The advantage is integration: simulation results can be correlated with real threat data to identify particularly at-risk employees (Very Attacked People) and train them specifically. The platform also offers compliance modules for GDPR, ISO 27001, and NIS2.

Cofense PhishMe

Cofense (formerly PhishMe) focuses heavily on the reporting rate as the central metric. The platform integrates a phishing report button into the email client and automatically analyzes reported emails. The goal is not just reducing the click rate but establishing a reporting culture where suspicious emails reach the IT security team before anyone clicks the link.

Hoxhunt

Hoxhunt relies on personalized, adaptive simulations. The platform automatically adjusts the difficulty level to the individual employee's level: those who perform well receive harder simulations. Those who struggle receive easier scenarios and additional training content. This adaptive approach avoids both beginner frustration and advanced boredom.

GoPhish (Open Source)

GoPhish is a free, open-source phishing simulation platform that you can self-host. It offers basic features: email templates, landing pages, campaign management, and reporting. The user interface is less polished than commercial solutions, and there are no pre-built training modules, but for organizations with limited budgets and technical know-how, GoPhish is a solid entry point.

Setup requires your own server, SMTP configuration, and creating your own templates. The effort is worthwhile if you want to maintain full control over the infrastructure and prefer not to transmit results to a third-party provider.

Execution: Step by Step

Preparation (2-4 Weeks Before the Simulation)

Define the simulation objective: is it a baseline measurement for a new awareness program? A progress check after a training? A test of a specific attack scenario?

Select the target group: the entire organization or a specific department? For the first simulation, a pilot run with a limited group (e.g., the IT department) is recommended to verify technical functionality and test the scenario.

Create or select the scenario and email templates. Configure whitelisting in the email filter. Inform the helpdesk. Prepare the landing page and learning page.

Sending

Do not send the simulation emails all at once but staggered over several hours. This prevents employees from warning each other ("Hey, I just got a weird email, you probably did too"). A window of four to six hours is a good compromise.

Choose a weekday and time that correspond to normal work routines. Tuesday morning through Thursday afternoon are typical windows. On Mondays and Fridays, employees are either still in the start-of-week rush or already in weekend mode and less attentive, which can skew results (or, depending on your objective, make them particularly interesting).

Tracking Period

Define a tracking period during which clicks and entries are measured. 48 to 72 hours is typical. After that, the campaign is closed and results are evaluated.

Evaluation and Metrics

The evaluation focuses on three core metrics that together provide a comprehensive picture.

Click Rate

The click rate measures the proportion of recipients who clicked on the link in the phishing email. A typical click rate in the first simulation is 20 to 40 percent, depending on the scenario difficulty and the existing awareness level.

However, the click rate alone is an incomplete indicator. An employee who clicks the link, recognizes the fake page, and immediately closes the browser has a different risk profile than an employee who clicks the link and enters their credentials.

Compromise Rate

The compromise rate measures the proportion of recipients who actually entered data on the landing page (username, password, personal information). This metric is more meaningful than the pure click rate because it reflects the actual risk.

Reporting Rate

The reporting rate measures the proportion of recipients who reported the phishing email through the defined channel (phishing button, email to the security team, call to the helpdesk). The reporting rate is the most positive metric because it measures proactive, security-conscious behavior.

A mature security culture is characterized by a reporting rate higher than the click rate. This means more employees report the email than click on the link.

Trends Over Time

A single simulation result is a data point, not a trend. The real value of phishing simulations shows only across multiple runs. Plan at least four simulations per year to obtain a meaningful trend. The click rate should decline over time; the reporting rate should increase.

Only compare comparable scenarios. If you increase the difficulty level between simulations, a rise in click rate is not a setback but the expected consequence of higher difficulty.

Follow-Up: The Most Important Part Comes After the Simulation

The simulation is just the trigger. The actual impact comes from the follow-up.

Immediate Feedback (Teachable Moment)

As described above, employees who click the link are immediately redirected to a learning page. This page explains the warning signs of the specific email and gives general tips for recognizing phishing. This immediate feedback is didactically most effective because the employee is still in the context of the experience.

Communication to All Employees

After the simulation concludes, communicate the aggregated results to all employees. This communication must strike the right tone: factual, appreciative, and constructive. No shaming, no blame, no threats.

A proven format is: "We conducted a phishing simulation last week. [X] percent of employees correctly identified the email and did not click. [Y] percent reported the email via the report button. Thank you to everyone who was attentive. For those who clicked the link: there is no reason for concern — this is a normal part of the learning process. The email was deliberately designed to look realistic. Here are the warning signs you can use to recognize similar emails in the future: [specific pointers about the email]."

Targeted Training

Employees who fell for the simulation receive targeted follow-up training. This can be a short online module (10 to 15 minutes) that reviews the key recognition features and practices with concrete examples. Some platforms (KnowBe4, Hoxhunt) offer this automatic assignment as an integrated feature.

Those who repeatedly fall for simulations do not need harsher punishment but a different form of support: personal coaching, an individual conversation with the department's Security Champion, or more intensive training. The cause of the behavior can vary: time pressure, limited reading skills (e.g., non-native speakers), uncertainty in handling emails, or simply disinterest. Each cause requires a different response.

Common Mistakes

Too difficult scenarios in the first simulation: If 80 percent of employees fall for the first simulation, it creates frustration and the impression that the simulation was unfair. Start with medium difficulty and increase gradually.

Publishing individual results by name: Publishing individual results ("Mr. Mueller from accounting entered his credentials") destroys trust and is in many cases problematic under data protection law. Results are aggregated at the department level.

Simulation without a training program: A phishing simulation without an accompanying awareness program measures a problem but does not solve it. The simulation is a tool within a comprehensive program, not a substitute for one.

Running too infrequently: An annual simulation provides a single data point. At least four simulations per year are needed to identify trends and demonstrate progress of the awareness program.

Not acting on results: If results are collected but not used to develop the awareness program further, the simulation is a waste of time. Results must lead to concrete actions: adapted training content, targeted follow-up training, improved technical protection measures.

Phishing Simulations in the ISMS Context

Phishing simulations are an integral part of your awareness program and should be documented in your ISMS. The documentation includes the simulation plan (frequency, scenarios, target groups), the results of each simulation (aggregated metrics, trends), the derived measures, and the effectiveness of the measures (change in metrics in the next simulation).

This documentation is your evidence for auditors that you are not just conducting training but verifying its effectiveness and continuously improving the program based on results. This is exactly the PDCA cycle (Plan-Do-Check-Act) that ISO 27001 requires, applied to employee sensitization. In ISMS Lite, simulation plans, results, and derived measures can be centrally documented and prepared for audits.