This year is seeing more phishing attacks in more places, having more detrimental impacts. One security messaging provider noted a 61% increase in phishing attacks between April and October 2022 compared to the same period the previous year.
Another report observed that when phishing attacks were successful in 2022, financial losses from those incidents increased by 76%.
In another trend, phishing attacks are moving to mobile personal devices. A 2023 survey of 300 employees revealed that 43% had experienced work-related phishing attacks on their personal phones.
In response to the growing threat of phishing attacks, many organizations have implemented phishing simulations to test and train their employees’ phishing detection skills and to cultivate cyber security awareness.
While phishing simulations give organizations a valuable measure of their phishing detection performance, that data is sometimes spoiled by false positive results. Many organizations use security tools that scan links and automatically click.
When a bot-click performs poorly on phishing simulation prompts, it can look like the end-user employee failed to spot the phishing attempt.
This article will explain what bot clicks are, how to avoid false positives in phishing simulations, and ensure your data fairly reflects employee performance.
What is a Phishing Simulation?
A phishing simulation is a test of phishing detection skills. To conduct a phishing simulation, an organization sends its employees email messages that mimic real-world phishing scams and contain dummy malicious links.
The objective of the test is to see how many employees click on the malicious link. Employees who click on a simulated phishing link are immediately provided with feedback, highlighting the importance of vigilance and teaching them how to spot real threats.
By measuring user clicks, organizations learn how effective employees are at spotting scams, assess whether they’re following the latest security best practices, and determine whether they would benefit from cyber security training.
In an actual attack, a single click could spell disaster. An employee could’ve unsuspectingly given away sensitive information to criminals, compromised the network, or downloaded malware to the system.
What is a Bot Click?
Organizations often use third-party security software in sandbox environments with bots that click on all URLs that form part of emails. This process is implemented to ensure that no malicious content enters the environment via email. The practice is common but can give inaccurate results of the phishing simulation and skew the resulting data.
What is a Phishing False Positive?
If you send a simulated phishing email to an employee as part of a phishing simulation, a real-time threat detection software might scan the message and "click" the links it contains. When those clicks are counted in phishing simulation results and assigned to human end-users, they are false positives. False positives can influence how your organization's phishing simulation reporting data is interpreted. In the worst case, they may culminate in imprecise testing insights that take your cyber security training strategy in unsubstantiated directions.
What Causes Bot Clicks?
While most bot clicks originate with third-party security solutions, they register in test results for several reasons. Some of the most common bot clicks happen when employees detect phishing emails and flag them using a default "Mark as Phishing" feature. Phishing reporting features like these are available as add-ons in Outlook and other email clients. When phishing emails are flagged this way, the third-party email provider scans the link for malicious content. In doing so, the test registers a "click." Another common scenario is when a user scans an email or attachment with a third-party security service, such as Microsoft SafeLink. Again, the third-party assessment registers a bot click that counts as a phishing false positive. Bot clicks also occur when:
- Endpoint security and antivirus software scan email links
- Mobile devices preview link content
- Users forward emails to other users, prompting the user's mail server to scan the email
- Poorly configured spam filters don't allowlist simulated phishing emails and scan the email links
Why is Spotting Bot Clicks Important?
Organizations and IT leaders need accurate phishing simulation data to gauge employee cyber security behaviors and implement an appropriate cyber security awareness training program.
When false positives from bot clicks artificially inflate the number of phishing simulation “fails,” this reflects poorly on your organization’s cyber security preparedness. It can suggest that your cyber security training program has been ineffective.
Additionally, false positives also make it challenging to monitor and record employee performance during tests.
Knowing that false positives can occur and how to spot them helps ensure your data are accurate and your phishing simulation is effective. Once you’ve separated the bot clicks from the employee clicks, you’ll have test results you can learn from and build on.
How to Spot False Positives
Check your phishing simulation test results. If you see a 100% or unusually high click rate, it’s a telltale sign of false positives. A high volume of external IP addresses among the results also suggests false positives.
The 100% click rate suggests that automated software was involved, and the external IPs tell you that a third-party remote software scanned your links.
Compare the delivery times of the original simulated phishing emails and their open and click times. If those times are close together or simultaneous, they are likely false positives from bot clicks.
Look closer at the IP addresses, the “user agent” operating systems, and browser combinations associated with clicks to understand what systems and devices are making them.
If they’re not from your organizational environment, the clicks likely originated with your security solutions or tools belonging to your providers (cloud services, Microsoft, etc.).
Finally, talk to your users. Ask for employee feedback on test results. If multiple employees fail a test but claim they didn’t click any links, it’s worth investigating further.
How to Prevent Phishing False Positives
Start by taking an inventory of all the software, security products, and services you use in your environment. Check the documentation and see if scanning, analysis, or probing tools are used.
If they do, look for a way to deactivate these capabilities for specific IP addresses and/or domains to ensure your simulated phishing emails can pass through for testing purposes.
For instance, if your email security solution has an allowlisting feature, you can set it up so that bots can’t scan or click on links from simulated phishing websites.
Make sure you run a test campaign before you start the live simulation to check whether your current configuration generates false positives. If it does, adjust your allowlisting and filtering settings to ensure accurate results.
Tell participants to report phishing emails through an approved mechanism rather than their email provider’s default reporting function.
Maximize Your Phishing Simulation's Accuracy & Impact
Phishing false positives are inconvenient but easy to mitigate with some preparation. Knowing what tools in your environment use bot clicks enables you to start allowlisting the links to keep your data accurate.
Remember that the goal of your phishing simulation is to gather data on how well your employees are detecting phishing emails. It helps identify which employees need extra support to combat the latest threats.
Can your users detect and avoid a phishing email?
Find out and get a firsthand look at what phishing simulations are about with a free demo.