_Part 11 of 12 in the Cyber security for the small business series._
Artificial intelligence is transforming every industry, and cyber crime is no exception. AI is not creating entirely new categories of attack, but it is making existing attacks faster, cheaper, more convincing, and harder to detect. At the same time, it is changing what your own staff might inadvertently share with services outside the firm. This month covers both sides.
AI-enhanced phishing
Traditional phishing emails often gave themselves away through poor grammar, awkward phrasing, or generic content. AI language models have largely eliminated this tell. Attackers can now generate phishing emails that are grammatically perfect, contextually appropriate, and personalised at scale.
An AI-generated phishing email can reference your industry, use your company's style of communication, and create plausible pretexts that would have required significant manual effort in the past. The volume of high-quality phishing emails is increasing because the marginal cost of producing each one has plummeted.
The defences remain the same — verification procedures, reporting culture, technical email filtering, all of which we covered in May. But the threshold of suspicion needs to be higher. "It was well written, so I assumed it was genuine" is no longer a safe assumption. The well-written email is now the default, not the exception.
The practical implication: do not train your team to look for spelling mistakes. Train them to look for unexpected requests and unusual senders and anything that wants money or credentials, regardless of how polished the email looks.
Deepfake voice and video
AI can now generate convincing fake audio of real people's voices based on relatively short samples. There are already documented cases of this being used for business fraud, including a reported $25 million loss at a Hong Kong firm via a deepfake video call. The technology is now in commodity tooling, which means it is available to anyone with motivation.
This has direct implications for business fraud:
CEO fraud calls. An attacker uses a deepfake voice to impersonate a senior executive, calling the finance team to authorise an urgent payment. The voice sounds authentic because it was generated from clips of the executive speaking at a conference, in a podcast, or in a public video.
Verification bypass. Some businesses use voice recognition or video calls as part of their verification processes. AI-generated voice and video can potentially defeat these measures.
The defence is procedural, the same as for business email compromise. Establish clear authorisation processes that do not rely solely on recognising someone's voice or face:
- Multi-person approval for significant payments.
- Callback procedures using pre-agreed numbers, not numbers given to you during the call.
- Code words or challenge-response protocols agreed in advance between people who will speak to each other in financial conversations.
The May rule — any change to payment details must be verified by a phone call to a number we already had — handles most of this. AI did not change the rule. AI made the rule more important.
Your staff and public AI tools
The other side of the AI coin is your own team's use of AI tools. Services like ChatGPT, Claude, Gemini, and Copilot are genuinely useful for business tasks. They also carry data risks that many businesses have not considered.
Data input risks. When staff paste customer data, financial information, internal documents, or proprietary business information into public AI tools, that data is being shared with a third-party service. Depending on the tool's terms of service, it may be used to train future models or stored in ways you cannot control. The default consumer-tier products typically include data in training; the paid business tiers typically do not. Both terms change regularly, so do not assume.
Confidentiality. Sensitive contract terms, pricing strategies, employee data, and client information should never be entered into public AI tools unless you have a specific business agreement in place addressing data handling. If you have a regulated obligation — under UK GDPR, under professional codes — that obligation does not pause when you open ChatGPT.
Accuracy. AI tools can produce confident, authoritative-sounding text that is factually wrong. If staff use AI to draft client communications, financial summaries, or legal documents without careful review, the business may inadvertently send inaccurate information. Sounds plausible is not the same as is correct.
A simple AI usage policy
You do not need a lengthy policy document. A single side of A4 will suffice:
Do use AI tools for general research, brainstorming, drafting generic content, learning new topics, and rewording text.
Do not paste customer personal data, financial records, employee information, contracts, supplier pricing, or any confidential business information into public AI tools.
Always review AI-generated content for accuracy before sending it to clients or relying on it for business decisions. You are accountable for what leaves your name, not the tool.
Ask if you are unsure whether something is appropriate to share with an AI tool. The asking is itself the control; there is no embarrassment in checking.
This policy should be communicated to all staff and revisited as AI tools and their terms of service evolve. Keep it on one page. Pin it to the wall.
What the next year of AI looks like (probably)
A short paragraph of honest speculation. The phishing emails will continue to improve. The deepfakes will get cheaper and more convincing. The line between human-written and machine-written will become unreadable for most business communications. The defences that work — procedure, verification, phone calls to known numbers, hardware authentication — will continue to work, because they do not depend on telling humans and machines apart. The defences that depend on can you spot the AI? will not work. Invest in the first set.
What November looks like
Two short pieces of work:
Write the one-page AI policy. Five bullet points are plenty. Email it to the team. Pin it where they will see it.
Add a question to your monthly team conversation: has anyone seen something this month that felt AI-generated? The conversation itself does most of the work.
Next month
December: the final post in the series. The law, the insurance, the incident response plan, and the culture that holds it all together. The year-end consolidation.