Keeping client details safe with AI

Picture a Thursday afternoon. A customer has sent a long, upset email about a bill. Your office manager wants to get the reply right, so they copy the whole thing into a free AI chat and ask for a calm answer. The email has the customer's name, address, phone number and a line about their health. The reply comes back in seconds and it's a good one. Nobody stops to think about where that email just went.
That's a made-up Thursday, but it's an easy mistake to make. And it's getting easier. In September Anthropic launched new Claude for Small Business workflows that connect to tools you may already use, like Xero, MYOB, Gmail and Google Drive. So AI can now get to client details directly, as well as whatever someone pastes in.
My short answer is to treat client details in AI the way you'd treat them anywhere else. Know where they're going, leave out what the job doesn't need, and use a business version of the tool with the right settings. Then write it down so your whole team does it the same way.
Is it OK to put client details into AI?
The Office of the Australian Information Commissioner (OAIC) has guidance on using AI products like chatbots. Its first point is that privacy obligations apply to any personal information you put into an AI system. They also apply to what comes out, when that has personal information in it.
The OAIC recommends, as best practice, that businesses don't put personal information, and particularly sensitive information, into publicly available AI chatbots and tools. Health details count as sensitive information, so the email in that made-up Thursday is exactly the kind it means.
Not every small business is covered by the Privacy Act. The OAIC says most small businesses aren't, but some are, whatever their turnover. Health service providers are one example. Even if the Act doesn't cover you, your clients still trust you with their details. I'd work to the OAIC's advice either way.
Does AI learn from what you type?
It depends on the tool and the plan, so check yours.
With Claude, Anthropic says it doesn't train on your business data by default on its Team and Enterprise plans. The exception is when you choose to share data with Anthropic, like sending feedback with the thumbs up or thumbs down button.
On the personal plans, which are Free, Pro and Max, there's a setting called "Help Improve our AI models". You'll find it in Settings, under Privacy. Have a look and set it the way you want before anyone uses it for work.
Other AI tools have their own rules, and those rules change. Read the tool's own privacy page rather than a forum post about it, and check again every so often. The OAIC says the checks you do on an AI product shouldn't be a "set and forget" job.
What about the files and accounts it can reach?
Once AI is connected to your email, files or accounts, it's worth asking what it can see.
Some tools stick to the permissions you've already set. Anthropic says that with Claude for Small Business, if a staff member can't see something in QuickBooks or Google Drive today, they can't see it through Claude either.
So the tidy-up starts in your own systems. If everyone in the office can open every client folder now, connecting AI makes those folders quicker to search too. Sort out who can see what before you connect anything.
What to check before client details go near AI
Which plan are you on? Make sure staff use the business account for client work, so the business plan's defaults apply.
Is the training setting where you want it? Check it on every personal account your team uses, including the app on their phones.
Does the job need the details at all? Plenty of jobs don't. You can ask for a reply to "a customer who's upset about a bill" without a name, address or phone number.
Who can see what? Check folder and account permissions first.
Have you written it down? A short AI use policy tells staff what they can put into AI and what gets checked before anything goes out.
Does your privacy policy mention AI? The OAIC says businesses should update their privacy policies and notices with clear information about how they use AI.
Would you leave a client's file on the front counter?
Most offices have a habit nobody needs reminding about. You don't leave a client's file open on the front counter, and you don't read someone's account details out loud in the waiting room. Client details in AI are the same habit in a new place. Once your team sees it that way, most of the rest follows.
Sources
- Office of the Australian Information Commissioner, "Guidance on privacy and the use of commercially available AI products", published 21 October 2024: https://www.oaic.gov.au/privacy/privacy-guidance-for-organisations-and-government-agencies/guidance-on-privacy-and-the-use-of-commercially-available-ai-products
- Office of the Australian Information Commissioner, "Small business", published 10 March 2023: https://www.oaic.gov.au/privacy/privacy-guidance-for-organisations-and-government-agencies/organisations/small-business
- Anthropic, "Claude for Small Business launches new workflows, integrations, and training programs", 15 September 2026: https://claude.com/resources/articles/claude-for-small-business-launches-new-workflows-integrations-and-training-programs
- Anthropic Privacy Center, "Is my data used for model training?", 18 August 2026: https://privacy.claude.com/en/articles/7996868
- Anthropic Privacy Center, "How do I change my model improvement privacy settings?", 3 August 2026: https://privacy.claude.com/en/articles/12109829
Newsletter
A short email from me now and then about AI changes that matter to small businesses.
Sign up to the newsletterWant to know where AI fits in your business? Book a call.