BlogAI adoption
Personal AI accounts at work: what your AI use policy should say
If a staff member pastes a referral letter into Gemini under a personal Google account, Google's Gemini privacy pages say that by default the chat can be used to train Google's AI models and may be read by a human reviewer. Google keeps reviewed chats for up to three years, according to its privacy notice of October 2026, and paid plans come under the same notice. So your AI use policy should name the accounts staff may use for work, as well as the tools.
3 years
the most Google's privacy notice says it keeps personal-account Gemini chats that human reviewers have read, even after the user deletes their Gemini activity
Google says Gemini chats from personal Google accounts that human reviewers have read are kept for up to three years, and are not deleted when the user deletes their Gemini activity.
Source: Google, Gemini Apps Privacy Hub.
A post on Hacker News on November 4, 2026 pointed this out for people who pay for Gemini, and Google's own pages bear it out. Our read is that the account decides which rules apply more than the app does. The same Gemini app runs under the Google Terms of Service in a personal account and under the organization's Workspace agreement in a work account on the right plan. Only that work account can be covered by a business associate agreement, the contract HIPAA requires before a vendor handles patient information for you.
The clauses below extend our short AI use policy template for a medical practice and fit a hospital or health system too. They are not legal advice, so have your compliance lead or counsel review your version.
What happens to a chat in a personal Gemini account?
By default, Google keeps the chat, uses it to improve its products and may have a person read it. The Gemini Apps Privacy Hub says that when a setting called Keep Activity is on, Google uses a person's activity to "provide, develop, and improve its services (including training generative AI models)." Keep Activity is on by default for anyone 18 or older, according to Google's help page on managing Gemini activity.
A subset of chats is read by human reviewers, including trained reviewers from Google's service providers. Reviewed chats are kept for up to three years, and deleting your Gemini activity does not delete them. The notice mentions a paid Google AI plan only as subscription information Google collects, and it sets no separate rules on training or review for people who pay.
The notice also asks users directly: "Please don't enter confidential information that you wouldn't want a reviewer to see or Google to use to improve our services, including machine-learning technologies." The hub says the Google Terms of Service apply to Gemini Apps, and those terms describe "the relationship between you and Google" and say they "don't create any legal rights for other people or organizations." The same terms say that to use Google's services on behalf of an organization, "an authorized representative of that organization must agree to these terms."
Our read is that Google leaves the decision about confidential information to the person typing, under terms no one agreed to on your organization's behalf. A practice's or hospital's policy should make that decision for everyone at work, because the confidential information there includes patients' records.
Does turning the setting off solve it?
It cuts what Google keeps and how it uses it, but a personal account still has no business associate agreement with your organization. With Keep Activity off, Google says future chats won't be used to train its AI models unless the person sends feedback, and temporary chats are not used for training either. Those chats still stay with the account for 72 hours, and Google still uses them to respond and to protect Google, its users and the public, "including with help from human reviewers." The notice adds that Gemini's settings "don't control processing of your chats to create anonymized data to improve Google services."
Staff also have a reason to turn it back on. With Keep Activity off, the privacy hub says connected apps "like Google Workspace are unavailable," so Gemini can't reach the person's Gmail or Drive, which Google's connected apps page lists as part of Google Workspace. In a work account, Google says the organization's Workspace administrator controls the Gemini activity settings, including Keep Activity. Our read is that a setting each person controls in a personal account is a weak safeguard, because the organization can neither see it nor lock it.
How is a work account different?
On the right plan, a work account runs under the organization's Workspace agreement with Google instead of the terms personal accounts use. Google's page on using Gemini with a work or school account says that for Workspace editions that include the Gemini app as a core service, "Your chats and uploaded files in Gemini Apps won't be reviewed by human reviewers or otherwise used to improve generative AI models," and the Workspace terms apply. Google's HIPAA page for Google Cloud and Google Workspace lists the Gemini app, excluding Gemini in Chrome, among the Workspace services its business associate agreement covers.
On the wrong plan, a work account gets the same terms as a personal account. For editions that offer the Gemini app only as an additional service, such as Workspace Individual, the same page says chats "may be reviewed by human reviewers and used to improve Google's products," under the Google Terms of Service and the Gemini privacy notice that govern personal accounts. The admin page for the Gemini app marks HIPAA support only for the core-service editions. Even on those, the Workspace privacy hub says Gemini in Chrome and Gemini Notebook are not covered by Google's business associate agreement.
Microsoft also separates personal and work accounts for Copilot. Its privacy article for individuals applies "only when you sign in with a Microsoft account, such as a personal email address," and says that use is governed by the Microsoft Services Agreement and the Copilot Terms of Use. Its page for work or school accounts says Copilot Chat provides "enterprise data protection" when you sign in with that account and that prompts and responses "aren't used to train foundation models," though the web searches it runs for a prompt go to Bing under an agreement "between each user and Microsoft."
Our read is that an approved list naming only apps leaves the real question open. Name the account and the plan for each approved tool, and check the agreement against the exact product, as what to settle before a chat assistant reads the patient record explains.
What does HIPAA require if patient information went into a personal account?
If the privacy rule did not permit the disclosure, HIPAA presumes a breach unless one of three exclusions in the rule's definition applies or the organization can show a low probability that the information was compromised. A covered practice or hospital may give patient information to a vendor that handles it on the organization's behalf only if it first gets satisfactory assurance that the vendor will appropriately safeguard it, documented in a written contract or other written agreement that meets HIPAA's requirements for business associate contracts (45 CFR 164.502(e) and 164.504(e)). A personal Google account comes with no such contract, because its terms are between the person and Google.
To show that low probability, the organization assesses at least four factors: what information was involved, who received it, whether anyone acquired or viewed it, and how far the risk has been mitigated (45 CFR 164.402). Our read is that the vendor's own terms bear on three of them: who received the information, whether anyone viewed it, and how far deleting the chat can reduce the risk, since Google says deleting activity does not delete a reviewed chat. So the account and its settings belong in the report to your compliance lead.
If it is a breach of unsecured patient information, the organization must notify each affected patient without unreasonable delay and no later than 60 calendar days after discovering it (45 CFR 164.404). It must also notify the Secretary of Health and Human Services, at the same time as the patients when 500 or more people are affected, or within 60 days after the end of the calendar year when fewer are (45 CFR 164.408). When a breach involves more than 500 residents of one state or jurisdiction, it must also notify prominent media outlets serving that state or jurisdiction, within the same 60 days (45 CFR 164.406).
The 60 days can start before leaders hear about it, because HIPAA treats a breach as discovered on the first day it is known, or would have been known with reasonable diligence, to any workforce member or agent of the organization other than the person who caused it. So the template below asks anyone who sees patient information go into the wrong account to report it the same day.
HIPAA also requires reasonable safeguards against unintentional disclosures as well as intentional ones, and appropriate sanctions for staff who break the organization's privacy policies. When a disclosure breaks those policies, the organization must reduce any harmful effect it knows of, as far as it practicably can (45 CFR 164.530). Your compliance lead or counsel decides whether a given case is a breach.
What should the policy say about accounts and settings?
Add these four short sections to your AI use policy, after the approved tools list if you use our template. Replace each bracket and delete any line that does not apply.
Accounts
- Use AI for work only while signed in to an account [Organization name] provides: [the work accounts and plan, for example our Google Workspace accounts on (edition)].
- Don't use a personal account for work, free or paid, even with its privacy settings changed.
- Before you paste anything, check that the account shown in the app is your work account.
Settings
- We use only plans whose terms say our chats and files are not used to train the vendor's AI models or read by its reviewers: [plan names].
- [Name and role] manages the AI settings on our work accounts, including chat history and which apps and files the AI can reach. Please don't change them.
- Features our business associate agreement does not cover stay off for anyone who handles patient information: [features].
What stays out of every other tool
- Patient information of any kind, including a note with the name removed.
- Staff records, contract terms, passwords and login codes.
- Connections to work email, files or calendars.
Checks and reports
- [Name] checks each vendor's privacy page, our plan and our business associate agreement at every policy review and whenever a vendor changes its terms.
- If patient information goes into a personal account or any tool not approved for it, whether you did it or saw it happen, tell [name] the same day, with the account used and its settings.
Fig. 1
Four sections to add to your AI use policy on accounts and settings
What a practice's or health system's AI use policy says about the accounts staff use for AI, their settings, what stays out and how it is checked
If your hospital or health system already licenses a work AI tool, the policy's main job is to send staff to it. A practice without one decides first whether to pay for a work plan a business associate agreement covers, or to keep AI to work with no patient information.
How do you find out which accounts staff use now?
Ask them, in the AI tool inventory that is the first of the five steps for putting our template in place. Add two questions: which account each person signs in with for work, and whether they have connected work email or files to an AI tool. Then check your Workspace edition against the editions table on Google's work account page, and your business associate agreement against the products staff use.
Check again at each review, because these pages change. Google updated its Gemini privacy hub on October 26, 2026 and its list of Workspace services under its business associate agreement on October 28, and Microsoft's privacy article for personal Copilot accounts covers only the updated Copilot app, available as of August 18, 2026. Our read is that any vendor setting your policy names should carry the date it was checked.
Personal accounts are hard to inventory, because the organization never set them up. The AI Governance Starter starts with an AI inventory that includes an anonymous staff survey asking what people use on their own, then writes a plain-language AI use policy, a risk tier for each tool and a vendor and BAA checklist. It takes three weeks and costs $8,500, fixed.
Sources
- Google, Gemini Apps Privacy Hub, last updated October 26, 2026.
- Google, Manage & delete your activity in Gemini Apps.
- Google, Use Gemini Apps with a work or school Google Account.
- Google, About personalization with Connected Apps.
- Google, Google Terms of Service, effective July 30, 2026.
- Google Workspace, Generative AI in Google Workspace Privacy Hub, last updated October 6, 2026.
- Google Workspace, Turn the Gemini App on or off.
- Google Cloud, HIPAA compliance on Google Cloud and Google Workspace.
- Microsoft, Microsoft Copilot for individuals, your data, privacy, and responsible AI.
- Microsoft, Data protection when using Microsoft Copilot Chat for work or school.
- Hacker News, post "Gemini trains on chat history, even for subscribers," November 4, 2026.
- 45 CFR 164.502, disclosures to business associates.
- 45 CFR 164.504, business associate contracts.
- 45 CFR 164.402, definition of breach.
- 45 CFR 164.404, notification to individuals.
- 45 CFR 164.406, notification to the media.
- 45 CFR 164.408, notification to the Secretary.
- 45 CFR 164.530, safeguards, sanctions and mitigation.
Related service
Related articles
- Before a hospital pilots an AI agent: what to check firstNovember 13, 2026, 4-minute read
- Healthcare AI in Q3 2026: what changed for healthcare leadersNovember 9, 2026, 6-minute read
More on AI adoption: 31 charts and 30 sourced facts.
Insights by email
Does this look like your organization?
Book a 20-minute call with Tyler to talk through your version of this problem and what fixing it would cost.
Prefer to skip the call? Get a written proposal