Hamburg’s Data Protection Authority Shows Its Work: Lessons From the LLMoin Chatbot Case Study

Most regulator publications about generative AI tell organizations what could go wrong. On November 5, 2026, Hamburg’s data protection authority did something less common: it published a step-by-step account of a large language model it had actually reviewed, approved and then switched on inside its own office. The Hamburg Commissioner for Data Protection and Freedom of Information (HmbBfDI) released a case study on LLMoin, the chatbot the City of Hamburg’s administration has run since December 2024, explaining which system-level safeguards made the deployment acceptable under the GDPR and how the authority now uses the same tool to help draft letters in complaint cases. The authority says the lessons carry over to private-sector controllers, which is why the document deserves a close read outside Germany.

What LLMoin is, and what Hamburg did not build

LLMoin is a chatbot tailored to public administration. According to the case study, the Senate Chancellery of the Free and Hanseatic City of Hamburg commissioned it a few months after ChatGPT appeared, and it is now available to all Hamburg authorities and, increasingly, to public bodies in several other German states. Each agency decides for itself whether and how to use it.

The most important design choice is what Hamburg left out. The city did not train its own model and does not fine-tune an existing one. LLMoin is a wrapper at the AI-system level that channels inputs and outputs to an OpenAI model hosted in a Microsoft Azure cloud under a data processing arrangement. The review was carried out while the system ran on GPT-4.1; the case study says a newer model, which it names GPT-5.6 Terra, is now in use, and that preparations are under way to run models hosted locally by the public IT provider Dataport. A further configuration planned for 2027 is meant to allow data with higher confidentiality requirements.

The interface offers four functions: summarizing texts, generating administrative-style documents, searching collections of documents supplied by the state or by the using authority, and free prompting in a chat window or through prepared templates. Within Hamburg, LLMoin is approved only for personal data with standard protection requirements, and use cases that would make it a high-risk AI system under the EU AI Act are excluded.

Why the review focused on the system, not the model

The legal backbone of the case study is the European Data Protection Board’s Opinion 28/2024 of December 18, 2024, on personal data in AI models. That opinion accepts that, depending on how training was done, personal data may be “absorbed” into model parameters, and it sets out scenarios for what happens when a model that may have been trained unlawfully is used by someone else. Readers who followed the Board’s wider work on anonymisation and AI web scraping will recognize the logic.

Hamburg’s practical move was to stop trying to prove the model was anonymous. The authority writes that such an assessment is hardly feasible without knowing the developer’s methods, so the pragmatic assumption is that the model is in all likelihood not anonymous. That places a deployer in the EDPB’s second scenario: it may use the model if it takes enough measures to prevent extraction of personal data that may have been unlawfully incorporated. The case study also states that the authority’s own 2024 discussion paper on personal data in LLMs, the so-called Hamburg Theses, is no longer authoritative for it now that the EDPB has spoken, even though both point toward system-level safeguards.

The document is candid about erasure. If personal data can be identified in parameters, Article 17 obligations would in principle reach the model, yet deleting data at parameter level is not currently feasible and machine unlearning remains immature. The authority notes the EDPB only asks providers to keep evaluating such techniques. It also cites the German regulators’ position that filtering an output is not the same as erasure under Article 17, a point that matters for anyone answering deletion requests about AI tools and one that echoes the European Board’s coordinated enforcement work on the right to erasure.

The safeguards Hamburg accepted as sufficient

The review concluded that the following system-level measures, taken together, were sufficient. No single control carried the decision; the authority stresses that measures must be chosen as a package and weighed against their cost.

1. Output filter. Suppresses or modifies responses classified as impermissible, including to honor objections under Article 21 GDPR and to block information known to be false or unlawfully represented.
2. Content filter. Classifies outputs into hate, sexuality, violence and self-harm and suppresses matching content, addressing misuse risks such as deepfakes, disinformation and fraud.
3. Provider-side protections. OpenAI’s post-training steps to reduce reproduction of training data, which the city relies on but does not control.
4. System prompt. Instructs the model not to make decisions and not to output personal data that the user did not supply. The authority calls this a contribution to the architecture, not a substitute for technical controls.
5. Retrieval and grounding. The search function uses retrieval-augmented generation over a vector database of material the authorities provide; elsewhere, users ground answers by pasting in the facts themselves. Retrieved data is easier to correct, delete and permission than anything in parameters.
6. Binding usage rules. City-wide guidelines prohibit high-protection data, automated decision-making and retrieving personal data from the model’s knowledge unless the person made it public. The authority says it would have preferred a complete ban on such retrieval.
7. Red teaming. The Senate Chancellery commissioned adversarial testing for prompt injection, filter bypass, regurgitation and training-data extraction, then implemented further measures.
8. Training through multipliers. A network of trained colleagues spreads knowledge about hallucinations, automation bias, permissible data handling and the duty to review outputs.
9. Usage-restricting functions. Three of the four functions rely on retrieval or grounding; only free prompting lacks that built-in mitigation.

For private-sector teams, that list maps neatly onto the documentation they already need before launch. It is effectively the control inventory an AI impact assessment should capture, with the useful addition that a regulator has now said in writing which combination it found adequate for one concrete system.

How a regulator uses a chatbot on complaints

The second half of the case study is the more unusual part. The authority says it received 4,220 complaints under Article 77 GDPR in 2025, a 62 percent rise on the previous year, without a matching increase in staff, and that 2026 is running at a similar rate of growth. Hamburg’s office is no stranger to heavy caseloads, having issued a series of midyear GDPR fines in past years, and it has now turned to AI assistance for correspondence.

It cites Article 6(1)(e) GDPR together with Hamburg and federal data protection statutes as the legal basis, reasoning that processing which makes task performance faster or more efficient can count as necessary. It then fences the tool tightly. Relying on the Court of Justice’s Schufa judgment of December 7, 2023, the authority accepts that AI-generated suggestions can amount to automated decisions if they play a decisive role in a later human assessment. Its internal rules therefore prohibit automated decision-making. Prompt templates are built so the system does not perform legal assessments; case officers must read the full file first, decide the outcome and key arguments themselves, and only then ask LLMoin to help structure and phrase the letter. If discretionary reasons are missing, the system asks the officer to supply them. That design reflects the concern behind protecting human judgment in automated workflows.

The rollout was phased. First the tool was used only without personal data, for tasks like drafting annual-report articles and summarizing anonymized court decisions. A test group then refined prompt templates on synthetic files and later on real case files, which the authority defends as permissible testing under CJEU case law and as consistent with Article 32(1)(d) GDPR’s demand for regular testing of safeguards. Findings fed the data protection impact assessment and a training campaign. Sensitive complaints, such as those involving psychotherapy practices or criminal proceedings, generally stay with humans, although the authority stresses that special-category data is an indicator of high protection needs rather than an automatic verdict.

Two smaller details are worth copying. The authority does not let LLMoin draft notices of appeal rights, because legal-remedy wording must match the legal department’s text word for word and even a precise prompt cannot guarantee that. And it tells data subjects in its privacy notice that it may use LLMoin in complaint handling, while concluding that it is not legally required to label each individual letter as AI-assisted.

What private-sector controllers can take from it

• Assume the model is not anonymous. Build your legal analysis around extraction controls rather than a vendor’s anonymity claim.
• Document layered controls. Filters, prompts, grounding, rules, testing and training should each appear in your DPIA with a stated purpose.
• Restrict functions where you can. Narrow features with retrieval or grounding are easier to defend than open chat.
• Keep humans deciding. Design templates so people supply conclusions and the model supplies wording.
• Classify data before it reaches the prompt. Hamburg inventoried its case types to give staff uniform criteria.
• Do not automate fixed legal text. Use static templates where exact wording matters.

The approach also complements supervisory experiments elsewhere, such as France’s PANAME project for auditing AI models, which looks at the model layer that Hamburg chose not to rely on. The primary sources are short enough to read in one sitting: the authority’s announcement in German and the English version of the LLMoin case study.

Frequently asked questions

Did the Hamburg authority certify LLMoin?

No. It concluded a supervisory review with a positive outcome and described the safeguards it found sufficient overall. That is a regulator’s assessment of one deployment, not a certification or a binding approval for other systems.

Does the case study say OpenAI models were trained lawfully?

No. It deliberately sidesteps that question, assumes the model is probably not anonymous, and asks whether the deployer’s controls make extraction of personal data unlikely.

Can a company copy the safeguard list and be compliant?

Not automatically. The authority says measures must be tailored to the use case and the impact on affected people. The list is a strong starting inventory, not a checklist that settles the GDPR analysis.

Must every AI-assisted letter be labeled?

The authority concluded that it does not need to label each letter, provided its privacy notice explains that the tool may be used. Other contexts, including AI Act transparency duties, may lead to different answers.

Hamburg’s real contribution is not a new rule but a worked example: a regulator showing its own homework, including the parts it would still like to tighten. For compliance teams tired of abstract AI guidance, that may be the most useful kind of document a supervisor can publish.