PrivacyPoint
Vibe coding and agentic AI are rapidly reshaping how organizations build software and automate decision‑making. They offer faster delivery, cheaper experimentation, and ambitious new services. They also introduce complex, cross‑system flows of personal data that sit squarely within the sights of California's CCPA Article 10 risk‑assessment obligations and the EU AI Act's risk‑management and impact‑assessment requirements.
For chief privacy officers (CPO) and privacy counsel, these technologies are no longer abstract. They are being piloted in marketing, product, human resources, and operations, often using the same underlying patterns: large language models, retrieval-augmented generation (RAG), vector databases, and increasingly autonomous artificial intelligence agents. The compliance question is not simply "Is this allowed?" It is "How do we evidence that we understand and have mitigated the risks in a way that satisfies California regulators and, where relevant, European Union (EU) supervisory authorities enforcing the Artificial Intelligence Act?"
When large language models enter a workflow, the risk profile changes. A large language model (LLM) is an AI system that takes text as input and generates text as output; teams connect to it via APIs (application programming interfaces) to sort documents, update records, or draft content. In typical hosted deployments, the instructions people type (prompts) and the underlying source data are sent to outside service providers, and the model's responses are then saved back into your own systems. Some enterprise deployments mitigate this by using zero-retention endpoints, on-prem models, or private cloud instances, but the default hosted model is what most teams encounter first.
Under the California Privacy Protection Agency's (CPPA) finalized Article 10 regulations (adopted by the CPPA Board on July 24, 2025 and approved by the Office of Administrative Law on September 22, 2025), a business must conduct a formal risk assessment when using personal information in ways that create significant privacy risk; for example, when automated decisionmaking technology (ADMT) helps make important decisions about people, build profiles of them, or handle sensitive categories of data. Risk assessments conducted in 2026 and 2027 must be submitted to the CPPA by April 1, 2028, with annual submissions thereafter. Many LLM‑powered workflows fall squarely within this scope.
In the European Union, the EU AI Act takes a risk-based approach, treating certain uses of AI as "high‑risk." The Act entered into force on August 1, 2024, and becomes fully applicable on August 2, 2026, with prohibited practices and AI literacy obligations already in force (since February 2, 2025), general-purpose AI model obligations effective from August 2, 2025, and high-risk AI systems embedded in regulated products having an extended transition period until August 2, 2027. High-risk categories under Annex III include distinct domains: employment (recruitment, candidate screening, performance evaluation); access to essential services (including creditworthiness evaluation and risk assessment and pricing for life and health insurance); education and vocational training (as its own separate category covering admissions, evaluations, and learning pathway decisions); and others. Once an AI system is high‑risk, the organization must follow strict rules on risk management, data quality, documentation, transparency, human oversight, and, for certain deployers, a Fundamental Rights Impact Assessment (FRIA), to show that the system is safe and rights‑respecting.
The picture changes once LLMs enter the workflow. When teams use them to classify documents, enrich records, or generate drafts, prompts, and source materials are often sent to external processors, and outputs are written back into internal systems. Under California law, Article 10 risk assessments must address processing activities that present significant risk to consumers' privacy, including automated decision-making, profiling, and the use of sensitive personal information. Section 7152 of the regulations lays out the detailed content requirements, and for many AI‑enabled workflows, that threshold will be crossed. Under the EU AI Act, systems that influence access to services, employment, credit, or other protected domains may fall into the "high‑risk" category, triggering mandatory risk management, data governance, and, for public-sector deployers and deployers of credit-scoring or insurance-pricing systems, FRIA duties.
Agentic AI further raises the stakes. Instead of a single, well‑bounded model call, agents plan, reason, and act across tools and systems in pursuit of a goal. An agent configured to "research and qualify new leads," for example, might search the web, query internal CRMs, enrich profiles with third‑party data, and trigger outreach sequences. From a California perspective, this can constitute automated profiling, targeted advertising, and systematic monitoring, all wrapped in a single capability. Under the EU AI Act, depending on context, such systems can qualify as high‑risk AI or, if mis‑scoped, approach prohibited-practice territory, where they use subliminal, manipulative, or deceptive techniques that cause significant harm to individuals' decision-making.
Technical details that might once have been delegated entirely to engineering now become core risk‑assessment inputs. Context windows limit how much combined input and output a model can handle, but they also shape the volume and granularity of personal data transferred to an external provider in any given interaction. When staff pastes entire PDFs, long email chains, or data exports into prompts, they effectively initiate batched transfers of personal data whose content is only loosely captured in logs.
RAG adds another dimension. By breaking documents into chunks, converting them into embeddings, and storing them in vector databases, organizations create new, highly queryable stores of transformed personal data. For Article 10 assessments, these stores must be treated as separate processing operations: they have their own purposes (semantic search and context retrieval), their own risks (reconstruction, unintended inference, cross‑dataset linkage), and their own retention and access‑control considerations. Under the EU AI Act, they fall within the data‑governance obligations for training, validation, and testing data, and they directly affect the quality and bias profile of high‑risk systems.
Prompting practices should also be explicitly included in the scope for risk assessments. Modern prompt frameworks encourage users to set context, define roles, and specify detailed tasks and outputs. In practice, that means individuals are drafting, on the fly, the functional equivalent of processing instructions: which data are relevant, how they should be interpreted, and which inferences are sought. Article 10 expects organizations to assess and document the nature, scope, context, and purposes of high‑risk processing, including the logic involved and potential impacts on consumers' rights and expectations. EU AI Act requirements similarly stress transparency, human oversight, robustness, and the prevention of unintended harmful outcomes.
If prompts are where staff encodes context, logic, and roles, then they must be addressed in both regimes' assessments. That includes examining whether prompts systematically encourage over‑collection, whether they nudge users to include sensitive categories, and whether they invite inferences that go beyond disclosed purposes. It also means reviewing system and agent prompts as part of the "logic" description that both California and EU AI Act‑style assessments demand.
Vibe coding multiplies these considerations on the development side. When AI‑assisted coding tools generate most of the code for applications that process personal data, risk assessments cannot treat the underlying logic as a black box. Article 10 calls for analysis of the benefits and risks of processing, including reasonably foreseeable adverse consequences. If consent flows, user dashboards, or automated decision rules are largely AI‑generated, CPOs must be able to explain and evidence how those components were tested, reviewed, and verified for compliance with data‑minimization, purpose limitation, and rights‑enablement obligations.
The EU AI Act goes further for high‑risk systems, requiring a risk‑management system (Article 9), data‑governance standards (Article 10), technical documentation (Article 11), logging (Article 19), transparency measures, human oversight, robustness, and cybersecurity. Where vibe‑coded applications form part of a high‑risk AI system, the development pipeline itself becomes part of the risk‑management story. That encompasses how requirements are expressed to AI coding tools, how outputs are reviewed, which tests are run, and how changes are version‑controlled.
Agentic AI architectures add still more layers that risk assessments must cover. One useful way to frame the architecture is as a layered stack spanning infrastructure, multi-agent communication, protocols, tooling, cognition, memory, applications, and governance, which maps directly to the questions California and the EU AI Act expect organizations to answer. Where is personal data stored and processed at each layer? Which entities act as controllers, joint controllers, or processors? How is human oversight exercised over agents' planning and actions? What logging exists to support post‑hoc analysis of a decision that adversely affected an individual?
For California's Article 10, these details inform the assessment of "reasonably foreseeable risks" to consumers' privacy and the evaluation of safeguards, including technical and organizational measures. For the EU AI Act, they form part of the required risk‑management process, which must be systematic, documented, and continuous across the lifecycle of high‑risk AI systems. In both cases, the non‑deterministic nature of agentic AI and the opacity of some LLM behaviors do not exempt organizations from explaining, in structured terms, how they mitigate those challenges.
This is where PrivacyPoint's work is deliberately focused. We help CPOs and privacy counsel translate the abstract requirements of California's Article 10 and the EU AI Act into concrete, AI‑aware risk‑assessment practices. That includes mapping RAG pipelines and vector stores into data inventories and RoPAs, structuring prompt and agent‑configuration reviews as repeatable assessment artifacts, and embedding privacy and security gates into vibe‑coding workflows so that AI‑generated code touching personal data is always subject to documented review. We also design logging and observability approaches that give you the evidence you need when a regulator asks, "How did this system reach that decision for this person?"
Used thoughtfully, vibe coding and agentic AI can become tools that help operationalize risk‑assessment obligations rather than undermine them. PrivacyPoint partners with organizations to build this alignment: designing architectures, policies, and workflows so that the same mechanisms that deliver speed and innovation also produce the documentation, controls, and assurance that California and the EU AI Act expect.