The Electronic Privacy Information Center (EPIC) recently published a comprehensive report, Assessing the Assessments: Maximizing the Effectiveness of Algorithmic & Privacy Risk Assessments, which dives into the critical need for robust risk assessment frameworks. Supported by the Rose Foundation, this report stems from a multi-year initiative to ensure that companies collecting and processing personal data are held accountable for the risks they create. By examining the California Privacy Protection Agency’s (CPPA) ongoing efforts to shape risk assessment rules under the California Consumer Privacy Act (CCPA), the report underscores the urgent need for transparency and accountability in today’s data-driven world.
The Hidden Dangers of Unchecked AI Systems
Every day, companies quietly gather vast amounts of personal information, feeding it into algorithms that influence decisions in areas like job hiring, healthcare access, housing approvals, and even law enforcement actions. These automated decision systems often operate behind a veil of secrecy, leaving consumers unaware of how their data is used, whether the outcomes are fair, or how to contest decisions that may be flawed. For example, behavioral advertising tracks your every click to tailor ads, while surveillance pricing adjusts costs based on your data profile often without your knowledge. These practices don’t just invade privacy; they can lead to discrimination, financial harm, and eroded trust. There was a recent NY Times piece about major banks like Bank of America & Chase shutting down accounts without evidence or a reason with the assumption that these are automated decision making being used and causing users harm.
A recent Consumer Reports survey found that 83% of Americans want to know exactly what data an algorithm uses when it decides something as critical as whether they get a job interview. This demand for clarity reflects a broader truth: opaque systems breed distrust. When businesses hide behind the complexity of their algorithms, they dodge accountability, leaving consumers vulnerable to errors or biases embedded in these systems.
“Automated decision systems can give companies a false sheen of objectivity,” said Mayu Tobin-Miyaji, EPIC Law Fellow and lead author of the report. “But without rigorous risk assessments, there’s no way to know if these systems are fair or even accurate. Consumers deserve to see the math behind decisions that shape their lives.”
Building a Better Risk Assessment Framework
EPIC’s report doesn’t just highlight problems—it proposes solutions. It outlines the key ingredients of an effective risk assessment framework, one that forces companies to be upfront about their data practices and the potential harms they pose. These assessments should act like a spotlight, illuminating how personal information is collected, processed, and used in automated systems. They should also provide clear pathways for consumers to challenge decisions and demand corrections when things go wrong.
The report also takes a close look at California’s proposed regulations under the CCPA, which aim to set rules for automated decision systems and risk assessments. While these rules could be a game-changer, there’s a catch: industry pressures and political influences threaten to water them down. “California has a chance to lead the way on consumer protections, but only if the CPPA stands firm against efforts to weaken these rules,” said John Davisson, EPIC’s Senior Counsel and Director of Litigation. “Robust risk assessments aren’t just a legal checkbox—they’re a critical shield against the harms of unchecked data practices.”
Why Businesses Should Care
Even without strict mandates, conducting thorough risk assessments is a smart move for companies. By proactively identifying and addressing privacy risks, businesses can build trust with consumers, avoid costly legal battles, and stay ahead of regulatory changes. In a world where data breaches and algorithmic missteps regularly make headlines, transparency isn’t just ethical—it’s good business.
EPIC’s report serves as a wake-up call. As AI and automated systems become more entrenched in our daily lives, the need for strong, consumer-focused risk assessments grows. Without them, the gap between corporate data practices and consumer rights will only widen, leaving individuals exposed to harm and businesses free from accountability. The time to act is now—before the next algorithm makes a decision that changes someone’s life without explanation.
If you’re business is dealing with AI and automated decision making you will want to book a demo right away to get your business compliant with our data privacy software solutions.
