Why the next era of technology must put people before progress

This article has been reviewed by a Digital Journal editor and may be licensed for reuse.

Artificial intelligence is entering a more consequential phase, and the industry’s old obsession with how quickly models can improve is colliding with a harder measure: what happens when those systems fail. The Stanford AI Incident Database recorded 233 AI-related incidents in 2024, a 56.4% increase from the previous year, as AI moved deeper into products and decisions that affect everyday life.

The regulatory response is arriving alongside the technology. As of August 2026, the European Commission has begun enforcing obligations for general-purpose AI providers under the EU AI Act, including requirements around risk assessment and technical documentation for models with systemic risk. The shift puts a new question on the industry’s agenda: whether safety can remain a principle discussed after a product is built, rather than a condition of building it.

The pressure is particularly sharp because AI systems are becoming more capable while remaining imperfect. Even AI developers acknowledge that hallucination, a model confidently producing information that is not true, remains a persistent problem, with current training and evaluation methods capable of rewarding guesses rather than uncertainty. 

The industry therefore may face a peculiar engineering challenge where systems can become dramatically better at producing an answer without becoming equally reliable at knowing whether the answer should be trusted.

That exposes a weakness in the dominant technology playbook. Speed, scale and performance are relatively easy to quantify; human consequences are harder to fit into a dashboard. Yet algorithms decide what information receives attention, what users encounter and which behaviours are reinforced. If those outcomes are treated as secondary to engagement or growth, the technology is already making a value judgment before anyone calls it one.

Aditi Shrikumar, founder of technology collective The Mechanical Soul, has spent nearly two decades watching those decisions take shape inside the industry. She began studying physics and computer science in 2006, later earning a PhD in human-centred AI, before spending a decade working on a popular search engine tool and moving into her current work at a visual search engine company. Her position is that the industry has spent years developing technical systems without giving equal weight to the human consequences attached to them.

Technology, in her view, has historically struggled to develop a moral center. “The problem is that choosing to ignore it is the easier thing you can do with information that harms people,” Shrikumar says, pointing out situations where companies know a product can cause harm but weigh that against its technological benefits. She believes technologists are often trained to solve technical problems without being equally prepared to reckon with the moral consequences of those solutions. 

Engineers who once argued they were “in the business of helping people,” she says, found themselves testifying before regulators, defending why certain search results ranked above others. According to Shrikumar, the right response wasn’t defensiveness, but opening the books and inviting scrutiny in. 

She argues that values become meaningful in technology when they can be translated into standards, tested before release, and measured after deployment. In her view, human-centered AI also starts with where products are tested and which problems they are designed to solve. She argues that only seeking feedback from those who are unusually immersed in technology is leaving companies with an incomplete picture of how products behave in everyday life. “The other way to build human-centered AI is to actually go out in the field, to places, to people, just to focus on real actual problems and not made up ones,” she says.

Algorithms sit at the center of that responsibility. Shrikumar argues that mathematical systems are not neutral simply because their mechanics are technical. Product teams choose what to measure, what to optimize, and which outcomes matter. Her recommendation is to measure harm alongside success. She believes companies should track signals such as people reporting content or abandoning an experience, then ask whether those numbers rise or fall after a product change.

According to Shrikumar, technology has been algorithmically optimized for rage and division, amplifying dissent to maximize engagement. True innovation, she posits, lies in reversing this formula to program for social cohesion. Human communication begins with emotion, yet she believes early tech culture dismissed emotional intelligence as unquantifiable. 

She believes universal emotional states are simpler to decode than complex language. Teaching systems to detect despair or rage early, in her view, could turn technology from an amplifier of division into a tool for repair. Shrikumar explains, “Integrating the healing power of art therapy and beauty into our digital architecture can dismantle collective echo chambers and restore human connection.”

The same perspective informs her concern about AI reliability. Shrikumar argues that hallucination rates should be disclosed prominently, rather than buried beneath the excitement around generative systems. For her, reliability is a product characteristic users should be able to assess plainly.

Ultimately, Shrikumar advocates for a change in the industry’s time horizon. She posits that she would rather see tech industries discussing advancement seven generations from now than that of the next quarter. She’s arguing for AI to measure itself by the lives it improves rather than the metrics it inflates, and to treat human consequence as a design requirement. Through that transformation, AI can begin with the idea that progress has to remain accountable to the people living with it.