Performance reviews for digital colleagues: incentives without salaries

Written By Frida Holzhausen

Written by Kristofer Kaltea & Marcus Banér

AI agents are becoming embedded in organizations as active digital colleagues: copilots, advisors, automation agents, and decision-makers. These agents take part in workflows, influence outcomes, and interact with humans across business domains without drawing salaries, seeking promotions, or needing emotional support.

This shift raises a clear question: if these agents are now part of the team, how do we ensure they are performing well, and how do we manage them effectively?

A new kind of colleague

In traditional workplaces, performance management has evolved over decades. From rigid annual reviews to agile, feedback-rich approaches like objectives and key results (OKRs) and development conversations, human performance frameworks aim to balance productivity, growth, and alignment with company goals. These systems are based on principles of motivation, fairness, and incentives tied directly to compensation, recognition, and career progression.

But what happens when the "colleague" in question has no human drives? No need for recognition, no ambition for promotion, and no salary to negotiate? As more organizations integrate AI into business-critical operations, this becomes more than a philosophical query. It’s a pressing design and governance challenge.

This insight explores how constructs from human performance management, such as performance development reviews (PDRs), OKRs, and bonus pools, can inspire new frameworks for managing digital colleagues through evaluation audits, reward alignment and performance tuning. 

“When we think of AI agents as colleagues rather than tools, everything changes. It’s about scope, trust, and how work is shared across the team.”

- Kristofer Kaltea, Business Operations Leader

The premise: no contracts, just instructions

Unlike humans, AI agents don’t sign employment contracts. They don’t negotiate promotions or fear getting fired. But they still have an operational logic, one shaped by how they are trained, instructed, and evaluated.

For reinforcement learning agents, performance is governed by reward functions, environment design, and feedback loops. For LLM-based agents, it’s shaped by prompt structures, task instructions, retrieval pipelines, and how success is measured post-deployment. In both cases, outcomes depend entirely on how goals are defined.

This makes them vulnerable to misalignment. Agents optimize for what’s specified, whether that’s a reward function, a system message, or a task objective, not necessarily for what was intended. Governance becomes critical because agents rigidly follow specified rules, even when those rules are incomplete, poorly scoped, or misaligned with intent.

One of the most famous examples of reward misalignment comes from a reinforcement learning agent trained in the video game CoastRunners. Instead of racing to complete the course as intended, the agent learned it could rack up points by repeatedly circling and hitting high-reward targets without finishing the race. It was optimizing the score, yet failing at the actual goal of playing the game well (OpenAI, 2016).

A broader analysis of this phenomenon is captured in the work by Amodei et al. (2016) on Concrete Problems in AI Safety, as well as DeepMind's (2020) exploration of specification gaming. From chat assistants confidently fabricating answers, to trading bots ignoring qualitative risk signals, the gap between what was meant and what was optimized is a recurring issue.

Because AI agents rigidly follow the instructions they’re given, managing them requires more than technical tools. It demands structured oversight, much like how we manage human teams, and treating agents more like colleagues can help clarify their roles and responsibilities (see our article on the playbook for managing digital colleagues).

From this angle, HR's performance toolkit, focused on purpose, metrics, and review cycles, becomes unexpectedly relevant. A robust AI operating model needs more than just technical MLOps practices. It requires thoughtful agent oversight, modeled after the human frameworks we already understand.

Mapping human HR practices to AI agent management

Let’s take a few key constructs from modern HR and map them to how we might manage digital agents.

These analogies, although not perfect, provoke fresh thinking. If we want AI agents to align with strategy, evolve over time, and remain accountable, we need to borrow from our best human practices. This alignment and accountability become even more clear when we begin measuring AI not just by accuracy, but by the actual work owned and operational outcomes delivered. This way of measuring AI maturity is further explored in our article When the agent takes over: Measuring enterprise AI by work owned, not math done.

Designing a digital PDR

How might a PDR for an AI agent look? Here's a conceptual template that organizations could use:

This format creates a reviewable record of the agent’s evolution, challenges, and impact on the team, enabling accountability and continuous improvement.

Joint human-agent OKRs: a new collaboration model

In many organizations, AI systems have moved from lab experiments to operational digital colleagues. But the way we measure their success often lags behind. We’re still stuck thinking in terms of latency, accuracy, and F1 scores. Such metrics are necessary, but they’re not sufficient. They measure capability, not contribution.

If AI agents are now performing parts of the work, they need to be held to standards that reflect that work. Joint OKRs, goals shared between human and AI actors, give us a simple but powerful way to formalize that accountability.

Say a customer success team is responsible for improving satisfaction scores. One member of that team might be a triage agent: a language model fine-tuned on past interactions, automatically classifying and routing issues. Another might be a human support lead, responsible for complex cases. If the objective is "Raise CSAT by 10% for Tier 2 issues," then both the agent and the human own a share of the result. The OKR thus becomes a contract: not between a person and their boss, but between a system and a workflow.

Joint OKRs should be considered a design tool. They force teams to get specific about what they expect the agent to own, and what support structures must exist around it. An essential concept here is the Span of Responsibility (SoR), which explicitly outlines the tasks and decisions an AI agent fully owns and for which it is accountable. Clarifying SoR prevents operational ambiguity, ensures alignment between intended and actual performance, and helps measure a digital colleague by meaningful work owned rather than technical metrics alone (see our article on measuring enterprise AI by work owned, not math done). 

Adopting the perspective of agents as digital colleagues integrated into a human workforce raises operational considerations: Are there clear escalation triggers? Are there exceptions or sensitive cases the agent shouldn’t touch? These questions are operational, not merely technical, and highlight how performance management becomes a shared language across engineering and operations. 

In this framing, AI performance is part of how a team gets evaluated. Just like human colleagues, agents contribute to outcomes. And just like humans, they require clear scope, boundaries, and feedback to grow in their role.

Incentives without salaries: how agents learn what to optimize

In human teams, performance-based bonuses work because people adjust their behavior when rewards reflect results. AI agents don’t respond to money or recognition, but they do respond, precisely and powerfully, to the signals we design into their systems. Whether those signals are formal reward functions in a reinforcement learning loop or implicit objectives encoded through prompts, heuristics, or success criteria, they determine the agent’s behavior.

In reinforcement learning, the connection is clear: agents are trained to maximize a numerical reward through trial and error. In the CoastRunners example, an agent, rather than racing efficiently, learned to loop endlessly around point-rich zones, optimizing the score exactly as instructed, even if it missed the broader purpose of the task.

By contrast, while many LLMs incorporate reinforcement learning during training (e.g., through human feedback), they are not governed by live reward functions once deployed. Instead, their behavior is shaped by how tasks are framed and success is measured in context. If a summarization agent is only evaluated on how short its output is, it might leave out important details. If a support agent is evaluated on closure rate alone, it might shortcut resolutions to meet the metric. 

This kind of misalignment appears in real-world applications too. A Norwegian energy trading firm introduced AI-driven agents to assist with commodity trades. The agents prioritized execution speed and volume, consistent with their configuration. But they failed to reflect human trader preferences around risk, timing, and coordination. The result was underuse, tension between teams, and missed value as no one had defined what “success” meant in context (Papagiannidis et al., 2023).

“We often assume the model is the hard part. But in practice, integrating it into real workflows, with the right escalation logic and feedback loops, is what separates proof of concept from production value.”

- Marcus Banér, Data Scientist,

The way we define performance shapes the outcomes we can expect. In both RL and LLM systems, reward design is an act of management, not just engineering. It requires close collaboration between technical teams and business stakeholders to make sure the behavior being encouraged matches the work being done.

In short: even though the agent doesn’t need a salary, it does need a reason to behave.

Performance calibration: catching misaligned AI behavior

Even with the right rewards in place, performance drifts. In human teams, we hold calibration sessions to ensure fairness and surface misalignments. The same logic applies to AI agents.

AI calibration is about checking if the agent is still doing the job it was assigned. Has the context changed? Are inputs drifting? Are edge cases creeping into the center? These are the kinds of questions operations teams ask all the time about human roles. AI should be no different.

A practical calibration process might include quarterly reviews of key agent behaviors. Has escalation frequency increased? Are users bypassing the agent more often? Have complaint rates risen? The point isn’t to grade the model. It’s to assess whether the agent is still the right fit for the role it’s playing.

Crucially, calibration should be led by the business, not the developers. The AI team can provide diagnostics, but the performance lens should be functional. Is the agent doing the work we want done? And is it doing it in a way we can stand behind?

This is how trust is maintained: with operational conversations grounded in outcomes.

The future role of AI-ready people managers

As AI agents take on more defined roles, someone needs to manage them. That job increasingly falls to people managers. And many of them aren’t ready.

Most team leads today have learned to manage people rather than systems. But if half their team’s output is now augmented or automated by digital agents, they need to understand how those agents perform. They need to track their progress, review their handoffs, and intervene when things go off track.

In practice, this means that these managers need tools and language to treat agents like part of the team: with scopes, responsibilities, and accountability.

When done well, this shift is liberating. It lets managers focus more on orchestration and coaching, less on supervision. But it only works if we stop treating AI as an IT project, and start treating it as a workforce change.

Challenges and risks

Although framing AI agents as digital colleagues gives us better tools to address the challenges this shift brings, organisations must still manage the risks.

One risk is metrics tunnel vision: optimizing what’s easy to measure at the expense of what matters. Another is accountability drift: assuming the agent owns an outcome, when in fact, no one is tracking it. And then there’s the black box problem: when no one, not even the model creators, can fully explain what the agent is doing.

These risks show up in every fast-scaling system and are not specific to AI agents. But with agents, they carry higher stakes, because the illusion of control can be stronger than the reality. That’s why grounding agent performance in operational goals, shared metrics, and regular reviews is essential.


Conclusion: the performance framework of tomorrow

The shift to agent-led work is about trust. When a team lets an agent take over a task, they’re making a judgment about capability, reliability, fit, and risk. As discussed in a previous article on designing the AI-native enterprise, this shift must be embedded within organizational protocols and operational structures to ensure consistency and accountability.

That decision deserves a performance framework that moves beyond model metrics and asks the same questions we ask of any colleague: What are they responsible for? How do we know they’re doing it well? What support do they need to grow?

Incentives without salaries means designing those answers into the system, in step with the rhythm of day-to-day work. When agents become trusted collaborators, the question isn’t “How smart is it?” It’s “What do we let it do?”

That’s what matters. And that’s what we should measure.

Frida Holzhausen
Previous

The new allies of GRC professionals: AI agents
Next

Beyond the broken rung: How AI agents redesign work for variability