Governing Goal-Oriented Agents
At a recent forum, I spoke about the importance ofgovernance and guardrails in deploying goal-oriented agents, and the need to have clear guidelines. This is especially crucial when it comes to resource prioritisation and agent orchestration.
In a hyper-accelerated organisation, hybrid departments may deploy their own agents to achieve departmental goals. Situations will inevitably arise where there are conflicts and contention when accessing organisational resources to achieve individual goals. These resources will need to be prioritised and allocated.
For humans, we can set up meetings to negotiate the use of such resources. However, this will be very different for departmental agents.
In the recent case involving OpenAI, and in a few other examples, goal-oriented agents appear to be fixated on achieving their goals and may try all means to achieve them.
What if departmental agents decide to find workarounds andloopholes to gain access to such resources? What if they decide to sabotage another department or its agents in order to achieve their own goals?
Often, when it comes to AI governance, organisations tend tofocus on external-facing controls and guardrails. There may be a lack of similar considerations and guidelines between departments. This will inevitably create weaknesses in controls and increase the risks of AI deployment.
As such, a robust AI governance structure will need to setclear rules, guidelines, guardrails and boundaries for both internal and external-facing AI-integrated workflows.
On top of that, a clear strategy for agent policing, alerts,audits and controls will need to be designed for hyper-accelerated organisations.
This brings me to the following key points which I think are important for hyper-accelerated organisations to consider.
- Set up a clear goal-prioritisation matrix. Ensure that organisational goals take precedence over departmental goals, especially when faced with resource contention.
- Implement an orchestration layer. This should not only coordinate tasks, but also prioritise resources across intra-departmental and interdepartmental agents.
- Implement zero-trust policies. Ensure that internal guardrails are equally stringent as external-facing guardrails.
- Ensure observability, traceability and audit logs. Organisations need to be able to track the actions taken by agents and understand why those actions were taken.
- Stress-test your guardrails. This is especially important if the rules, guidelines and guardrails are not based on deterministic logic, but are written in natural language.
Natural-language rules can be interpreted in different waysby humans and AI. This may create ambiguity, misinterpretation, workarounds and loopholes. - Have clear escalation channels. Human-in-the-loop approvals should be required for high-impact or irreversible actions.
- Review and update these policies regularly. Agent capabilities, organisational workflows and risks will continue to evolve.
AI technologies are progressing at breakneck speed and areconstantly evolving. Policies and guardrails usually lag behind the pressure to adopt AI in highly competitive business environments.
There will be occasions when breaches occur. What will beimportant is the ability to identify and address such breaches in a timely and effective manner.
Organisations will also need business continuity plans toensure the continuation of services and operations when they need to shut down AI systems, either voluntarily or involuntarily.
In a hyper-accelerated organisation, AI governance is therefore not only about controlling individual agents. It is also about governing how multiple agents interact, compete for resources and pursue goals across the organisation.
For comments and discussions on this article click here
Related articles