OMAFA: AI & XR Evaluation

Designing research ethics protocols and multi-dimensional evaluation frameworks for AI-driven XR agricultural decision-support systems.

Context

Research & Experiment Design

Role

Experiment Designer

Year

2025

Industry

AgTech, Human-Computer Interaction (HCI), AI Ethics, Sustainability

Project Link: Adaptive Context Environments (ACE) Lab - OMAFA

---

The Idea

The OMAFRA (now OMAFA) project is a Government of Canada and Ontario Ministry of Agriculture, Food and Agribusiness ‍initiative. It aimed to bridge the gap between advanced emerging technologies (XR and AI) and the practical needs of the agricultural sector. As these prototypes move from the lab to the field, there is a critical need to ensure they are ethically sound, user-centric, and technically effective. The goal was to develop a rigorous methodological foundation that allows researchers to evaluate how XR/AI tools can assist farmers, policymakers, and educators in resource optimization and sustainable decision-making.

OMAFRA: Ontario Ministry of Agriculture, Food and Rural Affairs.
OMAFA: Ontario Ministry of Agriculture, Food and Agribusiness.

Contributors: Dr.Alexis Morris, Aaditya Vaze, Wentian Zhu, Samarth Reddy, Jie Guan

Development

As the Experiment Designer, I was responsible for the structural integrity of the human-participant research phase, focusing on two primary pillars:

1. Research Ethics & Compliance (REB):

Authored the comprehensive Application for the Research Ethics Board (REB).
Navigated the complexities of testing immersive XR and AI technologies with human subjects, addressing data privacy, psychological safety (e.g., motion sickness in XR), and the ethical implications of AI-driven recommendations.
Established protocols for informed consent and data management to ensure the project met institutional and federal research standards.

2. Descriptive Evaluation Framework:I formulated a robust framework to evaluate XR/AI prototypes across 25+ distinct dimensions. This framework was categorized into four key evaluation pillars:

Core UX & Usability: Leveraging the Technology Acceptance Model (TAM) to measure Perceived Usefulness and Ease of Use, alongside Understandability, Engagement, and Usability for diverse stakeholders.
AI Transparency & Trust: Developed criteria to assess Explainability (why the AI behaved a certain way), Transparency of Accuracy, Bias Mitigation, and Ease of Correction when the system errs.
Human-AI Interaction Dynamics: Evaluated the system's Context Awareness, Proactive Assistance, and Multi-Modal Interaction (voice/gestures) to ensure the AI anticipates user needs without being intrusive.
XR Immersion & Real-World Impact: Defined metrics for Immersion Levels, Presence, and Real-World Alignment to ensure the virtual environment translates effectively to physical farming contexts.
Agricultural Sustainability: Focused on Decision Support and Sustainability Impact, measuring how the system contributes to resource optimization and emissions reduction.

Reflection

This project highlighted the necessity of "Human-in-the-Loop" design for specialized industries like agriculture. By formalizing the evaluation framework before the full-scale pilot, we ensured that technical development remained aligned with the actual cognitive and operational needs of farmers.

‍