Attorney AI Trainer- $140/hour

Location: U.S.-based experts only Engagement: Part-time, 1099 project-based expert evaluation work

Work Type: Remote

Project Summary: This project supports the development of an AI benchmarking framework designed to measure how effectively artificial intelligence models perform real-world, economically valuable knowledge work. The benchmark evaluates AI systems against tasks commonly performed by legal professionals in business environments. Contributors will create expert benchmark outputs, define evaluation criteria, and assess AI-generated work products across a range of professional scenarios. The goal is to establish rigorous, expert-driven standards for measuring AI performance on complex business and regulatory workflows. Consultant Engagement Terms This is a project-based consultant role. Consultants will be paid on a per-project basis; hourly rates are estimates based on anticipated completion time. Consultants control their own schedule, provide their own tools, and may simultaneously provide services to other vendors/employers (subject to those vendors’ allowances).

Responsibilities: Contributors will: Create benchmark answers for realistic legal scenarios Develop scoring rubrics defining high-quality legal work product Evaluate AI-generated legal analyses and written deliverables Identify legal inaccuracies, unsupported conclusions, and material omissions Assess legal reasoning, risk identification, and practical applicability Provide written justifications for scores and evaluation decisions Participate in calibration and quality review exercises Expected Outcomes High-quality benchmark answers reflecting professional legal practices Clear, defensible evaluation rubrics aligned with real-world business requirements Consistent and reliable assessment of AI-generated work products Identification of strengths, weaknesses, and risk areas in model performance Expert evaluations that support rigorous benchmarking of AI systems on economically valuable knowledge work Documentation that enables repeatable and scalable model evaluation

Qualifications: Active U.S. law license in good standing Minimum 3+ years of professional legal experience Strong legal research, analysis, and written communication skills Ability to assess legal arguments for accuracy, completeness, and sound reasoning Experience reviewing contracts, policies, memoranda, legal correspondence, regulatory guidance, or legal analyses Strong attention to detail and professional judgmentU.S.-based