virtual vocationsNew York, NY
Research Scientist, AI Benchmarks
Research Scientist, AI Benchmarks
Computer and Information Research ScientistsData ScientistsMarket Research Analysts and Marketing Specialists
Other Scientific and Technical Consulting ServicesAll Other Professional, Scientific, and Technical ServicesResearch and Development in the Physical, Engineering, and Life Sciences
Apply for this role (opens in a new tab)To support the development of AI benchmarks, the full-time remote Research Scientist, AI Benchmarks will design public benchmarks for image, video, and document models, build private test sets from real data, and run data-value experiments to measure the impact of Kled data on model performance.
Key responsibilities:
Design and ship public benchmarks for various AI models
Build private test sets using real, opt-in data
Run experiments to evaluate the value of Kled data in training modelsRequired qualifications 5-7+ years of ML research or research engineering experience
Proven experience creating benchmarks or evaluations used by others
Strong hands-on experience with training and fine-tuning image or video models using PyTorch
Deep understanding of experimental design and statistics
Experience running end-to-end experiments from GPUs to final reporting
Also on the board Same function, level within a rung
Level
Senior
Location
New York, NY
Occupation
Computer and Information Research Scientists
Industry
Other Scientific and Technical Consulting Services
Posted
3 days ago