PREP0004601 GenAI Robotics Researcher

GenAI Robotics Researcher

Project Description:

This position involves providing software and web development support for evaluation activities in the Artificial Intelligence Group (774.02) at NIST. Specifically, the work will support NIST GenAI evaluation series (https://ai-challenges.nist.gov/genai).  

The candidate will assist in developing cutting-edge human assessment infrastructure and social cognition assessments that support evaluations in various modalities (such as text, voice, image, and video). The infrastructure includes a web platform, evaluation pipeline (backend/frontend), database schema, and other related tasks. The development includes datasets using frontier AI/LLM tools, conducting experiments, performing statistical analysis, and writing technical reports. The candidate will work alongside NIST researchers to advance measurement science through cutting-edge research and evaluations in the field of Generative AI.

Key Responsibilities: 

• Evaluate Agentic AI applications in robotics and drones within the manufacturing sector
• Analyze AI‑generated content by extracting linguistic, syntactic, semantic, and structural features, and using computational methods to compare LLM judgments with human perception and behavioral responses
• Engineer prompt pipelines to generate controlled AI-generated content across models and other experimental conditions
• Apply pre/post-processing techniques for adversarial attacks on signals from multiple modalities (Audio, Image, and Text)
• Apply signal analysis methods (information theory, topological data analysis, spectral methods) on high-dimensional representation spaces
• Upgrade, maintain, and develop NIST GenAI software and evaluation pipelines
• Develop baseline systems to benchmark AI models on synthetic content generation and detection
• Curate datasets for synthetic content detection, aligned with real-world forensic applications
• Manage data, such as querying and translating data formats/specifications

Required Qualifications: 

• Proficiency in shell scripts (Bash, etc.) and coding (e.g., Python, Java, JavaScript, R, C, and C++).
• Proficiency in database management (e.g., PostgreSQL, MongoDB)
• Background in robotics engineering or AI evaluations involving LLM as judge, study design, and data analysis
• Real-time code development using a distributed computing environment
• Experience in applying machine learning and deep learning methods to audio and image processing tasks
• Experience with Computer Vision and Array Signal Processing (adaptive and non-adaptive filtering)
• Experience with open-source and API-based AI models in the domain of synthetic data
• Experience in statistical analysis methods to test and compare AI models on forensic tasks
• Experience with signal processing techniques like Cosine, Fourier, and Wavelet transforms for feature extraction of digital and time-series data
• Experience using GPU-accelerated clusters and CUDA for deploying open-source models
• Experience using cloud computing (such as Azure and GCP) and integrating cloud-based APIs
• Mastery of the principles, practices, and methods in a specialty of a technical professional field (in science, AI/ML engineering, computer science, robotics engineering, information technology, or mathematics, or related field)

Desired Qualifications: 

• U.S. Citizen Preferred
• Previous Experience as a full-stack web developer with tools potentially including using Ruby on Rails: Rails5, Postgres, HTML/CSS, JavaScript
• Experience with Jupyter notebooks, R, Shiny, and interactive plots 
• Experience in cross-platform software development (Linux, Mac OS X, Windows) 
• Experience/interest in machine learning and AI test and evaluation

Other Details:

• Full-time: the participant is expected to work 40 hours a week
• Location: the participant will work at the NIST Gaithersburg Campus and telework
• Duration: this is expected to be a ten-month position. Extensions are sometimes granted depending on the availability of funds.