Psychometrician Researcher

Position Overview

Role: Psychometrician (Independent Contractor)

Organization: SchoolOpsAI  Inc.

Reports To:  Chief Executive Officer in close collaboration with the Chief Technology Officer 

Engagement Type: 1099 independent contractor, hourly, part-time

Hours: Up to 10-20 hours per week (variable by milestone; invoiced monthly)

Term: ~12 months from execution date,

Location: Remote (U.S.); occasional virtual syncs with product team 

SchoolOpsAI is an AI-powered decision support platform for K–12 school leaders that integrates fragmented student data into equity-centered, curriculum-aligned recommendations. A priority of our current development is  to build the statistical and psychometric framework underlying our AI agents' recommendations — ensuring they are accurate, unbiased, and defensible. We are seeking a Psychometrician to lead the measurement science behind the platform, including the technical work of comparing and reconciling student proficiency scores across different assessment instruments that our partners use.

Key Responsibilities

• Design psychometric calibration algorithms that account for the differential accuracy of commonly used literacy and math assessments, including documented over- and under-identification of difficulties among students of color and multilingual learners.
• Lead cross-assessment proficiency comparability work: design and validate the statistical methodology (e.g., IRT-based linking/scaling, equipercentile equating, concordance/crosswalk tables, standard-setting) for comparing proficiency scores across different assessments (e.g., DIBELS, iReady, Amira, SIPPS, benchmark and HQIM-embedded formative assessments) and specify how these comparisons should be modeled and displayed within the platform.
• Design multi-source triangulation logic that statistically weights and reconciles data from multiple instruments when Tier 1, Tier 2, and Tier 3 signals disagree for a given student.
• Design equity-adjusted recommendation logic that illuminates or corrects for known assessment biases so that students of color, multilingual learners, and students with IEPs are not disproportionately over- or under-referred for services.
• Partner with the CTO  to translate psychometric specifications into production code; review implementation for fidelity to the underlying measurement model.
• Conduct validity, reliability, and fairness/bias studies (e.g., differential item/test functioning, subgroup calibration checks) on platform recommendations using de-identified synthetic data.
• Serve as the primary technical expert on measurement methodology.
• Author internal technical documentation and contribute to field-facing publications/white papers on SchoolOpsAI's psychometric framework, in line with the company's research and field-building goals.
• Advise leadership and the customer success manager on appropriate interpretation and limitations of assessment data and cross-assessment comparisons in leader- and educator-facing materials.
• Maintain FERPA and SOC-compliant data handling practices at all times, including background check,  secure storage, access controls, and de-identification protocols.

One-Year Scope of Work

The scope below is organized in quarterly milestones aligned to SchoolOpsAI's research  roadmap. Specific tasks may be re-sequenced by the CTO based on district onboarding and research-partner timelines, provided total scope and hours remain within the agreed budget.

Period

Focus & Milestones

Deliverables / Evidence

Q1(Months 1–3)

Audit current assessment instruments and scoring scales in use across partner districts. Draft the psychometric methodology for cross-assessment comparability (linking/equating approach) and the initial bias-calibration framework, in consultation with published research (e.g., Truckenmiller et al.) 

Assessment inventory with technical characteristics; draft technical methodology memo for cross-assessment comparability and calibration approach.

Q2(Months 4–6)

Validate linking/equating models on historical de-identified data for priority assessment pairs. Specify equity-adjustment logic for the recommendation engine. Begin joint QA with product team on staging implementation.

Validated concordance/linking tables for priority assessment pairs; equity-adjustment specification document; staging-environment QA sign-off.

Q3(Months 7–9)

Run subgroup fairness/bias analyses on production recommendations for partner schools. Support field study design and data requirements. Refine calibration models based on findings.

Subgroup fairness/bias report; research study protocol contribution; refined calibration parameters.

Q4(Months 10–12)

Finalize and document the full psychometric framework (calibration, triangulation, equity adjustment, cross-assessment comparability) for scaling. Co-author methodology write-up intended for field-facing publication.

Final, versioned psychometric framework documentation; field-facing methodology paper/white paper draft; renewal or scale-up recommendation.

Requirements & Qualifications

Required

• Ph.D. or Master's degree in Psychometrics, Educational Measurement, Quantitative Psychology, or a closely related field (or equivalent applied experience).
• Demonstrated expertise in test equating, linking, or scaling methods used to compare scores across different assessment instruments (e.g., IRT-based linking, equipercentile equating, concordance/crosswalk development).
• Strong background in classical test theory and/or item response theory, and in fairness/bias analysis methods (e.g., differential item functioning, subgroup calibration).
• Working knowledge of K–12 literacy and/or math assessment instruments (e.g., DIBELS, iReady, SIPPS, curriculum-embedded formative assessments) and MTSS/RTI frameworks.
• Comfort working in or alongside a statistical/programming environment (R or Python) to validate models built by the data science team; does not need to write production code.
• Experience translating psychometric findings into plain-language guidance for non-technical audiences (school leaders, educators, funders).
• Demonstrated commitment to educational equity and familiarity with research on assessment bias affecting students of color, multilingual learners, and students with disabilities.

Preferred

• Prior experience in an EdTech, assessment-publisher, or research-institution setting.
• Experience collaborating with research partners and contributing to peer-reviewed publications.
• Familiarity with FERPA-regulated student data environments and AI/ML-based tools.