_edited.jpg)
Where Assessment, Humans, and AI Converge
About Us
BroadMetrics
BroadMetrics is a consulting firm offering services in psychometrics and the applied AI sciences. ​We specialize in measurement science practices to support the assessment of individuals and evaluation of AI. We are committed to ethical and responsible statistical, measurement, and AI practices.
Our expertise includes the development of advanced assessment tools and the implementation of robust evaluation frameworks, ensuring precise assessments, principled data-driven solutions, and informed decision-making. We take pride in our adherence to principles of validity, reliability, and fairness, and are dedicated to advancing the field of psychometrics through innovative research and practical applications.
​
We recognize the unique utility of psychometric methods in evaluating AI, especially in high-stakes contexts. We leverage our psychometric sub-specialty in rating scales to develop human rating systems to evaluate AI. We also lean on this expertise to offer systems to collect human annotations to generate training data.
​
BroadMetrics was founded by Jodi Casabianca, a psychometrician and researcher with over 14 years of operational experience in large-scale assessments. We rely on a network of collaborative expert partners to offer comprehensive solutions. Together, our experts bring clarity to complex psychometric situations, offering guidance and coaching as well as implementation consultation.
Our Mission
Applying responsible psychometric methods to ensure assessments of people and evaluations of artificial intelligence lead to valid, reliable, and fair actions.
At BroadMetrics we are motivated to improve outcomes and lived experiences for all people. Given our expertise, the best way to do that is to apply methodologies to shepherd research and policies to effect change for individuals and groups, worldwide. Our work spans different domains: education, healthcare, skills, and sales. The end user in all of these areas is YOU and your family. Let's work together to ensure that the assessments and capabilities you are using are doing good and not harm. To these ends, we are dedicated to providing top-notch analytic and assessment services to help you make data-driven decisions. Our team works to ensure that our clients receive accurate and actionable insights. Let us guide you towards success with our expertise and commitment to quality.
Core Values
What do we care about???
QUALITY Without quality, what do you have? We are hyperaware of the importance of quality assessments and analyses. If the method is incorrect, or the analysis has one error, the impact of that could be very consequential.
RESEARCH We use research-backed methods to create assessment solutions. We conduct research to derive and understand methods to improve assessments.
COLLABORATION We value working together with clients and partners to come up with solutions that are responsible and effective.
TRANSPARENCY We believe it is crucially important for our clients to truly understand our methods, results, and implications. We work hard to bring meaning to the situation.
KNOWLEDGE SHARING In addition to building solutions and capacity, we believe in sharing and disseminating our research and effective approaches so that others in the scientific community can benefit.
Jodi Casabianca
Founder and Chief Scientist
Dr. Jodi Casabianca is Founder and Chief Scientist at BroadMetrics, a consultancy offering psychometrics and assessment support for researchers and organizations. With a rare blend of academic and industry experience, Dr. Casabianca excels in conducting research to address practical challenges, and in performing operational analyses grounded in rigorous, research-backed decision-making.
Dr. Casabianca founded BroadMetrics to help organizations, schools, and researchers develop and evaluate their assessment systems, with a focus on promoting and showcasing best practices. Previously, she worked as a senior measurement scientist at Educational Testing Service (ETS), where she contributed to psychometric efforts for various assessments, including TOEFL, GRE, Praxis, and supported K-12 clients. She has built a reputation as an expert in the scoring of open-ended tasks. Much of her work has focused on the design, implementation, and evaluation of observational and constructed response scoring systems in educational assessment.
​She has led research initiatives on a broad scale. At ETS she led research teams on various projects on AI in assessment and item response theory modeling to account for rater effects. She served as an assistant professor in the Quantitative Methods program within the Department of Educational Psychology at the University of Texas at Austin and was Associate Director of the Pearson Center for Applied Psychometric Research.​ She received a National Science Foundation grant to investigate hierarchical item response theory models for multi-rater designs. Dr. Casabianca has published research in several top-tier journals such as the Journal of Educational and Behavioral Statistics, the Journal of Educational Measurement, and Educational and Psychological Measurement, and she has presented her findings at numerous international and national conferences.
Dr. Casabianca earned a Ph.D. in Quantitative Psychology and Psychometrics from Fordham University, with additional degrees in Mathematical and Applied Statistics and Psychology from Rutgers University. With much of her experience within education and psychological testing, she is fully grounded in the Standards for Educational and Psychological and Educational Testing as governed by the American Psychological Association, American Educational Research Association, and the National Council on Measurement in Education. However she has analytics experience in other areas including healthcare and sales.