{"id":1003,"date":"2026-03-13T22:30:59","date_gmt":"2026-03-13T21:30:59","guid":{"rendered":"http:\/\/www.ingmar-hagemann.de\/wordpress\/?p=1003"},"modified":"2026-03-13T22:37:22","modified_gmt":"2026-03-13T21:37:22","slug":"halterman-keith-codebook-llms-evaluating-llms-as-measurement-tools-for-political-science-concepts","status":"publish","type":"post","link":"http:\/\/www.ingmar-hagemann.de\/wordpress\/halterman-keith-codebook-llms-evaluating-llms-as-measurement-tools-for-political-science-concepts\/","title":{"rendered":"Halterman ; Keith, Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts"},"content":{"rendered":"\n<div class=\"wp-block-group is-layout-constrained wp-block-group-is-layout-constrained\">\n<h6 class=\"wp-block-heading\">Zentrale Argumentation \/ Zitate<\/h6>\n\n\n\n<p>Guter Einstieg in die Frage der Nutzung von LLMs f\u00fcr Codebook-geleitete Analysen. Es werden konkrete Vorschl\u00e4ge f\u00fcr die Gestaltung von Codebooks gemacht und Evaluations-Systematiken f\u00fcr die Bewertung unterschiedlicher LLMs entwickelt.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>However, using LLMs for codebook-based measurement is different than many other applications of LLMs: it requires the LLM to follow the lengthy and detailed operationalization of concepts in a codebook and to faithfully apply the provided coding criteria rather than solely relying on what it learned from pretraining. (22)<\/p>\n<\/blockquote>\n\n\n\n<div style=\"height:51px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h6 class=\"wp-block-heading\">Ausf\u00fchrliche Quellenangabe<\/h6>\n\n\n\n<pre class=\"wp-block-code\"><code>Halterman A, Keith KA. Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts. in: Political Analysis. Published online 2025:1-17. doi:10.1017\/pan.2025.10017<\/code><\/pre>\n\n\n\n<div style=\"height:51px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h6 class=\"wp-block-heading\">Schlagw\u00f6rter<\/h6>\n\n\n<div class=\"taxonomy-post_tag wp-block-post-terms\"><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/tag\/ai-ki\/\" rel=\"tag\">AI \/ KI<\/a><span class=\"wp-block-post-terms__separator\">, <\/span><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/tag\/ai-ki-und-diskursanalyse\/\" rel=\"tag\">AI \/ KI und Diskursanalyse<\/a><span class=\"wp-block-post-terms__separator\">, <\/span><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/tag\/codebooks\/\" rel=\"tag\">Codebooks<\/a><span class=\"wp-block-post-terms__separator\">, <\/span><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/tag\/diskursanalyse\/\" rel=\"tag\">Diskursanalyse<\/a><span class=\"wp-block-post-terms__separator\">, <\/span><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/tag\/evaluation-von-llms\/\" rel=\"tag\">Evaluation von LLMs<\/a><span class=\"wp-block-post-terms__separator\">, <\/span><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/tag\/qualitative-methoden\/\" rel=\"tag\">Qualitative Methoden<\/a><\/div>\n\n\n<div style=\"height:51px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h6 class=\"wp-block-heading\">Einsortiert in folgende Lekt\u00fcrelisten<\/h6>\n\n\n<div class=\"taxonomy-category wp-block-post-terms\"><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/category\/ai-ki-und-diskursanalyse\/\" rel=\"tag\">AI \/ KI und Diskursanalyse<\/a><span class=\"wp-block-post-terms__separator\">, <\/span><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/category\/diskursanalyse\/\" rel=\"tag\">Diskursanalyse<\/a><span class=\"wp-block-post-terms__separator\">, <\/span><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/category\/methoden\/\" rel=\"tag\">Methoden<\/a><span class=\"wp-block-post-terms__separator\">, <\/span><a href=\"http:\/\/www.ingmar-hagemann.de\/wordpress\/category\/qualitative-methoden\/\" rel=\"tag\">Qualitative Methoden<\/a><\/div><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Zentrale Argumentation \/ Zitate Guter Einstieg in die Frage der Nutzung von LLMs f\u00fcr Codebook-geleitete Analysen. Es werden konkrete Vorschl\u00e4ge f\u00fcr die Gestaltung von Codebooks gemacht und Evaluations-Systematiken f\u00fcr die Bewertung unterschiedlicher LLMs entwickelt. However, using LLMs for codebook-based measurement is different than many other applications of LLMs: it requires the LLM to follow the [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[284,238,285,248],"tags":[286,287,289,240,288,256],"class_list":["post-1003","post","type-post","status-publish","format-standard","hentry","category-ai-ki-und-diskursanalyse","category-diskursanalyse","category-methoden","category-qualitative-methoden","tag-ai-ki","tag-ai-ki-und-diskursanalyse","tag-codebooks","tag-diskursanalyse","tag-evaluation-von-llms","tag-qualitative-methoden"],"_links":{"self":[{"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/posts\/1003","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/comments?post=1003"}],"version-history":[{"count":2,"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/posts\/1003\/revisions"}],"predecessor-version":[{"id":1007,"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/posts\/1003\/revisions\/1007"}],"wp:attachment":[{"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/media?parent=1003"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/categories?post=1003"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.ingmar-hagemann.de\/wordpress\/wp-json\/wp\/v2\/tags?post=1003"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}