The article examines whether contemporary large language models (LLMs) can assist with parliamentary committee scrutiny of legislation.
“Parliamentary committee scrutiny plays an important role in ensuring that proposed legislation and newly made regulations are subject to democratic oversight. However, this is labour-intensive work that is often undertaken under tight timeframes,” explains Professor Neudorf.
"This study examined whether AI could make that oversight more effective. While there were some promising results, the tests also revealed significant limitations, particularly where the work required consistent judgement and an understanding of institutional practice.”
The study tested several leading LLMs against real scrutiny tasks drawn from the work of two Senate committees: the Scrutiny of Bills Committee and the Scrutiny of Delegated Legislation Committee.
The findings suggest that, while current models can produce outputs that resemble committee analysis in some contexts, significant limitations remain, particularly in relation to threshold judgment, consistency and alignment with institutional practice.
