Asks: could LLMs be used to "create tools that sift and summarize scientific evidence for policymaking... [for] knowledge brokers providing presidents, prime ministers, civil servants and politicians with up-to-date information on how science and technology intersects with societal issues... [who must] nimbly navigate ... millions of scientific papers ... reports from advocacy organizations, industry and scientific academies... must work fast ... [but] Producing policy summaries in weeks, days or sometimes hours is a daunting task".
Obviously, there are quality concerns, so "US House of Representatives to impose limits on chatbot use in June" 2023, so a lot of work needs to go into guidelines and guardrails. The article explores two promising tasks: "synthesizing evidence and drafting briefing papers — and highlight areas needing closer attention".
"AI-based platforms should be able to ... free subject-matter experts to focus on more complex analytical aspects" by supporting two types of synthesis:
Early stage processes automisable by AI include "search, screening and data-extraction... especially useful in making sense of emerging domains [and] to detect emerging ‘clusters’ of research... Nonetheless, assessing data quality and drawing conclusions ... typically require human judgement."
These processes could also help "decision-making... creating possible options in a process known as solution scanning ... Advisers can then collate and synthesize the relevant evidence". AI can also help advisers overcome linguistic barriers.
Today Al "could be used to provide first drafts of discrete sections... plain-language summaries of technical information or complex legislation", although Elsevier's early experiment created bland text at the same (high) level of understanding as the papers it sourced - not that useful.
But tomorrow they might provide advice tailored to each policymaler, factoring in an MP's "political affiliation, voting record, educational ... background" and constituency (demographics, socio-economic information, and even "present content on science-informed issues in the voice of the policymaker" by "leverage the policymakers’ previous work as a training data... The level of technical detail might be dialled up or down by the reader themselves."
Developing this field properly will require a partnership: "technical know-how is likely to come from academia and technology companies, whereas demands for robust governance, transparency and accountability can only be met by governments... We still need old-school intelligence to make the most of the artificial kind. "
More Stuff I Like