It’s bizarre to me that there are still people claiming that LLMs are not reasoning or are not creative when by any objective measure they are obviously creative reasoners! By objective measure I mean a test that evaluates creativity and reasoning by evaluating outputs not by idle philosophical speculation that rules AIs out by definition. Here’s a good paper, Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers, which illustrates one such test. The authors asked top researchers in the field of natural language processing to propose research ideas which were then presented in a standardized format to a ratings panel of other NLP experts. The AI created ideas were judged more creative than the human ideas.
Now one might argue that the humans weren’t giving their best ideas–some data in the paper suggests they were giving ideas at the median of those for top researchers–and humans might also be looking for ideas that were perhaps easier to get funding precisely because they were less creative but more doable. Either way, however, the AIs are coming up with good ideas that could usefully supplement human generated ideas.
The post LLMs are Creative Reasoners appeared first on Marginal REVOLUTION.
You Might Also Like
History of Economics Summer Camp
About Me Name: Greg Mankiw Location: United States I am the Robert M. Beren Professor of Economics at Harvard University....
Summer Camp for Economic Theorists and Computer Scientists
Name: Greg Mankiw Location: United States I am the Robert M. Beren Professor of Economics at Harvard University. I use...
Greg Mankiw’s Blog: It’s all about shelter
Click on image to enlarge.This figure shows that the PPI for final demand tracks the CPI less shelter very closely....
I talk with Scott Cunningham
Name: Greg Mankiw Location: United States I am the Robert M. Beren Professor of Economics at Harvard University. I use...