Can LLMs Produce Original Astronomy Research in a Semester? A Graduate Class Experiment

This paper reports on a Fall 2025 graduate astronomy experiment where students used large language models to conduct original research, finding that while LLMs aided in drafting papers, they frequently failed to provide accurate insights, generate complex code, or avoid hallucinations, leading to mixed student perceptions regarding their overall value and impact on creativity.

The Gist

Imagine a group of graduate students in a university astronomy class. Their assignment is ambitious: by the end of the semester, they must write a brand-new scientific paper about galaxies that is good enough to be published in a real journal.

Usually, this is like trying to build a house from scratch without a blueprint, using only a hammer and a lot of patience. But for this specific semester, the professor gave them a super-powerful, all-knowing robot assistant (an AI called a Large Language Model, or LLM) and said, "Use this to help you build your house."

The paper you read is the students' report card on how well that robot actually helped. Here is the story of their experiment, told in simple terms.

The Setup: The "Super-Intern"

The students treated the AI like a super-fast, super-read intern. They asked it to do the heavy lifting: find old research papers, write computer code to analyze data, and even suggest what new discoveries they could make.

The Good News (The Magic Moments):

• The Speed Reader: If you asked a human to read 500 papers on galaxy bars, it would take weeks. The AI read them in seconds and gave a summary. It was like having a librarian who could instantly pull out the exact page you needed from a library of a million books.
• The Code Fixer: Sometimes, the students wrote a computer program that crashed. The AI could look at the error message and say, "Oh, you forgot a comma here," or "You used the wrong formula." It was like having a mechanic who could hear a car engine and instantly know which part was broken.
• The Brainstormer: When students were stuck and didn't know what to study next, the AI suggested ideas. It helped them narrow down a huge, scary topic into a manageable project.

The Bad News (The Hallucinations):
However, the AI wasn't perfect. In fact, it had some very dangerous habits:

• The Fake Librarian: About 20% of the time, the AI would confidently hand a student a "reference" to a scientific paper that didn't exist. It would give a fake title, a fake author, and a fake link. It was like a tour guide pointing to a building and saying, "That's the Museum of Art," when it was actually a gas station. The students had to check every single link manually.
• The Stubborn Chef: If a student told the AI, "This code is wrong," and the AI said, "No, it's right," and the student said, "No, look, it crashes," the AI would often double down and insist it was right. It was like a chef who keeps adding salt to a soup even after you tell him it's too salty, insisting, "No, it needs more salt."
• The Blind Navigator: The AI was great at saying, "Go to this website to get data," but terrible at actually going there. It couldn't log into the database, download the files, or understand the complex rules of the website. It was like a GPS that could tell you the address of a store but couldn't actually drive the car to get there.
• The Guessing Game: When asked for specific numbers (like the uncertainty in a measurement), the AI would sometimes just make up numbers if it didn't know the answer. It preferred to lie confidently rather than say, "I don't know."

The Verdict: Did it save time?

The answer is a split vote, like a coin toss.

• Team "Yes": About half the students said, "Yes, it saved me time." For them, the AI was a great tool to get started, to learn the basics of a new topic, and to fix small coding errors. It was like having a co-pilot for the boring parts of the flight.
• Team "No": The other half said, "No, it wasted my time." They spent hours fixing the AI's fake citations, debugging its broken code, and arguing with it when it was wrong. They felt that if they had just done the work themselves, they would have been faster and more accurate.

The Big Lesson: The "Human Touch"

The most important part of the paper isn't about the technology; it's about the students' feelings.

Many students worried that relying too much on the AI would make them lazy thinkers. They felt that if the AI does all the thinking, they lose the "spark" of discovery. One student said, "If the AI tells me what to do next, where is the fun in being a scientist?"

They realized that while the AI is a powerful tool, it lacks scientific taste. It doesn't have the intuition that comes from years of studying the universe. It can't tell the difference between a boring, solved problem and a thrilling, unsolved mystery.

The Conclusion

The students concluded that AI is like a very fast, very confident, but occasionally hallucinating intern.

• Do use it for: Summarizing old papers, fixing small typos in code, and brainstorming ideas.
• Don't use it for: Writing the final paper, finding real data, or making up facts.

The paper ends with a warning and a hope: As AI gets smarter, it will become a better intern. But until then, scientists must be the "managers" who check every single thing the intern does. If you don't check the work, the intern might build your house on a foundation of sand.

In short: The AI is a fantastic tool to speed up the work, but it cannot replace the human mind that asks the questions, checks the answers, and decides what is truly important.

Technical Summary

Technical Summary: "Can LLMs Produce Original Astronomy Research in a Semester? A Graduate Class Experiment"

1. Problem Statement
The paper investigates the efficacy of Large Language Models (LLMs) in facilitating original scientific research within an unfamiliar domain. Specifically, it addresses whether current-generation LLMs (as of Fall 2025) can assist graduate students in:

• Identifying novel, publishable research questions in astrophysics.
• Synthesizing literature and designing methodologies.
• Generating functional code for data analysis and simulations.
• Retrieving and processing data from astronomical archives.
• Producing draft manuscripts of sufficient quality for peer-reviewed submission.

The study highlights the tension between the potential for LLMs to accelerate research productivity and the risks of hallucination, lack of scientific "taste" (intuition), and the potential erosion of critical thinking and creativity in early-career researchers.

2. Methodology

• Participants: Seven first-year PhD students in the Astronomy and Astrophysics program at the University of Arizona. Most (5/7) had no prior research experience in galaxy formation/evolution.
• Setting: A graduate course (ASTR 540: Structure and Dynamics of Galaxies) during the Fall 2025 semester.
• Task: Students were required to produce a draft manuscript containing novel results on an unsolved galaxy-related problem, suitable for submission to a peer-reviewed journal.
• Tools: Students utilized various state-of-the-art LLMs, including OpenAI (ChatGPT-4o, 5, 5.2), Anthropic (Claude Sonnet 4, 4.5, Opus 4), and Google (Gemini Flash/Pro 2.5, 3).
• Usage Protocol: Students were encouraged to use LLMs for literature synthesis, topic refinement, code generation, and drafting. They were required to manually verify all citations, links, and code outputs.
• Duration: Most students spent 5–10 hours interacting with LLMs, though one student utilizing an "agentic" multi-model workflow spent ~30 hours.

3. Key Contributions
This paper provides a rare, empirical case study of LLMs in a high-stakes, open-ended scientific research environment. Its primary contributions include:

• Empirical Validation of Capabilities: Demonstrating that LLMs can effectively synthesize complex literature, generate initial research questions, and assist with basic coding and plotting.
• Identification of Critical Failure Modes: Systematically cataloging specific technical limitations, including hallucinated citations, inability to interact with APIs/databases, and the generation of non-physical simulation results.
• Pedagogical Framework: Offering a structured approach for integrating LLMs into graduate education, emphasizing verification, cross-model checking, and the distinction between "agentic" workflows and simple chat interactions.
• Ethical and Cognitive Reflection: Highlighting student concerns regarding the "dampening" of creativity and the loss of autonomy when AI predicts research steps.

4. Results

Successes:

• Literature Synthesis: LLMs significantly accelerated the initial literature review, transforming days of searching into instant summaries of complex topics.
• Topic Refinement: Models helped students narrow broad interests into specific, testable research questions.
• Code Assistance: LLMs were effective for debugging simple syntax errors, generating plotting scripts, and explaining existing codebases.
• Output: All seven students successfully completed draft papers addressing novel astronomical questions by the semester's end.

Failures and Limitations:

• Hallucinated Citations: Approximately 20% of the time, LLMs generated links to non-existent papers, misattributed authors/years, or provided summaries that did not match the actual content of the linked paper.
• Code and Simulation Errors: Models struggled to generate complex, functional code for data analysis or physically accurate simulations.
  • Example: Models frequently mixed parameters from different papers, leading to unphysical potentials in dynamical simulations.
  • Example: Models failed to correctly implement specific dataset formats (e.g., MaNGA) or access niche packages (e.g., STARBURST99).
• API and Data Retrieval: LLMs could not autonomously query online archives (e.g., VizieR) or download data. They often provided correct URLs but failed to write the necessary code to interact with the underlying data structures.
• Over-simplification and "Doubling Down": Models made implicit, unjustified assumptions (e.g., inventing error bars based on 10% of a value) and often persisted in incorrect reasoning even after being corrected.
• Lack of "Scientific Taste": Models struggled to identify truly novel, niche research questions that could be solved within a semester, often suggesting overly broad or already-solved problems.

Student Sentiment:

• Time Savings: Opinions were split; ~50% felt time was saved, primarily in literature review and initial coding. Others felt time was wasted debugging hallucinations.
• Future Use: All students indicated they would use LLMs again, but with strict limitations: primarily for literature discovery, syntax debugging, and plotting, while avoiding them for generating core scientific ideas, writing paper text, or accessing raw data.
• Creativity Concerns: Students expressed worry that over-reliance on LLMs for "next steps" could stifle the critical reflection and autonomy essential to scientific training.

5. Significance

• For the Scientific Community: The paper serves as a cautionary guide for researchers attempting to use LLMs for original research. It establishes that while LLMs are powerful "force multipliers" for information synthesis, they currently lack the reliability required for data retrieval, complex simulation, and citation verification without rigorous human oversight.
• For Education: It suggests a shift in graduate pedagogy. Future courses must explicitly teach "LLM literacy," including best practices for verification, prompt engineering for scientific contexts, and the ethical implications of AI use.
• For AI Development: The findings provide a clear roadmap for developers, prioritizing:
  • Truthfulness: Preventing the generation of fake citations and forcing reliance on verified databases (e.g., ADS).
  • Agency: Improving the ability to interact with APIs and execute code in niche scientific environments.
  • Uncertainty Quantification: Enabling models to explicitly state confidence levels or admit when data is unavailable.
  • Autonomy: Designing systems that assist rather than dictate research directions to preserve human creativity.

In conclusion, while LLMs in Fall 2025 showed promise as research assistants for literature review and basic coding, they were not yet autonomous researchers. Their effective use requires a "human-in-the-loop" approach where the scientist acts as the primary validator, editor, and creative driver.