UTS at CheckThat! 2026: Cite-Frame Engineering for Generated Fact-Checking Articles

In the modern information ecosystem, verifying a claim is only half the battle; explaining why it is true or false to a human reader is the other. This is the domain of automated fact-checking, a field where computers attempt to mimic the rigorous work of journalists. The challenge is not just to find the right evidence, but to weave that evidence into a coherent article that a human can trust and that a computer can verify. A recent competition, CheckThat! 2026, tasked teams with building systems to generate these fact-checking articles automatically. The systems had to take a specific claim, find supporting evidence from a list of web links, and write a short report citing those links. The scoring was unforgiving: a computer program graded the articles based on how well they matched human-written examples and, crucially, whether every single sentence citing a source was actually supported by the text at that source. If a system invented a detail or misquoted a source, even slightly, the computer would penalize it heavily.

A team from the University of Technology Sydney entered this competition with a system that took a different path than most. While other teams tried to use powerful artificial intelligence to write the entire article from scratch, the Sydney team built a system that relied on a rigid, predictable template for the main text, using artificial intelligence only as a strict gatekeeper to check the sources. Their approach placed them second out of eleven teams. The key to their success was not in generating more creative prose, but in understanding exactly how the computer judge worked. They discovered that the judge was incredibly conservative, rewarding articles that stuck closely to the facts provided and rejecting any sentence that added extra flair or unverified details. By designing their system to respect these strict rules, they created a machine that could reliably produce fact-checking articles that were accurate, even if they were not particularly elegant.

The researchers began by testing the most obvious idea: letting a large language model, a type of artificial intelligence capable of writing fluent text, draft the entire article. They tried this with models of different sizes, hoping the machine would learn to sound like a professional fact-checker. It did not work. The articles generated by these models consistently failed the computer's grading system. The problem was structural. The computer judge looked for specific patterns in the text, particularly how sources were cited. When the artificial intelligence tried to write naturally, it would often change the way a source was mentioned or add extra words that the computer could not verify against the original evidence. Even a small change, like adding a reviewer's name or a date that wasn't in the source text, caused the computer to reject the sentence. The more capable the artificial intelligence was at writing, the more it drifted away from the strict patterns the judge required. The team realized that trying to make the machine "write better" was actually making it perform worse.

Instead of trying to improve the writing, the team decided to improve the structure. They built a system that started with a simple, unchanging template. This template created a basic article with a headline, a single sentence for each piece of evidence, and a conclusion. This "stub" draft was boring, but it was safe. It followed the rules perfectly. The team then added two specific tools to this system, acting like a quality control inspector. The first tool, which they called a domain-attribution frame, simply wrapped each citation in a standard phrase like "According to the website, [fact]." This small change made the sentences sound more like the human-written examples the computer was comparing them against, without changing the facts. The second tool was a shadow validator. Before the system finalized a sentence, it asked a small artificial intelligence model to check if that specific sentence was actually supported by the source link. If the model said no, the system would try the next sentence in the source text. If that failed, it would stick with the original safe sentence. This ensured that every citation the system produced was something the computer judge would accept.

The results of this approach were clear and measurable. On the validation data used to test their system before the competition, their method improved the overall score by a small but significant margin compared to the basic template. When they combined the framing tool and the shadow validator, the improvement was even greater. The system did not try to be clever; it simply ensured that every sentence was verifiable and followed the correct format. In the final competition, their system achieved a score of 0.484, placing it second among eleven teams. The winning team had a higher score, but the gap between them was entirely due to one specific metric: citation precision. The winning team managed to cite sources with much higher accuracy, likely by choosing to omit citations they were not sure about, whereas the Sydney team cited every single source they were given. The Sydney team's system was so reliable that it cited almost all the available evidence, but because it did not drop the uncertain ones, its precision score was lower.

The team's work revealed two important lessons for the future of automated fact-checking. First, when a computer judge is designed to be strict about verification, adding extra words or trying to make the text sound more human often hurts the score. The most effective strategy was to keep the text simple and strictly tied to the evidence. Second, the tools used to select which sentence to cite were often misleading. The team found that standard methods for picking the "best" sentence from a source, such as looking for the longest sentence or the one most relevant to the claim, often picked sentences that the computer judge would reject. The only reliable way to pick a sentence was to ask the judge itself. This insight suggests that in tasks where accuracy is measured by strict verification, the best approach is not to generate new content, but to carefully curate and verify existing content.

The researchers also noted that their system had a limitation. Because they were required to cite every piece of evidence provided, they could not adopt the strategy of the winning team, which likely chose to skip uncertain sources to maintain a high accuracy score. The Sydney team hypothesized that if they had added a rule to drop any citation the judge was not confident about, their score might have been higher. However, they did not test this during the competition. Their work stands as a demonstration that in high-stakes verification tasks, a simple, deterministic system that prioritizes accuracy over creativity can outperform complex artificial intelligence models that try to write like humans. The paper concludes that for the next generation of these systems, the focus should shift from generating better prose to developing smarter ways of deciding what to include and what to leave out, ensuring that every word the machine writes can be proven true.