← Latest papers
🧠 neuroscience

Rtel1 in hypothalamic oxytocin neurons regulates oxytocin output and social behaviors in mice

This study reveals that RTEL1, a DNA helicase linked to autism risk, maintains social behavior and oxytocin output in mice by resolving G-quadruplex structures at the Oxt promoter to ensure proper oxytocin gene expression in hypothalamic neurons.

Original authors: Wang, J., Wang, X., Qiu, Z.

Published 2026-09-12
📖 5 min read🧠 Deep dive

Original authors: Wang, J., Wang, X., Qiu, Z.

Original paper dedicated to the public domain under CC0 1.0 (https://creativecommons.org/publicdomain/zero/1.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The human brain is a vast network of communication, where specific cells release chemical messengers to coordinate everything from heartbeat to mood. Among these messengers, oxytocin stands out as a key regulator of social connection. Produced in a small region deep within the brain called the hypothalamus, oxytocin helps animals recognize friends, bond with family, and navigate the complex world of social interaction. When this system works correctly, an animal feels comfortable approaching a stranger; when it falters, social engagement can become difficult or anxiety-inducing. Scientists have long searched for the genetic switches that control how much oxytocin is made and released, hoping to understand why some individuals struggle with social connection in ways that resemble autism spectrum disorder.

A team of researchers has now identified a specific gene, known as RTEL1, that plays a surprising role in this process. While this gene was previously famous for its job in maintaining the stability of DNA—the molecule that carries our genetic instructions—this study reveals a second, distinct function. The researchers found that RTEL1 acts as a molecular tool that helps turn on the gene responsible for making oxytocin. When the amount of this tool is reduced, the brain produces less oxytocin, and the animals behave as if they are more anxious and less interested in meeting new friends.

The investigation began with a look at human genetics. In a previous study of people with autism, scientists had noticed that some individuals carried variations in the RTEL1 gene. To understand what these variations meant, the researchers first tested them in a lab dish. They found that the specific changes found in people with autism caused the RTEL1 protein to become unstable or disappear entirely. This suggested that having less of this protein might be the root of the problem. To see how this played out in a living brain, the team created mice that carried only one working copy of the RTEL1 gene instead of the usual two, effectively mimicking the reduced dosage seen in humans.

These mice appeared healthy and moved normally, but when their behavior was tested, a clear pattern emerged. The mice with reduced RTEL1 were just as willing to approach a familiar mouse as any other mouse, showing that their basic desire to socialize was intact. However, when presented with a new, unfamiliar mouse, they failed to show the usual curiosity. Instead of investigating the newcomer, they spent as much time with the mouse they already knew. They also displayed signs of increased anxiety, such as avoiding open spaces and burying marbles in their bedding more than usual. These behaviors pointed to a specific difficulty in processing social novelty and a heightened state of worry, rather than a general inability to move or learn.

The researchers then traced the cause of these behaviors back to the brain's chemistry. They discovered that the RTEL1 protein is present in the very cells that produce oxytocin. In the mice with reduced RTEL1, these cells contained significantly less oxytocin. To confirm the link between the lack of oxytocin and the behavioral changes, the team used a specialized sensor to watch the brain in real time. When a normal mouse met a new friend, its oxytocin levels spiked. In the mice with reduced RTEL1, this spike was much weaker, indicating that the brain was not releasing enough of the chemical to support a confident social response.

To understand how a gene involved in DNA stability could control oxytocin production, the team looked at the structure of the DNA itself. They found a specific sequence of genetic code near the start of the oxytocin gene that tends to fold into a tight, four-stranded knot called a G-quadruplex. Normally, this knot might block the machinery that reads the gene and makes the protein. The researchers showed that the RTEL1 protein acts like a molecular tool that can unwind or resolve this knot, allowing the oxytocin gene to be read and turned on. In the mice with less RTEL1, this knot remained tied, the gene stayed silent, and less oxytocin was made.

The study went further to test the role of this mechanism in behavior. The researchers used a genetic trick to remove the RTEL1 gene only from the oxytocin-producing cells, leaving the rest of the body untouched. These mice developed the exact same social and anxiety problems as the mice with the gene reduced everywhere, confirming that the issue originated specifically in the oxytocin cells. Finally, they tested whether fixing the problem in adulthood could improve the effects. By injecting a virus carrying a healthy copy of the RTEL1 gene directly into the oxytocin cells of adult mice, they were able to restore the protein levels. This single intervention ameliorated the mice's social behavior, making them more curious about new friends and less anxious, though the study did not establish whether these improvements were directly mediated by a restoration of oxytocin production.

This work connects a specific genetic variation to a physical change in brain chemistry and a measurable shift in behavior. It suggests that for some individuals, difficulties with social novelty and anxiety may stem from a subtle breakdown in how a gene is read, rather than a total failure of the system. While the study does not claim to offer an immediate cure for human conditions, it provides a clear, biological explanation for how a single gene can influence the complex social world, highlighting a new path for understanding the molecular roots of social behavior.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →