Identification and mitigation of memory block timing issue in ITk ABCStar during ASIC production

This paper details the identification of a timing flaw in the ABCStar ASIC that threatened production yields, and the successful mitigation of this issue through a combination of increasing the core operating voltage and adjusting the clock duty cycle, thereby avoiding costly process changes or redesigns and enabling the continued production of ATLAS ITk detector modules.

The Gist

The Story of the "Star" Chip That Stuttered

Imagine the ATLAS experiment at CERN as a massive, high-speed camera trying to take pictures of particles colliding at nearly the speed of light. To do this, it needs millions of tiny, super-smart sensors called ABCStar chips. These chips are the "eyes" of the camera, reading data from silicon strips and sending it to a central computer.

Before the camera could be built, engineers had to manufacture these chips. They expected about 90% of the chips to work perfectly. However, during testing, they found a terrifying problem: on some batches of chips, only 2% worked. The rest were failing.

The Mystery: A "Silicon-Proven" Ghost

The engineers were confused. The failing chips weren't broken in a weird way; they were passing almost every test. They could read analog signals, handle power, and do complex math. The only thing they failed was a specific digital test that checked if they could remember and recall data correctly.

The data was being stored in SRAM blocks (think of these as the chip's short-term memory notebooks). These specific memory blocks had been used in many other successful chips before. In the industry, this is called being "silicon proven." It's like using a tire design that has been on millions of cars without ever having a blowout. Everyone assumed these tires were perfect.

The engineers suspected the memory itself was broken, but they were wrong. The memory was fine. The problem was the traffic controller (the "glue logic") that told the memory when to write and when to read.

The Root Cause: A Timing Mismatch

Here is the analogy: Imagine a relay race where a runner (the data) has to hand a baton to a teammate (the memory) exactly when a whistle blows.

• The Plan: The whistle blows, the runner sprints, and the teammate catches the baton.
• The Reality: In some of these chips, the runner was slightly slower than the engineers thought. Because the "silicon proven" memory models were based on older tools, they didn't account for the fact that the runner might be a little sluggish in this specific factory batch.
• The Result: The teammate tried to catch the baton too early. The runner wasn't there yet. The baton was dropped. In chip terms, this is a bit flip or a timing error. The data got corrupted.

This happened mostly on the edges of the silicon wafers (like the edges of a pizza), where the manufacturing process is slightly less uniform, making the "runners" even slower.

The Investigation: Finding the Fix

The team had to find a way to fix this without throwing away millions of dollars worth of chips or redesigning the whole thing from scratch (which would take years). They tested two main ideas:

1. The "Speed Boost" (Voltage Increase)

If the runner is slow, give them a caffeine shot.

• The Fix: They increased the electrical voltage supplied to the chip's digital brain from 1.20 Volts to 1.25 Volts.
• The Effect: Higher voltage makes the transistors (the runners) move faster. Suddenly, the runner was fast enough to catch the baton on time.
• The Result: Chips that were previously failing (2% yield) suddenly worked 80% of the time.

2. The "Longer Pause" (Clock Duty Cycle)

If the runner is still a bit slow, tell the teammate to wait a little longer before trying to catch the baton.

• The Fix: The chip runs on a clock signal that ticks back and forth. The engineers realized the "high" part of the tick (when the logic is active) was too short. They physically swapped two wires on the circuit board so the "high" part lasted longer.
• The Effect: This gave the logic more time to settle and get ready before the memory tried to grab the data.
• The Result: This added an extra layer of safety, ensuring the chips wouldn't fail even if they got a little older or colder.

The "What If" Scenario: Changing the Factory

The team also talked to the factory (the foundry) about changing the manufacturing process to make the transistors naturally faster.

• The Problem: They had already made 300 wafers with the "slow" process. You can't un-bake a cake. If they changed the process now, they would have to scrap all the existing wafers and start over, costing a fortune and delaying the project.
• The Decision: They tested "fast" transistors on new experimental wafers. While they worked, they caused other side effects (like changing the analog sensors' sensitivity).
• The Verdict: Since the "Speed Boost" (voltage) and "Longer Pause" (wiring swap) worked perfectly on the existing chips, they decided not to change the factory process. It was cheaper, faster, and safer to just tweak how the chips were used.

The Final Outcome

The team proved that by simply turning up the voltage slightly and swapping two wires, they could save the project.

• Yield: They went from a disaster (2% working) to a success (over 80% working).
• Power: The extra voltage used a tiny bit more power (about 3% more), which the cooling system of the detector could easily handle.
• Radiation: They tested the chips under heavy radiation (like they would face in the particle collider) and found the fix still worked.

The Big Lesson

The paper ends with a crucial lesson for all engineers: Don't assume "proven" is perfect.

Just because a component (like the memory block) worked in the past doesn't mean it will work perfectly in every new design, especially when combined with new manufacturing variations. The team learned that even "silicon proven" blocks need to be re-checked with the specific tools and conditions of the new project. If they had done this earlier, they might have caught the issue sooner.

Thanks to this detective work, the ATLAS ITk detector is now being assembled with these chips, and they are expected to run reliably for the lifetime of the experiment.

Technical Summary

Technical Summary: Identification and Mitigation of Memory Block Timing Issue in ITk ABCStar

Problem Identification
During pre-production testing of the ABCStar ASIC, a mixed-signal front-end readout chip for the ATLAS Inner Tracker (ITk) upgrade at the High-Luminosity LHC, a critical yield issue was discovered. While the ASIC design utilized "silicon proven" Static Random Access Memory (SRAM) blocks with a long heritage of reliable operation, specific manufacturing lots exhibited wafer yields dropping from an expected >90% to as low as 2%. The failures were concentrated in the digital test vectors (specifically tests A02, P03, and P99) and manifested as intermittent bit flips in the output data. Crucially, the output packet format remained correct, indicating that the error originated within the data payload stored in the SRAM buffers (L0 and Event Buffers) rather than the output circuitry. The failures were highly sensitive to operating conditions, particularly occurring more frequently at the nominal core voltage of 1.20 V and on dice located toward the periphery of the wafer, suggesting a process-variation-dependent timing margin issue.

Methodology and Root Cause Determination
An ad hoc, multi-disciplinary team employed a "shmoo plot" analysis to map test results across multi-dimensional operational parameters, specifically varying the digital core voltage and the clock duty cycle.

• Voltage Sensitivity: Tests revealed that increasing the digital core voltage from 1.20 V to 1.25 V significantly improved yields, suggesting the issue was related to transistor speed rather than a fundamental logic error.
• Duty Cycle Sensitivity: Increasing the clock duty cycle (keeping the clock high longer) also reduced failures, providing more time for logic states to propagate through the "glue logic" interfacing the SRAMs before the falling edge of the 40 MHz clock triggered a read/write operation.
• Root Cause: The investigation determined that the timing models for the "silicon proven" SRAM blocks were inaccurate. These models had been manually generated using legacy design tools and did not fully represent the blocks' performance under the specific manufacturing process variations of the 130 nm mixed-signal CMOS foundry. Consequently, the synthesized control logic met the provided (but inaccurate) timing constraints, resulting in marginal timing in the physical ASICs.

Investigation of Mitigation Options
Two primary mitigation strategies were pursued in parallel:

1. In-Situ Operating Parameter Changes: Modifying the operating conditions of the already manufactured ASICs.
2. Manufacturing Process Modifications: Working with the foundry to alter the transistor characteristics of future wafers.

Voltage and Duty Cycle Analysis:
Extensive testing on "bad lot" wafers demonstrated that raising the digital core voltage to 1.25 V restored yields to over 80% even on the worst-performing wafers. Thermoelectric studies confirmed that this voltage increase would result in only a ~2.8% increase in total power dissipation, well within the cooling and power capabilities of the ITk detector infrastructure. Additionally, radiation testing (up to 100 Mrad) and temperature cycling (down to -40°C) confirmed that the 1.25 V setting provided sufficient margin for the detector's operational lifetime, as lower temperatures and radiation effects were found to either improve transistor speed or require only minor voltage adjustments.

Regarding the clock, the HCCStar controller provided a duty cycle slightly below 50% (approx. 49%). Since a higher duty cycle improved reliability, the team proposed physically swapping the positive and negative signals of the Scalable Low-Voltage Signaling (SLVS) differential clock pair between the HCCStar and the ABCStar. This simple hardware modification guaranteed a duty cycle of >50% (specifically 51%), providing additional timing margin without redesigning the clock generator.

Process Modification Analysis:
The foundry proposed experimental process changes, including reducing transistor gate sizes by 2 nm or 4 nm and using a proprietary "faster transistor" technique. While these changes improved digital performance, they introduced significant shifts in analog circuitry behavior (e.g., bandgap reference voltage, gain) that would have required re-characterization of the entire chip. Furthermore, these changes could not be applied to the 300 wafers already manufactured, which would have necessitated scrapping them.

Key Results

• Yield Recovery: By implementing the voltage and duty cycle mitigations, the yield of the worst-performing wafers was recovered from ~2% to >80%.
• Final Production Metrics: With the new operating parameters (1.25 V core voltage and swapped clock signals), the final Category A yield reached 85.88%, and the Category A+B yield reached 89.94%, aligning with the original project expectations.
• Validation: Comprehensive testing confirmed that the modified operating conditions ensured reliable operation across the full range of expected temperatures (-16°C to -28°C) and radiation doses for the ITk detector lifetime.
• Decision: The team elected to proceed with the in-situ operating parameter changes for all existing and future wafers, avoiding the need for a costly ASIC redesign or the scrapping of already manufactured wafers.

Significance and Lessons Learned
The paper documents the successful resolution of a subtle timing issue that threatened the production of a critical component for the HL-LHC upgrade. The primary significance lies in the demonstration that a "silicon proven" design block, when reused without re-simulation against the specific process corner cases, can introduce marginal timing that only manifests under specific process variations.

The authors emphasize two key lessons for future ASIC development:

1. Re-simulation of Reused Blocks: Even for blocks with a long heritage, reliance on "silicon proven" status without re-simulation against the specific foundry process models carries risk.
2. Process Variation Testing: To catch such marginal designs before full production, the authors recommend requesting "striped" wafers from foundries (containing dice with extreme process variations on a single wafer) or conducting early testing at reduced voltages to simulate process corners and determine operational margins.

The successful mitigation allowed the ITk strip module production to proceed on schedule, with detector integration beginning in late 2025 and commissioning scheduled for 2028.