📊 statistics

Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networks

This paper proposes three knockoff-based variable screening methods for deep neural networks that effectively reduce computational complexity by identifying and removing irrelevant inputs while controlling the false discovery rate.

Original authors: Huiqi Zhang, Wenyu Liao, Yiqing Shi, Xiaobo Huang, Fang Xie

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Huiqi Zhang, Wenyu Liao, Yiqing Shi, Xiaobo Huang, Fang Xie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Noisy Kitchen"

Imagine you are a chef trying to make a perfect soup (the Deep Neural Network). You have a pantry with 100 different ingredients (the variables or inputs). Some of these ingredients are essential for the flavor (like salt and garlic), but many others are useless or even harmful (like a random rock or a bottle of motor oil).

The problem is that deep neural networks are like chefs who will try everything in the pantry. They use all 100 ingredients, which makes the cooking process slow, expensive, and the final soup messy. We need a way to figure out exactly which ingredients matter and throw the rest away, but we have to be careful not to throw away the good stuff by mistake.

The Solution: The "Knockoff" Twin

The authors propose a method to solve this using a concept called Knockoffs.

Think of a Knockoff as a "fake twin" for every real ingredient in your pantry.

If you have a real "Salt" shaker, you create a "Fake Salt" shaker that looks and feels exactly the same, but it has no flavor.
If you have a real "Garlic" bulb, you create a "Fake Garlic" bulb that is identical in texture but has no taste.

The goal is to feed both the real and fake ingredients into the cooking process (the neural network). If the chef (the AI) starts using the Real Salt but ignores the Fake Salt, we know Salt is important. If the chef uses the Fake Rock just as much as the Real Rock, we know the Rock is useless and should be discarded.

This method helps control the False Discovery Rate (FDR). In plain English, this is a safety guardrail that ensures we don't accidentally claim a useless ingredient is important too often. It keeps the "mistake rate" low.

The Three New Filters

The paper introduces three specific ways to run this "twin test" inside a neural network:

The One-Layer Filter (The Quick Scan):
This looks only at the very first step where ingredients enter the kitchen. It checks which real ingredients are used more than their fake twins right at the start. It's fast but might miss subtle clues that happen later in the cooking process.
The Multiple-Layers Filter (The Deep Dive):
This looks at the entire cooking process, from the first chop to the final simmer. It tracks how the importance of ingredients changes as they move through different layers of the network. This is more thorough and catches important ingredients that the "Quick Scan" might have missed.
The Variable Weight Aggregation Filter (The Panel of Judges):
Neural networks can be a bit random; if you cook the same soup twice, you might pick slightly different ingredients by chance. This method runs the "twin test" many times (like having a panel of judges taste the soup repeatedly). It only keeps an ingredient if most of the judges agree it's important. This makes the final list of ingredients much more stable and reliable.

Simplifying the Network (The "Weight Reduction")

Once the authors identify which ingredients (variables) are important, they do something else: they simplify the kitchen itself.

Imagine the kitchen has 100 counters and 500 knives. If you only need 10 ingredients, you don't need 500 knives. The authors use the data from their "twin test" to identify which "knives" (weights in the network) are barely used. They then delete those unused knives and even remove the empty counters (neurons).

The Result:

Faster Cooking: The network becomes smaller and runs much faster.
Same Taste: Surprisingly, the simplified network still makes the soup taste just as good (maintains accuracy).
Less Waste: It uses less computer power and memory.

Real-World Test: The Breast Cancer Dataset

To prove this works, the authors tested their method on a real dataset about breast cancer.

The Data: They had 30 different measurements taken from cell images (like the size of the nucleus, the texture, etc.).
The Goal: Predict if a cell is cancerous (Malignant) or not (Benign).
The Outcome: Their method successfully filtered out the unimportant measurements, keeping only the ones that truly mattered. They then built a smaller, faster version of the prediction model that performed just as well as the giant, complicated version.

Summary

In short, this paper teaches us how to use "fake twins" (Knockoffs) to trick a neural network into revealing which inputs are actually important. They offer three different ways to do this, and once the important inputs are found, they shrink the network down to its essential parts. This makes AI faster, cheaper to run, and easier to understand, without losing its ability to make accurate predictions.

Technical Summary: Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networks

Problem Statement
Deep neural networks (DNNs) are powerful tools for handling non-linear classification problems but often suffer from high computational complexity due to large numbers of parameters and input variables. Many of these inputs may be irrelevant to the target output, contributing to unnecessary computational costs and potential overfitting. While variable selection methods exist, controlling the False Discovery Rate (FDR) in non-linear, high-dimensional settings using DNNs remains challenging. Existing methods either lack FDR control, rely on attention mechanisms that may overlook marginal feature contributions, or involve complex implementations. Furthermore, there is a need for methods that not only select variables but also simplify the network structure to improve efficiency.

Methodology
The authors propose a framework that integrates the Knockoffs method (Barber and Candès, 2015) with $\ell_1$ -regularized neural networks to achieve simultaneous variable screening with FDR control and network simplification.

Knockoff Construction: For each original input variable $x_j$ , a knockoff variable $\tilde{x}_j$ is generated. These knockoffs share the same distributional properties and pairwise correlations as the original variables but are conditionally independent of the response. This creates a "negative control" for valid variable selection.
Regularized Neural Network: The model employs a DNN with $L$ hidden layers. The input layer consists of $2p$ neurons ( $p$ original variables and $p$ knockoffs). The network is trained using a loss function that combines binary cross-entropy (or categorical cross-entropy for multi-class) with an $\ell_1$ regularization term.
Importance Metric ( $Z_{jd}^{(l)}$ ): The core of the method involves determining the largest regularization parameter $\lambda$ for each weight $\theta_{jd}^{(l)}$ that prevents it from becoming zero. This value, denoted as $Z_{jd}^{(l)}$ , serves as a measure of the weight's importance. Larger $Z$ values indicate more critical weights.
Three Screening Algorithms:
- One Layer Filter (OL): Calculates variable importance based solely on the aggregated $Z$ values of weights connecting input variables to the first hidden layer.
- Multiple Layers Filter (ML): Aggregates importance by considering the product of weight matrices across all layers, effectively capturing the global structure of the network.
- Variable Weight Aggregation (VWA) Filter: An ensemble approach that runs the OL or ML filters multiple times (e.g., 10 iterations) with different random initializations. It uses hard voting to select variables that appear consistently across runs, thereby reducing the stochasticity inherent in neural network training.
FDR Control: The methods utilize the symmetry property between original variables and their knockoffs to construct a test statistic $W_j$ . A data-dependent threshold $T$ is calculated to ensure the empirical FDR remains below a target level $q$ .
Network Simplification (Reduce Weight): Based on the $Z$ values obtained during screening, weights with low importance are pruned. This process can also remove entire neurons if all incoming or outgoing weights are deleted, effectively simplifying the network architecture.

Key Contributions

FDR Control in Non-Linear Models: The paper extends the application of Knockoffs to deep neural networks, providing a rigorous method for variable selection that controls the FDR in non-linear settings where traditional linear knockoff methods fail.
Integrated Screening and Simplification: The proposed framework simultaneously identifies relevant features and simplifies the network structure by pruning low-importance weights and neurons, addressing both feature selection and model efficiency.
Three Distinct Algorithms: The introduction of OL, ML, and VWA filters offers flexibility. The VWA filter specifically addresses the instability of neural network training by aggregating results across multiple runs.
Empirical Validation: The authors provide extensive simulations and a real-world application demonstrating the efficacy of their approach compared to existing methods like DeepLINK, DeepPINK, and standard Knockoffs.

Results

Simulation Studies:
- The proposed methods (particularly VWA variants) achieved empirical FDR values close to or below the target level ( $q=0.1$ ) while maintaining high statistical power (the ability to detect true signals) across various data distributions (Normal, Chi-square) and network architectures.
- The VWA-OML (aggregating both OL and ML) showed robust performance, often achieving near-zero FDR with high power, outperforming single-layer or single-run methods in consistency.
- Compared to existing methods (Knockoffs, DeepLINK, DeepPINK, DFS, DeepLasso), the proposed VWA-OML method demonstrated superior power (0.97 vs. 0.12–0.30 for others in specific binary classification tests) while maintaining strict FDR control.
Network Simplification:
- Applying the "Reduce Weight" strategy significantly decreased the training time compared to using all variables or Dropout regularization, while maintaining or slightly improving prediction accuracy.
- On the Breast Cancer dataset, the method successfully reduced the number of input variables and simplified the network structure without sacrificing predictive performance.
Real-World Application (Breast Cancer Dataset):
- Applied to the Wisconsin Diagnostic Breast Cancer dataset (30 features, 569 instances), the filters selected varying subsets of features. The VWA-OML filter selected 21 variables, while OL and ML selected 13 and 10 respectively.
- The simplified models trained on these selected variables demonstrated competitive accuracy and reduced computational time compared to models using all original features.

Significance and Claims
The paper claims that its primary significance lies in bridging the gap between rigorous statistical variable selection (FDR control) and the practical needs of deep learning (handling non-linearity and model complexity). The authors assert that their methods provide a "satisfactory performance" in screening variables and simplifying networks, offering a viable alternative to attention-based mechanisms or complex ensemble methods that may overlook marginal feature contributions.

The authors modestly note that while the methods perform well under Normal and Chi-square distributions, empirical FDR can exceed the target level under skewed non-Gaussian distributions (e.g., Gamma), suggesting that the control is distribution-dependent. They emphasize that the VWA approach improves stability and reliability by mitigating the randomness of neural network training, making the feature selection process more interpretable and robust. The work is positioned as a tool applicable to diverse fields such as genomics, brain imaging, and particle physics, where high-dimensional non-linear data analysis is critical.