How much technical talent is there? A systematic estimate of the ML research pool among 3 million consultants

1. Problem Statement

The primary bottleneck for advancing AI safety, alignment, and assurance research is a lack of accessible technical talent. While capital expenditure for AI infrastructure is rising, the capacity to execute difficult alignment and evaluation work at production speed is currently concentrated in a handful of frontier labs and academic groups.

The authors investigate whether the IT consultancy sector represents a "latent capacity" of technically competent Machine Learning (ML) researchers. Specifically, they ask: Is there a scalable pool of engineers and researchers within global IT consultancies who possess the skills to perform high-level technical AI safety work (e.g., training models from scratch, debugging complex architectures, and implementing novel research methods)?

2. Methodology

The study employed a multi-stage, systematic approach combining large-scale data aggregation, probabilistic modeling, and empirical validation.

A. Data Collection & Screening

• Search Strategy: The team identified 2,121 organizations via personal networks, unstructured web searches (using LLMs), conference affiliations (ICLR, ICML, NeurIPS, arXiv), and Crunchbase keyword filtering.
• Inclusion Criteria: Companies were filtered to offer technical ML consulting services. Organizations with <10 employees or <2 years of operation were excluded.
• Final Sample: 403 organizations were retained, representing a total workforce of 3.269 million employees.
• Geographic Distribution: Heavily skewed toward North America and Europe, with significant representation in Southern and Eastern Asia.

B. Talent Estimation Pipeline

To estimate the number of "Highly Technical ML Research Talent" within these firms, the authors developed a Bootstrap Multivariate Probit Model based on the Dawid-Skene framework.

• Data Sources: LinkedIn resumes were scraped via Sales Navigator and BrightData.
• Annotators: Six imperfect annotators were used to label each employee as an "ML Expert" or "Non-Expert":
  1. Keyword Filters: Three variations (Broad, Strict, and Combined) based on top discriminative keywords (e.g., "transformer," "loss function," "reinforcement learning" vs. exclusionary terms like "certificate," "management").
  2. LLM Classifiers: Three large language models (Gemini 2.5 Flash, GPT-5 Mini, Claude Sonnet 4) prompted to evaluate CVs against a strict definition of technical research capability (e.g., ability to build architectures from scratch vs. fine-tuning).
• Statistical Model: The model combined these signals, accounting for correlated errors between annotators (e.g., if multiple LLMs fail on the same edge case). It used Beta priors stratified by company size to estimate the prevalence of talent.
• Validation: The pipeline was validated against a ground-truth dataset of 585 manually labeled CVs from known AI labs (OpenAI, Anthropic) and non-AI companies. The final combined model achieved 0.89 accuracy, 0.79 sensitivity, and 0.926 specificity.

C. Empirical Validation (Work Trials)

To verify the estimates, the authors conducted a 3-day paid work trial with a subset of companies.

• Task: Implement a "Sequential Unlearning" method (a wrapper around an existing RMU algorithm) and integrate it into an evaluation codebase.
• Participants: 8 organizations were invited; 5 completed the trial.
• Control: State-of-the-art AI agents (GPT-5, Claude Opus 4.1) were also tasked with the same problem.

3. Key Results

A. Talent Pool Estimates

• Total Estimate: Across the 403 consultancies, the 50th percentile aggregate estimate for highly technical ML Research Talent is 1,121 individuals (80% CI: 252–3,165).
• Distribution by Confidence:
  • Probable (80% CI excludes zero): 81 organizations, accounting for ~890 of the estimated talent (79.4% of the total pool).
  • Possible: 100 organizations.
  • Non-zero/Not Detected: The remaining organizations showed little to no signal of high-level research talent.
• Density: The overall density of ML research talent in the sample is low (~0.01% of total employees), but varies significantly by firm size and type. Smaller, specialized consultancies showed higher densities (up to 20% in some cases), while giant conglomerates (e.g., Accenture, Deloitte) showed very low percentages (<0.1%).

B. Work Trial Outcomes

• Human Performance: Of the 8 organizations invited, 5 (63%) received at least a conditional recommendation. Three organizations scored >90% (Recommendation), two scored 50-70% (Conditional), and three failed (<50%).
• AI Agent Performance: In contrast, 0% of the AI agents (GPT-5 and Claude Opus) passed the work trial. They scored between 30-40%, failing to execute the unlearning to completion or follow through with necessary evaluations.
• Implication: This suggests that while AI agents are advancing, they currently cannot replace the specific R&D capabilities of competent human consultants for novel, complex tasks.

C. Model Limitations & Anomalies

• Synthetic Imputation: For very large companies where individual CVs were unavailable, the model used synthetic data. This led to implausibly low estimates for known frontier labs (e.g., estimating only 9 ML researchers at Anthropic out of 2,000 employees), highlighting a mismatch between consultancy-calibrated priors and the talent density of dedicated AI labs.
• Geographic Bias: The search was English-only and reliant on LinkedIn, potentially underrepresenting non-Western markets.

4. Key Contributions

1. First Systematic Map: This is the first study to systematically quantify the "dormant" pool of technical ML research talent within the global IT consultancy sector.
2. Novel Estimation Pipeline: The authors introduced a robust, transparent pipeline combining keyword filtering, LLM classification, and a correlated probit model to estimate talent from noisy resume data.
3. Empirical Validation: Unlike purely theoretical studies, this work included a rigorous work trial benchmark, providing "proof of existence" that these consultants can execute challenging technical tasks that current AI agents cannot.
4. Strategic Insight: The study demonstrates that "consultancies" are not a homogeneous class; specific sub-units within them hold significant capacity for AI assurance work.

5. Significance and Conclusion

The paper concludes that there is a substantial, untapped pool of technically competent ML researchers (in the low thousands) within the IT consultancy sector.

• For Funders and Policymakers: This suggests a viable, scalable path to expand capacity for AI safety and alignment research without waiting for the next generation of academic graduates or hiring exclusively from frontier labs.
• Activation Requirements: To unlock this capacity, the study notes that activation requires deliberate scoping, credible sponsorship, and smaller, faster contracting paths. Crucially, these teams often require an external "vision holder" (e.g., a senior AI alignment researcher) to define outcomes and research direction.
• AI Safety Context: The finding that human consultants outperformed state-of-the-art AI agents on a complex research task reinforces the immediate need for human-centric research capacity in the AI safety ecosystem.

In summary, the study provides a data-driven argument that the IT consultancy sector represents a critical, underutilized resource for scaling technical AI assurance work.