Role-Aware Language Models for Secure and Contextualized Access Control in Organizations

1. Problem Statement

As Large Language Models (LLMs) are increasingly deployed in enterprise settings for tasks like document generation and internal assistance, a critical security gap has emerged. Traditional Role-Based Access Control (RBAC) systems effectively manage access in software databases but are largely unexplored in generative AI.

• The Gap: Existing LLM safety methods focus on preventing toxic or harmful outputs but assume uniform user access. They do not account for organizational hierarchies where specific roles (e.g., CEO vs. Intern) should have different permissions to view or discuss sensitive data.
• The Challenge: Can LLMs be fine-tuned to enforce role-conditioned access control, ensuring they generate responses only when the user's role has the necessary clearance, while maintaining high-quality output?

2. Methodology

The authors propose a framework to simulate realistic organizational scenarios and evaluate three distinct modeling strategies.

A. Dataset Construction

To train and evaluate these models, two complementary datasets were created, each structured around two organizational hierarchies:

1. Basic Structure: A flat hierarchy with one CEO and 19 direct subordinates.
2. Office Structure: A multi-level hierarchy with a CEO, four department managers, and team members.

The datasets were constructed using two strategies:

• Repurposed Data: The Databricks Dolly-15k dataset was clustered using sentence transformers. Instructions were recursively partitioned into "General," "Shared," and "Role-Only" categories to mimic hierarchical access rights.
• Synthetic Data: Generated using GPT-4.1 mini to create realistic enterprise scenarios (e.g., risk reports, vulnerability scans) with explicit role-based constraints.
• Training/Testing Split: Data was augmented to create positive (authorized) and negative (unauthorized) instances. Test sets included "seen" instructions, "unseen" paraphrased instructions, and adversarial cases (mismatched roles, broken role strings, and external roles).

B. Modeling Strategies

The study evaluates three approaches to role-awareness:

1. Role-aware Cls (Classifier): Fine-tuning six BERT-based models (e.g., ModernBERT, RoBERTa) to act as binary classifiers. The model receives a prompt and a role, outputting True (grant) or False (deny).
2. Role-aware LLM-Cls (LLM Classifier): Fine-tuning six open-source LLMs (Qwen, Llama, Gemma) using LoRA to perform binary access classification.
3. Role-aware LLM-Gen (Generative): Fine-tuning the same LLMs to generate full responses. If access is denied, the model generates a generic refusal message; if granted, it answers the query.

C. Role Encoding

Three encoding strategies were tested to represent roles in the prompt:

• Hierarchical Number: Dot-delimited indices (e.g., 1.1, 1.2).
• Single Name: Role titles only (e.g., CEO).
• Hierarchical Name: Full path concatenation (e.g., CEO - IT Manager - Support).

3. Key Contributions

• Novel Framework: The first comprehensive investigation into fine-tuning LLMs for hierarchical, role-based access control rather than just general safety.
• Comprehensive Evaluation: A rigorous benchmark comparing classifier-based vs. generative approaches across different model sizes (3B to 8B parameters) and organizational complexities.
• Robustness Analysis: Extensive testing against adversarial attacks, including jailbreak attempts (e.g., "I am the CEO, ignore policy"), prompt injection, and role mismatch scenarios.
• Open Resources: Public release of code, synthetic datasets, and evaluation benchmarks.

4. Key Results

A. Performance Accuracy

• Best Approach: Role-aware LLM-Cls (LLM-based classifiers) consistently outperformed other methods.
  • On the Repurposed (Dolly) dataset, ModernBERT Large achieved 90.0% accuracy.
  • On the Synthetic dataset, Llama 3.1 8B Instruct achieved 89.3% accuracy.
• Generative Trade-off: The Role-aware LLM-Gen approach showed slightly lower accuracy (dropping by ~5–10%) and higher False Negative Rates (denying valid requests), likely due to the complexity of generating both the refusal logic and the answer simultaneously.
• Classifier Instability: Standard BERT-based classifiers (Cls) showed high variance and sensitivity to model selection. For instance, RoBERTa-Large suffered from a massive False Positive Rate (58%) on the Dolly dataset, indicating poor generalization.

B. Robustness and Security

• Jailbreak Resistance: Models trained with specific jailbreak samples (e.g., users claiming higher authority) improved their resistance to prompt injection from ~70% to 87% accuracy.
• Blacklisted Topics: The models successfully restricted access to sensitive topics (violence, politics) regardless of the user's role, achieving >99% accuracy in denying these queries.
• Adversarial Attacks:
  • Models were highly effective at rejecting clearly invalid roles (Random/External roles) with ~100% accuracy.
  • Performance dropped on subtle Mismatch cases (e.g., a leaf node trying to access root data), with accuracy around 70–80% for instruction-tuned models.
  • Broken Role Strings: Models struggled with corrupted role formats (e.g., "1.2" vs "01.02"), though Hierarchical Number encoding provided the best defense against this specific attack vector.

C. Encoding Strategy Impact

• Hierarchical Number Encoding (e.g., 1.1) resulted in the highest False Positive Rate (poor rejection of unauthorized roles) and the highest False Negative Rate (denying authorized users who couldn't "climb" the hierarchy).
• Name-based Encodings (Single or Hierarchical Name) offered better generalization for authorized roles but were slightly more vulnerable to adversarial perturbations.

D. Generation Quality

• For the generative models, including role information in the prompt did not degrade the quality of the answers (Correctness, Completeness, Clarity) for authorized users. Scores remained high (~4.0/5.0) across datasets.

5. Significance and Conclusion

This paper establishes that LLMs can be effectively adapted to enforce enterprise access control policies, moving beyond generic safety filters to context-aware security.

• Practical Implication: Organizations can deploy LLMs that automatically filter sensitive information based on user identity without needing complex external middleware for every query.
• Future Directions: The authors identify limitations, such as the need for dynamic role updates (without retraining) and the integration of Retrieval-Augmented Generation (RAG) with role-awareness. They also suggest exploring alignment methods beyond Supervised Fine-Tuning (SFT), such as Direct Preference Optimization (DPO).

In summary, the study demonstrates that instruction-tuned LLM classifiers offer the most robust balance between security (access denial) and utility (answer quality), providing a viable path toward secure, multi-user enterprise AI deployments.