Best Practices: Managing Generative AI Systems
What is Managing Generative AI Systems?
Generative AI technologies are increasingly being integrated into enterprise applications, development platforms, automation tools, assistants, and AI agents. These systems may access data, interact with connected services, and perform actions across multiple environments.
Departments deploying or managing AI-enabled systems should evaluate them using the same security, privacy, governance, and risk-management principles applied to other enterprise technologies, while also accounting for AI-specific behavior such as generated outputs, prompt injection, memory, model changes, and autonomous actions.
Managing an AI-enabled system includes more than selecting a model or configuring prompts. Custodians should consider the system's data sources, connected tools, permissions, users, outputs, and ability to take actions. These considerations should be reviewed when the system is designed, changed, deployed, operated, and retired.
Common Risks
| Category | Description | Key Controls (At a Glance) |
|---|---|---|
Sensitive Data Exposure | Data may be entered, processed, stored, logged, retained, or used by AI systems in ways that are not appropriate for its classification. | Apply risk classification requirements, minimize sensitive inputs, review retention and training settings, and verify vendor protections. |
Data Integrity & Reliability | AI-generated outputs may be inaccurate, incomplete, or manipulated. | Validate inputs and outputs, test representative use cases, and require appropriate human review. |
Prompt Injection & Model Manipulation | Untrusted content may manipulate the system into disclosing information, bypassing controls, or taking unintended actions. | Use input and output controls, isolate trust boundaries, test malicious content, and limit tool permissions. |
Context & Memory Leakage | Information may persist across sessions, users, tools, or integrations unintentionally. | Limit context and memory, verify retention settings, isolate users and data, and review logs and stored content. |
Agents, Connectors & Integrations | AI agents, connectors, APIs, and MCP servers may expand access to systems and data. | Review integrations, maintain an allow list where appropriate, apply least privilege, and require approval for high-impact actions. |
Privilege & Identity Misuse | Excessive permissions, shared credentials, or untraceable automated activity may allow unauthorized access. | Use approved authentication, distinct attributable identities, credential management, logging, and prompt revocation. |
Governance & Visibility | Unmanaged AI deployments can create security, privacy, compliance, operational, and reputational risk. | Maintain inventory, ownership, documentation, review triggers, monitoring, and decommissioning procedures. |
Note:
The “Key Controls (At a Glance)” column is intended as a quick reference. Detailed expectations and implementation guidance are described in the Best Practices sections below.
General Best Practices
Documentation & Governance
- Maintain an inventory of AI-enabled systems, services, and integrations.
- Define ownership, approved use cases, and responsible administrators.
- Review new AI capabilities before enabling them.
- Reassess risks when systems, data sources, or integrations change.
Data Protection
- Apply Harvard data-classification requirements consistently.
- Treat prompts, outputs, logs, and retained context as organizational data.
- Review memory, retention, deletion, training, and data-location settings before deployment.
- Verify that vendor protections align with University requirements and approved contractual terms.
Identity & Access Management
- Use HarvardKey and approved authentication methods whenever possible.
- Apply least-privilege access controls to data, tools, APIs, repositories, and connected applications.
- Restrict AI systems to only the resources necessary for their intended function.
- Store credentials and secrets in approved enterprise solutions.
Agents, Connectors & Automation
- Review AI agents, connectors, plug-ins, APIs, MCP servers, and other integrations before deployment.
- Treat external connectors and services as third-party integrations requiring appropriate security and privacy review.
- Grant only the minimum permissions necessary.
- Require human approval for high-impact, sensitive, irreversible, or externally visible actions whenever feasible.
- Maintain the ability to promptly revoke access, disable the connector, or stop the system when it behaves unexpectedly.
Secure Development & Administration
- Review repository-level AI configuration files, agent instructions, system prompts, automation settings, and tool definitions before use.
- Disable unnecessary automated execution features.
- Validate AI-generated code, infrastructure changes, queries, scripts, and configuration before deployment.
- Follow secure development and change-management practices.
- Use approved libraries, dependencies, models, APIs, and deployment environments.
Environment Isolation
- Run AI coding agents and automation tools in isolated environments when feasible.
- Limit access to sensitive systems, credentials, production data, and administrative workstations.
- Separate development, testing, and production environments.
- Use sandboxing, network restrictions, and controlled test data for systems that can execute code or call external services.
Logging, Monitoring & Testing
- Maintain appropriate logging of AI-enabled activities and administrative actions.
- Monitor for unusual behavior, excessive usage, unexpected access patterns, data leakage, prompt injection, and anomalous automated actions.
- Test AI-enabled systems for prompt injection, malicious content, data exfiltration, misuse and unexpected behaviors.
- Evaluate systems for misuse, manipulation, and unexpected behavior before production deployment.
Vendor Management
- Use approved vendors and services.
- Review contractual protections related to data use, retention, model training, confidentiality, security, incident response, and deletion.
- Understand enterprise versus consumer service boundaries.
- Establish off-boarding and data-deletion processes for the primary and integrated systems in scope.