Managing for Quality
in the Age of AI
As AI use grows across the organization, maintaining quality requires both intentional leadership systems and a culture where everyone owns their output. This resource addresses both.
Sample AI Quality Control Risks by Role
Select a team to view the specific quality risks they face when using AI tools.
- Deceptive polish: AI-generated lesson plans or rubrics look complete and professional but may miss grade-level alignment, scaffold incorrectly, or reflect generic rather than standards-specific content.
- Factual inaccuracy: AI can produce incorrect content knowledge, misattribute quotes, fabricate source texts, or generate problems with wrong answers—all packaged in confident, teacher-ready formatting.
- Loss of voice: AI-drafted IEP notes, parent emails, or progress reports may lose the relational tone families expect and strip out the nuance that reflects actual knowledge of a child.
- Rigor collapse: AI may suggest (or plan) differentiation moves that actually undermine rigor, such as text leveling.
- Erosion of pedagogical judgment: Over-reliance on AI for planning may lessen teacher's intellectual preparation, impacting their ability to anticipate misconceptions, sequence instruction responsively, or make in-the-moment instructional decisions.
- Privacy exposure: Teachers may paste student names, IEP details, behavioral notes, or assessment data into unapproved AI tools, creating FERPA violations and data security risks.
- Deceptive polish: AI-generated observation feedback, professional development materials, or curriculum reviews can look impressive while missing the specific instructional context that makes feedback/plans actionable.
- Factual inaccuracy: AI may misrepresent research findings, cite retracted or non-existent studies, or mischaracterize what a standards progression actually requires when generating PD content or curricular guidance.
- Loss of coaching voice: AI-drafted feedback or walkthrough notes can flatten the relational, developmental tone that effective coaching requires.
- Superficial analysis: AI can produce polished classroom observation summaries or trend analyses that pattern-match to frameworks without capturing the instructional moves that actually matter in a given context.
- Erosion of instructional expertise: Leaders who rely on AI to identify look-fors, write feedback, or analyze instruction risk atrophying the very expertise their role demands: the ability to see, name, and develop quality teaching.
- Privacy exposure: Instructional leaders may input teacher evaluation data, coaching notes, or student performance details into AI tools, crossing both FERPA and personnel confidentiality boundaries.
- Deceptive polish: AI-generated dashboards, data summaries, or technical documentation can look clean and complete while containing calculation errors or misleading visualizations.
- Factual inaccuracy: AI can produce queries that run without errors but return wrong results, generate reports with incorrect statistics, or misinterpret data.
- Loss of institutional voice: AI-drafted communications to schools about system changes, data protocols, or compliance requirements may miss the organizational context and relationship dynamics that determine whether guidance actually gets followed.
- Superficial analysis: AI can generate what appears to be a thorough data analysis—complete with charts and narratives—while missing the contextual factors (demographic shifts, policy changes, data collection irregularities) that an experienced analyst would flag.
- Privacy exposure: Data and IT staff routinely handle the most sensitive system data—student records, network configurations, vendor credentials—and pasting these into AI tools creates the highest-severity security and compliance risks in the organization.
- Deceptive polish: AI-generated job postings, interview questions, or evaluation summaries can appear thorough and well-crafted while embedding bias, misrepresenting role requirements, or using language that inadvertently discourages diverse applicants.
- Factual inaccuracy: AI may generate interview protocols that conflict with org culture, produce inaccurate summaries of HR policies, or fabricate policy references in employee communications.
- Loss of institutional voice: AI-drafted offer letters, corrective action documents, or culture communications can lose the specific tone and relational awareness that HR communications require.
- Superficial analysis: AI can produce what looks like a rigorous analysis of hiring pipelines, retention patterns, or equity metrics while missing the contextual factors (school culture, leadership transitions, community dynamics) that explain the numbers.
- Erosion of people expertise: Over-reliance on AI for screening, writing evaluations, or drafting personnel recommendations can weaken HR professionals' judgment about candidate fit, employee development needs, and organizational culture.
- Privacy exposure: HR teams handle highly sensitive personnel data—social security numbers, medical documentation, disciplinary records, salary information—and any use of unapproved AI tools with this data creates serious legal liability and trust violations.
The Management Dilemma
Both over-managing and under-managing AI adoption carry real risks. The goal is to find the approach that maximizes opportunity while managing risk—and adjust as your organization's AI maturity grows.
Over-Managing Risks
Blanket restrictions, burdensome approvals, and a culture of suspicion will stifle adoption, frustrate staff, and cause your organization to fall behind.
Under-Managing Risks
A laissez-faire approach with no guardrails risks quality failures, reputational harm, and erosion of stakeholder trust.
The Quality Control Framework
These strategies are organized into three phases. Start with Phase 1 and build from there as your organization's comfort and capability grows.
Guidelines should cover approved software, encouraged use cases, areas where AI use is risky (such as with sensitive student data), and expectations for transparency when work has been meaningfully assisted by AI.
Clearly state that while AI use is encouraged, employees, not AI, are responsible for their work. This means carefully reviewing and revising AI output. "The AI did it" is never an acceptable excuse. Leaders should reinforce this in team meetings, in feedback conversations, and in how they respond when quality issues surface.
Training should cover AI benefits and risks, how to critically evaluate AI output, the organization's use guidelines and quality expectations, and practical techniques for getting better results from AI tools.
Staff should know which software is approved for which data types and why using an unapproved tool with sensitive data creates risk. A simple one-page reference card goes a long way here.
One of the biggest quality risks with AI is that it produces work that looks polished but is shallow, generic, or just off. Normalize colleagues flagging this in each other's work. This isn't about catching people; it's about building a collective sense of what "good" actually looks like in your context versus what AI defaults to.
Help staff develop workflow techniques that produce higher-quality, more context-specific results. Encourage sharing across teams. When someone figures out an approach that works well, make it easy for others to learn from it.
Which software and use cases are producing real value? Where are quality issues recurring? Collect this information regularly, use it to refine your guidelines and training, and share what you're learning with staff.
For work that is high-volume, high-impact, or high-risk, have organizational leaders or designated teams design and test workflows centrally. This might look like shared prompt templates for common communications, standardized AI-assisted processes for recurring analytical tasks, or clear approval workflows for AI applications touching sensitive areas.
When a team or department finds an AI approach that improves quality and efficiency, make that knowledge available broadly. Build an internal repository of effective practices, prompts, and workflows. Celebrate innovation and improvement, not just compliance.
Revisit guidance regularly, which may become less restrictive over time as organizational maturity grows. What requires centralized approval in year one may need only peer review in year two and individual judgment in year three. Build in at least annual review points to assess whether your governance matches your organization's current capabilities.
Calibrating for Risk
As you implement these strategies, calibrate the level of oversight based on the stakes involved. Not all AI-assisted work requires the same controls.
Maximum oversight and documentation required
Examples
- IEPs and special education documents
- Board presentations and policy documents
- Family/community communications
- Data analysis informing major decisions
- Legal or compliance-related content
Quality Controls
- Human ownership required (staff member accountable as author)
- AI may support drafting, but not final decision-making
- Mandatory expert review before use/publishing
- Source verification required (citations, cross-checking)
- Equity/tone review required (risk of harm)
- Disclosure required (internal and external when appropriate)
- Version control + documentation of review steps
Human review with peer oversight
Examples
- Staff-facing communications
- Curriculum support materials
- Meeting summaries and agendas
- Routine reports and updates
Quality Controls
- Human review required before sharing
- Peer or supervisor review encouraged (required for sensitive topics)
- Use of rubrics/checklists (accuracy, tone, audience fit)
- Spot-checking by managers to prevent quality drift
- Required edits/revisions (not "copy-paste AI output")
Individual judgment with basic verification
Examples
- Internal brainstorming and ideation
- First drafts for further revision
- Personal productivity tasks
- Research and information gathering
Quality Controls
- Human judgment required (AI as thought partner, not source of truth)
- Basic fact-checking required before reuse
- No confidential/student-identifiable information
- Encourage experimentation and iteration
- Training + norms are usually sufficient
Building a Culture That Supports Quality
Culture shows up in the small moments when no one is looking. These three questions are designed to become those moments. When teams share them as common language, quality is not something a manager enforces and becomes something a team expects of itself.
For further guidance, see Fostering a Culture of Innovation and Amplifying Power Users and Creating a Culture of Innovation.
Who is the audience, and what's at stake?
A quick survey summary for the team and a polished report for the board are both good uses of AI. They just need different levels of care. A survey tally for an internal meeting? Skim it, make sure the numbers look right, move on. A data narrative going to the board? That needs real scrutiny — every claim checked, every interpretation pressure-tested.
What do I know that the AI doesn't?
AI can help prepare for a difficult coaching conversation — but it doesn't know the person being coached the way their manager does. What motivates them? What doesn't? What mood do they seem to be in today? These are the subtle things that leaders pick up on and intuit that no AI script can capture. This applies everywhere: the context behind a data trend, the history with a particular family, the unwritten dynamics on a team. AI provides a strong starting point. Professional judgment is what makes the output actually fit the situation.
Would I put my name on this?
AI can help teams work faster, brainstorm new ideas, and unlock creative potential. But at the end of the day, anything submitted is the submitter's work. That means making sure information is accurate, plans are not just polished but actually actionable, and the tone reflects the person — not just what a model generated. If it's not ready for a name on it, it's not done yet.
These aren't compliance questions — they're professional habits. The goal is a team that uses AI confidently because they take ownership of what it produces.