Large Language Models and Generative AI

LLM applications across customer-facing, agent-facing, and back-office operations.

Large language models (LLMs) are a class of artificial intelligence systems trained on massive text datasets to generate, summarize, translate, and reason about human language. Built on the transformer architecture introduced in 2017, LLMs represent a step change from earlier AI approaches. Where previous systems required hand-crafted rules or task-specific training data, a single large language model can perform thousands of language tasks—from drafting emails to analyzing policies to holding multi-turn conversations—without being explicitly programmed for any of them.

Generative AI is the broader category of AI systems that create new content—text, images, code, audio, video—rather than merely classifying or predicting from existing data. LLMs are the most commercially significant form of generative AI as of 2026, and the form most directly relevant to workforce management operations.

For WFM practitioners and leaders, understanding LLMs is no longer optional. These models power the conversational AI agents handling customer interactions, the real-time assist tools supporting human agents, the automated summarization reducing after-call work, and an emerging generation of agentic AI systems capable of autonomous planning and execution. Gartner projects that more than 80% of enterprises will have deployed generative AI-enabled applications by 2026, and McKinsey estimates generative AI could add $2.6 to $4.4 trillion in annual economic value across industries, with customer operations among the most impacted functions.

This article explains how LLMs work, what they can and cannot do, and how they apply specifically to contact center operations and workforce management. It is written for practitioners who need working knowledge, not computer science expertise.

How Large Language Models Work

Large language models are built on the transformer architecture, a neural network design that revolutionized natural language processing when it was introduced by researchers at Google Brain in 2017. Understanding three core concepts explains most of what practitioners need to know about how these systems function.

The Attention Mechanism

The breakthrough innovation in the transformer was the attention mechanism — a method that allows the model to weigh the relevance of every word in a passage against every other word simultaneously. Before transformers, language models processed text sequentially, one word at a time, like reading a sentence left to right. The attention mechanism lets the model consider the entire context at once.

Consider the sentence: "The agent said she would transfer the caller to billing because that department handles refund disputes." When processing the word "she," the attention mechanism determines that "she" refers to "agent" (not "caller") by examining the relationships between all words in the sentence. When processing "that department," it connects the phrase to "billing." This ability to track long-range dependencies in text is what makes LLMs effective at understanding and generating coherent language.

In technical terms, the attention mechanism computes a set of weights that determine how much each word should "attend to" every other word. These weights are learned during training, not programmed by hand. The original transformer paper described this as "self-attention" because each position in a sequence attends to all other positions in the same sequence.

Training on Text

LLMs are trained on enormous corpora of text — books, websites, articles, code repositories, and other written material. The scale is difficult to overstate: models like GPT-4 and Claude were trained on datasets measured in trillions of tokens (roughly, word fragments).

During training, the model processes this text and adjusts billions of internal parameters (also called weights) to become better at the core task described below. Training a frontier LLM requires thousands of specialized processors (GPUs or TPUs) running for weeks or months, at costs that can reach tens or hundreds of millions of dollars. This is why most organizations use LLMs created by foundation model providers rather than training their own.

Next-Token Prediction

The fundamental task during training is deceptively simple: given a sequence of text, predict the next word (technically, the next token — see Tokens below). The model sees "The workforce planner reviewed the forecast and noticed that Tuesday's predicted volume was" and learns to predict plausible continuations like "higher" or "significantly" or "30%."

By performing this prediction task across trillions of text examples, the model develops sophisticated internal representations of language, facts, reasoning patterns, and even writing styles. It does not memorize specific texts; rather, it learns statistical patterns across the entire training corpus. The resulting model can then generate coherent, contextually appropriate text by predicting one token at a time, each prediction conditioned on all previous tokens.

This is important to understand: LLMs generate text sequentially, one token at a time. A 500-word response involves hundreds of individual predictions, each informed by everything that came before. The fluency of the output can obscure this underlying process, leading users to attribute understanding or intent where the mechanism is sophisticated pattern completion.

Key Concepts

Practitioners working with LLMs or evaluating LLM-powered tools encounter a specific vocabulary. The following concepts appear regularly in vendor documentation, product evaluations, and internal deployment discussions.

Pre-Training

Pre-training is the initial phase where a model learns from a massive general-purpose text corpus. This phase produces a foundation model — a general-purpose system that understands language broadly but is not specialized for any particular task. Pre-training is extremely expensive and is performed by foundation model companies (OpenAI, Anthropic, Google, Meta, and others). WFM organizations consume pre-trained models; they do not create them.

Fine-Tuning

Fine-tuning takes a pre-trained model and trains it further on a smaller, task-specific dataset. A contact center vendor might fine-tune a foundation model on thousands of customer service transcripts to improve its performance at call summarization or intent classification. Fine-tuning is far less expensive than pre-training and is within reach of organizations with sufficient data and technical capability. However, the rapid improvement of prompting techniques (below) has reduced the need for fine-tuning in many use cases.

Prompting and Prompt Engineering

A prompt is the input text given to an LLM to elicit a desired response. Prompt engineering is the practice of crafting prompts that reliably produce useful outputs. Unlike traditional software where behavior is determined by code, an LLM's behavior is significantly shaped by how the request is worded.

For example, asking an LLM "Summarize this call" will produce a different result than "Summarize this call in three bullet points, focusing on the customer's issue, the resolution provided, and any follow-up actions." The second prompt is more constrained and typically produces more consistent, useful output. Prompt engineering has emerged as a practical skill for operations teams deploying LLM-powered tools, and is a key component of the AI scaffolding framework that determines how effectively organizations deploy these systems.

Context Window

The context window is the maximum amount of text an LLM can process in a single interaction — both the input prompt and the generated response combined. Context windows are measured in tokens and vary by model: early GPT-3 models supported 4,096 tokens (roughly 3,000 words), while models in 2025–2026 commonly support 128,000 to 1,000,000 tokens or more.

The context window determines what an LLM can "see" at any given moment. A model with a 128,000-token window can process an entire policy manual in a single query. A model with a 4,000-token window cannot. For WFM applications such as policy document Q&A, knowledge article generation, or multi-document analysis, context window size directly affects feasibility and quality.

Tokens

Tokens are the fundamental units that LLMs process. A token is not exactly a word — it is a sub-word fragment determined by the model's tokenizer. Common words like "the" or "call" are single tokens. Less common words are split: "workforce" might become "work" + "force," and "shrinkage" might become "shrink" + "age." On average, one token corresponds to roughly 0.75 English words, or conversely, 100 words is approximately 130–140 tokens.

Tokens matter practically because LLM pricing is typically per-token (both input and output), and the context window is measured in tokens. Understanding tokenization helps practitioners estimate costs and plan for context window limitations.

Temperature

Temperature is a parameter that controls the randomness of an LLM's output. At low temperature (e.g., 0.0–0.2), the model consistently selects the most probable next token, producing deterministic, predictable responses. At high temperature (e.g., 0.8–1.0), the model samples more broadly from probable tokens, producing varied and creative — but less predictable — output.

For WFM applications, low temperature is generally preferred. A call summarization system should produce consistent, reliable summaries. A forecast narrative generator should be factual and repeatable. High temperature is appropriate when variety is desired, such as generating multiple draft versions of training content.

Capabilities and Limitations

LLMs exhibit striking capabilities alongside fundamental limitations. Understanding both is essential for realistic deployment planning and for evaluating the claims of vendors incorporating LLMs into their products.

What LLMs Do Well

• Text generation and summarization — LLMs excel at producing fluent, coherent text and at condensing longer documents into shorter summaries. Call summarization, report generation, and knowledge article drafting are natural applications.
• Translation and reformulation — Converting text between languages, registers, or formats. A technical WFM policy can be restated for a frontline audience. A customer complaint in Spanish can be summarized in English.
• Information extraction — Identifying and structuring specific information from unstructured text. Extracting action items from meeting transcripts, categorizing call reasons from conversation logs, or pulling key metrics from narrative reports.
• Conversational interaction — Maintaining multi-turn conversations that respond coherently to context. This capability underlies conversational AI agents in customer-facing and employee-facing applications.
• Pattern recognition in text — Identifying sentiment, tone, intent, and topic across large volumes of text. This complements and in some cases replaces traditional speech analytics approaches.
• Code generation — Writing, explaining, and debugging software code. This capability accelerates WFM tool development and custom reporting.

What LLMs Do Poorly

• Hallucination — LLMs sometimes generate plausible-sounding but factually incorrect information. The model may cite a policy that does not exist, invent a statistic, or confidently describe a procedure incorrectly. Hallucination is not a bug that will be fixed in the next version; it is an inherent consequence of how next-token prediction works. Any deployment that uses LLM output as factual reference must include verification mechanisms.
• Reasoning limitations — Despite impressive performance on many reasoning tasks, LLMs can fail at multi-step logical reasoning, mathematical computation, and problems requiring systematic analysis. They are better at pattern-matching against reasoning they have seen than at novel reasoning from first principles. For WFM calculations (Erlang models, optimization, statistical analysis), traditional software remains more reliable than LLM-generated computation.
• Knowledge cutoffs — A model's knowledge reflects its training data, which has a cutoff date. A model trained through mid-2025 does not know about events, policies, or data from late 2025 onward. This is particularly relevant for WFM applications that reference current schedules, policies, or operational data — the LLM must be provided this information through the prompt or connected data sources, not relied upon to "know" it.
• Inconsistency — The same prompt can produce different outputs on repeated runs (especially at higher temperature settings), and slightly different phrasings of the same question can produce meaningfully different answers. This stochastic behavior makes LLMs unreliable for tasks requiring strict determinism. See also Deterministic vs Probabilistic Models for a broader treatment of this distinction.
• Lack of real-time awareness — LLMs do not observe the world in real time. They do not know the current service level, today's staffing count, or whether a particular agent is logged in. Any real-time application must feed current data into the model through integration.

Generative AI Beyond Text

While LLMs dominate the generative AI conversation, the same transformer-based approaches have been extended to other modalities.

Image generation models (such as DALL-E, Midjourney, and Stable Diffusion) create images from text descriptions using diffusion-based architectures. Code generation models (such as GitHub Copilot) apply LLM capabilities specifically to programming languages. Multimodal models combine text understanding with image, audio, or video processing — for example, analyzing a screenshot of a dashboard or transcribing and summarizing a recorded meeting.

Audio and speech models generate natural-sounding speech from text (text-to-speech) or convert speech to text with high accuracy (speech-to-text), enabling conversational AI systems that can handle voice interactions alongside chat.

For workforce management, text-based generative AI is the primary application area. Contact center operations are fundamentally text-heavy: transcripts, policies, knowledge articles, forecasts, schedules, performance reviews, and reports are all textual. Image and video generation have niche applications (training content, visual aids) but do not represent the core WFM use case for generative AI. The remainder of this article focuses on LLM-based text generation and its contact center and WFM applications.

LLMs in Contact Center Operations

Contact centers were among the first operational environments to deploy LLMs at scale, driven by the natural fit between language models and language-intensive work. Applications fall into three categories: customer-facing, agent-facing, and back-office.

Customer-Facing Applications

LLMs power a new generation of conversational AI agents that handle customer interactions directly — through chat, email, and increasingly through voice. Unlike earlier chatbots built on rigid decision trees and intent-classification models, LLM-powered agents can understand nuanced customer language, handle unexpected turns in conversation, maintain context across long interactions, and generate natural, human-sounding responses.

These AI agents operate within guardrails defined by the deploying organization: they follow specific policies, escalate when uncertainty is high, and log interactions for review. The containment rate — the percentage of interactions fully resolved by AI without human intervention — has become a key operational metric. Gartner predicts that agentic AI will autonomously resolve 80% of common customer service issues by 2029.

The supervision and escalation framework governing these AI agents determines when and how interactions are handed to human agents. This framework directly affects workforce planning: higher containment rates reduce the volume of human-handled contacts, changing staffing requirements, skill mix, and scheduling patterns. See Workforce Planning with AI Agents for a detailed treatment.

Agent-Facing Applications

LLMs deployed to assist human agents during and after interactions represent the most widely adopted contact center application as of 2026.

Real-time assist systems monitor live customer interactions and provide agents with suggested responses, relevant knowledge articles, policy guidance, and next-best-action recommendations. These systems reduce handle time, improve first-contact resolution, and flatten the learning curve for new agents. The global real-time AI agent assist market was valued at $4.4 billion in 2024.

Auto-summarization uses LLMs to generate structured call summaries immediately after (or during) an interaction, reducing after-call work (ACW) by 45–50% in documented deployments. For a 1,000-seat center where agents spend 60 seconds on average completing post-call documentation, eliminating half of that work recovers approximately 8,300 productive agent-hours per year — a meaningful workforce planning input.

Knowledge retrieval systems use LLMs to search, interpret, and present information from knowledge bases in conversational form. Rather than an agent searching for a keyword and scanning articles, the agent asks a question in natural language and receives a direct, contextualized answer drawn from the organization's knowledge repository.

Back-Office Applications

Behind the operational front line, LLMs assist with:

• Report generation — Transforming structured data (performance metrics, trend analyses, forecast accuracy reports) into written narratives. A WFM team can generate a weekly operations summary from raw data rather than writing it manually.
• Policy document Q&A — Enabling supervisors and analysts to query dense policy documentation in natural language. "What is our overtime authorization policy for agents under 90 days?" returns a direct answer with source citation rather than requiring manual search.
• Training content creation — Generating draft training materials, scenario-based exercises, and knowledge checks from policy documents and interaction examples. Human subject matter experts review and refine the output rather than creating from scratch.
• Quality management — Analyzing 100% of interactions for quality signals rather than the traditional 2–5% sample. LLMs can evaluate interactions against quality criteria, flag exceptions, identify coaching opportunities, and generate coaching summaries.

WFM-Specific Applications

Beyond general contact center use cases, LLMs enable several applications specific to workforce management practice.

Forecast Narrative Generation

WFM analysts regularly produce written explanations accompanying numerical forecasts: why next week's volume is projected higher, what drove the variance between forecast and actual, what assumptions underlie the quarterly capacity plan. LLMs can draft these narratives from structured forecast data, saving analyst time while maintaining consistency. The analyst reviews and adjusts the narrative rather than composing it from a blank page. This application directly leverages the LLM's strength (fluent text from structured inputs) while avoiding its weakness (the underlying calculations remain in purpose-built forecasting systems).

Schedule Explanation Generation

When agents or supervisors ask "why did I get this schedule?" the answer typically involves a complex interplay of business rules, seniority, preferences, coverage requirements, and optimization constraints. LLMs can translate the technical optimization output into plain-language explanations: "Your Tuesday start time is 10:00 AM because your seniority rank (47 of 312) combined with the high coverage requirement during the 10:00–14:00 peak means earlier shifts were assigned to higher-seniority agents. Your Wednesday off-day was assigned because Wednesday has the lowest forecasted volume this week."

This capability improves schedule transparency and reduces the volume of supervisor inquiries, a measurable operational benefit.

Automated WFM Reporting

Weekly business reviews, monthly capacity reports, and ad hoc variance analyses all involve converting data into narrative. LLMs can generate first drafts of these reports from standardized data inputs, applying consistent structure, tone, and formatting. Analysts shift from writing to reviewing and editing — a faster workflow that also ensures reports maintain organizational standards.

Knowledge Article Generation

Contact center knowledge bases require continuous maintenance as products, policies, and procedures change. LLMs can draft new knowledge articles from source material (policy documents, product specifications, training decks), update existing articles when source material changes, and identify gaps in knowledge coverage by analyzing interaction transcripts for questions that existing articles do not address. Human review remains essential — the consequences of an incorrect knowledge article propagated to agents or customers are significant — but the drafting and updating workflow is substantially accelerated.

The Agentic AI Frontier

The most significant evolution of LLMs as of 2025–2026 is the emergence of agentic AI — systems where an LLM acts not just as a text generator but as an autonomous agent capable of using tools, making decisions, executing multi-step plans, and operating with limited human supervision.

An agentic AI system differs from a simple LLM interaction in several ways:

• Tool use — The LLM can invoke external tools: querying a database, calling an API, running a calculation, searching a document repository. It is not limited to the information in its training data or its context window.
• Planning — The system can decompose a complex goal into sub-tasks and execute them in sequence or in parallel. "Generate next week's staffing recommendations" might involve retrieving the volume forecast, checking current agent availability, applying business rules, running an optimization, and producing a written summary.
• Memory and state — Agentic systems maintain state across interactions, remembering previous actions, decisions, and their outcomes. This enables iterative refinement and learning from feedback within a session.
• Autonomy — Within defined guardrails, the system operates without requiring human approval at every step. The degree of autonomy varies by implementation and risk tolerance.

For workforce management, agentic AI represents the path from "AI that answers questions" to "AI that does work." See Agentic AI Workforce Planning and AI Agent Orchestration for WFM for detailed coverage of architectures, use cases, and governance frameworks. The human supervision framework becomes critical as these systems move from advisory roles to operational execution.

Gartner projects that by 2028, 33% of enterprise software applications will incorporate agentic AI capabilities, up from less than 1% in 2024.

Workforce Implications

The deployment of LLMs and generative AI in contact center operations carries significant implications for workforce composition, job design, and employment.

Job Displacement vs. Augmentation

The debate over whether AI displaces or augments workers is not binary — it is role-specific and implementation-dependent. McKinsey's analysis estimates that generative AI could automate 60–70% of worker activities (up from a prior estimate of 50% before generative AI), with customer operations among the most affected functions. However, automating activities is not the same as eliminating jobs. Most roles comprise a mix of automatable and non-automatable activities.

In contact center operations, the pattern emerging through 2025–2026 is:

• Routine, repetitive interactions are increasingly handled by AI agents, reducing demand for agents performing simple, script-driven work.
• Complex, judgment-intensive interactions continue to require human agents, often augmented by AI-powered assist tools that improve their effectiveness.
• Supervisory and quality roles are transformed rather than eliminated — the work shifts from manually reviewing a sample of interactions to overseeing AI-driven quality systems and handling exceptions.
• WFM roles are augmented: forecasting, scheduling, and real-time management benefit from AI-generated insights and draft outputs, but human judgment remains essential for interpreting context, managing exceptions, and making trade-off decisions.

New Roles Created

Generative AI deployment creates new roles and skill requirements:

• Prompt engineers and AI configuration specialists who design, test, and maintain the prompts and guardrails governing AI behavior.
• AI trainers and quality analysts who evaluate AI outputs, identify failure patterns, and provide feedback that improves system performance.
• AI operations managers who monitor AI agent performance, manage escalation thresholds, and coordinate between AI systems and human workforces.
• Conversation designers who design the flows, personas, and boundaries of AI-powered customer interactions.

See Organizational Change Management for AI Workforce Transitions for frameworks addressing the organizational transformation required to integrate AI into contact center operations.

The Transition Challenge

The workforce impact is not evenly distributed. Entry-level roles that historically served as the on-ramp to contact center careers are most affected by AI containment. Organizations face a pipeline challenge: if AI handles the simple interactions that new agents once cut their teeth on, how do future supervisors, quality analysts, and WFM professionals develop the operational judgment that comes from direct customer interaction experience? This question has no settled answer and represents an active challenge for workforce strategists.

Implementation Considerations

Organizations evaluating or deploying LLM-powered solutions face several practical decisions.

Build vs. Buy

Few organizations will train their own foundation models — the cost and expertise required are prohibitive. The build-vs-buy decision instead centers on:

• Buy a packaged solution — Use an LLM-powered product from a CCaaS vendor or specialized provider. Fastest to deploy, least flexibility, ongoing subscription cost. Best for standard use cases (summarization, quality scoring, conversational AI).
• Build on a foundation model API — Use APIs from foundation model providers (OpenAI, Anthropic, Google, AWS Bedrock) to build custom applications tailored to organizational needs. Requires technical capability but offers full control over prompts, data flows, and business logic. The AI scaffolding framework provides a structured approach to this build pattern.
• Fine-tune a model — Take an open-weight model (such as Meta's Llama series) or a commercial model and fine-tune it on organizational data. Requires significant ML engineering capability and sufficient high-quality training data, but can produce superior performance for specific tasks.

Data Privacy and Security

LLMs process text, and contact center text contains customer data — names, account numbers, complaint details, health information, financial records. Key considerations include:

• Data residency — Where is the data processed and stored? Cloud-based LLM APIs typically process data in the provider's infrastructure. Regulated industries may require on-premise or sovereign cloud deployments.
• Data retention — Do LLM providers retain input data? Most commercial API providers offer zero-retention options, but the terms must be verified and contractually bound.
• Training data use — Will customer interaction data be used to train or improve the model? Reputable providers offer opt-out guarantees for API customers, but free-tier and consumer-facing products often do use input data for training.
• PII handling — Sensitive data should be masked or redacted before being sent to LLMs where possible. Architectures that separate PII from the text processed by the LLM reduce exposure.

Cost Structure

LLM costs follow a per-token pricing model for API-based usage. Organizations pay for both input tokens (the prompt and any context provided) and output tokens (the model's response). Output tokens are typically 3–5× more expensive than input tokens. Costs vary by model capability: more powerful models cost more per token.

For volume applications like call summarization, the economics must be calculated carefully. Summarizing 10,000 calls per day, each with a 2,000-token transcript generating a 200-token summary, involves approximately 22 million tokens per day. At typical 2025 pricing for a capable model, this might cost $50–200 per day — modest relative to the labor cost savings, but requiring monitoring as volumes scale.

On-premise or private cloud deployments involve different economics: high upfront infrastructure costs (GPU servers) but no per-token charges. The break-even point depends on usage volume and the specific deployment architecture.

Vendor Evaluation

When evaluating LLM-powered products, WFM and operations leaders should assess:

• Accuracy on relevant tasks — Vendor demonstrations on cherry-picked examples are insufficient. Request evaluation on the organization's actual data, including edge cases and failure modes.
• Latency — Real-time applications (agent assist, conversational AI) require sub-second response times. Batch applications (report generation, quality analysis) are more tolerant.
• Customization — Can the system be tuned to the organization's terminology, policies, and quality standards? A generic summarization model may not use the organization's preferred format or terminology.
• Integration — Does the solution integrate with existing telephony, CRM, workforce management, and knowledge management systems?
• Transparency — Can the organization inspect prompts, review model versions, and understand when changes are made? Black-box AI systems create operational risk.

Responsible Deployment

Deploying LLMs in customer-facing and employee-affecting operations requires deliberate attention to safety, fairness, and accountability. Anthropic's research on Constitutional AI demonstrates one approach to building safety constraints directly into model training, but organizational responsibility extends well beyond the model itself.

Bias and Fairness

LLMs trained on internet text inherit the biases present in that text. In contact center applications, this can manifest as:

• Differential treatment of customers based on dialect, language proficiency, or name
• Biased quality scoring that penalizes non-native English speakers
• Scheduling or routing recommendations that correlate with protected characteristics

Organizations must test for bias in their specific deployment context, monitor for drift over time, and maintain human oversight of consequential decisions. The human oversight framework should include bias monitoring as a core function.

Transparency and Explainability

Agents, supervisors, and customers affected by AI-driven decisions deserve transparency about when and how AI is involved. Best practices include:

• Disclosing to customers when they are interacting with an AI agent
• Making AI-generated content (summaries, quality scores, schedule explanations) identifiable as AI-generated
• Providing appeal or override mechanisms when AI decisions affect employees (scheduling, quality evaluation, performance scoring)

Safety and Guardrails

LLMs can generate harmful, inappropriate, or off-policy content if not properly constrained. Production deployments require:

• Prompt-level guardrails — System prompts that constrain the model's behavior, define its role, and specify prohibited actions
• Output filtering — Automated checks on model output before it reaches customers or agents
• Escalation triggers — Conditions under which the system stops generating and hands off to a human
• Regular evaluation — Ongoing testing against adversarial inputs and edge cases, not just pre-deployment validation

Regulatory Environment

The regulatory landscape for AI, including LLMs, is evolving rapidly. The EU AI Act (effective 2024–2026) classifies AI systems by risk level and imposes requirements on high-risk applications, which may include certain customer service and employment-related AI uses. Organizations deploying LLMs in contact center operations should monitor regulatory developments and ensure their governance frameworks can adapt.

See Also

• Artificial Intelligence Fundamentals
• Machine Learning Concepts
• Neural Networks and Deep Learning
• Natural Language Processing
• AI in Workforce Management
• AI Scaffolding Framework
• Conversational AI
• Speech Analytics
• Agentic AI Workforce Planning
• AI Agent Orchestration for WFM
• Human AI Supervision and Escalation Frameworks
• AI Containment Rate and Its Workforce Implications
• Workforce Planning with AI Agents
• AI and Employment
• Organizational Change Management for AI Workforce Transitions
• Knowledge Management
• Quality Management
• Deterministic vs Probabilistic Models