Skip to main content

Semantic Kernel Core Concepts

Semantic Kernel (SK) is Microsoft’s open‑source AI orchestration framework that enables developers to integrate large language models (LLMs), tools, memory, and enterprise systems into intelligent applications. It provides a consistent SDK across C#, Python, and Java, allowing you to build AI agents and workflows that connect to your existing infrastructure. This handbook article is a practical, implementation‑focused guide to the foundational abstractions of Semantic Kernel—the Kernel, AI services, functions, plugins, prompts, memory, connectors, and agents—and how they work together at runtime.

What Is Semantic Kernel​

Semantic Kernel is an AI orchestration platform packaged as a multi‑language SDK. It lets you:

  • Register AI models (OpenAI, Azure OpenAI, Hugging Face, etc.) as services.
  • Define functions – both native (C#/Python/Java code) and prompt‑based (LLM prompts).
  • Organise functions into plugins for reuse.
  • Use memory connectors to store and retrieve contextual information from vector databases and text search services.
  • Build agents that automatically select and execute the right functions to fulfil a user’s goal.
  • Integrate with enterprise systems through connectors and standard authentication.

A minimal Python example:

import asyncio
from semantic_kernel import Kernel
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from semantic_kernel.functions import KernelArguments

kernel = Kernel()
kernel.add_service(OpenAIChatCompletion(service_id="gpt", api_key="..."))

# A simple prompt function
prompt = """Answer the user's question in a concise way.
User: {{$input}}
Assistant:"""

answer = kernel.add_function(
    function_name="Answer",
    plugin_name="QAPlugin",
    prompt=prompt,
    execution_settings=...
)

result = await kernel.invoke(answer, KernelArguments(input="What is Semantic Kernel?"))
print(result)

This illustrates the core pattern: a Kernel with an AI service, a prompt function, and an invocation that produces a result.

Why Semantic Kernel Exists​

Building production AI applications requires more than a simple API call. Semantic Kernel exists to:

  • Bridge AI and traditional software – bring LLMs into your existing codebase, not replace it.
  • Simplify AI integration – provide a unified abstraction over multiple model providers.
  • Standardise tool orchestration – let the kernel handle function calling, prompt rendering, and context management automatically.
  • Connect enterprise systems – natively work with Azure services, SQL databases, REST APIs, and Microsoft 365.
  • Improve developer productivity – offer a familiar SDK experience with dependency injection, logging, and telemetry.

By providing a common runtime, Semantic Kernel allows you to focus on business logic while the framework handles the orchestration plumbing.

Core Concepts Overview​

ConceptPurpose
KernelCentral execution engine that manages services, plugins, and function invocation.
AI ServicesAbstractions for chat completion, text generation, and embeddings (OpenAI, Azure, etc.).
FunctionsExecutable units of logic—native code or prompt templates.
PluginsContainers that group related functions for reuse and dependency injection.
PromptsTemplates with placeholders that guide LLM behaviour.
MemoryMechanism to store and retrieve information using vector stores or text search.
ConnectorsIntegrations with external data sources and services (databases, APIs, search engines).
AgentsHigher‑level abstractions that autonomously use functions and memory to complete tasks.

Kernel​

What Is the Kernel​

The Kernel is the heart of Semantic Kernel. It acts as a lightweight dependency injection container and an execution coordinator. You register all services (AI models, logging, HTTP clients) and plugins with the kernel. When you invoke a function, the kernel resolves its dependencies, renders prompts, calls AI services, handles function calls, and manages context.

Key responsibilities:

  • Service registration – holds references to AI services, memory connectors, and custom services.
  • Plugin management – keeps a catalogue of available plugins and their functions.
  • Function invocation – orchestrates the execution pipeline: argument building, prompt rendering, AI service calls, native function execution, and output processing.
  • Context management – manages the KernelArguments and FunctionResult that flow through the pipeline.

Creating a kernel and adding services:

kernel = Kernel()
kernel.add_service(OpenAIChatCompletion(service_id="gpt", api_key="..."))
# Add other services, e.g., logging, HTTP client, text generation

The kernel is fully extensible. In .NET, you typically use the builder pattern:

var builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion("gpt-4o", apiKey);
var kernel = builder.Build();

AI Services​

AI services provide access to language models. Semantic Kernel abstracts the underlying provider behind common interfaces.

Supported service types:

  • Chat completion – for conversational agents (OpenAI, Azure OpenAI, local models via Ollama).
  • Text generation – legacy completion models.
  • Text embedding generation – for creating embeddings used in semantic memory.

You register a service with the kernel and reference it by a service_id. Functions can specify which service to use, or you can rely on auto‑selection if you configure execution settings.

from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion

kernel.add_service(OpenAIChatCompletion(service_id="gpt-4o", api_key="..."))

Multiple services can be added, and you can select the model per function invocation by passing an execution_settings object.

Functions​

Functions are the fundamental building blocks. They are what the kernel executes. There are two main types:

1. Native Functions​

Ordinary methods written in C#, Python, or Java, decorated with [KernelFunction] (or the Python equivalent). They can accept Kernel, KernelArguments, and custom parameters.

Python example:

from semantic_kernel.functions import kernel_function

class MathPlugin:
    @kernel_function(name="Add", description="Add two numbers")
    def add(self, a: int, b: int) -> int:
        return a + b

Native functions are ideal for deterministic operations, API calls, or business logic.

2. Prompt Functions (Semantic Functions)​

Defined as a prompt template string (often with YAML metadata for configuration). They leverage an AI service to generate a response.

Prompt template:

prompt_template = """
Summarise the following text in 3 sentences:
Text: {{$input}}
Summary:
"""

You can add a prompt function to the kernel via kernel.add_function(...). The kernel renders the template, sends it to the AI service, and returns the generated text.

Functions are invoked with kernel.invoke(function, KernelArguments(...)). The kernel automatically resolves the correct AI service for prompt functions or calls native functions directly.

Plugins​

Plugins are collections of related functions. They promote reuse and modularity. A plugin can contain both native and prompt functions.

Creating a plugin in Python by defining a class:

class TimePlugin:
    @kernel_function(description="Get the current time")
    def get_time(self) -> str:
        return datetime.now().isoformat()

Register the plugin with the kernel:

kernel.add_plugin(TimePlugin(), plugin_name="Time")

Now you can invoke Time.GetTime. Plugins can be loaded from separate files (prompt templates in YAML) or from classes. In .NET, you can import OpenAPI specs as plugins automatically.

Prompts​

Prompts define the behaviour of an AI agent. They are templates with placeholders ({{$input}}, {{$parameter}}) that get populated at runtime from the KernelArguments. Semantic Kernel uses a powerful template language inspired by Handlebars to create dynamic prompts.

You can also define execution settings (model, temperature, max tokens) alongside the prompt in a YAML file.

Example of a YAML‑defined prompt function:

name: Summarize
description: Summarize a long text
template: |
  Summarize the following text in a concise paragraph:
  {{$input}}
template_format: semantic-kernel
execution_settings:
  default:
    model_id: gpt-4o
    temperature: 0.3
    max_tokens: 200

Loading the prompt:

plugin = kernel.add_plugin_from_prompt_directory("./Prompts")

Prompts are first‑class citizens; the kernel handles rendering and execution automatically.

Memory​

Semantic Kernel memory provides a way to store and retrieve information across conversations. It is not a database itself; rather, it integrates with memory connectors that talk to vector databases (Azure AI Search, Chroma, Pinecone, etc.) or text search services.

Key components:

  • Memory store – an abstraction for storing and searching text records with embeddings.
  • Semantic text memory – a higher‑level API that allows you to save information, search by similarity, and retrieve context.

Example of using semantic memory:

from semantic_kernel.memory import SemanticTextMemory, VolatileMemoryStore
from semantic_kernel.connectors.ai.open_ai import OpenAITextEmbedding

kernel.add_service(OpenAITextEmbedding(service_id="embed", ...))
memory = SemanticTextMemory(storage=VolatileMemoryStore(), embeddings_generator=kernel.get_service("embed"))

await memory.save_information(collection="conversations", id="1", text="User prefers email communication.")
results = await memory.search("conversations", "How does the user like to be contacted?")

Plugins and agents can access memory to ground responses. Memory is typically injected into the kernel and then accessed by functions.

Connectors​

Connectors are pre‑built integrations with external services. They come in two flavours:

  • AI connectors – to various model providers (OpenAI, Azure, Hugging Face, Ollama).
  • Memory connectors – to vector databases and search services (Azure AI Search, Redis, Qdrant, etc.).
  • Service connectors – to Microsoft Graph, SharePoint, OneDrive, SQL, etc. (more enterprise‑focused).

Connectors allow your agents to interact with the world without writing boilerplate. For example, you can use a MicrosoftGraphConnector to read emails, or a SqlConnector to query a database.

Agents​

In the latest Semantic Kernel versions, agents are higher‑level abstractions that combine a kernel, plugins, and memory to autonomously complete tasks. The most common is the ChatCompletionAgent, which can use the chat completion API and function calling to determine which plugin functions to invoke.

Creating an agent:

from semantic_kernel.agents import ChatCompletionAgent
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion

agent = ChatCompletionAgent(
    kernel=kernel,
    name="SupportAgent",
    instructions="You help customers with orders.",
    plugins=[OrderPlugin()],
    service=OpenAIChatCompletion(service_id="agent-gpt", api_key="...")
)

response = await agent.get_response(messages="Where is my order 123?")

The agent’s lifecycle:

  1. Receives a user message.
  2. The kernel sends the message + instructions + available functions to the AI service.
  3. If the model decides to call a function, the kernel executes it and feeds the result back.
  4. The loop continues until the agent returns a final text response.

Agents can also be used in multi‑agent scenarios with group chats and handoffs (via the experimental AgentGroupChat).

Semantic Kernel Execution Lifecycle​

Understanding how a single function invocation flows through the kernel is key.

The kernel abstracts away the complexity of model selection, prompt rendering, and tool orchestration. All you do is invoke a function.

Data Flow Inside Semantic Kernel​

The data flow can be visualised as a pipeline:

  • Input arrives as KernelArguments, a dictionary of key‑value pairs.
  • The kernel resolves which function to call, renders the prompt, and invokes the AI service (if prompt‑based) or calls native code.
  • The result is returned as a FunctionResult, which contains the output string and metadata.
  • For chat agents, the conversation history is maintained in a ChatHistory object that is passed in the arguments.

Context and Memory Management​

Context is passed through KernelArguments. For chat scenarios, the ChatHistory is usually stored in a Kernel data collection or on the agent. Memory is a separate subsystem that allows retrieval of past interactions. In an agent loop, you can:

  1. Retrieve relevant memories using a search function before prompting the AI.
  2. Include the retrieved context in the KernelArguments.
  3. After the interaction, save new memories.

This “retrieve – augment – generate” pattern is easily implemented by creating a plugin that wraps memory operations.

Semantic Kernel Development Model​

The development model follows these steps:

  1. Create the kernel and add AI services, logging, and other infrastructure.
  2. Define plugins – native functions for business logic, prompt functions for AI‑driven tasks.
  3. Import plugins into the kernel (from classes, YAML directories, or OpenAPI specs).
  4. Build an agent (or directly invoke functions) to orchestrate the workflow.
  5. Set up memory connectors for long‑term context.
  6. Deploy the kernel as a service (e.g., ASP.NET Core Minimal API, Azure Function, or container).

This model encourages separation of concerns: the kernel is configured once, plugins are reusable, and agents bring everything together.

Semantic Kernel vs Other Frameworks​

FrameworkCore ApproachStrengthsTypical Use
Semantic KernelKernel + plugins + connectorsDeep enterprise integration, native .NET/Java/Python SDK, strong Azure ecosystemEnterprise AI assistants, process automation, Microsoft 365 integration
LangGraphStateful graph executionFull control over workflows, checkpoints, human‑in‑the‑loopComplex multi‑step agents needing custom state management
CrewAIRole‑based multi‑agent teamsSimple, declarative, fast to prototypeContent generation, research teams, quick multi‑agent setups
AutoGenConversation‑centric, event‑drivenAsynchronous, scalable, rich event modelMulti‑party dialogues, research, tool‑augmented agents
OpenAI Agents SDKAgent‑centric, OpenAI‑nativeLightweight, built‑in tracing, quick setupOpenAI‑focused applications, fast prototyping

Semantic Kernel’s distinguishing strength is its enterprise‑grade plugin model and native integration with the Microsoft ecosystem. It is the go‑to choice for organisations already invested in .NET or Azure who need AI woven into their existing applications.

Common Beginner Mistakes​

  1. Overusing plugins – creating dozens of tiny plugins that are never reused. Group functions logically.
  2. Weak prompt design – not using the template language or failing to provide clear system instructions.
  3. Ignoring memory management – expecting the agent to remember everything without configuring a memory connector.
  4. Poor function organisation – mixing native and prompt functions in a single class without purpose.
  5. Excessive context size – passing the entire conversation history to every prompt, leading to token bloat.
  6. Not setting execution settings – relying on defaults that may not match the model (e.g., using gpt-4o for simple classification).

Best Practices​

  • Keep plugins modular – one domain concept per plugin (e.g., CalendarPlugin, OrderPlugin).
  • Design reusable functions – native functions should be stateless and idempotent.
  • Optimise prompts – use the Handlebars template engine for complex logic; keep templates concise.
  • Manage memory carefully – store only key facts; use a vector database for semantic search.
  • Use connectors appropriately – leverage built‑in connectors instead of writing custom HTTP calls.
  • Version your prompts and plugins – treat them as code, store in source control.
  • Monitor and log – use the kernel’s logging and telemetry to trace function invocations.

Practical Example: Enterprise Knowledge Assistant​

Let’s build an assistant that answers HR policy questions. It uses a knowledge base stored in an Azure AI Search index (memory connector) and a native plugin to fetch current employee data.

1. Set up the kernel with services and memory​

from semantic_kernel import Kernel
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAITextEmbedding
from semantic_kernel.memory import SemanticTextMemory
from semantic_kernel.connectors.memory.azure_cognitive_search import AzureCognitiveSearchMemoryStore

kernel = Kernel()
kernel.add_service(OpenAIChatCompletion(service_id="gpt", api_key="...", deployment_name="gpt-4o"))
kernel.add_service(OpenAITextEmbedding(service_id="embed", api_key="...", deployment_name="text-embedding-ada-002"))

# Memory connector to Azure AI Search
acs_store = AzureCognitiveSearchMemoryStore(vector_size=1536, ...)
memory = SemanticTextMemory(storage=acs_store, embeddings_generator=kernel.get_service("embed"))
kernel.add_memory(memory)

2. Define plugins​

from semantic_kernel.functions import kernel_function

class HRPolicyPlugin:
    @kernel_function(description="Search HR policies for a given query")
    async def search_policies(self, query: str, kernel: Kernel) -> str:
        memories = await kernel.memory.search("policies", query, limit=3)
        return "\n".join([m.text for m in memories])

class EmployeeDataPlugin:
    @kernel_function(description="Get employee details from HR system")
    async def get_employee(self, employee_id: str) -> str:
        # Call internal HR API
        return "{'name': 'Alice', 'department': 'Engineering'}"

Register plugins:

kernel.add_plugin(HRPolicyPlugin(), "HR")
kernel.add_plugin(EmployeeDataPlugin(), "Employee")

3. Create the agent​

from semantic_kernel.agents import ChatCompletionAgent

agent = ChatCompletionAgent(
    kernel=kernel,
    name="HR_Assistant",
    instructions="You are an HR assistant. Use the available plugins to answer questions accurately. If you don't know, say so.",
    plugins=["HR", "Employee"]
)

4. Handle a conversation turn​

chat_history = []
while True:
    user_input = input("User: ")
    if user_input.lower() == "exit":
        break
    response = await agent.get_response(messages=chat_history + [{"role": "user", "content": user_input}])
    print(f"Assistant: {response}")
    chat_history.append({"role": "user", "content": user_input})
    chat_history.append({"role": "assistant", "content": response})

What happens internally:

  1. The user asks “What is the remote work policy?”
  2. The agent’s kernel sends the instructions, history, and available plugins to the AI service.
  3. The model decides to call HR.search_policies with the query “remote work policy”.
  4. The kernel invokes the function, which searches the Azure AI Search index and returns relevant policy texts.
  5. The AI receives the policy text and generates a summarised answer.
  6. The answer is returned to the user.

This example demonstrates how the Kernel, plugins, memory, and agent work together to build a grounded, enterprise‑ready AI assistant.

Semantic Kernel and Modern Agent Development​

Semantic Kernel aligns with current trends in agentic AI:

  • Tool Calling – native functions serve as tools; the kernel handles the function‑calling loop automatically.
  • Memory Systems – integrates with vector databases for long‑term memory, enabling RAG patterns.
  • MCP Integration – you can create an MCP client as a plugin or connector, exposing external MCP tools to the kernel. See the MCP Guide.
  • Workflow Automation – can be combined with Microsoft Power Automate, Azure Logic Apps, or custom workflows.
  • Production Operations – supports OpenTelemetry, logging, and dependency injection for robust production services.

For deeper dives, the handbook continues with:

FAQ​

1. What is Semantic Kernel?​

It’s Microsoft’s open‑source AI orchestration framework that provides a unified SDK for integrating LLMs, tools, memory, and enterprise systems into applications.

2. What is a Kernel?​

The Kernel is the central runtime that manages AI services, plugins, functions, and memory. It coordinates function invocation and context flow.

3. What are Plugins?​

Plugins are containers that group related functions together. They can be written in code (native) or defined via prompt templates.

4. What are Functions?​

Functions are executable units of logic. Native functions are written in a programming language; prompt functions are AI prompts that generate output.

5. How does Memory work?​

Memory integrates with vector stores or text search services. You can save information and later retrieve it semantically, grounding responses in past context.

6. What are Connectors?​

Connectors are pre‑built integrations with AI models, memory stores, and external services (e.g., Microsoft Graph, SQL databases).

7. Is Semantic Kernel production‑ready?​

Yes. It’s used in enterprise products, supports logging, telemetry, and can be deployed as a microservice.

8. What languages does Semantic Kernel support?​

C#, Python, and Java are officially supported. The Python and .NET versions have the most complete feature sets.

9. How does Semantic Kernel compare to LangChain?​

Semantic Kernel provides a more structured, enterprise‑oriented plugin model and deeper Azure integration, while LangChain offers a broader ecosystem of community connectors. Both are viable; the choice often depends on existing Microsoft investments.

10. Can I use non‑OpenAI models with Semantic Kernel?​

Yes, through various connectors (Azure OpenAI, Hugging Face, local models via Ollama, and custom connectors).

11. What is a ChatCompletionAgent?​

An agent that uses the chat completion API and function calling to autonomously interact with users and invoke plugins.

12. How do I handle conversation history?​

You maintain a list of messages and pass it to the agent’s get_response method. The history is typically managed by your application logic.

13. How do I add authentication to connectors?​

Most connectors support Azure AD authentication, API keys, or connection strings. You configure them when creating the connector instance.

14. Can Semantic Kernel work with MCP?​

Yes, you can build an MCP client as a native plugin, exposing MCP tools as kernel functions. This allows Semantic Kernel agents to use any MCP server.

15. Where can I find more examples?​

The official Semantic Kernel GitHub repository contains numerous samples in all three languages. The AgentDevPro Handbook also offers dedicated articles on plugins, planners, memory, and production.

Conclusion​

Semantic Kernel provides a robust, extensible platform for building AI applications that integrate deeply with existing enterprise systems. Its core abstractions—the Kernel, AI services, functions, plugins, prompts, memory, connectors, and agents—form a coherent execution model that simplifies orchestration while offering fine‑grained control.

Key takeaways:

  • The Kernel is the central orchestrator that wires everything together.
  • Functions (native and prompt) are the building blocks of agent behaviour.
  • Plugins organise functions and promote reuse.
  • Memory and connectors ground agents in real‑world data and systems.
  • Agents provide autonomous execution with built‑in function calling.

Now, deepen your expertise with these handbook articles:

For cross‑framework comparisons, explore the LangGraph, CrewAI, AutoGen, and OpenAI Agents SDK guides, and the complete framework comparison. To standardise tool integration, visit the MCP Guide.

With Semantic Kernel, you have an enterprise‑grade toolkit to bring AI into every corner of your applications.