# Classifying Prompt Patterns in Chat Logs

Canonical: https://hyperparam.app/docs/usecases/data-transformation

Use AI-generated columns to categorize unstructured fields in your chat or agent logs, and understand what your system is actually being asked to do.

## Overview

Chat and agent logs often contain free-text system prompts or user messages that are hard to analyze at scale. In this example, we load a large conversation dataset (Orca), use Hyperparam's AI agent to classify each system prompt into a category, and review the distribution of prompt types across the dataset.

![Demo showing prompt classification in Hyperparam](https://s3.hyperparam.app/video/Data_Transformation.gif)

## Steps

1. **Load the log dataset**

   Open [OpenOrca/partial-train/0000](https://hyperparam.app/files?key=https%3A%2F%2Fhuggingface.co%2Fdatasets%2FOpen-Orca%2FOpenOrca%2Fresolve%2Frefs%252Fconvert%252Fparquet%2Fdefault%2Fpartial-train%2F0000.parquet)

2. **Examine the system prompts**

   Review the `system_prompt` column. These are the instructions each conversation was given, but as free text they're hard to analyze in aggregate.

3. **Classify with AI**

   In chat, request: "create a new column that categorizes the system prompt"

   The agent analyzes each system prompt and generates categories (e.g., "Education/Tutor", "General Assistant", "Information retrieval/QA"). A new `system_prompt_category` column appears.

4. **Review the distribution**

   Scroll through rows to verify category assignments. Sort or create a view to see which prompt patterns dominate your dataset.

5. **Export the results**

   Export the dataset with the new `system_prompt_category` column included. Export processes the full 100k+ dataset.

## Expected Results

- **Categorical column**: `system_prompt_category` classifying each system prompt
- **Pattern visibility**: See which prompt types dominate and which are rare
- **Actionable metadata**: Filter to specific prompt categories, compare behavior across categories, or identify prompt patterns that correlate with failures

---

## Other Use Cases

- [Dataset Discovery](https://hyperparam.app/docs/usecases/dataset-discovery): Use natural language to find public datasets
- [Patient Data Workflow](https://hyperparam.app/docs/usecases/patient-data-workflow): Extract, filter, and export structured medical data
- [Quality Filtering](https://hyperparam.app/docs/usecases/quality-filtering): Score and filter low-quality, sycophantic responses in chat logs
- [Deep Research](https://hyperparam.app/docs/usecases/summary-comparison): Multi-step AI workflow for comparing model outputs