Log Detective: AI-Powered Build Failure Analysis in Packit

Introduction

Packit, the tool that streamlines integration between upstream projects and downstream distributions, has just gained a powerful new ally: Log Detective. Starting this month, Log Detective will automatically analyze failed Packit-triggered scratch Koji builds on dist-git pull requests, offering clear explanations of what went wrong—without any extra configuration from users.

How Log Detective Works in Packit

The integration is seamless. In Copr, users could already request Log Detective analysis by clicking an "Ask AI" button. Now, in Packit, whenever a build failure occurs, the system automatically triggers an analysis request. Results are displayed in the Packit dashboard as soon as they are ready.

No Configuration Needed

You don’t need to choose which logs to send or tune any prompts. The service handles everything behind the scenes, making it accessible for everyone—from beginners to seasoned package maintainers.

Log Parsing and Analysis Derivation

Log Detective version 4.0 is built as an agent using the BeeAI Framework. When an analysis request arrives, the agent receives all build logs and artifacts. It then applies a variety of tools—primarily the Drain template mining algorithm—to extract only the most relevant snippets from the log files. These snippets represent a small fraction of the original log size.

By working with snippets instead of full logs, Log Detective saves tokens and reduces analysis time. It also limits useless information in the model’s context, allowing the system to use relatively small AI models while still producing accurate results.

Communication Architecture

Packit handles failed Koji builds just as it always has—but now it additionally requests a Log Detective analysis. To facilitate this, a lightweight containerized interface server mediates communication between Packit and Log Detective. Packit sends the analysis request to this server; once the analysis is complete, the interface server posts the results on the Fedora Messaging bus, where Packit collects them and displays them in the dashboard next to the relevant pull request.

Result Presentation

Each analysis includes a statement of what, if anything, went wrong during the build, and optionally a suggestion for a solution. In its current configuration, Log Detective only uses build logs as information sources—no external repositories or historical data are consulted.

The Packit dashboard links these results directly to the PR that triggered the build failure, making it easy to find and act on the diagnosis.

Purpose and Limitations

Log Detective uses a general-purpose AI model and does not have access to broader context about packages or the Fedora ecosystem. Therefore, it is not a substitute for years of hands-on package maintenance experience. If you’ve been building packages for a while, you may find the insights are basic.

The real value is for newcomers and those who haven’t yet internalized common build failure patterns. Log Detective aims to make their work easier by providing immediate, understandable explanations of what went wrong and how to fix it.

Future Development

Log Detective is still evolving. The team plans to expand its capabilities, for example by integrating additional data sources beyond build logs. While the current implementation is already useful, future versions may offer deeper insights and solution suggestions tailored to specific packages.

Stay tuned for updates as Log Detective continues to mature, bringing AI-assisted debugging to even more Fedora developers.

---

For more details, see the introduction or jump to the How It Works section.