IT & AI Technology

Building a Coding Agent from Scratch, Part 6: How to Build the Evaluation, and Doubt the Grader Itself

AI Agents

Building a Coding Agent from Scratch, Part 6: How to Build the Evaluation. Have It Build Something That Runs, Auto-Grade It, and Doubt the Grader Itself

The grader's 10 points for tests were 0/0 for three whole days, invisible in the score table. How we built a rig that has the agent build a web system and grades it by running it, the five-kinds-times-three pass/fail criteria, running the control first, and contracts decided only by the tests.

By Qualiteg Product Development Team
Understanding Active Directory: The Technology Behind Enterprise AI Security - Part 7: Security and Best Practices

LLM Security

Understanding Active Directory: The Technology Behind Enterprise AI Security - Part 7: Security and Best Practices

Hello! In this seventh installment of the series, we cover Security and Best Practices - considerations for production environments. Series overview * Chapter 1: Understanding the basics - Fundamentals of Active Directory and Kerberos/NTLM authentication * Chapter 2: Building a domain environment - Setting up a lab environment * Chapter 3: Joining

By Qualiteg Consulting
Building a Coding Agent from Scratch, Part 3: Count Idle Turns, and Cap the Thinking Without Cutting It

AI Agents

Building a Coding Agent from Scratch, Part 3: "It Didn't Stop" Is Not "It Made Progress." Count Idle Turns, and Cap the Thinking Without Cutting It

300 turns, zero APIs. Re-reading the log showed 234 turns had never called a tool. We cover the idle rate as a metric, idling that shifts form each time one is closed, how turning thinking off dropped 95 to 71 without saving time, and counting the other side after widening the output window.

By Qualiteg Product Development Team
Suspended Three Days After Release: What the U.S. Government Directive on Fable 5 / Mythos 5 Reveals About a New AI Availability Risk

Generative AI Frontlines

Suspended Three Days After Release: What the U.S. Government Directive on Fable 5 / Mythos 5 Reveals About a New AI Availability Risk

Hello! In our previous article, we covered Claude Fable 5 and Claude Mythos 5, announced by Anthropic on June 9, 2026. Fable 5 took Mythos-class capability, added safeguards, and brought it down into a form that could be offered to general users. We described it as "the myth descending

By Qualiteg Consulting, Qualiteg AI Security Team
How to Talk Honestly About PII Detection Accuracy: Evaluation Design Through Recall, Confidence Intervals, and Representativeness

AI Mathematics

How to Talk Honestly About PII Detection Accuracy: Evaluation Design Through Recall, Confidence Intervals, and Representativeness

Hello. This is the Qualiteg Research Team. We work on technology for detecting and masking sensitive information — personal information (PII), confidential information, and special-care-required personal data (https://pii-fi.com). In that work, the question we face every day is: 「how can we talk about accuracy figures honestly and correctly」 ? Suppose,

By Qualiteg Research Team
Understanding Active Directory: The Technology Behind Enterprise AI Security - Part 6: Common Problems and Solutions

LLM Security

Understanding Active Directory: The Technology Behind Enterprise AI Security - Part 6: Common Problems and Solutions

In Part 5, we examined browser authentication settings and how to verify them. This installment explains how to investigate authentication and domain-join failures. A slow file share or a failed join despite working name resolution does not identify a single cause. Record the initial state, then separate naming, time, service

By Qualiteg Consulting, Qualiteg AI Security Team
What the Confusion Matrix Doesn't Show in PII Detection: Cross-Recognizer Collisions and Integration Testing

LLM Security

What the Confusion Matrix Doesn't Show in PII Detection: Cross-Recognizer Collisions and Integration Testing

Hello! This is the Qualiteg Research Team. Automatic detection of personally identifiable information (PII) can be framed as the problem of extracting specific expressions from text and determining which type of PII each one represents. As the range of PII types to detect grows — phone numbers, personal names, account numbers,

By Qualiteg Research Team, Qualiteg AI Security Team
Understanding Active Directory: The Technology Behind Enterprise AI Security - Part 5: Browser Configuration and Authentication

IT & AI Technology

Understanding Active Directory: The Technology Behind Enterprise AI Security - Part 5: Browser Configuration and Authentication

Hello! Part 4 covered proxy authentication and access control. This time, we look at what to check in browsers and how to distribute their settings. The same page may trigger an authentication prompt in one browser or connection path but not another. Before widening the allowed targets or authentication methods,

By Qualiteg AI Security Team, Qualiteg Consulting