Skip to content

[Feature request] Support decision/classification models (e.g. Microsoft-Decision-1 on Foundry) as a lightweight engine or pre-agent step #74832

Open

Description

@DeagleGross

Several workflows use a full agent run only to classify an input, such as triaging an issue into one of N labels. In dotnet/aspnetcore, issue-triage-agent runs a full agent job plus a detection job for every new issue. A recent run consumed about 22 AI credits, roughly $0.22, and took about 4.5 minutes. Most of that was the agent job writing a ~70k-token prompt to the cache, which holds the area reference.

Decision models are built for this kind of work. They take a state and a fixed set of typed questions (yes/no, choice, score) and return calibrated probabilities in one pass, without generating text. For example, Microsoft-Decision-1 on Microsoft Foundry costs $0.042 per million input tokens, and output is free. In a quick test, classifying real aspnetcore issues with their full title and body took 340-610 input tokens, about $0.00002 per call, in under a second.

Current workaround

I can call the model from a custom jobs: entry or steps: block with curl and a repo secret. That works, but every repo has to reimplement it. It also has these gaps:

  • No first-class way to feed the probabilities into the prompt, safe-outputs, or an if: condition.
  • No built-in confidence-threshold gate that skips the agent job.
  • No usage or cost reporting. The Copilot engine's token-usage.jsonl and AI credit accounting don't cover these calls.
  • Authentication and firewall setup are manual (a Foundry key or Entra OIDC, and the Foundry domain).

Proposal

Any of these would help, in rough order of effort:

  1. Frontmatter for a decision step, for example:

    decisions:
      provider: azure-foundry
      endpoint: ${{ secrets.FOUNDRY_ENDPOINT }}
      deployment: decision1
      questions:
        area:
          type: choice
          instructions: Which area owns this issue?
          criteria:
            area-blazor: "..."
            area-mvc: "..."

    with typed outputs (needs.decision.outputs.area, .confidence, .probabilities).

  2. A confidence gate such as skip-agent-if: area.confidence >= 0.95, so that high-confidence cases can go straight to safe-outputs without an LLM run.

  3. A non-generative engine type, or provider support for OpenAI-compatible and Foundry endpoints, with usage reporting in the run summary.

Activity

  1. locked and limited conversation to collaborators on Nov 4, 2026
  2. unlocked this conversation on Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions