Skip to content
Noorscale

AI Agents & Automation

AI governance and evaluation that turns AI risk into tested, documented controls.

For companies with AI in production, or about to be, whose board, customers or auditors are asking how it's controlled. We build evaluation sets from your real cases, red-team your assistants and agents under written authorization, add guardrails, and write the policies and risk register that map your work to the NIST AI RMF, ISO/IEC 42001 and the EU AI Act.

  • A person replies within one business day
  • A plan you can act on
  • You own every asset
Timeline
Baseline evaluation and risk register in 4 to 8 weeks
Engagement
Fixed-price assessment, then optional scheduled reviews
Frameworks
NIST AI RMF, ISO/IEC 42001, EU AI Act
Testing
Red-teaming under written authorization

Reviewed by Noor Khan

AI controls

Sample

Three AI systems are listed by risk, a red-team attack gets the support bot to leak customer data, a guardrail then blocks the same attack and the re-test passes, and the controls are mapped to the NIST AI RMF, ISO/IEC 42001 and the EU AI Act.

  1. AI systems ranked by risk
  2. Red team finds a leak
  3. Guardrail blocks it
  4. Evidence ready for audit

Illustration with sample data. Your version uses your accounts and numbers.

Brands Noor has worked with

  • Brand North
  • Olympic Behavioral Health
  • White Light Behavioral Health
  • Rubicon Recovery Center
  • Quest 2 Recovery
  • The Nestled

Overview

What is AI governance and evaluation?

AI governance and evaluation is the set of tests, controls and records that show an AI system behaves as intended and that its risks are known and managed. It covers evaluation sets, red-teaming, guardrails, monitoring, policies and a risk register, often mapped to frameworks such as the NIST AI RMF and ISO/IEC 42001. Noorscale builds and maintains this evidence, while any certification comes from an independent certification body.

See it in action

What does an AI governance engagement look like when it's working?

It looks like the live sample below, built with illustrative data. Your version runs on your own accounts and your numbers.

AI chat agent · website widgetNorthwind Accounting · pricing page

Sample

Chat ready

northwind.example/pricing
NNorthwind assistantAI · replies in seconds · a person on request
Sample conversation, plays automatically

CRM record · created by the agent

Filling in
Source
Industry
Locations
Transactions
Email
Owner
Fit score86
StatusCall booked · Wed 10:00

Routing rules

Waiting
  • Good fit: offer a call with a specialist
  • Later stage: send a resource, add to nurture
  • Discount, complaint or 'a person, please': a teammate joins

Illustrative chat. Your agent answers only from your approved content, books into your team's calendars and brings in a person by the rules you set.

Sound familiar?

What problems does an AI governance engagement solve?

It solves the three problems teams raise most often on a first call, and each one has a fix.

The problem

  • “Customers and auditors ask how our AI is controlled”Security questionnaires and procurement reviews now ask about AI testing, data use and human oversight, and the answers live in a few people's heads.
  • “We can't say how accurate our AI really is”Nobody built a test set, so quality gets judged by a handful of demos and whichever complaint arrived last.
  • “AI rules are tightening and ownership is unclear”Legal tracks regulation, engineering tracks models, and the risk register has no line for the assistant that went live last quarter.

What changes

  • Clear answers for customers and auditorsSecurity questionnaires and audits get documented answers, test results and named owners instead of best guesses.
  • Quality you can measureEvery priority AI system has a score, so you know whether each change made it better or worse.
  • Weak spots found in testingRed-teaming surfaces injection, leakage and unsafe actions before release, where fixes are quick and cheap.
  • One view of AI riskLeadership sees every AI system, its risk level, its owner and its controls in one register.

Compare your options

Your own team, a policy consultant or Noorscale for AI governance?

Each option suits a different stage. Here's how they compare on the points buyers ask about most, including where the others do well.

Your own team, a policy consultant or Noorscale for AI governance?
Knows your systemsYour own teamBest, they built themPolicy-only consultantLearns them from interviewsNoorscaleLearns them in the inventory
Policies and framework mappingYour own teamSlow without specialist timePolicy-only consultantTheir main strengthNoorscaleWritten and mapped to controls
Evaluation and red-teamingYour own teamPossible with security skillsPolicy-only consultantOutside their scopeNoorscaleTest sets and authorized red-teaming
GuardrailsYour own teamBuilt if engineers have timePolicy-only consultantOutside their scopeNoorscaleBuilt and tuned on test traffic
Legal interpretationYour own teamYour legal counselPolicy-only consultantVaries by firmNoorscaleStays with your legal counsel
Best whenYour own teamSpare AI and security skills in-housePolicy-only consultantYou need policies, not testingNoorscaleYou need tests and evidence together

On the free call, Noor will tell you which option suits your stage, even when it's one of the others.

What's included

What does an AI governance engagement include?

An AI governance engagement includes six deliverables, each one working material your team can use, not a slide deck.

Scope · AI Governance & Evaluation6 deliverables

  1. 01

    AI system inventory and risk tiering

    Every AI system you build or buy, such as assistants, agents and AI features in vendor tools, listed with its owner, purpose, data and risk level.

  2. 02

    Evaluation sets and baseline scores

    Real cases with agreed answers for each priority system, scored for accuracy, groundedness, refusals and bias where relevant, so quality becomes a number you can track.

  3. 03

    Red-teaming under written authorization

    Adversarial testing for prompt injection, data leakage, jailbreaks, unsafe actions and harmful output, run within an agreed scope and mapped to the OWASP Top 10 for LLM Applications and MITRE ATLAS.

  4. 04

    Guardrails and human oversight

    Input and output filters, grounding checks, tool permission limits and approval steps, tuned to stop what matters while normal use flows through.

  5. 05

    Monitoring plan and thresholds

    What to measure in production, the thresholds that trigger a review, and a schedule for re-running evaluations after model or prompt changes.

  6. 06

    Policies, risk register and framework mapping

    An AI policy, a risk register with owners and treatments, and a control map against the NIST AI RMF, ISO/IEC 42001 and the EU AI Act, ready for your auditors and legal counsel.

Tools and platforms we work in

  • NNIST AI RMF
  • IISO/IEC 42001
  • EEU AI Act
  • OOWASP Top 10 for LLM Applications
  • MMITRE ATLAS
  • PPromptfoo
  • Ggarak
  • PPyRIT
  • DDeepEval
  • RRagas
  • Langfuse
  • NNeMo Guardrails
  • LLlama Guard
  • Amazon Bedrock Guardrails
  • AAzure AI Content Safety

Process

How long does an AI governance engagement take?

The work runs in five steps, starting in weeks 1 to 2 and then running as ongoing work. Every step ends with something you can use, so you always know what's landed and what comes next.

Timeline: Baseline evaluation and risk register in 4 to 8 weeks

  1. Step 01Weeks 1 to 2

    Inventory and risk tiering

    We list your AI systems, their owners, data and users, and rank them by risk so effort goes where it matters most.

    You get: AI system inventory ranked by risk

  2. Step 02Weeks 2 to 4

    Build evaluation sets

    We collect real cases with your experts, agree what a good answer looks like and score each priority system to set a baseline.

    You get: Evaluation sets and baseline scores

  3. Step 03Weeks 3 to 6

    Red-team and add guardrails

    Under written authorization and an agreed scope, we test how each system fails, then add or tune guardrails and test again.

    You get: Red-team findings and re-tested guardrails

  4. Step 04Weeks 5 to 8

    Write policies and map controls

    We write the policy and risk register, map controls to the frameworks you've chosen and walk your leadership through the gaps.

    You get: AI policy, risk register and control map

  5. Step 05Ongoing

    Re-evaluate on a schedule

    Quarterly or after major changes, we re-run evaluations and red-team tests, update the register and refresh the evidence.

    You get: Updated scores, register and evidence

Reporting

How will you know an AI governance engagement is working?

You'll see it in four numbers on a live dashboard, with a written review every month. Leading indicators come first, then the numbers your finance team cares about.

Monthly reportSample

Evaluation scores per systemHigher is better

Accuracy, groundedness and refusal scores on each system's test set, tracked from baseline through every change.

Red-team findings closedHigher is better

Issues found in adversarial testing, by severity, and how many have been fixed and re-tested.

Guardrail precisionHigher is better

How well guardrails catch harmful or off-policy requests while letting legitimate ones through, measured on test traffic.

Control coverageHigher is better

Share of framework controls with an owner and evidence in place, for each framework you've chosen.

Noor Khan

15 minutes with Noor

Noor Khan, Founder & CEO

You'll leave with

  • Your biggest growth opportunity, in plain terms
  • The first two or three moves worth making
  • A clear plan for your first milestone

Noor listens first, then shows you the fastest route to the result you want.

Useful to have with you

  • Your website address
  • The goal or problem you want to solve
  • Any numbers you already track
See available times

Booking opens on Cal.com. Or send a call request

Who it's for

Who is an AI governance engagement for?

An AI governance engagement works best for teams in one or more of these situations.

  • Companies with customer-facing AI assistants or agents in production
  • Teams answering AI questions in security reviews and procurement
  • Regulated firms preparing for ISO/IEC 42001 certification or EU AI Act duties
  • Leadership teams that want one clear view of AI risk across the business

Engagement

How is an AI governance engagement scoped and priced?

Pricing is shared after the free call, in a written scope with deliverables, timeline and fee. Never a package, never a surprise.

Book a free AI governance call
  • Typical for this service

    Monthly retainer

    Ongoing work against a 90-day plan, for channels that compound month over month.

    Usually: Fixed-price assessment, then an optional retainer for scheduled re-evaluation

    • Monthly report and strategy call
    • Priorities reset every quarter
    • No long lock-in
  • Fixed-scope project

    A defined scope, timeline and fee for builds, audits, migrations and launches.

    • Deliverables agreed in writing
    • Milestones you can check
    • Optional care plan after launch
  • Focused sprint

    A short, time-boxed piece of work on one priority, to prove the approach first.

    • One clear goal
    • Working output, not a deck
    • Roll into a retainer or stop

Every engagement includes

  • AI system inventory and risk register
  • Evaluation sets and baseline scores for priority systems
  • Red-teaming within an agreed, written scope
  • Guardrails, policies and a framework control map

What shapes the scope

  • AI systems in scope
  • How much risk each system carries
  • Frameworks you map to
  • Depth of red-team testing
  • Bought tools as well as your own builds

You'll see how each one applies to you in the written scope.

Noor Khan, founder of Noorscale

Who leads the work?

Founder-led by Noor Khan

A software engineer turned AI and growth strategist. Noor sets the strategy and architecture on every engagement, reviews every plan before it ships and built the method the whole team works by.

FAQ

Questions about AI Governance & Evaluation

Something else on your mind? Email grow@noorscale.com and a person will reply within one business day.

How much does AI governance and evaluation cost?

Cost depends on how many AI systems are in scope, how much risk they carry, how many frameworks you map to and how deep the red-teaming goes. A single customer-facing assistant takes far less work than a portfolio of agents that change records. The first assessment is usually a fixed price, with optional scheduled reviews after it. You'll get a written scope after a free call.

How is this different from managed AI agents?

Managed AI agents is the day-to-day operation of live agents: monitoring runs, fixing failures and shipping tested updates. AI governance and evaluation sets the bar those agents must meet and produces the evidence, meaning test sets, red-team results, guardrails, policies and the risk register. Many clients use both, with governance defining the tests and thresholds and the managed service running them every week.

Can you certify us for ISO/IEC 42001 or the EU AI Act?

Certification comes from an independent, accredited certification body, and our role is to get you ready for it. We prepare the controls, policies, risk register and evidence an ISO/IEC 42001 audit looks for. For the EU AI Act, we build the technical documentation and testing records, while your legal counsel confirms how the Act applies to each system.

What is AI red-teaming?

AI red-teaming is structured adversarial testing that tries to make an AI system fail, for example by injecting instructions, extracting private data, bypassing its rules or triggering unsafe actions. We run it only under written authorization and an agreed scope, combining manual testing with tools such as garak, PyRIT and Promptfoo. Every finding comes with a severity, evidence and a fix to re-test.

What is an LLM evaluation set?

An LLM evaluation set is a collection of real inputs with agreed good answers or behaviors, used to score an AI system the same way every time. We build ours with your subject experts, covering normal cases, edge cases and cases where the system should refuse or hand off. It turns "it seems fine" into a score you can track after every change.

Do guardrails slow the AI down or block normal use?

Well-tuned guardrails add little delay and let normal requests through. We measure both sides on test traffic: how much harmful or off-policy input they catch, and how many legitimate requests they stop by mistake. Then we tune the thresholds until the balance fits each system's risk.

Which frameworks do you work with?

We map controls to the NIST AI Risk Management Framework, ISO/IEC 42001 and the EU AI Act, and test against the OWASP Top 10 for LLM Applications and MITRE ATLAS. You choose the frameworks that matter to your customers and regulators, and we keep one control set mapped to all of them, so each control is done once.

Does this cover AI tools we buy, as well as ones we build?

Yes, the inventory and risk register cover vendor tools too, such as Microsoft 365 Copilot or AI features inside your CRM. For bought tools, we review the vendor's documentation, data terms and settings, and test behavior where the vendor's terms and your written authorization allow it.

What do you need from our team?

A list of the AI systems you know about, access to the ones in scope, and subject experts who can agree what good answers look like. We also need a sponsor, often from risk, security or legal, who signs the written authorization for red-teaming and owns the register after handover.

Free 15-minute call

Tell us what you want to build.

In fifteen minutes you'll have a clear first step for AI Governance & Evaluation and a plan to reach it. Pick a time below, or write if that's easier.

Prefer email? grow@noorscale.com

Noor Khan

15 minutes with Noor

Noor Khan, Founder & CEO

You'll leave with

  • Your biggest growth opportunity, in plain terms
  • The first two or three moves worth making
  • A clear plan for your first milestone

Noor listens first, then shows you the fastest route to the result you want.

Useful to have with you

  • Your website address
  • The goal or problem you want to solve
  • Any numbers you already track
See available times

Booking opens on Cal.com. Or send a call request

Book a free 15-minute call