Learn More
Assessment Intelligence
How AIM measures assessment quality, extracts signals from free text, and generates personalized follow-up questions to sharpen recommendations and reports.
Assessment Confidence Tiers
AIM scores every assessment against a set of weighted signals to produce a signal score from 0 to 100. That score maps to one of three confidence tiers, which appears on every generated report and determines how precise the language, cost ranges, and recommendations can be.
Signals required
Core fields: organization type, environment, project size.
Report effect
Reports include explicit data gap disclosures and broad assumption ranges. Recommendations are directionally correct but cost ranges are wide.
Example
A new client submits their first assessment with just the organization name, environment type, and a brief description of scope.
Signals required
Most key signals: scope areas, regulatory requirements, cloud preference, project timeline, basic staffing profile.
Report effect
Recommendations are well-scoped with moderate confidence. Cost models use real data but may carry ±30–40% variance bands.
Example
An agency submits an assessment identifying HIPAA compliance, Azure preference, a 24-month timeline, and a 10-person IT team.
Signals required
Full signal set: all scope areas answered, user/endpoint counts, identity provider, budget band, risk tolerance, state location, domain archetype.
Report effect
RAO scoring and cost modeling reach maximum precision. Reports use confident language. Suitable for executive presentations and procurement packages.
Example
A hospital network provides detailed environment data, confirms existing Okta SSO, specifies 2,400 endpoints, identifies HIPAA + CJIS scope, and sets a $2–5M budget band.
Assumptions & Data Basis Section
Every AIM-generated report includes an Assumptions & Data Basis section that documents exactly which signals were provided, which were estimated, and what assumptions underlie any cost or staffing figures. This section is designed to satisfy auditor and procurement reviewer scrutiny in government and regulated environments.
Signal Map: What Moves the Score
Each signal carries a weight that reflects how much it affects recommendation precision and cost model accuracy. Higher-weight signals have the most impact when missing or present.
| Signal | Weight | Why it matters |
|---|---|---|
| Scope areas | High | Which technology domains are in scope (Identity, Data Platform, Cybersecurity, AI/ML, etc.). Determines which recommendations are generated. |
| Project size | High | Small, medium, or large. Drives team sizing, phase planning, and implementation cost models. |
| Known constraints | High | The free-text field. Signal Extraction mines this for cloud providers, regulatory frameworks, vendors, timelines, and more. |
| User & endpoint counts | High | Powers licensing cost models, infrastructure sizing, and FTE estimates. |
| Regulatory scope | High | Ensures only compliant, certified products appear in results. Critical for government and healthcare. |
| Cloud preference | Medium | Filters recommendations toward your preferred platforms and away from incompatible ones. |
| Identity provider | Medium | Prevents duplicate identity solutions from appearing in recommendations. |
| State location | Medium | Enables BLS-sourced regional labor rate benchmarks. |
| Project timeline | Medium | Enables phase planning and per-month cost amortization. |
| Organization domain | Medium | Applies sector-specific scoring patterns (federal, healthcare, financial, etc.). |
| Staffing capacity | Medium | Shapes implementation approach and partner engagement recommendations. |
| Current IT capabilities | Medium | Reveals staffing gaps and informs build-vs-buy decisions. |
| Risk tolerance | Low | Adjusts RAO scoring weights toward conservative or innovative options. |
| Budget band | Low | Filters cost-prohibitive options before recommendations run. |
Free-Text Signal Extraction
The Known Constraints field in every assessment accepts plain-language descriptions of your environment, requirements, and restrictions. AIM's Signal Extraction engine reads this text automatically and populates structured fields — deterministically, at zero AI cost, without overwriting anything you already set.
You can also trigger re-extraction manually from the Smart Follow-Up tab any time you update your constraints text.
Cloud providers
"We run on Azure" → azure set in cloud_providers_preference
Regulatory scope
"HIPAA and CJIS compliance required" → hipaa, cjis added to regulatory_scopes
Technology domains
"Okta SSO integration" → identity_access added to scope_areas
Identity provider
"Active Directory environment" → idp_present = true
Project timeline
"18-month modernization" → expected_duration_months = 18
Organization domain
"hospital network" → domain_archetype = healthcare
Non-destructive guarantee
- ›Only fills fields that are currently null or empty.
- ›For array fields (like regulatory scopes or scope areas), detected values are added to the existing set — never removed.
- ›Explicit answers you provided in the wizard always take priority.
- ›Extraction results are stored on the assessment and visible in the Smart Follow-Up tab.
Smart Follow-Up
After an assessment is submitted, the Smart Follow-Up tab in the assessment workspace generates a personalized list of up to 12 targeted questions — ordered by their confidence impact — to help you move from Baseline to Standard or Standard to High Confidence without re-doing your entire intake.
Signal gap detection
AIM checks every weighted signal against your current assessment data. Any signal that is missing or empty becomes a candidate question.
Signal Extraction cross-check
Signals that were auto-detected from your constraints text (e.g., cloud provider, identity provider) are pre-answered or marked as confirmed — skipping redundant questions.
Scope filtering
Questions that are only relevant to specific technology domains (like networking questions for Identity scope) are shown only if those scope areas are selected for your assessment.
Priority ordering
Remaining candidate questions are ranked by signal weight. Questions with the highest confidence impact appear first.
Answer & save
You answer as many or as few questions as you want and hit Save. Your signal score updates immediately. Regenerate any report to reflect the improved confidence tier.
Scope-Conditional Questions
In addition to main signal questions, the Smart Follow-Up tab surfaces scope-specific sub-questions for each technology domain you selected. These questions are only shown if the relevant scope area is part of your assessment and the corresponding field is still empty.
Identity & Access
Current IdP, SSO protocol, MFA status, federation requirements
Data Platform
Data volume, existing warehouse, BI tool, streaming vs. batch
Cybersecurity
SIEM presence, zero-trust maturity, endpoint count, SOC model
Cloud Infrastructure
Current hypervisor, container strategy, IaC tooling
AI / ML
Existing ML platforms, data science team size, model hosting preference
CRM / ERP
Current ERP/CRM vendor, customization depth, integration footprint
Frequently Asked Questions
Does answering Smart Follow-Up questions duplicate work I already did in the wizard?
No. Smart Follow-Up only shows questions for fields you did not already answer. If you selected scope areas in the wizard, those scope-conditional questions will not appear. If you described your environment in free text and Signal Extraction detected your cloud provider, that question is skipped automatically.
What happens if I answer a follow-up question and then want to change it?
Saved answers are written to the assessment record. You can re-open the Smart Follow-Up tab at any time and submit a new answer — it will overwrite the previous value. The signal score recalculates on every save.
Does a higher confidence tier change which products are recommended?
Yes, indirectly. More complete signals allow the RAO engine to apply tighter constraint filtering, which can change which products pass the cut and their relative ranking. A Baseline assessment may show a broader set with wider score variance; a High Confidence assessment will show a tighter, more precisely ranked set.
Can Signal Extraction overwrite answers I already provided?
Never. Signal Extraction is strictly non-destructive. It only fills fields that are null or empty, and it only adds to array fields (like regulatory_scopes) — it never removes items you explicitly set.
Does the confidence tier affect the cost estimates in reports?
Yes. At Baseline, cost models use broad regional averages and wide variance bands (often ±40–60%). At High Confidence, BLS labor rate data, endpoint counts, and timeline data allow point estimates with tighter bands. Every report documents the data basis and any assumptions made.
Can I regenerate a report after improving my confidence tier?
Yes — that is the intended workflow. Answer follow-up questions, save, then regenerate any report. The updated confidence tier and signal data will be reflected throughout. Note: regenerating consumes a new snapshot credit if the decision state has changed.
Ready to improve your assessment?
Open any assessment, navigate to the Smart Follow-Up tab, and answer a few targeted questions to move your confidence tier and sharpen every report.