Razor Auditor — Evaluate AI Systems

A structured diagnostic and evaluation interface for examining how a declared AI system uses compression, expression, memory, recursion, infrastructure, and total computational cost.

The Auditor organizes public evidence or directly supplied measurements into bounded findings. It helps identify what is documented, what can be calculated, what remains inferred, and what cannot be determined without controlled telemetry.

Canonical definitions remain governed by the Grand Compression Master Reference Document. The Razor Auditor is an applied diagnostic and evaluation-intake layer.

1 · Purpose & Boundary

What the Razor Auditor Is

The Razor Auditor is a structured method for examining a specifically identified AI system through Robbie’s Razor. It turns available evidence into a transparent profile of system behavior, resource use, memory, reuse, recursive stability, infrastructure dependency, and unanswered questions.

The Auditor may be used with public disclosures, user-supplied documentation, controlled telemetry, or formal benchmark results. The strength of its output must never exceed the strength of the evidence supplied.

The Correct Unit of Audit

The minimum valid audit subject is not an entire company or industry. It is a declared system operating within a declared boundary:

Audit Finding Labels

These labels describe the basis of an individual audit finding. They are separate from the formal GC-MRD-v2.0 evidence states: Proposed, Testing, Provisionally Supported, Supported, Challenged, Inconclusive, and Retired.

What Changed from the Earlier Auditor

Continue to Audit Modes →

2 · Evaluation Maturity

Four Razor Audit Modes

The word “audit” can describe several levels of evaluation. The Razor Auditor must identify which mode is being used so that an exploratory public profile is not mistaken for a measured or independently replicated result.

Mode Comparison

Continue to Evidence Inputs →

3 · Source Registration

Evidence Inputs & Source Hierarchy

Every Razor audit begins with an evidence register. Each source must be identified, dated, scoped, and linked to the finding it supports so that documentation, calculation, inference, and unknowns remain distinguishable.

Preferred Evidence Hierarchy

Minimum Source Record

Evidence Handling Rules

• prefer primary sources for factual and technical claims;
• attribute company-reported performance to the company;
• separate announced, planned, contracted, operating, and utilized capacity;
• record the exact product and version whenever available;
• preserve conflicting evidence rather than silently selecting one source;
• label calculated findings with the formula and source values used;
• label inferred findings and explain the reasoning boundary;
• keep missing internal telemetry explicitly unknown;
• recheck time-sensitive facts before publishing or updating an audit; and
• do not treat licensing, payment, inclusion, or publication as evidence.

Continue to Audit Model →

4 · Evaluation Architecture

The Razor Auditor Model: Reference Before Evaluation

The Razor Auditor evaluates a declared system in sequence. Before asking whether a system compresses, expresses, remembers, or recursively reuses information efficiently, the Auditor first asks whether the system and the proposed evaluation can establish what must be preserved.

This creates an upstream diagnostic layer for canonical authority, identity, provenance, relationships, versions, rights, normalization, equivalence, binding, and fail-closed behavior. Only after that reference architecture is sufficiently defined should an evaluator proceed toward quality, CEMR, cost, and formal benchmark design.

The Auditor therefore does not begin by asking whether a system is efficient. It begins by asking whether the proposed evaluation has a sufficiently governed reference against which preservation and performance can later be measured.

Three Readiness Layers Before Economic Interpretation

Reference Readiness Diagnostic

When the proposed evaluation depends on governed knowledge, the Auditor should inspect the following reference conditions before recommending benchmark escalation.

Appropriate Reference-Readiness Outputs

Nine Razor Auditor Dimensions

CEMR Remains the Core Preservation Sequence

Reference readiness does not replace Robbie’s Razor. Once the governed preservation target is sufficiently defined, the Auditor can examine how the system handles that structure through the canonical CEMR sequence.

Dimension-Specific Diagnostic Findings

The Auditor reports each dimension separately. A system may have strong reference documentation, incomplete binding rules, measurable output quality, unknown memory behavior, and unresolved total cost at the same time.

Continue to Quality Gate →

Block 7 · Evaluation Control

The Quality Gate: Efficiency Begins After Acceptance

The Razor Auditor does not treat a shorter, faster, or less expensive result as more efficient unless it first satisfies the same declared acceptance standard as the comparison result. Quality is the gate through which every efficiency claim must pass.

This protects the audit from rewarding systems that reduce visible computation by omitting requirements, lowering accuracy, transferring work to a human reviewer, or increasing downstream correction costs.

Define Acceptance Before Testing

The evaluation record should define what counts as an accepted result before the test is run. Depending on the task, acceptance may include:

Accepted Sources of Judgment

Operational Measures

The following measures may be used within a declared evaluation protocol. They are operational audit measures—not universal canonical constants:

• Accepted-task rate: accepted results divided by all attempted tasks.
• Repair burden: human and machine effort required after the initial result.
• Cost per accepted task: total measured cost divided by the number of accepted results.
• Time to accepted result: elapsed time through generation, verification, correction, and final acceptance.
• Failure rate: rejected, incomplete, unsafe, or unusable results divided by all attempts.

Next: Once equivalent accepted outcomes are established, the audit can examine whether the system preserved task-relevant structure with less unnecessary expansion.

↑ Back to top

Block 8 · CEMR Analysis

Compression Analysis: Preserve Structure, Reduce Redundancy

Within the Razor Auditor, compression means reducing unnecessary representational burden while preserving the structure required to produce an accepted result. It is not synonymous with shorter text, fewer tokens, smaller files, or lower latency.

Useful compression preserves what the task needs. If a shorter representation removes constraints, relationships, provenance, identity, or retrieval pathways that must later be reconstructed, the apparent saving may be displacement rather than efficiency.

Compression Audit Questions

1. What raw information, context, records, or state enters the system?
2. How is that information represented before and during execution?
3. Which details, relationships, and constraints must survive compression?
4. What information is removed, summarized, indexed, cached, or externalized?
5. Can the compressed structure be retrieved and used without ambiguity?
6. Does the process reduce repeated expansion across multiple accepted tasks?
7. What compression, retrieval, decompression, and verification costs are added?
8. Does the apparent benefit persist when quality and total cost are held constant?

Representation Types to Inspect

Compression Classification

Candidate Measures

• Active context or input volume per accepted task.
• Repeated or semantically redundant material within the active representation.
• Representation size before and after compression.
• Fidelity or quality retention after compression and restoration.
• Retrieval precision, retrieval recall, and unsuccessful retrieval attempts.
• Compression, indexing, storage, decompression, and verification overhead.
• Expansion avoided across repeated tasks—not merely within one isolated run.

↑ Back to top

Block 9 · CEMR Analysis

Expression Analysis: Turn Preserved Structure Into Usable Output

Expression tests whether preserved or compressed structure can generate a correct, usable result under the declared task conditions. A compact representation has limited value if it cannot reliably produce the answer, action, tool call, decision, or control signal the task requires.

The Razor Auditor therefore evaluates expression as an observable system behavior. It does not infer private chain-of-thought or claim access to hidden internal reasoning.

Expression Forms

Expression Audit Questions

1. What observable output or action is required?
2. Which preserved information, constraints, and relationships must the output express?
3. Does the result satisfy the quality gate on the first attempt?
4. Does the output comply with the required format, schema, tool, or execution boundary?
5. How much correction, retrying, reformatting, or human intervention is required?
6. Does the system introduce details that are unsupported by its evidence or stored state?
7. Can the same representation support consistent expression across equivalent tasks?
8. What latency, compute, tool, and verification costs are incurred before acceptance?

Representation Is Not Execution

Implementation is not validation. Producing an answer, plan, or tool call demonstrates expression only. It does not establish correctness, effectiveness, or support for a broader theory until the result passes the declared evaluation process.

Candidate Measures

• Accepted output rate and first-attempt acceptance rate.
• Schema, format, and constraint-compliance rate.
• Successful tool execution or downstream validation rate.
• Time and measured cost to an accepted result.
• Retries, corrections, regenerations, and human interventions per task.
• Unsupported assertions or details not grounded in registered evidence.
• Consistency across repeated or equivalent task conditions.

Next: The audit examines whether useful structure and accepted outcomes are preserved as durable memory—and whether that memory can support later recursive reuse.

↑ Back to top

Block 10 · CEMR Analysis

Memory Analysis: Preserve Structure for Reliable Reuse

Memory determines whether useful structure survives beyond the immediate task. The Razor Auditor examines what is preserved, how it is identified, where it is stored, how it is retrieved, and whether later systems can reuse it without reconstructing the original work.

A transcript, cache, database, summary, or vector store is not automatically effective memory. It becomes useful memory only when its retained structure can be located, interpreted, verified, and applied within the declared scope.

Minimum Structure to Preserve

Under the current Grand Compression framework, a reusable memory object should preserve the following elements when they are relevant to the task:

Memory Forms the Auditor Can Inspect

Memory Audit Questions

1. What structure is preserved after an accepted task is completed?
2. Does the memory retain identity, relationships, provenance, constraints, version state, and retrieval pathways?
3. Can the preserved state be retrieved under defined later conditions?
4. Does retrieval return the correct object, source, and version?
5. Can the retrieved structure support an accepted result without reconstructing the original task?
6. What storage, indexing, synchronization, retrieval, validation, and maintenance costs are incurred?
7. How are conflicting, stale, incomplete, or superseded memories handled?
8. Does the memory remain useful outside the original evaluation window?

Candidate Measures

• Successful retrieval rate under preregistered conditions.
• Correct-version retrieval rate.
• Provenance and constraint retention rate.
• Time and cost required to retrieve usable state.
• Accepted-task performance with memory compared with the declared baseline.
• Reconstruction avoided across repeated tasks.
• Stale, conflicting, missing, or incorrectly resolved memory events.
• Storage, indexing, synchronization, validation, and maintenance burden.

Next: Once structure can be preserved and retrieved, the audit tests whether later operations can reuse that memory across repeated state transitions.

↑ Back to top

Block 11 · CEMR Analysis

Recursion Analysis: Reuse Structure Across State Transitions

Recursion begins when the result or preserved state from one operation becomes a governed input to a later operation. The Razor Auditor tests whether this reuse improves task performance or reduces repeated work without allowing error, ambiguity, or cost to accumulate across cycles.

Repeated execution alone is not recursion in the audit sense. A system must preserve and reuse relevant structure across identifiable state transitions. The audit does not assume that recursion produces learning, self-improvement, autonomy, or intelligence unless those properties are independently defined and measured.

Define the Recursive Unit

Every recursion analysis should identify the state transition being tested:

The transition should declare which elements may change, which must remain invariant, which evidence is inherited, and what conditions cause the successor state to be rejected.

Recursive Patterns to Inspect

Recursion Audit Questions

1. What prior state is being reused, and how was it validated?
2. What new input, evidence, or constraint causes the next transition?
3. Which identities, relationships, provenance records, and constraints must remain invariant?
4. What is allowed to change in the successor state?
5. Does reuse reduce measured reconstruction, latency, or total cost?
6. Does quality remain above the same accepted-result threshold across cycles?
7. How are errors, conflicts, uncertainty, and failed transitions contained?
8. Can an evaluator reproduce the transition from the registered inputs and state?

Candidate Measures

• Accepted successor-state rate across declared transitions.
• Prior structure successfully reused per accepted task.
• Reconstruction or repeated expansion avoided.
• Quality change across successive cycles.
• Error propagation and correction rate.
• Constraint, provenance, and version retention across transitions.
• Human intervention required per cycle.
• Total cost per accepted transition and across the complete recursive sequence.

Next: The auditor combines quality, computation, memory, retrieval, infrastructure, correction, and operational burden into a total-cost record.

↑ Back to top

Block 12 · Cost Discipline

Total-Cost Analysis: Count the Full Path to an Accepted Result

The Razor Auditor evaluates cost across the complete path to an accepted result. A reduction in one visible measure—such as output tokens or generation time—does not establish net efficiency if cost is transferred to retrieval, storage, verification, retries, human review, infrastructure, or downstream repair.

Total-cost analysis keeps the audit focused on measured system behavior rather than a single convenient proxy.

Cost Categories

Normalize Cost by Accepted Outcomes

A defensible comparison should report cost relative to accepted work rather than successful-looking output alone.

Comparison Requirements

• Use the same task set, workload distribution, and acceptance threshold.
• Register the baseline before examining comparative results.
• Use equivalent system boundaries and observation periods.
• Include failed, retried, rejected, and abandoned attempts.
• Document caching, batching, routing, quantization, retrieval, and other configuration differences.
• Report unavailable telemetry as Unknown rather than estimating without a defensible method.
• Separate directly measured values from calculated, inferred, or proposed values.

Tokens, Energy, Water, and Emissions

Required Cost Record

• System, model, version, configuration, and provider.
• Workload, sample size, task conditions, and measurement period.
• Baseline and comparison configuration.
• Accepted-result definition and judging method.
• Measured cost categories and excluded categories.
• Units, instrumentation, calculations, assumptions, and uncertainty.
• Failed runs, missing telemetry, and known confounders.
• Finding label, evidence state when applicable, and scope limitation.

Next: The audit separates documented facts, calculations, inferences, proposals, and unknowns—then assigns formal evidence states only where a registered test supports them.

↑ Back to top

Block 13 · Evidence Discipline

Evidence and Confidence: Separate Observation From Conclusion

Every Razor Auditor output must distinguish what was directly documented, what was calculated, what was inferred, what remains proposed, and what is unknown. This prevents a plausible interpretation from being presented as a measured result.

Audit finding labels describe the basis of an individual statement. Formal evidence states describe the status of a registered claim after an appropriate evaluation. These systems serve different purposes and should not be merged.

Audit Finding Labels

Formal Evidence States

A formal evidence state applies only when a claim has been registered with a testable prediction, baseline, metrics, thresholds, failure conditions, and result record.

Confidence Record

Each material finding should identify:

• The finding label and evidence source.
• The system, version, workload, and measurement period.
• The applicable CEMR or total-cost dimension.
• The observed result and declared comparison baseline.
• Uncertainty, missing telemetry, and known confounders.
• Alternative explanations consistent with the evidence.
• The conditions under which the finding would no longer apply.
• The formal evidence state, if—and only if—a registered claim was tested.

↑ Back to top

Block 14 · Reporting Standard

Audit Output: A Scoped, Reproducible Evaluation Record

A Razor Auditor report should explain exactly what was evaluated, which evidence was available, how each finding was derived, and where the audit must stop. The output is dimensional and evidence-bounded—not a single company-wide score or label.

The same organization may show strong measured memory reuse in one workflow, unknown infrastructure cost in another, and unsupported public claims in a third. Those findings should remain separate.

Required Audit Header

Dimensional Findings Matrix

Required Report Sections

1. Executive summary: the evaluated system, mode, strongest finding, principal limitation, and next test.
2. Scope and boundary: what the audit includes and explicitly excludes.
3. Evidence register: sources, telemetry, versions, access dates, and finding labels.
4. Quality-gate results: accepted-task criteria and observed performance.
5. CEMR findings: separate compression, expression, memory, and recursion records.
6. Total-cost record: measured categories, units, exclusions, and cost per accepted task.
7. Unknowns and alternatives: missing evidence and competing explanations.
8. Failure conditions: results that would challenge or invalidate the conclusion.
9. Evidence state: included only for claims tested under a registered protocol.
10. Escalation pathway: the measurements or replication needed next.

↑ Back to top

Block 15 · Diagnostic & Evaluation Pathway

Run a Razor Audit

Choose the audit pathway that matches the evidence you actually possess. A public-source or structured diagnostic can identify authority gaps, reference-readiness issues, CEMR mechanisms, measurement requirements, and unknowns. A formal measured evaluation requires a separate preregistered protocol and may proceed only through the stages authorized by its frozen rules.

The Razor Auditor should begin with the narrowest defensible system boundary and ask an upstream question before recommending model-level testing: Can the proposed evaluation define what must be preserved strongly enough to test whether it was preserved?

That means reference readiness now comes before formal baseline execution, Quality Gate measurement, CEMR performance interpretation, and economic comparison.

Choose the Correct Audit Pathway

Ten-Step Razor Audit Intake

Minimum Reference-Readiness Intake

Recommended Auditor Readiness Record

Continue to Audit Examples →

Block 16 · Worked Examples

Audit Examples: From Broad Claims to Testable Systems

A useful audit begins by narrowing a broad description into a specific system, configuration, workload, and evidence boundary. The following hypothetical examples demonstrate how to frame Razor Auditor questions without assigning unsupported conclusions.

Important: These examples illustrate audit design only. They are not findings about any named company, model, platform, or product.

↑ Back to top

Block 17 · Integrity, Censorship & Failure Controls

Failure Conditions: Know Where the Evaluation Must Stop

Failure conditions protect the Razor Auditor from producing conclusions stronger than the available evidence. They identify when an audit should stop, narrow its claim, record an Unknown, request corrective work, or escalate a question into a more controlled evaluation.

A failure condition does not automatically mean that the evaluated model, architecture, or implementation failed. The problem may instead belong to the reference layer, target-construction process, evaluation design, Quality Gate, economic boundary, instrumentation, evidence record, or reporting process.

The Auditor should therefore identify which layer failed and which later questions were never reached.

Failure Architecture

Each stage answers a different question. A failure at an earlier stage prevents later stages from being interpreted as though they occurred.

Reference-Readiness Failure Conditions

These conditions generally indicate that a proposed governed evaluation is not yet ready for formal authority capture.

Formal Reference-Gate Failure Classes

Once a governed evaluation has been formally preregistered, the authority layer can produce several different bounded outcomes. These should remain distinct.

Target-Construction Failures

• A hidden target is constructed before the required Reference Gate passes.
• The target incorporates a source, value, relationship, correction, or interpretation not permitted by the frozen Reference Contract.
• The target is silently repaired after the evaluator discovers an authority discrepancy.
• Target provenance cannot be traced back to the governing authority package.
• Model-visible input leaks the hidden target or grants one comparison arm undeclared target information.
• The model is asked to repair an authority defect that the benchmark itself defined as part of the target.

System and Scope Failures

• The evaluated system, model, version, configuration, workload, or evidence boundary cannot be identified.
• Material components change during testing without a new evaluation version or preregistered allowance.
• The conclusion extends beyond the system, workload, domain, language, scale, or observation period actually examined.
• Results from materially different configurations are combined as though they describe one system.
• A feature-level observation is generalized to an entire organization, model family, architecture, or industry.

Quality and Comparison Failures

• The baseline is selected or changed after comparative outcomes are known.
• Compared systems receive materially different tasks, tools, authority access, constraints, context, or acceptance thresholds.
• Failed, rejected, abandoned, or unscorable runs are removed contrary to the registered treatment.
• The evaluator, rubric, threshold, adjudication procedure, or correction rule changes after results are observed.
• Apparent efficiency is created by reducing correctness, completeness, safety, usefulness, fidelity, or constraint compliance.
• Human verification or correction burden is omitted from the quality comparison.

CEMR Failure Conditions

Economic Failure Conditions

Environmental-Claim Failure Conditions

Evidence and Measurement Failures

• Material evidence lacks source identity, provenance, date, version, or system linkage.
• Calculated results omit inputs, units, equations, assumptions, or uncertainty.
• Inferred findings are presented as directly measured or documented facts.
• Missing telemetry or authority evidence is replaced with unsupported estimates.
• Measurement instruments, data pipelines, judges, or transformations cannot be reconstructed.
• Private internal reasoning is claimed without observable evidence or authorized access.
• Contradictory or adverse evidence is omitted from the final record.

Governance and Reporting Failures

• A diagnostic Razor Auditor output is represented as certification or formal validation.
• Reference Readiness is reported as though it were a formally executed Reference-Gate result.
• Licensing, payment, publication, adoption, deployment, or implementation is presented as empirical validation.
• A formal evidence state is assigned without the governing claim-review process.
• An authority-layer result is mislabeled as a model-performance result.
• A Quality Gate failure is reported as an economic result.
• An economic negative result is generalized into universal architectural failure.
• A benchmark result is published without stating the deepest valid stage reached.
• A prior result is silently overwritten instead of being versioned, corrected, challenged, superseded, or retired.

Keep Failure Layers Separate

Required Response to a Material Failure Condition

1. Identify the exact layer where the condition occurred.
2. Stop or isolate downstream stages that the failed condition no longer authorizes.
3. Preserve the authority capture, input, target state, run, logs, configuration, and evidence associated with the event.
4. Record the applicable finding label and bounded failure classification.
5. State the deepest valid stage reached.
6. Explicitly identify every downstream stage that was not reached or not tested.
7. Reduce the scope of any public conclusion accordingly.
8. Determine whether corrective action, governance correction, a successor study, or independent replication is appropriate.

Continue to Benchmark Escalation →

Block 18 · Validation Pathway

Benchmark Escalation: Move From Diagnostic Questions to Testable Evidence

A diagnostic audit organizes the evidence and identifies what must be tested. Benchmark escalation begins when a material proposed finding can be converted into a preregistered comparison with a baseline, metrics, thresholds, and failure conditions.

Not every diagnostic observation requires a formal benchmark. Escalation is most valuable when a claim affects architecture, procurement, deployment, licensing, environmental reporting, safety, or a broader theoretical conclusion.

When to Escalate

Minimum Benchmark Registration

Escalation Levels

↑ Back to top

Block 19 · System Navigation

Canonical Links and Evaluation Resources

The Razor Auditor is an applied diagnostic and evaluation-intake framework. It does not replace the governing canon, claims register, lab protocol, benchmark records, or independent replication.

Use the following resources to move between canonical definitions, testable claims, evaluation procedures, reproducible benchmark assets, and applied examples.

Public Benchmark Repository

Authority and Evidence Flow

↑ Back to top

Block 20 · Frequently Asked Questions

Razor Auditor FAQ

These answers clarify what the Razor Auditor does, what evidence it requires, and where its conclusions must stop.

↑ Back to top

Block 21 · Author and Framework Origin

About Robbie George

Robbie George

Robbie George is a National Geographic–published nature photographer, writer, and creator of the Grand Compression Cosmology and Robbie’s Razor.

His work examines how systems preserve usable structure across change through compression, expression, memory, and recursion. The Razor Auditor extends that framework into a disciplined method for asking measurable questions about AI reasoning systems, reusable memory, recursive workflows, infrastructure, and total cost.

Robbie’s field experience in nature photography and ecological observation informs his interest in systems that operate under real limits. His current work connects visible human-readable explanations with versioned claims, machine-readable resources, public benchmark assets, and reproducible evaluation pathways.

The Grand Compression framework remains evidence-bounded: implementation does not equal validation, domain transfer requires new testing, and formal claims must remain open to support, challenge, revision, or retirement.

About Robbie George Read GC-MRD-v2.0 GitHub Benchmarks

↑ Back to top