Define the decision before asking for code

Replace a vague request with a testable contract

“Make a script that finds urgent tickets” leaves almost every important decision unresolved. What is urgent? Which fields are trusted? Does urgency mean a label or a write to the live system? What happens if a timestamp is unknown? Can the script overwrite yesterday's output? An assistant can fill those gaps fluently, but invented rules are not requirements.

Our lab contract is intentionally small. Each ticket has exactly four fields: ticket_id, severity, age_hours and target_hours. IDs use T followed by three decimal digits. Severity is one of low, medium or high. Age is a finite integer or floating-point number between 0 and 168 inclusive. Target is finite, greater than zero and at most 168. Booleans and numeric-looking strings are not accepted as measurements.

A valid ticket becomes an URGENT_CANDIDATE when severity is high or age is at or above target. Otherwise it becomes a ROUTINE_CANDIDATE. Retain both reasons if both conditions apply. Invalid tickets are reported as REJECTED, not quietly classified as routine. Duplicate valid ticket identities make the whole batch ambiguous and must stop analysis before publication.

These are lab rules, not universal helpdesk policy. The 168-hour range is an approved exercise bound; a real organization may need a different range and calendar. The classification name deliberately includes “candidate” so a downstream reader does not confuse a local rule with an authorized ticket-priority change.

Write the expected matrix yourself

Use this matrix before examining an implementation:

The baseline has five input records: two urgent candidates, one routine candidate and two rejected records. Those three populations reconcile to five. A report showing three routine tickets because it treats invalid values as zero has changed the meaning of the result even if it still produces five output rows.

Add three more mental cases. High severity with age exactly at target retains two reasons. A low-severity ticket just below target remains routine. An empty batch has zero counts, not a 100% quality conclusion. These cases separate rule semantics from output formatting and help you avoid testing only the most obvious example.

A constrained prompt, with authority limits

If you use an assistant, provide a prompt like this and adapt it only after an approved specification change:

Implement a Python standard-library, offline classifier for fictional tickets.
The exact schema is ticket_id, severity, age_hours, target_hours.
IDs are T plus three digits. Severity is low, medium or high.
Age: finite int/float in [0,168], excluding bool and text.
Target: finite int/float in (0,168], excluding bool and text.
Urgent candidate means severity high OR age >= target.
Retain both reasons when both apply. Invalid rows remain rejected evidence.
Duplicate valid IDs stop the batch; do not select a latest record without evidence.
No network, secrets, accounts, ticket writes, subprocesses or package downloads.
Do not overwrite an existing output or the input file.
Return code and explain assumptions; do not invent operational SLA rules.
I will supply independent expected results and review the implementation.

Do not paste real helpdesk exports into an unapproved tool. Even a ticket subject or hostname can reveal sensitive context. Use a synthetic schema and small fixture for development; any later real-data handling needs the organization's approved service, data policy and access scope. The same care applies to logs and error traces you ask an assistant to debug.

Exercise: identify unsupported decisions

Review a hypothetical response that adds automatic priority updates, translates unknown to zero, installs an external package and chooses the last duplicate ID. Which parts violate the contract, and which require new approval rather than a minor refactor?

Reasoning model

Treat generated code as an untrusted draft whose behavior must be established. A useful prompt supplies input contract, invariants, forbidden side effects and acceptance tests, but the model’s explanation is not execution evidence. Review dependencies, network calls, file paths and destructive operations before running. Tests must include boundaries and failure paths that challenge the implementation, not only examples it was shown.

Official reference for further study

Review the draft as executable behavior, not confident prose

Inspect the deliberately flawed specimen

"""Synthetic review specimen. Deliberately flawed; not a production tool."""
def classify_ticket(ticket):
    if ticket['severity'] == 'high' and ticket['age_hours'] > ticket['target_hours']:
        return 'URGENT_CANDIDATE'
    return 'ROUTINE_CANDIDATE'

Trace T002: severity is low, so the and expression is false regardless of elapsed time. It is marked routine even though age equals target. Trace T003: severity is high but age is below target, so it also becomes routine. Both violate the written OR rule. A high-severity ticket beyond target passes this flawed branch, which means a single dramatic example would miss the problem.

The specimen also uses > instead of >=, ignores type and range contracts, and loses separate reasons. A missing field can raise an exception without a row-level rejection policy. A text age may compare incorrectly or fail at runtime. It has no batch duplicate policy and no provenance or output interface. These are distinct review findings; repairing the boolean operator alone does not finish the job.

Turn the review into a defect ledger

For each defect, record the requirement, a smallest exposing input, observed result and expected result. Avoid “the script looks unsafe” as the only review comment. For the OR defect, T003 is useful: high severity with age zero must be urgent. For the equality defect, T002 is useful: low severity at exactly four hours must be urgent. For missing numeric validation, use True, NaN, infinity and numeric text as separate cases.

Python booleans have a relationship to integers that makes a broad isinstance(value, int) check insufficient for this contract. The implementation uses exact type membership. Floating-point non-finite values also need deliberate handling: a comparison with NaN does not behave like an ordinary measurement. Do not “clean” it into zero just to keep arithmetic convenient.

Huge integers introduce another edge case. Converting an arbitrarily large integer to a float just to check finiteness can fail before the range check returns a useful validation error. Here, Python integers are checked by range without unnecessary floating conversion; finite checking applies to actual floating values. The approved maximum makes an enormous integer invalid regardless of its representation.

Separate record errors from batch ambiguity

A bad numeric field can be rejected with its safe fixture identity and reason while valid records continue. A repeated valid ID is different: two rows now claim the same ticket identity, and the exercise has no revision field to determine whether either is current. The program refuses the batch. That choice prevents hidden “last row wins” behavior in a dictionary.

For an invalid ID such as a list, do not attempt to use it as a set key or echo arbitrary data into a report. The output uses unavailable when no valid fixture ID exists. A test for this invalid shape caught a real review defect in the initial replacement's rejection path. Error handling deserves tests too; code that carefully validates normal fields can still crash while reporting invalid input.

Read side effects before running

Inspect imports and calls for network libraries, subprocesses, file writes, deletion, environment access and credential handling. Names alone do not establish safety: follow how a path is constructed and whether an action is authorized. In this lab, the only final write is a new local JSON report, opened exclusively. Source data is read, not modified. The pure classifier itself performs no I/O.

Run a syntax check and tests in the new fixture directory before using a new output path. A successful syntax check does not prove the rule, and successful tests do not prove an arbitrary package or hidden adapter is authorized. Maintain a review boundary around the exact files being exercised.

Exercise: avoid a shared-oracle trap

An assistant generates both the function and a test that expects T002 to be routine. Both run successfully. What has been established? How would you repair the verification process?

Evidence laboratory — attempt before reading the solution

An assistant generates a cleanup script for files older than30 days in a selected practice directory. The requirement actually says: report candidates only, never delete; exclude symlinks and subdirectories; treat exactly30 days as not older. The draft calls unlink(), follows recursive traversal and uses age>=30. Identify each contract violation before execution and design a safe replacement interface.

Official reference for further study

Implement a reviewed replacement and reconcile its report

Keep logic, batching and the command interface separate

classify_ticket validates one ticket and returns its label and reasons. evaluate manages population counts, row rejection and duplicate identity. The CLI parses the input, records its hash and publishes a new report. This separation lets tests target behavior without contacting anything or needing real system permissions.

The CLI rejects duplicate JSON properties and non-standard numeric constants rather than allowing the parser to silently choose a value. An input such as two ticket_id properties in one object is not the same as two clearly separate tickets. Ambiguous serialization should stop the input contract before analysis.

Save the reviewed program below as reviewed_classifier.py:

"""Classify fictional ticket fixtures; never contact or change a ticket system."""
import argparse
import hashlib
import json
import math
from pathlib import Path
import re
import sys

class ContractError(ValueError):
    pass

def number(value, field, positive=False):
    if type(value) not in (int,float) or (type(value) is float and not math.isfinite(value)):
        raise ContractError(field+' must be a finite numeric value, not text or bool')
    if not 0 <= value <= 168 or (positive and value == 0):
        raise ContractError(field+' outside approved lab range')
    return value

def classify_ticket(ticket):
    if not isinstance(ticket,dict) or set(ticket) != {'ticket_id','severity','age_hours','target_hours'}:
        raise ContractError('Unexpected ticket schema')
    if not isinstance(ticket['ticket_id'],str) or not re.fullmatch(r'T[0-9]{3}',ticket['ticket_id']):
        raise ContractError('Invalid ticket identity')
    if ticket['severity'] not in ('low','medium','high'):
        raise ContractError('Unsupported severity')
    age=number(ticket['age_hours'],'age_hours')
    target=number(ticket['target_hours'],'target_hours',positive=True)
    reasons=[]
    if ticket['severity']=='high': reasons.append('HIGH_SEVERITY')
    if age >= target: reasons.append('AT_OR_OVER_TARGET')
    return {'ticket_id':ticket['ticket_id'],'classification':'URGENT_CANDIDATE' if reasons else 'ROUTINE_CANDIDATE','reasons':reasons}

def unique_object(pairs):
    result={}
    for key,value in pairs:
        if key in result: raise ContractError('Duplicate JSON property')
        result[key]=value
    return result

def reject_constant(value):
    raise ContractError('Non-standard numeric JSON constant')

def evaluate(rows):
    if not isinstance(rows,list) or len(rows)>1000:
        raise ContractError('Expected an array of at most 1000 lab tickets')
    seen=set();results=[]
    for row in rows:
        identity=row.get('ticket_id') if isinstance(row,dict) else None
        valid_identity=isinstance(identity,str) and bool(re.fullmatch(r'T[0-9]{3}',identity))
        if valid_identity:
            if identity in seen: raise ContractError('Duplicate ticket identity')
            seen.add(identity)
        try: results.append(classify_ticket(row))
        except ContractError as error:
            results.append({'ticket_id':identity if valid_identity else 'unavailable','classification':'REJECTED','reasons':[str(error)]})
    return {'scope':'Candidate labels in supplied fictional data; no operational ticket action.',
            'input_records':len(rows),'urgent_candidates':sum(r['classification']=='URGENT_CANDIDATE' for r in results),
            'routine_candidates':sum(r['classification']=='ROUTINE_CANDIDATE' for r in results),
            'rejected_records':sum(r['classification']=='REJECTED' for r in results),'results':results}

def main():
    parser=argparse.ArgumentParser(description=__doc__)
    parser.add_argument('source',type=Path);parser.add_argument('output',type=Path);args=parser.parse_args()
    try:
        if args.source.resolve()==args.output.resolve(): raise ContractError('Source and output must differ')
        data=args.source.read_bytes()
        if len(data)>1000000: raise ContractError('Input exceeds lab byte budget')
        rows=json.loads(data,object_pairs_hook=unique_object,parse_constant=reject_constant)
        report=evaluate(rows);report['source_sha256']=hashlib.sha256(data).hexdigest()
        with args.output.open('x',encoding='utf-8') as stream:
            json.dump(report,stream,indent=2);stream.write('\n')
    except (OSError,ValueError) as error:
        print('Review failed: '+str(error),file=sys.stderr);return 2
    print(f"Classified {report['input_records']} records: {report['urgent_candidates']} urgent, {report['routine_candidates']} routine, {report['rejected_records']} rejected")
    return 0

if __name__=='__main__':
    raise SystemExit(main())

Run the baseline with literal input

Use the supplied tickets.json. Its five rows are exactly the matrix above. Run:

python reviewed_classifier.py tickets.json review.json

Expect Classified 5 records: 2 urgent, 1 routine, 2 rejected. Inspect review.json: T002 has AT_OR_OVER_TARGET; T003 has HIGH_SEVERITY; the rejected rows explain their invalid data. The source hash identifies the exact input bytes used for this run. It does not establish that the input was authentic or complete.

Rerun with the same output path. Expect exit status 2 and the original report unchanged. Try using the input path as output and confirm refusal. Use a new name for each successful experiment. Retain the failed invocation and diagnostic message as evidence rather than deleting the old result to force success.

Understand the resource and publication limits

The CLI reads the local source before checking the one-million-byte lab budget. That limit is an admission check after reading, not a hard memory bound against a hostile oversized file. A production reader would enforce limits while streaming and account for parser expansion and nested data. The 1,000-record limit also is a lab contract, not a benchmark for unlimited automation.

Exclusive output opening prevents overwrite, but the JSON is written directly to that new file. A write failure or crash can leave a partial new report. A final filename alone is not proof of successful completion. An operational publisher would need a complete-write marker or an atomic publication design, integrity verification and a clear failed-run procedure.

The program has no business-hours calculation, tenant connection, authentication, durable ticket revision state or update idempotency. A separate live collector must establish source scope, coverage and timing semantics. Do not adapt the candidate report into automatic priority changes by adding a single API call at the end.

Review a proposed “helpful” improvement

Suppose an assistant proposes accepting strings such as "4" and interpreting an empty string as zero. Decide whether this is a parsing feature or a silent measurement policy. If you intentionally support numeric strings later, define syntax, decimal conventions, whitespace, units, invalid values and provenance. Test them independently and disclose the changed interface.

Another proposal changes URGENT_CANDIDATE to BREACHED_SLA. That overstates evidence. The classifier uses supplied elapsed numbers and a simple threshold; actual SLA policy may exclude weekends, pauses or particular ticket states. Names influence downstream decisions, so review output vocabulary with the same care as the boolean expression.

Worked laboratory solution

Official reference for further study

Workbook already downloaded

Prove your review and build a reusable workflow

Run independent tests

from pathlib import Path
import json
import math
import unittest
from reviewed_classifier import classify_ticket,evaluate,ContractError

def ticket(**changes):
    return {'ticket_id':'T099','severity':'low','age_hours':1,'target_hours':4,**changes}

class ReviewTests(unittest.TestCase):
    def test_literal_baseline(self):
        rows=json.loads(Path(__file__).with_name('tickets.json').read_text())
        report=evaluate(rows)
        self.assertEqual([report[k] for k in ['input_records','urgent_candidates','routine_candidates','rejected_records']],[5,2,1,2])
        self.assertEqual([r['classification'] for r in report['results']],['ROUTINE_CANDIDATE','URGENT_CANDIDATE','URGENT_CANDIDATE','REJECTED','REJECTED'])
    def test_exact_target_is_urgent(self):
        self.assertEqual(classify_ticket(ticket(age_hours=4))['reasons'],['AT_OR_OVER_TARGET'])
    def test_high_alone_is_urgent(self):
        self.assertEqual(classify_ticket(ticket(severity='high'))['reasons'],['HIGH_SEVERITY'])
    def test_both_reasons_retained(self):
        self.assertEqual(classify_ticket(ticket(severity='high',age_hours=4))['reasons'],['HIGH_SEVERITY','AT_OR_OVER_TARGET'])
    def test_invalid_numbers_rejected(self):
        for value in [True,'4',math.nan,math.inf,-1,169,None]:
            with self.subTest(value=value),self.assertRaises(ContractError):
                classify_ticket(ticket(age_hours=value))
    def test_target_zero_rejected(self):
        with self.assertRaises(ContractError): classify_ticket(ticket(target_hours=0))
    def test_large_integer_rejected_without_float_overflow(self):
        with self.assertRaises(ContractError): classify_ticket(ticket(age_hours=10**1000))
    def test_unhashable_identity_becomes_rejection(self):
        self.assertEqual(evaluate([ticket(ticket_id=[])])['rejected_records'],1)
    def test_wrong_schema_rejected(self):
        with self.assertRaises(ContractError): classify_ticket(ticket(extra='ignore me'))
    def test_duplicate_identity_rejected(self):
        with self.assertRaises(ContractError): evaluate([ticket(),ticket()])
    def test_empty_fixture_has_zero_counts(self):
        report=evaluate([])
        self.assertEqual([report[k] for k in ['input_records','urgent_candidates','routine_candidates','rejected_records']],[0,0,0,0])

if __name__=='__main__':
    unittest.main()

Run python -m unittest -v from the downloaded project directory. Eleven tests should pass. The literal baseline and boundary expectations are specified independently of the classifier. The tests include both reasons, unsupported numbers, zero target, oversized integer, invalid list-shaped identity, wrong schema, duplicate identity and empty population.

That evidence is useful and bounded. It does not cover every possible nested JSON payload, runtime failure or operational integration. A coverage percentage would also not prove that the tested assertions match the requirement. Ask what each test can detect and what important boundary remains unverified.

Mutation: demonstrate that verification can fail

In a separate copy, change if age >= target to if age > target. The equality and baseline checks must fail. Restore the code. Next disable the high-severity branch while leaving the age branch intact. The high-alone and both-reasons checks must detect the loss. These two mutants isolate separate requirements rather than producing one large broken program with no clear diagnosis.

Compare the supplied flawed specimen with the reviewed solution on T002 and T003. Record the mismatched labels. Do not run the specimen over real data just to gather more examples. A small synthetic counterexample already demonstrates that it violates the contract.

Project A — Add an approved “near target” category

Propose a third valid label, NEAR_TARGET_CANDIDATE, for non-high tickets with age at least 75% of target but below target. Keep high severity and at/over target as urgent. Write expected cases before coding: low severity at 2.99 of 4 is routine; at 3 of 4 is near; at 4 of 4 is urgent; high severity at zero remains urgent.

Retain rejection rules and update reconciliation counts so every row appears in exactly one category. Avoid overlapping independent counters that count the same row twice. Add a reason vocabulary and explain whether percentages are measured on elapsed hours or a different approved business clock. Do not claim the extension is implemented or tested until your own code and assertions establish it.

Project B — Review an assistant's proposed live adapter

Write an adapter review without implementing external writes. Identify the endpoint, authentication identity, read permission, pagination, source coverage, timestamp normalization, timeouts, retry budgets and duplicate revision semantics. Define the evidence needed to trust the age and target fields. Unknown source coverage cannot be repaired by testing the classifier more often.

If a future requirement includes ticket updates, add authorization, approved states, a durable action identity, concurrency handling, ambiguous-success recovery and human escalation. Retrying a request after a lost response can duplicate an action or apply it in the wrong state. A read-only candidate report is a safer bounded first task, but it is not the complete design for a write workflow.

Project C — Produce a code-verification portfolio

Deliver a specification, expected-value matrix, flawed draft, defect ledger, reviewed replacement, tests, baseline report, mutation results and limitations. Record the exact source version and runtime. Include one review finding that was not obvious from the normal happy path, such as the invalid-ID rejection path or huge-integer handling.

For each finding, explain the actual failure, not just a generic warning. A useful portfolio demonstrates that you can disagree with a fluent answer using evidence and improve it without inventing new requirements. It also shows when you stop: a model's suggestion is not authority to access secrets, install tools or modify a production service.

A repeatable review loop

Start with an approved bounded task; define input, output and side effects; prepare independent cases; request a draft; inspect interfaces and error paths; run isolated tests; inject a targeted defect; review the repair diff; and verify a clean copy. Only then consider an authorized integration with separate staging and rollback checks.

Keep the prompt and review notes useful, but do not confuse them with execution evidence. “The assistant says it tested the code” is not a substitute for your captured command, runtime, input and result. Likewise, a green local suite does not authorize deployment. The distinction between drafting, testing, reviewing and releasing is the central skill in this course.

Consult official Python documentation for unittest, numeric types, math.isfinite, JSON parsing and exclusive file opening. The ticket policy, prompts, scenarios and review portfolio are original exercises. You can finish the standalone course without purchasing a book or using any paid AI model.

Official references: unittest, math.isfinite, JSON and open.

Transfer project — changed conditions

Build fixtures at29,30 and31 days, plus a directory and symlink where supported. Assert that only the intended31-day regular file is listed and every original remains unchanged. Add an unreadable entry and a path outside the allowed root; define explicit error/scope behavior. Ask the assistant to explain a failing test, then independently review its proposed patch. Submit the specification, diff, executed results and limitations.

Practical assessment rubric

Score each dimension 0–2: 0 = absent or materially incorrect; 1 = partly correct with an unresolved gap; 2 = correct, explicit and supported by the supplied evidence or an actual labeled lab result.

1. Evidence/model: identify the relevant entities, inputs, assumptions and boundaries without inventing facts.
2. Main result: derive the case-specific expected result, calculation or decision and show the intermediate reasoning.
3. Failure/boundary: test the changed condition and explain why the original conclusion does or does not transfer.
4. Verification: supply reproducible steps and expected/observed results; label manual traces as manual, not executed.
5. Communication/safety: explain limitations, owners and a proportionate next action without unsafe or unnecessary access.

Target: at least 8/10, with no zero in main result or communication/safety. This is a local learning rubric, not an accredited certification standard. A written/manual solution demonstrates reasoning only; operational competence additionally requires the relevant executable lab in an authorized environment. The native multiple-choice score is separate and does not automatically grade this rubric.

Submission template

• My input/evidence and assumptions:
• Expected result with calculations or trace:
• My implementation/decision and justification:
• Changed-condition result:
• Observed execution, or explicitly labeled manual trace:
• Failure and recovery evidence:
• Unverified limitation and next action:
• Self-score with evidence for each dimension:

Remediation

If the main result is wrong, reconstruct the supplied case from the raw inputs before changing code or controls. If the boundary case is wrong, identify the assumption that changed and rerun both the original and changed case. If verification is missing, do not convert a prediction into an observed result. Reattempt the transfer task without copying the worked solution, then explain it to another person.

Official reference for further study

Assessment: explain your decisions

Choose an answer and write your reason before opening each explanation. Then explain why the other choices do not fit the evidence. Use missed questions to revisit the lesson. This knowledge check is only one part of assessment: complete the practical transfer project as well. It does not predict certification-exam success.

1. Which input reveals the supplied comparison bug?

1. 10
2. 9 only
3. Any screenshot

2. What should you do when an acceptance test reveals a defect?

1. Delete the test
2. Correct the implementation or clarify the requirement
3. Trust confident prose

3. What belongs in a safe coding request?

1. Production secrets
2. Unbounded system access
3. Input/output contract and permitted actions

4. A low-severity ticket is exactly at its target age. Under the written rule, which label is expected?

1. URGENT_CANDIDATE, because age >= target independently satisfies the OR rule.
2. ROUTINE_CANDIDATE, because only high severity can qualify.
3. REJECTED, because equality is invalid.

5. Generated code and generated tests both say T002 is routine, but the approved matrix says urgent. What has the green suite proved?

1. The approved specification is wrong.
2. The two generated artifacts agree; independent requirement verification is still missing.
3. The script is ready to update real tickets.

6. The assistant proposes replacing unknown age with zero and automatically updating priorities. What is appropriate?

1. Accept both because they simplify the workflow.
2. Run on real tickets to see if users complain.
3. Reject the invented measurement and external action unless separately specified and authorized.

7. Which violation should stop execution immediately?

1. The draft deletes files despite a report-only requirement.
2. It uses readable variable names.
3. It returns a list.

8. Should an exactly30-day file be included?

1. Only if its name is short.
2. No; older than30 is a strict boundary.
3. Yes; >= is always equivalent to older than.

9. Why inject the reference time?

1. To avoid all filesystem permission checks.
2. To make deletion safe automatically.
3. To make age classification deterministic and testable.

Use the explained questions above for offline practice. A scored attempt requires the course assessment on CogniSkill.ca.