The product
Cognanimus turns broad learning goals into scoped paths, lessons, practice and cumulative assessments. Prepared material makes the ordinary learning flow immediate; metered generation adds custom content and written feedback. The live catalog contains 17 paths, 85 milestones, 255 lessons and 1,530 questions.
The design problem is larger than generating a lesson. A useful learning system must keep the question, grading criteria, learner’s submission and resulting progress consistent—even when a model disagrees, a provider fails or content changes.
Make the application own the grade
Numerical, choice, ordering and circuit answers use deterministic checks. Written responses go to a model with the submitted question’s rubric and relevant public context. The application validates question identities, point allocations and outcomes, then computes weighted totals itself.
An inconsistent rubric or an ambiguous judgment can remain visibly ungraded. Provider failure preserves the saved answer and exposes a retry path. Repeated submissions reuse the original attempt instead of multiplying grades or rewards. Content revisions preserve submitted snapshots; explicit corrections carry an audit record.
Treat grader failures as product findings
A broad 100-case run matched 99 declared score expectations. The disagreement exposed an ambiguous quantum partial-credit criterion. The expected score stayed fixed; the public rubric was clarified and a complementary valid-input case added. A focused 25-case run then passed.
A separate reported biology grading issue revealed a hidden requirement: the rubric demanded information the question had not asked for. Three related questions were repaired, and a later nine-case regression passed. Those are distinct diagnostic runs, not a single 110-case result or a real-world accuracy estimate.
| Evaluation | Observed result | What it supports |
|---|---|---|
| Broad grading sweep | 99 / 100 | Detected partial-credit ambiguity |
| Focused rubric / feedback regression | 25 / 25 | Checked the bounded repair |
| Later hidden-requirement regression | 9 / 9 | Checked three revised biology questions |
Keep progress separate from learning evidence
First attempts, fresh follow-ups and delayed recall are reported separately. The evidence view records content versions, available references, extra help and repeated questions. An unresolved check stays visible; an empty denominator says there is no evidence yet.
This makes the product’s claims narrower and more useful. Completing an activity and answering a fresh question after a delay are different observations. Neither alone establishes a causal learning benefit or a calibrated mastery probability.
Reliability beyond the happy path
The system includes isolated accounts, durable jobs, spending reservations, export and deletion, graceful shutdown and backup/restore verification. A local reliability exercise used real HTTP requests, SQLite and process crashes with a synthetic delayed provider. Sixteen- and 24-learner bursts exposed a shared-IP throttle that could block unrelated signed-in learners; authenticated API limits were moved to verified accounts.
The public application is an invite-only alpha. The engineering evidence covers recovery and bounded operation; a real learner study is still needed to evaluate educational effectiveness.
Limitations
- Most grading fixtures are constructed and labels are not independently supplied by human subject-matter experts. The complete 110-case suite was not rerun as one final benchmark.
- Passing regression cases does not establish prompt-injection immunity, comprehensive curriculum correctness or population grading accuracy.
- Reliability observations use a local single-instance application and a synthetic provider, not a production capacity benchmark.
- No causal learning-effectiveness result is claimed. The freshness detector checks exact normalized repeats, not semantic paraphrases.
Source code & evidence
Results reflect the linked evidence snapshots. Source artifacts retain their dates and experimental conditions.