Changelog¶
All notable changes to looplet are documented here. The format is
based on Keep a Changelog, and
this project adheres to Semantic Versioning.
[Unreleased]¶
Security¶
- Cartridge YAML loading rejects unconsumed lines instead of silently dropping later budgets or permissions. Comments before nested lists remain supported.
- MCP structured arguments are prepared before sync/async permission hooks and sent without later type conversion. Direct registry dispatch retains JSON-string repair; raw tool executors reject structured arguments that still need repair.
- MCP adapter and cartridge imports reject duplicate tool names before registering any tool from that server. Rejected cartridge servers are closed, including in permissive loading mode.
- MCP protocol requests have configurable message, queued-response, and skipped-message limits, with one deadline spanning writes and reads. Closing an adapter discards prior-session messages and tool definitions.
- MCP supports argv commands without a shell and exact child environments with
inherit_env=False. Existing shell-string and inherited-environment defaults remain supported. - CI and package builds use read-only workflow permissions and non-persisted checkout credentials. Direct action dependencies are commit-pinned, Pages privileges are deployment-only, and Dependabot maintains action references.
- Saved assessment and experiment report inspection reject links outside the selected report directory. Experiment histories reject report and child-directory links outside the selected history root. Contained links, JSON values, and saved evidence remain unchanged.
- Saved-experiment and history inspection escape terminal controls in report fields, trace links, and diagnostics. JSON values, Unicode, and saved evidence remain unchanged.
- Saved-assessment inspection escapes terminal controls in report fields, evidence links, and diagnostics. JSON values, Unicode, and saved evidence remain unchanged.
- Saved-evidence readers reject selected FIFOs, sockets, devices, and directories before opening them. Missing optional files and contained links to regular files remain supported.
- Human
showand saved-traceinspectescape terminal controls in displayed evidence and errors. JSON values and saved files remain unchanged. - Trace inspection rejects escaping and broken selected component links, including legacy trajectory, manifest, and optional eval files. Contained links remain supported.
- Checkpoint auto-resume without an explicit run ID rejects every corrupt file, including legacy keys containing
__, before model calls or tool effects. Explicit run IDs still isolate unrelated run namespaces. - Saved eval and replay readers reject component links outside the selected artifact directory, including legacy layouts and optional files.
- Contained component links remain supported. Broken metadata links fail closed instead of selecting legacy fallback.
- Optional provider extras require
certifi>=2026.10.19for the corrected CA bundle described in GHSA-6q48-6c7w-9gw2. - Matching
DEFAULTpermission rules resolve through the configured engine default instead of permitting an unresolved decision. - Approval checks preserve nested argument types when comparing the approved action with the action submitted for execution.
- Permission failures retain diagnostic categories without logging raw exception text, approval tracebacks, or invalid approval payloads.
- File checkpoint stores reject unsafe run IDs before access and treat lookup-key metacharacters literally.
- File checkpoint stores reject cross-run filename collisions and preserve corrupt existing records. Atomic publication requires filesystem hard-link support.
- The security policy documents trust boundaries, secure deployment requirements, residual risks, and controlled regression checks.
Changed¶
- Sync, async, and cartridge runs resolve
execution-v1by default: deterministic compaction and native typed history without implicit stagnation guidance or a step-derived tool quota. Explicit policies take precedence; the namedstandard-v1policy andminimalprofile remain available without changing positional config arguments. - Original tool results are retained independently of conversation previews and compaction. Base registries expose detached recall and checkpoint snapshots, with a host-owned retention opt-out.
- Loop configuration leaves temperature unset by default; built-in SDK adapters omit an unset temperature and preserve explicit sampling values.
- Shared model admission rechecks cancellation and deadlines after acquiring its lock. Run measurements retain observed request counts separately from legacy reported helper totals.
- Loop API documentation states the existing live-config ownership contract. Explicit loop argument overrides update the supplied config without replacing cancellation tokens or other host controls.
- OpenAI text streams yield visible text from multipart responses and exclude structured provider metadata in sync and async paths.
- Cartridge loading reads and validates config and runtime declarations before importing or building shared resources. Reference resolution and resource cleanup retain their existing ownership rules.
- Eval root help introduces Python factories before cartridge execution.
- Saved trace inspection labels grades as capture-time observations rather than current assessments.
--versionincludes the doctor's Git-independent source fingerprint, distinguishing same-version preview builds without selecting a provider.- Human Python-factory inspection lists tools, the completion tool, step limit, protocol, and hook count, with an equivalent full-blueprint command.
- The Quickstart starts with the released package and ordinary Python tests. The complete experiment starter and factory CLI are optional, with a separate commit-pinned preview environment and matching provider extras.
- The packaged Quickstart labels its output, completion state, and successful grade instead of printing unexplained booleans. Intentionally stopped and failed runs do not claim a passing grade.
- CLI
run, its format aliases, andreplayrequire accepted completion by default. Use--allow-incompletefor the earlier command-success policy. --require-completedremains supported. Strict bundle runs use the shared runner; custom human runners require--allow-incomplete.- README, docs home, and Quickstart introduce
minimal_presetandrun_resultconsistently. Cartridge and skill-bundle packaging remain optional.
Added¶
- Inspectable
resolve_harness()and preset-level resolution compose deterministic context compaction, actual-result stagnation nudges, and a bounded dispatch budget without implicit I/O or summary-model calls. ToolCallBudgetHookreserves before concurrent dispatch and counts successful free calls. It is a per-invocation tool guard, not a provider-request or billing limit.- Independently graded default-harness examples exercise fresh Python and strict-exported cartridge agents; a downstream qualification script covers Analytics profiles and both proxy transports.
is_loop_hook()recognizes partial, duck-typed hooks without mandatory inheritance.LoopHookremains the complete interface for protocol checks.- Public optional backend contracts preserve transparent wrappers, retry ownership, typed context and provider state without required inheritance.
- Shared model budgets admit individual requests, avoid duplicate nested charges and restore consumed allowances without replenishment.
- Built-in adapters accept explicit provider request options and expose detached requested and submitted settings without claiming provider confirmation.
- Named checkpoint participants validate complete versioned snapshots before restoration and block reuse after partial restoration failure.
- Opt-in host assistant-turn policies support bounded continuation without fake tool steps. Request ceilings cover admitted retries and helper inference.
- Typed working history, correlated progress, copied message checkpoints, and explicit replay bindings preserve assistant-turn semantics without changing strict defaults.
- The packaged hello example supports offline assistant continuation with an explicit model-call limit.
- Structured model-call snapshots preserve returned content blocks, provider metadata, stop reasons, usage, and errors after redaction. Replay prefers canonical captures over bounded text projections.
- Opt-in assistant-message completion uses the existing terminal gates and output schema without executing a terminal tool body. Native truncation recovery is bounded, checkpoint-safe, and records each request and its usage.
- Trajectories retain completion/recovery settings for default Python replay and snapshot observed tool payloads against later mutation.
inspectlists saved experiment history with status, stage, and shell-safe child inspection commands, retaining unreadable reports as errors.inspectlists saved eval suite cases and opens separate assessment reports without rerunning graders or inventing a historical gate policy.- Saved-evidence rescoring supports
--out, andEvalReport.save()writes private, exclusive assessment files with relative evidence links without replacing capture-time grades. inspectaccepts Quickstart experiment directories and reports without rerunning code, while preserving legacy reports and trace inspection.inspect module:factorysupports safe reference-only declarations and explicit, resource-managed runtime wiring validation without selecting a model.- Factory CLI replay restores the captured tool protocol and carries saved expectations into fresh, regradable eval records.
- The one-file starter shares tasks and expectations through eval cases, supports
--case, and records cases for CLI replay. - One editable Quickstart agent supports factory execution, provider selection, and repeat-safe experiments with saved reports.
- Quickstart replay saves fresh evidence and grades, honors explicit limits, and preserves the original capture.
- Python replay accepts an optional recording sink and step limit for both synchronous and asynchronous hosts.
- Commit-pinned preview installation separates unreleased factory commands from the published Python workflow.
- Quickstart exports a complete editable Python agent and provides repeat-safe capture, eval, replay, and grading recipes.
- Cartridge exports report detected external Python imports in metadata and a warning. Factory bundles report their required factory module.
- Export guidance explains host dependencies and distinguishes strict conversion from a self-contained distribution.
- Direct Python factory commands run, evaluate seeded cases, and replay captured responses without packaging. Eval and replay accept explicit
--factorytargets. - Shared completion payloads report
grading_statusfrom actual eval-hook results. Definition checks reportbehavior_run: falseinstead of implying behavioral success.
Fixed¶
- Context-pressure estimates include structured tool arguments, result values, and errors instead of counting only text messages.
- Stock completion and thinking tools use equivalent importable implementations so strict cartridge exports remain runnable.
- Native tool schemas normalize optional markers and type aliases in cartridge dictionary descriptors without mutating authored declarations or dropping constraints.
- Sync and async loops share default-state budget resolution and terminal retry classification. Async loops synchronize mismatched default-state limits and stop on exhausted scripted responses without retry backoff.
- Pre-push checks clear Git-local repository variables so test fixtures cannot overwrite the caller's Git configuration or identity.
- The data-agent example preserves its CSV and scripted position when resuming incomplete checkpoints, and explicit terminal resumes no longer repeat tools or recreate data.
- Human saved-evidence rescoring shows each failing check's explanation when available, including with
--verbose. - Exhausted captured-response replay stops without retry backoff and records
replay_exhaustedas a failed run rather than a genericllm_error. - Saved-evidence rescoring reports missing components as integrity failures instead of silently skipping known cases.
- Assessment evidence links exclude colocated reports and unrelated files.
- Human saved-assessment inspection shows stored scores and labels beside result states.
- Experiment trace paths are report-relative. Human inspection of copied legacy reports prefers matching colocated capture and replay directories.
- Malformed saved starter evidence returns concise preservation and recovery guidance for rescoring and replay without hiding unrelated runtime errors.
- Starter grades explain incomplete runs and expected-versus-observed mismatches in console errors, saved evidence, inspection, and rescoring.
- Experiment reports record per-stage status and errors. Uncreated capture or replay directories use null paths instead of missing destinations.
- The one-file starter rejects unsupported flags such as
--jsoninstead of silently abbreviating them to--json-textand changing the model protocol. - Expected provider setup failures print concise configuration guidance without a Python traceback or empty experiment directory.
- Failed Quickstart captures, including the copyable recipe, report their stop reason and a shell-safe inspection command while preserving evidence and nonzero exits.
- Unsaved eval workspaces are cleaned after reporting and setup failures. Factory build failures return actionable CLI errors.
[0.4.1] - 2026-10-17¶
Added¶
- A complete keyless Quickstart module runs, grades, captures, rescores, replays, and stops without authored Python files.
async_replay_loop()replays captured responses with awaited async tools and hooks, shared replay validation, and early-close cleanup.- Documented application-state checkpoint and restore callbacks with nested-copy and legacy-snapshot tests.
looplet --versionprints the installed package version without a command or backend.
Changed¶
- Cartridge export stages a complete result before replacing the destination. Failed serialization and installation preserve existing output for retries.
- Unsupported callable exports name the actual type and explain top-level wrappers, arguments, and declared resources instead of mislabeling partials and instances.
- Installation guides pin the fixed
0.4.1package and matching provider extras instead of a moving development archive. - Cartridge round trips preserve supported per-tool argument validation without changing unvalidated tools. Nullable type tags and enum strings retain their YAML types.
- Strict script-tool export rejects omitted dependencies with actionable guidance. Loose export records warnings and writes editable placeholders instead of broken function copies.
- Capture documentation distinguishes provider-input scrubbing from record-only redaction before first use, without changing privacy defaults.
- Golden-test guidance uses a reviewed synthetic replay fixture with a narrow Git allowlist instead of committing raw captures.
- Inferred tool schemas reject variadic parameters during construction. Explicit schemas and list/dict wrappers remain supported.
- Graders and collectors reject unsupported async callables with host-code guidance and close unexpected coroutine returns.
- The Quickstart places its optional real-model handoff before grading, capture, and replay. The eval guide documents pre-awaited collection for synchronous grading.
- The Quickstart keeps provider extras on the same fixed package version.
- Tool schemas preserve every mixed or nullable
Literalchoice. Enum checks distinguish booleans from numeric values. toolandtools_fromaccept partials and callable instances, preserving effective signatures, resolved annotations, and explicit metadata overrides.- Trajectory JSON redacts string values before encoding, preserving valid JSON for escaped text and replacement strings.
- Captures redact backend errors, recorded tool schemas, and manifest metadata without changing live arguments or exceptions.
- Tool registration rejects positional-only signatures. Inferred container annotations require JSON-compatible inputs and explicit conversion for sets or tuples.
- Introductory scripted examples use strict response exhaustion while mock defaults remain unchanged.
- Public sync and async loop help documents parameters, budget overrides, and iterator cleanup.
- Introductory guides recommend a non-editable package install without a repository checkout.
- The CLI guide compares format support before packaging and uses strict completion in primary run examples.
- The quality-gate example checkpoints its successful-call counter across resumes, and its copied unit tests use real steps.
DefaultStaterestores charged-step budgets across checkpoint resumes, including free-call accounting. Explicit cap changes retain prior charged work.- Automatic resume rejects corrupt or unreadable checkpoints before model or tool calls. Checkpoints namespaced to other runs remain isolated.
- The hook-sharing guide uses persistent metadata across turns and tests the copied example.
- Cached
pre_dispatchresults pass custom permission gates in sync and async loops. Explicit denials still stop dispatch. - The quality-gate example counts successful non-completion calls and supports custom completion names. Tests execute the copied snippet.
- The migration guide configures JSON-text mode for its sample adapter and tests the copied snippets together.
- Prompts show the total step number separately from the charged-step budget.
- Package metadata, the public version, and published installation commands identify release
0.4.1. - Cartridge compatibility checks place development builds before the matching final release and preserve numeric version constraints.
doctorreports a SHA-256 fingerprint of library Python sources without requiring Git metadata.- Budget help describes charged steps and links to a cooperative deadline example for repeated free calls.
- Loops warn before model execution when no configured terminal tool is registered, without changing registry defaults.
- Sync and async loops reject failed terminal dispatches instead of reporting accepted completion.
- Default child registries install an isolated completion tool when cloning excludes the parent terminal. Explicit child registries remain caller-owned.
DefaultStatepreserves budget for successful free tools while charging failures, intercepted calls, and terminal attempts.- Cartridge and generic bundle completion output includes bounded last-error details without tool arguments or result payloads.
- The report tutorial asserts the full required assessment, including completion. Copied README and site examples use typed result access and pass Pyright.
- Python eval starters use unique test filenames so multiple generated suites collect with default pytest imports.
eval init --pythonadds a test to existing case and grader definitions without importing or replacing them.- Cartridge eval commands validate selected case IDs before constructing agent or judge backends.
- The report-agent tutorial uses installed demo files without requiring a repository checkout.
- Generated Python eval tests preserve case marks for pytest selection.
- Invalid case entries report their file and JSON entry number or JSONL line.
- Human trace inspection prints complete final results for completed runs. Plain-trace JSON and saved artifacts remain unchanged.
- Eval cases reject unknown fields and suggest close spellings.
- Saved eval inspection shows recorded grades and collected artifacts without rerunning them. Eval JSON output adds
evalsandartifacts. - Cartridge eval runs and case listings accept repeatable
--case-markfilters. Invalid selections fail before backend construction. - Native response captures apply configured redaction to nested strings without changing live tool arguments.
- Python replay uses recorded tool-step budgets and counts native batches in call-only captures.
- Python replay restores saved task data by default. Explicit tasks, including
{}, and config budgets remain authoritative. - Serial native batches check permissions, dispatch tools, record steps, and yield one call at a time.
- Serial runs honor cancellation, deadlines, and hook stops before the next call in a batch.
- Installation and capture guides distinguish
0.4.0limitations from the0.4.1safety fixes. - Python introductions use callable functions and show direct unit tests. The optional decorator's non-callable
ToolSpecbehavior is explicit. - Checkpoint errors link to a tested SQLite reconciliation recipe. Recovery guides distinguish retry, verified effects, and unresolved evidence.
- Tool timeout errors warn that effects may continue and require reconciliation. They no longer imply retry safety.
- Tool schema inference preserves
Annotated,Any, and simple unions. Unsupported annotations fail with explicit-schema guidance. minimal_presettypes its existing callable-tool support, including reusable sequences.- Quickstart capture, rescoring, replay, and early-stop scripts run separately. Extracted Python examples are type-checked, and the budget-hook example renders its count.
[0.4.0] - 2026-10-16¶
Added¶
looplet eval init --pythoncreates a runnable pytest starter and prints validation and test commands.- Raw shell replay accepts
--seed-fromfor an explicit starting input tree, preserving inputs and excluding Git and Looplet metadata. looplet replayuses captured responses with a trusted cartridge and fresh workspace, preserving original evidence and excluding grader-only expectations.tools_fromandminimal_presetacceptvalidate=Truefor top-level typed argument checks before tool effects.ProvenanceSinkchooses unique directories by default and requiresoverwrite=Trueto replace existing evidence.- Scaffold inspection lists unimplemented tools and separates draft readiness from structural runtime validation.
runand its explicit aliases accept--require-completedto fail incomplete runs without changing default exit codes.load_evalsloads ordinary grader and collector functions from one file or directory without requiring a cartridge.looplet eval initcreates a non-clobbering outcome contract;eval checkvalidates trusted definitions without running an agent.EvalContext.expectedexposes grader-only expectations while retaining the existing task view.EvalReportandeval_assessprovide one assessment policy across Python, hooks, saved records, and CLI evaluation.- Both eval CLI routes support JSON assessments and
--assessment-version;--summary-versionremains an alias. - Version 2 separates failed behavior from invalid evaluation and refuses empty or measurement-only release gates.
- Async child loops and bounded parallel children use the existing loop drivers.
- Child results expose completion, accepted output, identity, trace, and pending approval fields.
- Optional host-owned child recovery reuses saved output and refuses changed requests or uncertain effects.
- Offline parallel delegation example and real-tool dogfood cover Python and portable cartridges.
- Documentation tests execute the README, Quickstart, site example, and report tutorial without a model service.
- Optional
AgentPreset.run_result/run_result_asyncconsume the existing step iterator and projectRunResultwithout changing caller-owned resources. Explicitcomposable_loop/async_composable_loopremain the primary API. inspect,describe, andblueprintshare explicit--mode declaration|runtimevocabulary. Runtime reports validate one built preset and close resources; declaration inspection stays import-free and legacy defaults remain supported.- Default help promotes
new,run,inspect,eval, andreplay;--help-alllists every advanced command and compatibility alias without removing capabilities. -
Installed-wheel qualification executes paired raw-loop/preset sync/async clients, checking outcome parity, visible state, fresh executions, early exit, and exactly-once cleanup without claiming human usability or model quality.
-
Unified
runaccepts cartridge--quiet,--pretty, and--parent-tracethrough the same option definitions as legacy commands; bundles reject these unsupported options before importing authored code. - Bundle
blueprintexposes--declaration-onlyand explicit--instantiatemodes while preserving live inspection as the default. - Unified
looplet rundispatch for cartridges and skill bundles, an explicitrun-bundlealias, and safeinspectfor declarations, metadata, and traces. -
looplet new --offlinecreates a non-clobbering cartridge draft without a provider; bundle runs now accept stdin tasks,--project-root, and--json. -
Added optional host-layer records and lifecycle primitives:
AgentRuntime,RunHandle,RunStore,RunEvent,ArtifactRef,ContextPlan, andExecutionSession. - No-output cartridge eval records now expose explicit sandbox cleanup through
EvalRunRecord.cleanup()and the context-manager protocol. - Async loops now await hook results across lifecycle slots, share the same policy decisions and terminal output-schema checks as sync loops, and stop cleanly when a host deadline expires during an LLM call.
- Prompt renderers can consume the immutable
ContextProjectionsnapshot; blueprint inspection closes temporary presets after extracting structure. - Multi-tool model turns now preserve ordered turn metadata through steps,
trajectories, conversations, and streaming events; dispatch is bounded by
the remaining step budget and
max_parallel_callswhen concurrent dispatch is enabled. - Private-loop migration recipe.
examples/private_loop_migration/evolves one tiny agent from a hand-written loop tocomposable_loop(), captured provenance, and a required outcome grader, with no provider, network call, or cartridge in any of the four stages.docs/migrate.mdwalks the stages and states when the original raw loop remains the better choice.
Changed¶
- Human cartridge eval reports print failed grader explanations and details beside case and grader identities.
newaccepts the same model and endpoint overrides as run and eval without changing environment variables.- Case browsing accepts eval roots as well as direct case paths and previews
task.goalwhen no description is provided. - Published
0.4.0installs support the current tutorial and CLI without Git or source-only dependencies. - Shell replay restores saved tasks and defaults to recorded tool-step counts, including budget-stopped captures and multi-tool turns.
- Run and eval commands share model and endpoint options without changing environment variables.
- Saved-case replay preserves grader-only expectations, collects fresh outcome artifacts, and prints a separate grading command.
- Replay trajectories record their source run and saved case identity without modifying original evidence.
- Shell replay explains its cartridge-only format and links a tested Python recipe for skill bundles.
- New saved evidence directories own wildcard Git ignore policies; existing policies are preserved, and guides describe reviewed summary exports.
- Human eval output prints executable per-case inspection commands instead of leaving users to discover child trace paths.
doctorreports effective endpoint origins and configuration sources without URL credentials, paths, queries, or fragments.- Automatic provider selection warns about conflicting Anthropic and OpenAI settings;
doctor --strictrequires an explicit choice. - CLI cartridge evals save evidence in unique directories by default;
--no-saveexplicitly disables persistence without changing Python API defaults. - Human eval tables size and separate columns for long grader names and metric values.
- Cartridge evals refuse existing case output paths before execution instead of deleting previous evidence and workspaces.
- Cartridge trace collisions return actionable CLI errors before loading providers or authored code.
- The Quickstart grades structured completion facts instead of exact prose and exposes a configurable call budget for live runs.
- Saved-run evaluation accepts either a single trace or a parent directory without changing assessment rules.
- The Quickstart saves grader-owned expectations and outcomes, reloads them, and regrades without rerunning the agent.
- One pinned source install supports the current Python and CLI guides without requiring a contributor checkout.
- Tutorial installs use one immutable source revision until a published package contains the required APIs.
- The Quickstart shows native-provider selection and an executable Python-to-pytest eval handoff around the same agent.
- Python scripts and standalone eval definitions receive actionable CLI guidance instead of packaging errors.
- Run help separates cartridge-only and bundle-only options. Offline creation rejects the live-factory
--max-stepsoption before writing files. - Both eval CLI routes now default to assessment policy version 2, matching Python. Use
--assessment-version 1for legacy gates. Blockdenies execution at tool gates in Python and LEP. It also rejects completion incheck_done.- Published-install guidance names PyPI
0.3.0separately from the source checkout version. Native-tool failures require explicit protocol selection. assert_evals_passuses shared assessment failures and rejects empty suites by default;version=1retains the previous policy.- Eval hooks preserve grader inventories in existing run records; assessments and offline rescoring do not repeat collection.
- Built-in children inherit parent permission decisions, execution grants, and nested call limits.
- Async tool contexts expose the async backend without removing the synchronous compatibility bridge.
- Shared child call budgets preserve provenance without charging a delegated call twice.
- Delayed cartridge LEP hooks retain their preset-owned process environments.
- Standard child state follows the effective step limit in both drivers.
- Router-selected child backends spend the same shared model-call allowance.
- Built-in children prepare their model gateways, which await async backend deliveries.
- OpenAI-compatible multipart responses preserve visible text and native tool calls without exposing metadata as text.
- Resumed child prompts include committed tool receipts and preserve caller checkpoint-state callbacks.
- Git report examples retain their bounded analysis history across the full workflow.
- The README keeps explicit loop wiring. The Quickstart adds results, grading, capture, and replay around one agent with fresh per-task state.
- Source distributions include the guides required by documentation tests.
- Sync/async teardown finalizes initialized hooks exactly once on completion, failure, and early stream close, preserving primary errors and attempting all cleanup hooks. Started early-closed streams are stopped, not failed.
- Host runtime and RPC use preset execution; repeated loaded RPC runs rebuild
fresh state/resources. Python RPC hosts can supply
preset_factoryfor repeated runs instead of reusing live hooks, clients, or state. Wire v1 remains unchanged. -
The shipped hello-world example exposes explicit loop wiring and prints steps as they occur, with truthful incomplete exit status and unchanged provider policy.
-
Required tool arguments with explicit nullable schema types accept
Nonewithout treating it as an omitted value; legacy empty-input checks remain. - Stagnation hook exports preserve built-in fingerprint policies. Unsupported callable policies fail strict export and produce explicit loose-export warnings.
- LEP pipe exchanges and MGP response waits honor host run deadlines; timed-out policy processes are reaped and gateway connections are closed.
- RPC completion returns the actual accepted terminal output, including tools
configured through
done_tools, without changing the wire schema. - Bundle
inspecthonors the exact requested path rather than a nested child.newreports incomplete factory runs as failures and retains partial drafts. - Bundle and cartridge examples use context-managed preset ownership; live validation examples reuse their validated preset instead of building twice.
- Cartridge eval reports and bundle blueprints keep ordinary authored Python diagnostics on stderr, including import, build, execution, and cleanup output.
- Human
showandinspectsummaries tolerate missing or null timings without changing the original trace or JSON inspection output. - CLI creation, execution, doctor, evals, and general-purpose examples now share provider resolution and model defaults with Python. Explicit compatible endpoint/model overrides and legacy command names remain available.
-
Bundle and cartridge CLI completion use the same accepted
RunResultview; generic runs reject missing workspaces, report actual stops, and fail on fatal model errors. Human custom bundle runners remain supported; JSON usesbuild. -
Backend wrappers now share supported-option and capability forwarding, so recording, resilience, routing, and legacy cost tracking preserve provider usage/state and cancellation/cache options. Explicit resilience owns retries without multiplication by the core call helper or nested wrappers; provider SDK retries remain separately configured. Routing and fallback wrappers also forward mutable provider handles to the backend that owns their exposed state.
ValidatingToolRegistrynow uses the base registry's single preparation and execution path for sync/async calls and batches. Optional schemas fail closed with structured validation errors, and mixed batches preserve per-call contexts, resources, order, and call IDs without a second dispatcher.- Skill-bundle execution now delegates to
AgentPreset.run()for validation, single-use claims, envelope and gateway binding, and trace returns. Unstarted and failed runs clean up helper-owned presets; caller-supplied presets remain caller-owned, and bundle-local import context is preserved. - Sync and async turns now share post-step stop and observer rules for normal, rejected, and synthetic/error steps. Pending stops prevent extra parse-repair calls; async rejected steps notify observers and emit rejected classifications. Accepted completion and fatal model errors retain their distinct precedence.
- Persisted cartridge self-tests now capture agent model calls through the
existing provenance/save path, enabling fresh-effects captured-response
replay beside offline grades.
capture=False/--no-captureretain grading-only records, redaction remains independent of capture, and interrupted runs save available incomplete evidence before re-raising. Captured case directories use owner-only permissions on POSIX; hosts still control evidence retention and access when exporting it to other storage. - Cartridge self-tests now own evaluation and replace embedded
EvalHookobservers; reference coder defaults no longer grade normal runs implicitly. Explicit host hooks and reusable eval helpers remain supported. - Eval CLI thresholds now default to
0.5, matching pytest, and apply to each scored result rather than an average.--threshold 0retains the old numeric default; required graders still enforce their0.5floor. - Pytest eval assertions use the shared result-state classifier: passive
metrics and optional skipped judges are neutral, while missing or filtered
required graders fail closed. The eval guide now teaches one entry path per
audience and one durable
save_eval_run()/load_eval_run()round trip. -
Collector failures now survive offline rescoring and batch CLI aggregation; older eval records recover those failures from their saved results. Missing grader identities and malformed recorded failure evidence cannot pass a gate.
-
Expanded the hook guide to document the complete hook extension surface, including async-capable methods, permission/compaction hooks, prompt builders, lifecycle events, and
pre_loop(tools=...). - Checkpoint auto-resume is now terminal-authoritative and scoped by
RunEnvelope.run_id;FileRunStorecoordinates read/modify/write transactions across processes. AgentPreset.run()andrun_async()now enforce a single-use lifecycle, preventing mutable state and live resource wiring from leaking across sequential or concurrent runs.LoopConfig.max_tokensis now unset by default, preserving provider decisions, and oversized prompts are blocked before provider calls when reactive recovery is disabled.- Native tool calls now require structured
tool_useblocks; unavailable native APIs and text-only native responses fail visibly instead of switching to JSON-text parsing. Setuse_native_tools=Falseto opt in to text mode. Brokencheck_donequality gates fail closed in sync and async loops.
Fixed¶
- Cartridge and factory CLI runs now use the preset lifecycle for validation, model-gateway binding, and accepted terminal output. Factory and generated validation presets close their owned resources on completion or failure.
looplet new --max-stepsnow overrides the authoritative factory config and limits actual execution. Fatal cartridge/provider failures return non-zero; intentional budget and hook stops retain their existing exit-zero contract.
Deprecated¶
EvalHook.save()retains its standalone report but warns; usesave_eval_run()for durable evidence.- Dictionary gate-score inference from
f1,accuracy, oroverallwarns while preserving legacy behavior. New graders should return a boolean or explicitEvalResult(score=..., metrics=...).
Earlier 0.4.0 Changes¶
Added¶
looplet-proofruns the complete network-free failure-to-regression workflow from an installed wheel; its cartridge, collector, grader, and before/after tool assets now ship in both distribution formats.- Saved trace and eval directories now carry a versioned
artifact.jsondescriptor with producer and required-component metadata. Readers retain explicit compatibility with the supported descriptor-free legacy shape. looplet conformnow ships its schema-v2 fixtures in the wheel and checks both successful loads and required rejection cases without a source checkout.
Changed¶
- The public README now follows one Python-first adoption path, uses the installed proof as its primary action, and moves protocol and skill-bundle details into advanced guides. Removing the Mermaid block also prevents PyPI from displaying raw diagram source.
- Cartridge Spec v2 and its machine-readable manifest schema now match the
v2-only loader. Stale v1,
setup.py,${py:...}, plural-sentinel, and field tier guidance has been removed from the current contract. - Schema-v2 cartridges now reject runtime keys in
config.yamleven when the same key appears inruntime.yaml; there is one unambiguous file boundary. - Host-only approval, cancellation, and message-render capabilities are now rejected from cartridge files and omitted with a warning during loose serialization instead of being converted into resource stubs.
preset_to_cartridge()no longer writes non-empty host-onlydone_toolsconfiguration that the schema-v2 loader would reject.- Programmatic tool tags now produce a clear serialization warning, or a
strict-mode error, instead of writing a
tags:field rejected by schema v2. - State-service declarations and an explicit
llm_gateway: falsenow survive cartridge round trips without generating resource stubs for service clients. - Obsolete generated demo recordings that showed retired
.workspacepaths andrun-workspacecommands were removed; maintained scripts can regenerate current recordings when needed.
CLI¶
- Re-running
looplet migrateon an already-current cartridge is now a true no-op and reportsalready currentinstead of claiming files were rewritten. looplet diffnow labels manifest, runtime, and eval changes explicitly; binary changes are represented by content digest instead of being hidden;looplet hash --helpnow describes its actual all-versioned-files scope.looplet run-cartridgenow captures a provenance trace under the target project's.looplet/traces/directory by default. Use--trace-dirto choose a location or--no-traceto opt out.looplet run-cartridge <cartridge> -reads the task from standard input for shell pipelines and rejects empty input before starting a run.looplet show <trace-dir> --jsonemits parsed trajectory and manifest records for shell pipelines and external tooling.looplet run-cartridge --parent-trace <dir>records the parent run ID in trajectory metadata without adding orchestration semantics.looplet run-cartridge --jsonemits one machine-readable completion object, including explicitcompletedandtermination_reasonfields, while suppressing human progress output.looplet eval run --jsonemits the existing threshold and integrity verdict with per-case completion and grader results.
Fixed¶
- Tool dispatch now preserves string arguments byte-for-byte, derives optional simple-schema parameters from callable defaults, and retains MCP JSON Schema requiredness. MCP stdio reads now skip notifications and queue responses that arrive out of order instead of treating them as missing responses.
AgentRuntimenow prevents unsafe reuse of mutable preset state, serializes file-backed run-store updates, and rejects malformed compatibility constraints.- State Service request reads now honor run deadlines, malformed requests receive structured errors, non-JSON results are reported without dropping the connection, and service instances can be served again after shutdown.
- SSP, MGP, and MCP clients now reject mismatched response IDs; State Service environment ownership is stacked safely across same-name handles, and failed MGP connection retries close their sockets.
- Model gateway run envelopes now reach the host server for deadline checks, and duplicate live gateway sockets are rejected instead of stolen.
- RPC runs now bind the selected backend to optional model gateways; RPC/MGP shutdown closes active client sockets, and malformed MGP frames receive protocol errors instead of hanging callers.
- RPC servers now close loaded presets on workspace replacement,
quit, and EOF, and reject malformedmax_stepsvalues instead of coercing them. - Checkpoint and conversation serialization now make defensive JSON-safe copies and reject invalid scalar/nested shapes early.
- State-service startup no longer unlinks an active socket owned by another live service.
- Tool registries can now opt into checkpointed result recall via
snapshot_results()andrestore_results(); sync/async resumes restore those JSON-safe snapshots, and terminal result keys are preserved. - Validating tool registries now forward result-store snapshot/restore hooks, so checkpoint-enabled loops remain compatible with schema validation.
- Checkpoint loading now validates JSON object shapes consistently, and
CheckpointHookrejects non-positive save intervals at construction. CheckpointHooknow snapshots after the current step/history update, and batched terminal sentinels report their actual step number tocheck_done.- Checkpoint files are now written atomically, completed checkpoints are excluded from automatic resume, and queued terminal tools stop honoring cancellation/deadline signals after dispatch.
- Async history now preserves tool-result recall keys, and async streaming
- Async history now preserves tool-result recall keys, and async streaming
avoids duplicating lifecycle start/end events when
StreamingHookis used. RunResult.to_dict()now recursively normalizes arbitrary output and metadata into JSON-safe values as documented.- Async
pre_loophooks now receive the tool registry when their signature declarestools=, matching synchronous loop behavior. - Cartridge loads now clear ordinary helper modules imported by the previous cartridge, preventing same-named helpers from leaking across reloads.
async_llm_callis now available from the package-levelloopletAPI.- Tool-result pruning now clears the structured payload as well as rendered
content, and
keep_recent=0clears every eligible result. - Fallback routers no longer advertise native-tool support unless both backends can provide it.
- Resilient backends no longer retry abandoned local timeout workers by default; timeout retries remain available through an explicit predicate.
AgentsMdMemorySourcerejects a stop path outside the start path, and context-based eval saves remove stale per-step files on reuse.- Async tool timeouts no longer wait for cancellation-resistant coroutine cleanup when dispatch is bridged from a running event loop.
- Non-positive recent-context windows now render no steps instead of exposing the entire step history when the value is zero.
- Unknown
FieldSpectype tags are rejected instead of silently disabling type validation. - RPC, LEP, and MCP transports now reject non-object JSON frames and malformed response envelopes as controlled protocol failures instead of leaking attribute errors or bypassing failure policy.
- LEP and state-service shutdown now remain bounded, reap forcibly terminated child processes, and clear process handles during cleanup.
- Async
LoopEndEventemission now follows awaitedon_loop_endcleanup and includes integer extra-call counts, matching the sync loop contract. looplet shownow reports unreadable manifest paths as controlled CLI errors instead of leaking an uncaught traceback.- Manifestless replay now rejects non-contiguous response files instead of silently truncating the recorded call stream, and replay preserves recorded LLM failures so native-tool fallback behavior remains observable.
- Invalid
save_eval_run()calls no longer leave a pending artifact marker behind or invalidate an existing saved artifact before trajectory validation. - Synchronous checkpoint resume now restores serialized conversation history before binding the active history recorder, matching async resume behavior.
- Sync and async loops now tolerate protocol-compatible custom states that reject dynamic private attributes, and host results recover lifecycle status and phase from the state's metadata fallback.
- Cartridge manifests now reject malformed JSON, non-object values, invalid
names, missing required fields, and invalid field types before any body is
imported.
Cartridgealso preserves the optional opaque artifact version across metadata reads and writes. - Cartridge load failures now close model gateways, state services, and MCP
adapters already started during that load.
looplet describeandlooplet conformalso close successfully loaded presets after inspection, and explicit preset close now terminates LEP hook processes. - State-service startup failures now reap their child and temporary socket directory, and failed connection retries close their sockets. State-service and model-gateway close now restore any prior exported socket value instead of clobbering caller environment state.
- MCP server
enventries now extend the host environment, preserving PATH and Looplet model/state socket variables required by portable tool servers. - Model-gateway startup now fails and cleans its temporary directory when the server thread does not create a usable socket.
- Default and explicitly selected long-term memory files now load once instead of also being included as generic ordered Markdown memory. Explicit paths must exist inside the cartridge root.
- Legacy
check_donehooks no longer intermittently receive an unsupportedtool_callkeyword after short-lived hook classes are collected. - Trace inspection and replay now reject malformed structures, invalid replay indices or methods, missing responses, and corrupt native content blocks before rendering or execution.
- Reusing a trace directory removes stale Looplet-owned call and step files while preserving unrelated files.
looplet run-cartridgenow rejects missing project roots before execution, preserves results from custom terminal tools, reports incomplete human runs honestly, closes cartridge-owned subprocesses before returning, and labels human output with the canonical command rather than the skill-bundlerun.
[0.3.0] - 2026-07-16¶
Added¶
- Cartridge-native behavioral contracts. Cartridges can ship cases,
independent outcome collectors, and required graders under
evals/.run_cartridge_evals()executes the complete contract in fresh per-case workspaces and persists trajectories, artifacts, grader-only expectations, and results for offline review. - Out-of-process control surfaces. The RPC protocol now covers lifecycle events, cancellation, completion payloads, checkpoint/resume, backend selection, capability negotiation, and protocol versioning. LEP hooks, SSP state service, and MGP model gateway provide explicit cross-runtime boundaries without changing the core loop.
- Cartridge portability analysis and portable examples.
looplet portabilityreports blockers and supported tiers. Portable twins for the shipped cartridges demonstrate MCP/LEP boundaries and are protected by cross-runtime conformance tests. - Runtime interaction primitives.
SkillRuntime.ask_user()andbatch_ask_user()support host-supplied structured questions while retaining a sequential fallback for adapters without native form rendering. - Declarative hook contracts.
ViewSpec, thread-rewrite support, and related hook surfaces make context views and rewrites inspectable. - Provider usage telemetry. Model token usage is recorded on state and lifecycle events so hooks and host applications can account for real calls.
- Environment-selected backend construction.
make_backend()and the RPC backend command support Anthropic, OpenAI, OpenAI-compatible proxy, and explicit deterministic mock configuration through documented environment variables. - Public regression proof. A network-free example captures fixed model responses, changes one tool implementation, independently collects the fresh artifact, and moves a required grader from red to green.
- Task-oriented documentation. New guides cover installation, migration from a private tool loop, troubleshooting, experiment selection, runtime operations, the CLI and Python API surfaces, and saved artifact layouts.
- Bundled portable coder reference. Wheels and sdists include
coder_portable.cartridge;bundled_cartridge_path()resolves shipped cartridges in source and installed environments. The portable coder moves 16 tools, five hooks, shared cache state, and model access behind MCP, LEP, SSP, and MGP with zero in-process analyzer blockers.
Changed¶
- Looplet is now positioned as test-driven harness engineering. The README, documentation site, package metadata, roadmap, contribution guidance, and issue forms lead with the post-prototype workflow: own the loop, capture failures, grade independently observed outcomes, and gate harness changes in pytest or CI.
- Agent-factory output is explicitly a draft.
looplet newdefaults to./agent.cartridge, recommendsrun-cartridge, and requires review plus behavioral contracts before release.run-workspaceremains a compatibility alias. - Replay terminology is precise. Documentation says captured-response replay and states that tools and side effects execute again. Prompt and model changes require fresh sampled runs.
- The shipped coder harness is more capable and cache-friendly. It adds a
reviewable
web_searchtool, sharper safety and answer-quality guidance, and keeps stable prompt prefixes to improve provider cache reuse. - Same-model harness evidence is documented with caveats. Reproducible benchmark reports compare harness scaffolding rather than claiming universal model or framework superiority.
- Documentation front door and navigation. The site now routes readers by task, uses the custom domain as its canonical URL, preserves the network-free regression proof as the primary demonstration, and adopts a restrained engineering-focused visual system.
- Portability claims match the evidence. The portable coder is a flagship reference architecture while the Python-host coder remains the factory default. Documentation distinguishes loader portability from subprocess language, Unix-socket support, and full non-Python runtime execution.
Fixed¶
- Eval integrity boundaries fail closed. Zero scores remain scores; invalid returns, evaluator errors, collector failures, explicit failing labels, discovered required graders that are filtered, skipped, errored, invalid, or below the pass boundary, empty grader suites, malformed trajectories/metrics, duplicate eval modules, invalid thresholds, unsafe case IDs, seed traversal, symlink escapes, and stale workspaces no longer produce false greens. Detecting a grader deleted before discovery still requires a trusted expected-grader manifest.
- Online and offline grading use the same evidence. Expected data stays grader-only, terminal payloads survive persistence, and session logs, metadata, artifacts, and stop reasons round-trip.
- Provider cache policy now reaches real requests. OpenAI and Anthropic adapters translate configured cache breakpoints into provider request fields while preserving byte-equivalent behavior when caching is disabled.
- RPC and MCP edges are hardened. MCP JSON-string arguments and non-dict tool results are normalized safely; MCP response reads and startup cleanup honor configured timeouts; RPC checkpoint capabilities, completion reasons, and backend construction now reflect actual runtime behavior.
- Installed
looplet newincludes its factory. Wheels and sdists now carryagent_factory.cartridgeplus itscoder.cartridgeparent, with a clean-wheel CI smoke test covering factory loading and CLI help.
Developer experience¶
- Repository and launch documentation now use ASCII punctuation consistently; CI enforces the no-em-dash rule across tracked UTF-8 files.
- Release metadata, mirrored documentation, lockfile version, changelog date, strict docs build, and publication tag are covered by automated gates.
- Tag publication creates an idempotent GitHub Release after PyPI succeeds, attaches the built wheel and sdist, and uses the matching changelog section as release notes.
- The public issue backlog is split into independent evidence, compatibility, and provider work with explicit dependencies and non-goals.
[0.2.0] - 2026-05-14¶
Added¶
mcp_servers:slot inconfig.yaml- declarative MCP (Model Context Protocol) transport for cross-language tools. Loader spawns one stdio subprocess per entry, runs MCP discovery, and registers every discovered tool into the agent’s registry alongside in-process Python tools. Optional per-servertools:allow-list.AgentPresetis now a context manager (__exit__terminates spawned MCP subprocesses cleanly).examples/mcp_demo.cartridge/- fully self-contained demo. The MCP server is a ~60-line Python stdio process bundled at_server/calc.py(no Node, no npm, no external deps).cartridge_rootauto-injected into the runtime dict, so YAML fields can reference${runtime.cartridge_root}(most importantly inmcp_servers.<name>.command).language:field in cartridge metadata + per-tooldescription.mdfiles (closes paper gaps).
Changed¶
- Cartridge spec v2 is now the only supported shape. Loader
hard-fails on
schema_version != 2. RUNTIME-tier keys must live inruntime.yaml, notconfig.yaml. - Round-2 cleanup: extracted three pure helpers from
_load.py, trimmed re-exports, tightened render→runtime boundary. - Four v2 cuts:
tags, single-file constraints,${py:}references,done_toolsplural.
Fixed¶
MCPToolAdapternow uses newline-delimited JSON framing per the MCP stdio spec. Previous LSP-styleContent-Length:framing silently failed against every real MCP server. Init-failure errors now include exit code and stderr tail.
Removed¶
- All v1 back-compat. Public surface trimmed:
Workspace,WorkspaceLayout,WorkspaceSerializationError,workspace_to_preset,preset_to_workspace,scaffold_workspace,ContextManagerHook. UseCartridge*equivalents. - Magic
prompts/briefing.md/prompts/recovery.mdauto-load andsetup.pyescape hatch. - Alias modules:
scaffold.py,prompt_files.py,hot_reload.py,spec_slots.py,workspace.py.
Added¶
- Unified workspace reference grammar. Three forms, one resolver,
applied uniformly to every string value the cartridge loader
processes:
max_steps: ${runtime.max_steps:-15} # per-invocation data compact_service: ${ref:compact_service} # resource registry state: ${py:my.app.state:MyState} # imported Python symbol${runtime.x}supports nested lookup (${runtime.a.b}) and defaults (${runtime.x:-15}). The legacy"@name"form continues to work as an alias for${ref:name}so existing cartridges load unchanged. Seedocs/workspace.md#reference-grammarfor the full spec. AgentPreset.resourcesfield. the cartridge loader now populates this dict with every resource it built. Callers that need post-load access to live objects (benchmarks, evidence-bundle writers, SDK shims) read frompreset.resourcesby name - no more module-hunting to reach a resource the cartridge constructed. Empty for presets built directly in code.- Declarative
state:directive inconfig.yaml. Cartridges can now describe their state class via the same grammar instead of relying on thestate_factoryconstructor arg ofcartridge_to_preset:Priority:state: ${py:my.app.state:MyAgentState} # → MyAgentState(max_steps=...) state: ${ref:my_state} # → resource as-isstate:directive >state_factoryarg >DefaultState. Closes the last gap where a non-trivial workspace had to writesetup.pyjust to attach a custom state class.
Fixed¶
- YAML parser now skips full-line comments.
#lines used to raiseCartridgeSerializationError. Same fix applied to the runtime-substitution pre-pass so${runtime.x}inside a YAML comment doesn't fire the regex.
Removed (BREAKING)¶
looplet.flagsmodule deleted. All feature flags migrated toLoopConfigfields in 0.1.6 and the module had been deprecated since. TheFLAGSsingleton,_Flagsclass, andLOOPLET_*environment variables are gone. Use the equivalentLoopConfigfields directly:LoopConfig(concurrent_dispatch=True),LoopConfig(reactive_recovery=True), etc.looplet.scaffolding.StallDetectorclass deleted. Superseded byStagnationHookinlooplet.stagnation. The back-compat bridge methods onStepProgressTracker(consecutive_empty,is_diminishing,record_step,guidance_text) are also removed - use the nativeconsecutive_unproductive/is_stagnatingproperties.
Changed¶
- Documentation cleanup. Stale "v1 / v2 / legacy / cartridge" framing trimmed from public docstrings and comments now that the migration to the cartridge format is complete. No behavioural change. ROADMAP entries that have shipped are dropped.
- Coding and research presets now use
DefaultCompactService. The preset path gets the same production compaction policy users can import directly: prune old tool payloads, summarize older context, keep recent steps verbatim, and report stage-level outcomes. - Docs and shipped examples now teach
DefaultCompactServicefirst. README, tutorial, AGENTS guide, packaged examples, and bundled workspacecompact_service.pyresources now point to the coherent default service, leavingcompact_chain(...)as the custom-policy escape hatch.
Added¶
DefaultCompactServiceanddefault_compact_service(...). A clear production default for context compaction that composesPruneToolResults,SummarizeCompact, and deterministic truncate fallback into one inspectable service.CompactOutcomenow reports session-log entry counts, compacted step ranges, summaries, and a JSON-ableto_dict()used inPOST_COMPACTevent payloads.extends:workspace composition. A cartridge'sconfig.yamlmay now declareextends: <path>. At load time the parent workspace is recursively materialized and overlaid with the child via a tempdir; child files override parent files at matching paths. Multi-level inheritance works (grandparent → parent → child), cycles raiseCartridgeSerializationError, missing parent paths raise a clear error. (#44)examples/agent_factory.cartridge. First product built onextends:. Inherits allcoder.cartridgetools and adds avalidate_workspacetool that callscartridge_to_preset()and returns structured success/error. ~4500-char system prompt teaching the cartridge v2 grammar, robustness rules, andextends:usage. (#44, #45)looplet.scaffold.scaffold_cartridge(). Plain Python helper that creates a working but stubbed cartridge skeleton in one call:cartridge.json+config.yaml+prompts/system.md+tools/<name>/{tool.yaml, execute.py}stubs (raiseNotImplementedError) + the standarddonetool. Idempotent - re-running preserves existing files via_write_if_absent. (#46)builtin_tools:directive inconfig.yaml. Cartridges can opt into looplet-shipped tools without writing atools/<name>/dir: Resolved at load time vialooplet.builtin_tools.AVAILABLE. (#46)subagentbuilt-in tool. Invokes another workspace as a sub-loop, sharing the parent's LLM and forwarding the parent'sworkspace_config.pathas the sub-loop'sruntime["workspace"]. Returns the sub-loop's finaldonesummary. Recursion-guarded viacontextvars.ContextVar(defaultmax_depth=5). Sequential only - parallel fan-out deferred. (#46)scaffold_cartridgebuilt-in tool. Agent-callable wrapper around the scaffold helper. The agent factory uses it as the very first tool call. (#46)
Fixed¶
scaffold_cartridgewrote invalid JSON (cartridge.jsonemitted single-quoted Python repr instead of double-quoted JSON).Workspace.from_directory()and any externaljson.loads()failed. Now usesjson.dumps(name)to emit RFC-compliant JSON.agent_factory_extract_jsonexample inprompts/system.mdwas double-escaped inside an r-string (\\sinstead of\s,\\[instead of\[), so the regex matched nothing. Agents that copied the helper verbatim got a broken extractor. Fixed.subagentdid not actually inherit parent runtime. Docstring promisedworkspace_configpropagation; code readctx.metadata["runtime"]which the loop never sets. Now reads the parent'sworkspace_configresource and forwardsruntime["workspace"]to the sub-loop's resource builders.subagentrecursion depth via process-global env var (LOOPLET_SUBAGENT_DEPTH) - two parallel parent loops in the same process raced. Replaced with aContextVar(threadsafe and per-async-task). The sub-loop receives a freshly-constructedruntime(it does NOT share the parent'sresources/file_cacheinstances - only the cartridge path is forwarded).validate_workspacewas silent on TODO-laden scaffolds. Now scans the system prompt for<TODO:markers and tool execute.py files forNotImplementedError("scaffold:and surfaces both as warnings - agents can no longerdoneon an unfilled skeleton. (#48)subagentcwd-fallback was silent. When neither the parent'sworkspace_configresource norctx.metadata['runtime']is present, the sub-loop'sruntime['workspace']falls back toPath.cwd()AND the response now includes a structuredwarningfield with explicit recovery hints. (#48)- Tool name vs directory name mismatches (e.g.
tools/foo/whosetool.yamldeclaresname: WRONG_NAME) used to silently register the wrong name and leave the agent unable to usefoo. The loader now warns in loose mode and raisesCartridgeSerializationErrorin strict mode. (#48) - Documentation cleanups (#48).
subagentmodule docstring no longer claims to "share the parent's runtime" (it constructs a fresh one).builtin_tools/__init__.pynow lists both shipped built-ins (subagent,scaffold_cartridge). -
Internal cleanup (#48). Removed redundant
extends:line check; rewrote tempdir registry as module-level state with singleatexit.register; inlined a one-line_is_absolutehelper; removed duplicate import; switchedsubagent.max_stepssentinel from0toNone. -
examples/coder.cartridgeper-tool guidance + safety. Three information-additive improvements modelled on patterns observed in production coding agents: - Read-required-first on
edit_file.FileCachenow tracks every path passed toread_file;edit_filerefuses with a model-actionable error ({error, missing: "prior_read", recovery: "read_file(...)"}) when called on a file that hasn't been read in the current session. Editing without reading is the #1 cause ofold_stringmismatch failures. bashsafety classifiers. Newclassify_bash_commandandclassify_sed_commandhelpers incoder_lib_tools.pyflag destructive command/flag combinations (rm -rf,git push --force,git reset --hard,shutdown,mkfs, …) andsed -iin-place edits (which bypass the file_cache and cause stale reads). The bash tool refuses both with a structured error pointing at a safer alternative (edit_filefor in-place edits). The classifiers are exported so other cartridges can reuse them.- Rich per-tool descriptions. Every
tool.yamlinexamples/coder.cartridgerewritten as a multi-paragraph description (Usage / Refusals / Examples / Recovery sections) using YAML block scalars (|-). The looplet workspace YAML loader gained|/|-/>block-scalar support so these descriptions round-trip correctly. ToolError.recovery_hint- structured suggestion (dict or str) for how the caller could recover. The dispatcher now populates it on the four self-correctable errors: unknown-tool ("did you mean?"), unexpected-argument ({unexpected, did_you_mean, expected}), missing-argument ({missing, provided, expected}), and empty required-string-argument ({empty_param, expected}). Information- additive: smarter models exploit the structured hint to self-correct without re-discovering the catalog from prose; existing models still see the same human-readable error message.looplet.LLMResponsesExhausted+MockLLMBackend(cycle=False)- opt-in test ergonomics. The default still cycles for backward
compatibility; passing
cycle=Falsemakes the mock raise instead of silently re-usingresponses[0]past the last scripted answer (which previously made over-running loops look "stuck on step 1"). Same flag onAsyncMockLLMBackend. run_sub_loop(parent_hooks=...)- opt-in event forwarding from a sub-loop to the parent's observability stack. When supplied, the parent's hooks (e.g.MetricsHook,StreamingHook,TrajectoryRecorder) receive every lifecycle event the sub-loop emits via theiron_eventmethod, tagged withsubagent_idin the payload'sextradict so consumers can route / nest. Defaults toNone- no forwarding, sub-loop fully isolated.
Changed¶
tool.yaml requires:validated at workspace-load time. A typo inrequires: [my_resoruce](missing or mistyped resource name) used to silently setctx.resources["my_resoruce"] = Noneat dispatch and crash deep inside the tool body withAttributeError. The loader now warns in loose mode (default) and raisesCartridgeSerializationErrorin strict mode, naming the unresolvable resource and listing the available ones - surfaces the bug at its source.
Changed¶
- Naming consolidation. Dropped the legacy "cartridge" /
"Composable Harness Workspace (CHW)" / "workspace v2" terminology
in favour of the two canonical names already used in code:
Workspace(the round-trippable directory format fromlooplet.cartridge) andSkillBundle(the runnable folder format fromlooplet.bundles). All docstrings, doc pages, README, and comments now use these names. TheClaudeSkillCompatibility.levelstring"looplet-cartridge"is renamed to"looplet-bundle"- the only minor breaking change in this consolidation. Renamedtests/test_cartridge_round_trip_smoke.pytotests/test_skill_bundle_round_trip_smoke.py. No behavioural change; the_chw_*synthetic module-name prefixes (used internally by the cartridge loader) keep their names.
Removed (BREAKING)¶
- All
setup.pyfiles removed from shipped example cartridges. Every workspace underexamples/*.workspace/is now fully declarative; the imperativesetup.pymechanism remains in the loader as the documented escape hatch for callers with truly imperative load-time wiring needs but no shipped example needs one. Migrations: coder.cartridge: tools moved fromWORKSPACE_CONFIG/FILE_CACHEmodule-globals torequires: [...]intool.yamlctx.resources[...]inexecute.py.compact_servicemoved toresources/compact_service.py.
threat_intel.cartridge,dep_doctor.cartridge,git_detective.cartridge:compact_servicemoved toresources/compact_service.py.git_detectivetools moved fromREPO_CONFIGmodule-globals torequires: [repo_config]ctx.resources["repo_config"].
hello.cartridge:greettool moved from_GREETING_LOGmodule-global torequires: [greeting_log]+ctx.resources["greeting_log"].
v1 example modules deleted¶
- The legacy
examples/coder/,examples/dep_doctor/,examples/git_detective/, andexamples/threat_intel/agent-CLI directories have been removed. Their tool functions, hook classes, and helpers now live inside the matchingexamples/<name>.workspace/Composable Harness Cartridges as co-located helper modules (<wsname>_lib.pyfor the simple examples,coder_lib_{tools,hooks,wiring}.pyfor the coder one). The v2 cartridges are now the only published agent surface. - The
examples/coder/skill/SkillBundle was relocated totests/fixtures/coder_skill_bundle/(with vendored sibling modules so it loads without anyexamples.coder.*import). Alllooplet.bundles/looplet.blueprintstest coverage continues to exercise it via the new fixture path. - Removed tests that targeted the deleted v1 modules:
tests/test_coder_example_smoke.py,tests/test_coder_reliability_smoke.py,tests/test_dep_doctor_example_smoke.py,tests/test_git_detective_example_smoke.py,tests/test_threat_intel_example_smoke.py, andtest_distributions_include_coder_cartridge_and_dependencyfromtests/test_cartridge_round_trip_smoke.py.
Changed¶
- Workspace loader pushes the cartridge root onto
sys.pathfor the duration ofcartridge_to_preset, so a cartridge's tools / hooks / resources / setup.py canfrom <wsname>_lib import Xwithout a dedicated import shim. The path is removed on exit. Cartridges should pick a unique top-level lib filename (<wsname>_lib.py, not barelib.py) to avoid sys.modules cache collisions when two cartridges are loaded back-to-back in the same process.
Added¶
- Cartridge discovery without import.
discover_skill_bundles(roots)walks one or more roots and returnsBundleCardrecords (name, description, entrypoint, tags, metadata, ok/errors) without importing the entrypoint. Powers the newpython -m looplet list-bundles <roots…>CLI for product UIs and agent menus, with--jsonand--include-invalidmodes. - Eval cases as data.
EvalCase,load_cases,save_case,pytest_param_cases, and theparametrize_cases(path)decorator let users write hand-edited JSON/JSONL cases that round-trip into pytest with theirmarkscarried through.assert_evals_pass(ctx, evals)collapses the run/filter/pretty-print failure idiom into one call (with cached discovery for parametrized tests). looplet eval cases ls|showCLI subcommands for browsing case corpora directly from the terminal.- Outcome-grounded evals.
EvalContext.artifactsandEvalHook(collectors=…)let you grade what changed in the world (test results, repo diff) instead of grepping the trajectory. Trajectory directories may now ship anartifacts.jsonnext totrajectory.json;EvalContext.from_trajectory_dirloads it automatically. Collectors that raise or return non-dicts are skipped silently - observers must never break a run. AgentPreset.run(llm, …)convenience method drivescomposable_loopwith the preset's wiring in one call.composable_loop(…, max_steps=N, system_prompt=…)keyword shorthands for inline agents that don't construct aLoopConfig.OpenAIBackend.from_env()/AnthropicBackend.from_env()/AsyncOpenAIBackend.from_env()classmethods that readOPENAI_*/ANTHROPIC_*env vars in one line.OpenAIBackend(api_key=…)no longer requiresbase_url- the cloud path now works with just an API key (or env vars).BaseToolRegistry.tooldecorator registers aToolSpecin one step, mirroring the module-level@tooldecorator.save_cases(cases, directory)plural form symmetric with :func:load_cases. Refuses to write when two cases share anid(which would silently overwrite each other on disk).metadatadict onToolCallandToolResult(PR #24) for carrying out-of-band tags through the loop without subclassing. Round-trips throughto_dict().metadatadict onStepRecordandLLMCall(PR #19) for per-step / per-call annotations on saved trajectories.LifecycleEvent.HOOK_DECISION(PR #20) fires whenever a hook returns a non-noopHookDecision. Payload carries the slot, hook name, and serialized decision - single observation point for every gate, redaction, or short-circuit in the run.LifecycleEvent.DONE_ACCEPTED(PR #21) fires aftercheck_doneaccepts thedone()call and the final payload is committed. Payload includes thetool_callandtool_resultof the accepted termination - observer-only, fired right before STOP.serialize_harness(...)+TrajectoryRecorder(harness_snapshot=…)(PR #22) record a stable JSON-friendly snapshot of the agent config, tool list, hooks, and LLM backend on every saved trajectory. Lands intrajectory.metadata["harness_snapshot"].tool_callkwarg onLoopHook.check_done(PR #23) so quality gates can inspect the agent's pending answer before it terminates. Backward-compatible: existingcheck_done(self, state, log, ctx, step_num)signatures continue to work viainspect.signaturedetection.
Changed¶
- Coder example split into modules.
examples/coder/agent.pynow delegates toexamples/coder/{tools,hooks,wiring}.pyso the library entrypoint and the runnable cartridge inexamples/coder/skill/share exactly the same composition. Modify behavior inwiring.pyonce and both surfaces pick it up. Public symbols re-exported for back-compat. looplet evalsubcommand now routes tolooplet.evals.eval_cliwith full-h/--helpsupport; the top-level CLI no longer eats option-like tokens before they reach the eval parser. The eval help text now also documents thecases ls|showsubcommands.- Coder hooks default to "steer, don't restrict".
TestGuardHookships in observe-only mode (failures inject a briefing nudge butdone()is never blocked);StagnationHookusesresult_size_fingerprintwith a lenient threshold so legitimate retries don't trip a stall warning. Passtest_strict=Trueto recover the legacy hard-block behavior. - Coder example ships an outcome-grounded
EvalHookthat re-runs the project's pytest suite after the loop and surfacestests_passingviactx.artifacts.
Fixed¶
EvalContext.from_trajectory_dirnow preservestrajectory.metadata. Previously only four well-known top-level fields (run_id,started_at,ended_at,termination_reason) were copied intoEvalContext.metadata, silently droppingharness_snapshot(added by PR #22'sTrajectoryRecorder(harness_snapshot=…)kwarg) and any user-attached metadata. The fulltrajectory.metadatadict is now overlaid first, with the four top-level fields applied on top.TrajectoryRecordernow reflects latemetadatamutations. When a downstream hook ranpost_dispatchafterTrajectoryRecorderand taggedtool_call.metadata/tool_result.metadata(the documented annotation point added by PR #24), the mutations were silently lost because the recorder had already snapshotted viato_dict(). The recorder now sweepsstate.stepsinon_loop_endand refreshes the metadata fields on every capturedStepRecordso hook ordering no longer affects the saved trajectory.OpenAIBackend.from_env/AnthropicBackend.from_env/AsyncOpenAIBackend.from_envraise clean errors upfront. Previously, the OpenAI variants leaked an SDK-levelOpenAIErrorwhen neitherOPENAI_API_KEYnorOPENAI_BASE_URLwas set, andAnthropicBackend.from_envraised aTypeErrorfrom looplet's own constructor. Both now raise a singleRuntimeErrorwith an actionable message naming the env var to set. The OpenAI variants also now defaultapi_keyto a sentinel when onlyOPENAI_BASE_URLis set, so local-server flows (Ollama / vLLM / llama.cpp) work without settingOPENAI_API_KEY=ollama.- Coder example: clarified the
eval_tests_passedskip reason. The label now says "no Python project (pyproject.toml/setup.py) detected in workspace; collector cannot re-run tests" instead of the misleading "no test runner detected", since the collector needs a project file to re-run pytest against. discover_skill_bundlesacceptson_duplicate=andlooplet list-bundlesno longer crashes on collisions. Previously, two bundles claiming the samenamefield always raisedValueError, so a single dirty discovery root (e.g. left-over pytest fixtures under/tmp) made the entirelooplet list-bundlesCLI unusable. The function now acceptson_duplicate="raise"(default, back-compat),"first_wins"(silent), or"warn"(logs each collision tolooplet.bundles). The CLI passes"warn"so users see what was dropped but still get a list of valid bundles.Conversationserialization now round-tripsToolCall/ToolResultmetadata. PR #24 added the field but the Conversation serializer dropped it silently;Conversation.deserializealso never restored it. Both sides now plumb the field, so saved conversations preserve any out-of-band tags hooks attached.Message(role="system", …)no longer breaks serialization.MessageRoleis astr, Enum, so callers naturally pass plain strings - but_serialize_messagedidmsg.role.value, which raisedAttributeErrorwhenrolewas a plainstr.Message.__post_init__now coerces toMessageRoleso both call styles work identically.check_donesignature cache no longer poisoned by id reuse. PR #23's backward-compat dispatch cached_accepts_tool_call_kwargresults keyed onid(bound_method). Bound methods are ephemeral in CPython (obj.methodcreates a fresh object each access), so they get garbage-collected and their ids get reused for unrelated methods on other classes - leaving the cache claiming a legacy-signature hook acceptstool_call, then raisingTypeError: ...check_done() got an unexpected keyword argument 'tool_call'. The cache now keys onid(method.__func__)(the stable underlying function) with a fallback toid(method)for callables that lack__func__.async_composable_loopnow acceptsmax_steps=andsystem_prompt=shorthands. The synccomposable_loopgot these convenience kwargs but the async version was missed - callers had to construct aLoopConfigeven for one-liner async agents. The signatures now match.generate_kwargsnow reach backends declared as**kwargs._accepts_kwargonly matched explicitly-named parameters in the backend'sgenerate(...)signature, so any backend written asdef generate(self, prompt, **kw)(a common permissive pattern) silently dropped every entry ofLoopConfig.generate_kwargs. The helper now also returns True when the function declares aVAR_KEYWORDparameter, sotop_p,response_format,chat_template_kwargs, etc. propagate as documented.save_case(case, "evals/cases/")no longer creates a file literally namedcases. The "treat as directory" branch only fired when the path already existed, so a non-existent trailing-slash path (the obvious "I want a directory" convention shown indocs/evals.md) wrote the case content into a single file at the path. The helper now also treats trailing path separators as directory intent and creates the parent directories before writing<dir>/<case.id>.json.MetricsCollector.total_llm_callsis now populated by default. The field was advertised in the report but no built-in hook updated it, so it sat at 0 unless callers wired their own counter.MetricsHook.on_eventnow increments it on everyPOST_LLM_RESPONSElifecycle event.
[0.1.8] - 2026-04-24¶
Added¶
ctx.llm: tools receive the loop's LLM backend for internal calls. Tracked byRecordingLLMBackendwithscope="tool:<name>"for nested provenance.LLMCall.scope: provenance field for loop vs tool-internal calls.state.step_context: per-step ephemeral dict for hook-to-hook communication.LoopConfig.tool_metadata: static dict merged into everyToolContext.metadata.LoopConfig.generate_kwargs: extra kwargs passed through to every LLM call. Can overridetemperature,max_tokens,system_prompt. Supports provider-specific params (chat_template_kwargs,response_format,top_p).async_composable_loop: async generator for async LLM backends._SyncBridgeLLM: sync tools can usectx.llm.generate()even with async backends.OpenAIBackend.tool_choice: configurabletool_choiceparameter.PerToolLimitHook.default_limit: blanket cap for all tools.CompactOutcome.compacted: property indicating if compaction reduced context.register_done_tool(): convenience for registering the done tool.EvalResult.passed: property for pass/fail determination.- Async tool dispatch in sync loop:
dispatch()detects coroutine returns. - 3 example agents: threat intel briefing, git history detective, dependency doctor.
Changed¶
ToolContextis now always created (neverNone), withmetadatapopulated fromstate.metadata(copy, not reference).default_max_tokensdefaults toNoneacross all backends - lets the provider API decide instead of forcing 2000.- All docs and examples updated to use convenience
OpenAIBackend(base_url=...)andregister_done_tool().
Fixed¶
Step.to_dict()key names:call→tool_call,result→tool_result.- Tool validation error now shows what args were provided.
Step.summary()shows dict preview instead of?.Trajectory.taskfield preserved in trajectory.json for eval round-trip.TrajectoryRecorder(output_dir=...)auto-saves on loop end.RecoveryRegistry.registerwarns on overwrite.clone_tools_excludingwarns on missing names (typo detection).- Permission audit strips
__…scaffolding keys. - Conversation
compact()marks summary as compaction boundary.
[0.1.7] - 2026-04-21¶
First public release of looplet.
Added (launch polish)¶
ROADMAP.mdwith a frozen v1.0 API contract and explicit out-of-scope list.docs/site scaffold (tutorial, evals, recipes, hooks, good-first-issues, discussions-seed, demo-script) + mkdocs-material config + GitHub Pages workflow.THIRD_PARTY_USERS.mdsocial-proof seed.src/looplet/examples/ollama_hello.py- zero-API-key onboarding.- Codecov upload step in CI (non-blocking).
- Leaner README (<170 lines) with the pydantic-ai-harness disambiguation moved to the top.
Added (evals - pytest-style agent evaluation)¶
- Eval framework (
looplet.evals). Writeeval_*functions that takeEvalContextand return any offloat,bool,str,dict, orEvalResult. The framework normalizes all return types. eval_discover(path)- auto-discovers eval functions ineval_*.pyfiles (like pytest discoverstest_*).eval_run(evals, ctx)- runs evaluators, auto-detectsllmparameter for LLM-as-judge, catches errors gracefully.eval_run_batch(evals, contexts)- runs same evals across multiple trajectories with per-eval avg/min/max aggregation.eval_mark(*tags)- decorator for categorizing evals.eval_runandeval_run_batchacceptinclude=/exclude=to filter by marks.eval_cli(args)- CLI runner with threshold-based pass/fail exit codes for CI integration.EvalHook- LoopHook that builds EvalContext aton_loop_endand runs all evaluators automatically during development.EvalContext.from_trajectory_dir()- loads context from saved trajectories with support for both looplet and benchmark formats.
Added (MCP + skills)¶
MCPToolAdapter- wraps MCP server tools asToolSpecinstances via JSON-RPC over stdio. No MCP SDK required.Skill- bundles tools + context + prompt fragment into one loadable unit.skill.register(registry)adds all tools.
Added (approval)¶
ApprovalHook- stops the loop when a tool returnsneeds_approval=True. Combined withcheckpoint_dirfor crash-safe async human-in-the-loop approval.- Renamed
elicit→approvaluniformly:LoopConfig.approval_handler,ToolContext.request_approval,ToolContext.approve().
Changed (naming cleanup)¶
- Renamed internal names for clarity:
coerce_text→to_text,DiminishingReturnsTracker→StallDetector,reactive_compact→emergency_truncate,compress_session_log→age_session_entries,enforce_result_budget→trim_results,should_compress_context→is_context_oversized,HEAVY_BLOCK_KINDS→LARGE_CONTENT_TYPES,DefaultSummarizer→default_summarizer. - Renamed compact services:
DefaultCompactService→TruncateCompact,LLMCompactService→SummarizeCompact. - Renamed
normalise_hook_return→normalize_hook_return. - Moved
concurrent_dispatchandreactive_recoveryfromFLAGSglobal singleton toLoopConfigfields. - Trimmed
__all__from 154 → 54 symbols organized into labeled tiers.
Changed (developer experience)¶
- Added
preview_prompt()- shows what the LLM sees before the first call. Invaluable for debugging. - Added
TrajectoryRecorder.summary()- one-liner run summary. - Added
--trace DIRto coding_agent example for trajectory recording. - Added step-by-step tutorial to README (5 progressive steps).
- Added
LoopConfigdocstring with "start here" guide listing the 4 essential fields. - Added
FileCheckpointStore.load_latest()+ auto-resume wiring incomposable_loop- crash-resume is now one line:LoopConfig(checkpoint_dir="./ckpt").
Removed¶
- Removed
async_loop.py(feature-frozen, no consumers). - Removed 3 mock examples (calculator, code_review, research).
Replaced with
hello_world.py(real LLM) +coding_agent.py(Claude Code-equivalent tools: bash, read, write, edit, glob, grep, think, done). - Removed all back-compat aliases.
- Removed all internal project references (cadence, primal_security).
Added (compaction strategies)¶
PruneToolResults- new zero-LLM-call compaction service that clears old tool-result content while keeping conversation structure intact. Configurablekeep_recent(how many recent tool results to preserve) andcompactable_tools(restrict to specific tools). Cheapest possible compaction - use as the first stage in a chain.compact_chain(*services)- combinator that tries compaction services in order; first stage that has an effect wins. Replaces the need for a separateChainedCompactServiceclass. Usage:compact_chain(PruneToolResults(), SummarizeCompact(), TruncateCompact()).CompactOutcome.cleanup- optional post-compact callback. When set,run_compact()invokes it after firingPOST_COMPACT. Use for domain-specific state resets (clear caches, re-inject context, reset token baselines) without the loop knowing details.
Changed (renames - back-compat aliases kept)¶
DefaultCompactService→TruncateCompact- clearer name for "drop old entries, keep N recent, zero LLM calls."LLMCompactService→SummarizeCompact- clearer name for "LLM summarizes middle, keeps N recent."- Old names (
DefaultCompactService,LLMCompactService) remain as aliases and continue to work.
Added (context management pt. 2)¶
- Prompt caching infrastructure (
looplet.cache). NewCachePolicydataclass declares which stable prompt sections (system prompt, tool schemas, memory) should carry Anthropic-stylecache_controlmarkers, with per-section TTL (ephemeral/1h).LoopConfig.cache_policythreads per-turnCacheBreakpointlists (label + SHA-256 hash + TTL) to backends that exposegenerate_with_cache(..., cache_breakpoints=[...]). Backends without the kwarg keep working unchanged - caching is strictly additive.CacheBreakDetectorships as a drop-in observer hook that records section-hash changes across turns for cache-miss telemetry. LLMCompactService- new compaction strategy that spends one LLM call to summarise the session. Produces a dense 4-section summary (task goal, findings, open questions, recent decisions) spliced into the session log as a synthetic entry after keep-recent pruning. Falls back to deterministic keep-recent on any summariser error. Trade-off vsDefaultCompactService: one LLM call per compaction for preserved reasoning chains.- Threshold-tier context budgeting (
looplet.budget). NewContextBudgetdataclass withwarning_at/error_at/compact_buffertiers.ThresholdCompactHookis a ready-to-registershould_compactimplementation that fires proactive compaction once estimated tokens cross the configured tier.BudgetTelemetryobserver records per-step tier samples and exposespeak_tierfor production dashboards.
Added (architecture improvements)¶
- Proactive compact hook slot -
LoopHook.should_compact(state, session_log, conversation, step_num) -> bool. Fires at the top of each step, before prompt build. Any hook returningTruetriggers the configuredCompactServicepreemptively. Complements the reactiveprompt_too_longpath - use for message-count or token-estimate heuristics.StreamingHookgets a no-op stub. - Tool-result streaming via
TOOL_PROGRESS- newLifecycleEvent.TOOL_PROGRESS. When hooks are present, the loop builds aToolContext.on_progresscallback per tool-call that emitsTOOL_PROGRESS(with the originatingtool_call) whenever the tool invokesctx.report_progress(stage, data). Observers can stream intermediate output from long-running tools without blocking dispatch. - Budget-aware turn continuation - new
LoopConfig.max_turn_continuations: int = 0. When> 0and the backend exposeslast_stop_reason,llm_call_with_retrywill re-prompt up to N times onstop_reason == "max_tokens"and concatenate outputs so long thoughts aren't truncated mid-message.LLMResultgainsstop_reasonandcontinuationsfields. build_briefing/build_promptas hook slots - both are now optional methods onLoopHook. First hook returning a non-Nonestring wins; the loop falls back toLoopConfig.build_briefing/config.build_prompt/ the built-in default. Lets domain hooks own prompt construction without threading callables throughLoopConfigseparately.DomainAdapter- new dataclass bundling the five domain callables (build_briefing,extract_entities,build_trace,build_prompt,extract_step_metadata) into a single object.LoopConfig.domain: DomainAdapter | None = Noneseeds matching flat fields when they areNone. Flat fields still win over the adapter, which wins over built-in defaults - use the adapter to package a reusable agent in one handle instead of five kwargs.
Removed (breaking)¶
InvestigationLogbackward-compat alias is gone - useSessionLogdirectly.HARNESS_FLAGSbackward-compat alias is gone - useFLAGS.- Legacy
CADENCE_*environment variables for feature flags are no longer read; use theLOOPLET_*prefix. _clone_tools_excludingprivate alias is gone - useclone_tools_excluding.LoopConfig.permissionsis gone. Register aPermissionHook(PermissionEngine(...))inhooks=[...]instead - it flows through the same unifiedHookDecision+ event bus as every other hook.
Added¶
- Unified hook vocabulary -
HookDecision(looplet.hook_decision). All hook slots now accept a singleHookDecisionreturn type (legacyNone/bool/strreturns still work vianormalise_hook_return). HelpersAllow(),Deny(reason),Block(reason),Stop(reason),Continue(),InjectContext(text)make intent explicit at the call site. - Lifecycle events -
on_event(payload)(looplet.events).LoopHookgained an optionalon_event(EventPayload)method. The loop now fires 10 named events:SESSION_START,PRE_LLM_CALL,POST_LLM_RESPONSE,PRE_TOOL_USE,POST_TOOL_USE,POST_TOOL_FAILURE,PRE_COMPACT,POST_COMPACT,STOP,SUBAGENT_START,SUBAGENT_STOP. Any hook can subscribe with a single method instead of implementing every slot. PermissionHook(looplet.permissions) - wrapsPermissionEngineand plugs it into the event bus so policy decisions flow through the sameHookDecisionpath as custom hooks.CompactService+DefaultCompactService+run_compact(...)(looplet.compact) - reactive compaction is now a swappable service withPRE_COMPACT/POST_COMPACTevents.LoopConfig.render_messages_override- byte-exact escape hatch. Receives(messages, default_prompt, step_num)and returns the exact prompt string sent to the LLM. Lets advanced callers take full control of prompt rendering without forking the loop.- First-class subagents -
run_sub_loop(..., subagent_id=...)now firesSUBAGENT_START/SUBAGENT_STOPevents on the parent's hooks and returnssubagent_idin the result dict for correlation. replay_loop(trace_dir, tools=...)- rerun a captured trace through a freshcomposable_loopwithout calling the LLM again. Useful for golden-trajectory regression tests, hook A/Bs, and cost-free loop diffs. RaisesRuntimeErrorif the replay loop requests more calls than were recorded or diverges in method (generatevsgenerate_with_tools). Falls back tocall_NN_response.txtfiles whenmanifest.jsonlis missing.python -m looplet show <trace-dir>- stdlib-only CLI that prints a one-page summary of a captured trace (run id, termination, per-step tool calls with durations, LLM totals). Exit code 1 when the directory is missing or malformed.looplet.provenance- new module for debugging agent runs:RecordingLLMBackend/AsyncRecordingLLMBackendwrap any backend and capture every prompt, system prompt, tool schema, response, duration, and error asLLMCallrecords.generate_with_toolsis surfaced only when the wrapped backend supports it, soNativeToolBackenddetection stays honest.TrajectoryRecorderhook captures a structuredTrajectoryper run (steps, context-before, termination reason, embeddedTracerspans) and writestrajectory.json+steps/step_NN.json.ProvenanceSinkis a 3-line facade:wrap_llm(...),trajectory_hook(),flush().- On-disk layout is diff-friendly:
call_NN_prompt.txt/call_NN_response.txtper LLM call plus amanifest.jsonl. - Both recorders accept
redact=for secret scrubbing andmax_chars_per_call=for bounded memory. - See Provenance guide for API reference, recipes, and performance notes.
Step.pretty()- human-readable CLI formatter complementingStep.summary()(which is tuned for LLM context assembly).
[0.1.6] - 2026-04-17¶
Added¶
looplet.testing- public test-utility module exposingMockLLMBackendandAsyncMockLLMBackend(scripted, zero-dependency) so downstream packages can unit-test hooks, tools, and backends without a real LLM provider.- PyPI publish workflow (
.github/workflows/publish.yml) that builds + publishes on version tags via PyPI trusted publishing. - README positioning matrix comparing
loopletto LangGraph, DSPy, and smolagents; observability/OTel wiring example; stability & versioning policy; realAnthropicBackendusage in quick-start.
Fixed¶
resume_loop_state()now restores the checkpointedConversationthread (was silently dropping multi-turn message history on resume).RoutingLLMBackend.generate_with_toolsis now gated dynamically via__getattr__sohasattr(llm, "generate_with_tools")returns a truthful answer for the currently-selected backend (consistent with_FallbackLLMandCostTracker).- Async
__llm_error__step is now recorded through_historyto match the sync loop (previously caused session-log/conversation drift on LLM failure).
Previously added in this release¶
ToolErrortaxonomy - structuredErrorKindenum (PERMISSION_DENIED,TIMEOUT,VALIDATION,EXECUTION,PARSE,CONTEXT_OVERFLOW,RATE_LIMIT,NETWORK,CANCELLED) plus aToolErrordataclass.ToolResultnow carries botherror: str(for JSON-safe display) anderror_detail: ToolError(for introspection).PermissionEngine- declarativeALLOW/DENY/ASK/DEFAULTrules with fail-closedarg_matcher, plug-inask_handlerfor human-in-the-loop, and an append-only denial audit log.CancelToken- cooperative cancellation is now threaded throughLoopConfig→llm_call_with_retry/async_llm_call_with_retry→ToolContext.cancel_token, so both the next LLM call and any in-flight tool can stop cleanly.ToolContext.elicit-LoopConfig.elicit_handlersurfaces a genericelicit(prompt) → strprotocol to tools for interactive prompts.- Multi-block messages -
Message.contentsupports alistofContentBlock(kind, data)alongside plainstr.HEAVY_BLOCK_KINDS(image/audio/video/binary) are stripped before summarization. - Async
build_trace-async_composable_loopnow stashes the built trace onstate.traceat exit (async generators can'treturna value). SyncToAsyncAdapter.generate_with_tools- router-selected sync backends keep native-tools support in the async loop.- Preflight context check - async loop matches sync by skipping a
doomed LLM call when the prompt is already too long under
FLAGS.reactive_recovery. - Checkpoint state counters -
resume_loop_statenow round-tripsstate.queries_usedandstate.budget_remainingso budget enforcement continues across resume.
Changed¶
ToolResult.errornarrowed back tostr | None(JSON-safe). UseToolResult.error_detailfor structured introspection.PermissionRule.matches()now fails closed per decision type:DENYrules match on matcher errors (block),ALLOW/ASKrules do not (don't accidentally grant).PermissionEngine._resolve_defaultcollapses ambiguous engine defaults (ASK/DEFAULT) toDENYso a decision never leaks into aPermissionOutcomewhere both.allowedand.deniedare False.ToolSpec._accepts_ctxis computed eagerly atregister()time (and self-heals indispatch()for specs inserted directly)._backend_accepts_cancel_tokencache keyed by(type, method_name)instead ofid()(eliminates id-recycling hazard)._classify_exceptionbroadened to detectasyncio.CancelledError, rate-limit, context-overflow, and parse exceptions by class name / message content.SyncToAsyncAdapter._adapter_cachenow prefers the backend object itself as the dict key, withid()as a fallback for unhashable backends.SessionLog.to_list()includesrecall_keyfor full round-trip through checkpoints.ToolError.contextnow round-trips throughConversation.serialize/deserialize.- Permission-denied results from hooks now populate
error_detailwithErrorKind.PERMISSION_DENIED(parity with thePermissionEnginepath) in both sync and async loops.
Fixed¶
_rebuild_promptnow rendersmemoryand falls back to the structuredbuild_promptfromlooplet.promptsinstead of a bare f-string, restoring parity with the first-pass build._deserialize_messagenow reconstructsToolErrorfrom serializederror_kind/error_retriable/error_contextfields._NullSessionLog(async) gained the attributes the async loop expects:entries,current_theory,to_list(),compact().
[0.1.5] - initial public import¶
- Initial release as a standalone package. See the extraction commit history for the pre-extraction development timeline.