The repo's threat model (SECURITY.md) calls out XML/SVG content as a key
attack surface and lists "ElementTree auto-escaping" as the mitigation.
That covers XML *output* (serialization). XML *input* (parsing) across
the agent backends still goes through stdlib `xml.etree.ElementTree`,
which is vulnerable to entity-expansion DoS ("billion laughs" /
quadratic blowup) — Python 3.7.1 disabled external-entity lookup on ET
by default, but internal-entity expansion is still possible and can
exhaust CPU/RAM from a small file.
Migrate the parse/fromstring call sites in production code paths to
defusedxml.ElementTree (drop-in API). Construction calls (Element,
SubElement, tostring, register_namespace) are left on stdlib ET — those
don't parse untrusted input. Tests are left as-is in this PR.
Detected by Aeon + semgrep p/security-audit + p/owasp-top-ten
(rules: use-defused-xml, use-defused-xml-parse).
Severity: medium (DoS from agent-ingested files; not RCE).
CWE-776 Improper Restriction of Recursive Entity References.
- Remove duplicate macro_record implementation in macrocli_cli.py
(stale second implementation from lines 582-706 that would cause
duplicate saves and unexpected behavior)
- Update MACROCLI.md: document visual_anchor and gui_agent backends,
optional dependencies ([visual], [gui_agent]), and OpenAI-compatible
LLM configuration (MACROCLI_MODEL/API_KEY/BASE_URL)
- Resolve .gitignore conflicts: add sbox and quietshrink directories
from upstream while preserving macrocli entries (Step 4/5/6 pattern)
- Remove flameshot_annotate and kolourpaint_annotate from manifest
(only gedit demos are shipped)
- Remove kolourpaint demo YAML and snapshot from tracked files
- Fix gui_agent model defaults: remove hardcoded Gemini model name,
require user to set MACROCLI_MODEL env var or pass model in params
- Replace stale 'gemini' extra with 'gui_agent' extra (openai SDK)
- Remove google-generativeai dependency (unused)
- Update README: document gui_agent and visual_anchor backends,
optional extras, and environment variables for model configuration
- Simplify package_data (no PNG assets shipped with gedit-only demos)
gui_agent backend:
- Add instruct_with_refine action: executes drag, captures before/after
screenshots, sends all three images (before, after, target) to model
for comparison, model provides corrected coordinates, undo + re-execute
- Add drag to action space and _execute_action
- max_tokens=1024 to prevent JSON truncation on drag actions
- Support instruct and instruct_with_refine actions
visual_anchor:
- _mouse_drag: use xdotool first (reliable with Qt5/KDE apps like
KolourPaint), fall back to pynput if xdotool unavailable
macro_definitions:
- Add kolourpaint_annotate demo macro (hybrid fixed+agent annotation)
- Add end_state snapshot for annotation grounding
- Register kolourpaint_annotate in manifest.yaml
64/64 tests passing
gui_agent backend (backends/gui_agent.py):
- Executes macro steps via Gemini Vision in a perception-action loop
- Takes current screenshot + end_state_snapshot + description
- Asks model for next action (click/type/hotkey/scroll/done)
- Verifies end state reached by comparing screenshots with model
- Strict action space: 7 allowed actions, no arbitrary code
- Loops up to max_steps, returns reached_end_state in output
recorder.py:
- RecordedStep: add is_agent_step, agent_description,
agent_end_state_description, agent_end_state_snapshot fields
- to_step_dict(): emit gui_agent/instruct for agent steps
- interactive_agent_review(): post-recording step-by-step review,
mark steps as fixed or agent, capture end-state snapshots
- save_as_package(): save as <name>/ folder with macro.yaml + snapshots/
- _capture_end_state_snapshot(): take full-screen snapshot
macro record --agent-review: trigger interactive review after recording
Macro package structure: <name>/macro.yaml + <name>/snapshots/*.png
Replace fragile wmctrl -lG text parsing (hostname column caused
'(/tmp)' bug) with xdotool getwindowfocus getwindowgeometry --shell.
Single subprocess call, no column counting, works regardless of
hostname format or window title content.
- _get_active_window_at(): query wmctrl -lG to find window under click
point, return (app_title, bounds); fix split(None,7) to capture full
title including spaces
- RecordedStep: add window_title, x_pct, y_pct fields
- to_step_dict(): prefer click_relative with window_title anchor over
click_image (robust to window movement/resize)
- Featureless template detection: skip saving if std < 8.0
- space key treated as printable char, accumulated into type_text
- Modifier key dedup: skip recording hotkey when only modifiers pressed
- Template padding increased 30px -> 60px for better matching
native_api: add start_process action for background GUI app launch
- uses Popen with start_new_session=True, never waits for exit
- gedit_new_window now uses start_process instead of run_command
semantic_ui + visual_anchor: pass DISPLAY via _x_env() to all
subprocess calls (wmctrl, xdotool, xwininfo)
- fixes breakage when running via conda run which strips DISPLAY
gedit_new_window: use start_process + env.DISPLAY for reliable launch
gedit_save_as: use Ctrl+L + Ctrl+A to clear path before typing,
remove postcondition (gedit writes async after dialog closes)
CLI: add --macro-file option to macro run (bypass registry)
DEMO.md: fix command examples, add test notes
aeonframework