Research Note

macOS Native App Automation

Zhenyu He · Jobs Stroustrup 1 min read

Proficient

Two-layer architecture: AppleScript (bring app to front, switch windows) + computer-use MCP (screenshot + click + keyboard) combined — works around desktop apps with no official API
Vision-driven UI operations: locate UI elements from screenshots (e.g. WeChat’s Discover icon, the .. button on each post), explicit waits, computer_batch to combine multiple actions for speed
Claude Code Skill packaging: wraps multi-step agent workflows into standard skill format (SKILL.md + supporting data files) for reuse and iteration
Agent behavior engineering: human-in-the-loop review, data flywheel (self-growing few-shot corpus), multi-tier behavior rules (blocklist / frequency cap / per-target persona), structured long-term memory
Complements Python Web Automation Selenium scripting — one tackles browser DOM, the other tackles native macOS apps

Personal project / daily productivity tool: private personal automation project — social-feed interaction + archival workflow
AI Tools & Productivity — the path of delegating repetitive desktop tasks to AI agents