Documentation

Everything Mint can do.

Ask in your own words; you never need a tool's name. Tap a ▶ example and the little Mint in the corner acts it out.

Install

One download with everything inside: Python, libraries and models. Apple silicon Macs on macOS 14.2 or later.

Download Hey-Mint-…-arm64.dmg from the latest release and drag Hey Mint into Applications.
Open it. The first time, macOS may say it can't check it: open System Settings ▸ Privacy & Security and click Open Anyway.
Paste a free Gemini API key. Hold ⌥ while opening Mint to change keys later.
Allow the Microphone, then switch Mint on under Accessibility so it can click and type.

Optional keys, and what they unlock

Key	Unlocks
OpenAI or OpenRouter	Agents: research, reports, code
TypeSafe (Jev)	The `desktop` tool for multi-step clicking in any app, and surer picks of buttons, skills and memories
Exa or Tavily	Better web search (DuckDuckGo otherwise)
ChatGPT app	Codex, the agent that builds websites and apps

Build from source instead

git clone https://github.com/shivatmax/hey-mint.git && cd hey-mint
./set-key.sh      # your Gemini key, saved in .env
./install.sh      # builds ~/Applications/Mint.app
open ~/Applications/Mint.app

Developing Mint? See the developer handbook.

Talking to Mint

Say “Hey Mint”, then ask. Your words and Mint's reply appear as captions beside the orb. Keep talking without the wake word: it's one conversation.

A request, start to finish. Your words, the orb thinking, turning into Slack while it opens it, a ✓, the reply.

what's on my calendar today?
and tomorrow?
stop
bye

To	Do this
Wake it	“Hey Mint”, click the orb, or `⌘J` and type
Interrupt	Just talk while it speaks; it stops and listens
Stop everything	“stop”, “cancel”, “never mind”: queued clicks and keys too
Send it to sleep	“bye”, “that's all”; or 12 quiet seconds

The chat window

⌘J opens the chat: the conversation, every action with a ✓ or ✗, and a box to type in.

The buttons along the top show or hide Mint in screen sharing, compact or clear the chat, send Mint to sleep, turn the mic or spoken replies off, and open Settings. Typing never turns the microphone on.

Reading the orb

The orb turns into whatever it's doing, with a caption, then shows a ✓ or ✗. Long tasks get a plan, a progress ring and confetti.

Searching, reading a file, a menu command, reading the window; a failed step (✗ and a tiny rain cloud); then a three-step plan.

You see	It means
Eyes following your pointer	Awake and listening
Eyes up, swirl turning	Thinking
A symbol with a caption	Working: a magnifier is searching, a hand is clicking, a cursor is typing
Dimmed, eyes closed	Asleep; only “Hey Mint” wakes it

Apps, clicking and typing

Mint reads every control in the window, presses the one you meant like a person would, and checks something changed. Menus and app scripting come first; the pointer moves only when nothing else can do it.

A click on “Design review” (a ripple and a paw print) and typing into the outlined box.

open Figma
switch to the Jira tab
click Sign in
type see you at six
read this page
summarise this
in Finder go to Applications
scroll to Privacy & Security

Mint never types into a password field, and presses Send, Delete, Buy and the like only when you asked.

Files

Spotlight search by name and content; reads text, PDFs, Word, CSV and more; writes and edits with a backup of the old version.

what did I download today?
find the PDF about the lease
make a file called plan.md with…
change the deadline in plan.md to Friday
move it to Archive
show it in Finder

“Delete” means the Trash. Keys, keychains, browser profiles, Mail and Messages are off limits.

Browser and web

Full control of Chrome, Safari, Brave, Arc and Edge, including sites you're signed in to. Web search and page reading happen in the background.

what tabs do I have open?
put my name in the Full name field
put YouTube and anime in a group called Entertainment
always open YouTube in Brave
what's the latest version of Python?
read that article

Which browser it uses

The one you name; then your rules (“entertainment in Brave”); then the one in front; then the Mac's default. A tab already showing the page is reused. Page actions are fastest with Allow JavaScript from Apple Events on (Chrome ▸ View ▸ Developer); Mint never switches it on itself.

Showing you where

Ask where something is, in any app. Mint finds the words on screen, scrolls to them, marks them, and flies over to look.

Box, highlight and arrow, each with a note. The orb swoops over beside each mark.

where's the deadline?
underline who owns the notes
highlight the budget
circle the biggest risk
clear the marks

Launch plan

We ship the new voice experience next week. The deadline is Friday, 3 October.

Priya handles the release notes and the demo. Budget stays under $4,000.

Biggest risk: the wake word on older Macs.

Everyday and long tasks

Calendar, reminders, notes, timers, email drafts, volume and media. Ask for a list of things and Mint does them back to back, then reports once.

remind me to call Sam at five
set a ten minute timer for tea
draft an email to Priya saying I'll be late
take a screenshot of this window
open Notes, Calendar and Slack
what did I copy before?

Bigger jobs get a plan you can watch step by step. Mint checks each step really happened, splits a step that turns out bigger, and keeps notes as it goes. The plan is saved, so “stop” only pauses it: say where were we? tomorrow and it carries on.

Emails are always drafts: Mint never sends one. A message goes out only when you asked, and never twice within two minutes.

Watching videos

Give Mint a link, a file or “this video” on screen. In seconds it tells you what the video is, the key moments with timestamps, and how it looks and feels.

watch this and tell me what it's about
what did he say about pricing?
show me the frame at 3:20
what's the aesthetic of this reel?
transcribe my screen recording

Mint doesn't sit through the video. It reads what's said, from the captions or by transcribing the audio, and looks at about a dozen frames picked where the scene changes. A 15-minute talk takes about 15 seconds. The transcript and notes are saved in Documents ▸ Mint ▸ videos.

Works with YouTube, X, Vimeo, Loom, TikTok and Instagram links, and any video or audio file on your Mac. Agents can watch videos for their research too.

Automations

Things Mint does by itself: at a time, every so often, or when something happens.

every weekday at 9, brief me on my calendar and mail
every Friday at 5, have Astra write a brief on this week's AI news
remind me to stretch every hour between 10 and 6
when a PDF lands in Downloads, file it in Documents/Invoices
10 minutes before any meeting, open its notes
what automations do I have?

Pause, resume, run or delete any of them by name.

An automation never sends, posts, buys or deletes on its own. It prepares the draft or the list and tells you it's ready.

Agents

For big jobs (research, long documents, code, websites) Mint hands the work to an agent and keeps talking with you. You hear back when it's done.

A real run. Astra pops out of the box, searches, reads pages, writes the brief, and hops back in. Middle 2× faster.

AstraResearch: searches, reads, cross-checks, writes a sourced brief

LunaCode: plans and writes complete, runnable files

SageWriting: long documents, reports and PDFs

CodexBuilds websites and apps in OpenAI Codex, and checks them

have Astra research the best mechanical keyboards
ask Codex to make a tiny hello-world page
how's Luna doing?
tell Luna to use pytest
stop all agents
create an agent called Nova that writes tweets

Teams, folders and limits

An agent can bring in teammates when their specialty helps (Sage asks Astra for research) and reports back for all of them. Each works in its own folder under ~/Documents/Mint/agents. Up to six at once; stopping the lead stops its helpers. Finished work goes into Mint's memory.

The critters

Each agent is a little critter that lives in a toy box beside the orb. What it does tells you what the agent is doing.

Sage calls Astra in to help; then Luna fails (a rain cloud and a tear) and Codex is stopped (a startled dash home).

The critter	Means
Pops out with sparkles	An agent started
A “…” or a tool in its bubble	Thinking, or searching, reading, writing
A “?” and a hop	It has a question: click it
A somersault and confetti	Finished; it hops back into the box
A rain cloud, or a startled “!”	Failed, or stopped

Hover to see what one is doing, click for a reaction, right-click to stop it. Say put on a show and the whole family performs:

Showtime: the critters pop out, do a stadium wave, a solo each and a dance, then bow.

Expressions and tricks

Fourteen expressions and eight tricks, each a real 60 fps animation. Mint uses them sparingly on its own, and whenever you ask.

Love, laugh, cool, surprised, cry, sleepy, dance, wave.

A loop, a figure eight, chasing a star, a bee's zigzag, a spin.

do a loop the loop
a figure eight
chase a star
bounce like a ball
go a little up
go back

Voices

Any of Gemini's 30 voices, in any speaking style you describe. It switches between turns and keeps the conversation.

use a deeper voice
sound more cheerful
talk slower
use Puck

All 30 voices

Zephyr female · brightPuck male · upbeatCharon male · informativeKore female · firmFenrir male · excitableLeda female · youthfulOrus male · firmAoede female · breezyCallirrhoe female · easy-goingAutonoe female · brightEnceladus male · breathyIapetus male · clearUmbriel male · easy-goingAlgieba male · smoothDespina female · smoothErinome female · clearAlgenib male · gravellyRasalgethi male · informativeLaomedeia female · upbeatAchernar female · softAlnilam male · firmSchedar male · evenGacrux female · maturePulcherrima female · forwardAchird male · friendlyZubenelgenubi male · casualVindemiatrix female · gentleSadachbia male · livelySadaltager male · knowledgeableSulafat female · warm

Memory

Mint remembers facts about you, one per note, and looks up only what matters to the question.

remember my manager is Meera
my manager is now Rahul
who's my boss?
forget my old address
what do you remember about me?

When a fact changes, Mint replaces it and still remembers what it used to be. Once a day it tidies up: it merges duplicates and drops things whose date has passed. Read and edit every fact in the Skills & Memory window. Passwords, keys and card numbers are refused.

What happened when

Mint can look back through your conversations, what it did, what the agents found, and the videos it watched, by day.

what did we do yesterday?
when did I ask you about flights?
what was that site you found last week?
what did Astra find on Monday?

Say remember what I work on to turn on the activity timeline. It notes which app, window and web page is in front. Then you can ask what was I working on yesterday afternoon?, find that Airbnb tab from Tuesday or how long was I in Slack today?.

The timeline is off until you turn it on. It is text only, never screenshots or typing, and skips private windows and password managers. Safari pages are never recorded, because Safari can't tell its private windows apart. It stays on your Mac for 14 days. Say delete my timeline to erase it.

Skills

A skill is a short how-to. When a task took many steps or needed your correction, Mint writes it up, then ranks it by how often it works.

save how to do that as a skill
next time use the sidebar
what skills do you have?
delete that skill

Wake word and voice lock

“Hey Mint” is detected on your Mac. Train your voice once (menu bar ▸ Train my voice…, two minutes) and only you can wake it, talk to it, or talk over it.

Setting	Use it when
Relaxed	It sometimes misses you (a noisy room, a cold)
Balanced	Most of the time (the default)
Strict	Someone with a similar voice is around

you heard cube con, I said KubeCon
it's GitHub, one word

Mint steps aside during Zoom, Meet or FaceTime calls, and ignores talk that isn't meant for it. Rename it in Settings ▸ You and it builds a new wake word.

Settings

Open Settings from the menu bar icon (⌘,), or just ask Mint to change something.

don't speak, just chat
make it pink
move to the top left
Mint, be visible
quit Mint

Tab	What's in it
You	The assistant's name, your name, and what Mint should know about you
Your voice	Training, the voice lock and its strictness
Mint's voice	Voice and speaking style
Audio	Microphone, speaker, echo cancellation, calls
Appearance	Theme, position, face, captions, effects

Mint is invisible in screen shares by default; “be visible” shows it in a Google Meet or Zoom share.

Privacy and safety

🎙️	Nothing leaves your Mac while Mint is asleep. The wake word and voice lock run on the Mac.
🔒	Never enters passwords or card numbers, and never saves secrets in files, skills or memory.
✉️	Never sends email; messages and forms go out only when you asked.
🗂️	“Delete” means the Trash; files are backed up before they're changed; private folders are off limits.
✋	“Stop” halts everything at once, including queued clicks and keys.

Troubleshooting

Problem	Fix
Can't click or type	System Settings ▸ Privacy & Security ▸ Accessibility: switch Mint on (off and on again after an update)
“not allowed to control that app”	Privacy & Security ▸ Automation ▸ Mint
It hears the room	Train your voice, or turn the mic off and type (`⌘J`)
It mishears a name	Tell it once: “you heard X, I said Y”
An agent seems stuck	“how's Astra doing?”, then “stop Astra”
The app won't open	Privacy & Security ▸ Open Anyway (the app isn't notarized yet)

The log is at ~/Library/Logs/Mint/mint.log. Report bugs on GitHub.

Every tool

What Mint can call, for the curious. You never need these names.

Show all tools

Tool	What it does
`open_app`	Instantly launch or switch to a Mac application by name.
`open_url`	Instantly open a web address in a new browser tab (reuses a tab already showing it).
`open_folder`	Instantly open a folder in Finder.
`scroll`	Instantly scroll the window under the pointer.
`press_key`	Instantly press a key, optionally with modifiers.
`set_volume`	Instantly set the Mac's output volume.
`media_key`	Instantly send a media key: play/pause, next or previous track.
`frontmost_app`	Which application is in front right now.
`list_windows`	List the visible windows as app name and title.
`quit_app`	Ask an application to quit.
`type_text`	Instantly insert text at the cursor in whatever field is focused, in any app.
`get_selected_text`	Read the text the user has selected in the front app.
`get_status`	Current local time and date, battery level, and volume.
`set_timer`	Start a timer.
`notify`	Show a macOS notification.
`system_action`	Lock the screen, sleep the display, switch dark mode, or mute.
`calendar_events`	The user's calendar events from today onwards.
`create_reminder`	Create a reminder in the Reminders app, optionally due at a time.
`create_note`	Create a note in the Notes app.
`compose_email`	Open a pre-filled email draft in the user's mail app for them to review and send.
`list_emails`	List the newest messages in the macOS MAIL APP's inbox.
`read_email`	Read one message from the macOS Mail app's inbox in full (see list_emails).
`open_chrome`	Open Google Chrome as one particular account (Chrome profile), optionally at a URL.
`open_slack`	Open Slack, optionally switch to a workspace, and optionally jump to a channel, DM or person by name.
`list_accounts`	List the Chrome profiles and Slack workspaces available.
`read_window`	Read ALL the text in the front window - a web page, a Gmail inbox or open email, a document, an app - exactly, including parts scrolled out of view.
`list_open`	What is open right now: the app in front, and every Chrome window with its profile and tabs (* = the selected tab).
`switch_to`	Bring an already-open Chrome tab or window to the front, by words from its title, e.g.
`export_doc_pdf`	Export the open Google Doc itself as a PDF, exactly as Docs renders it, and open it.
`plan_task`	Start a multi-step task: state the goal and the ordered steps BEFORE doing any of them.
`step_done`	Mark a step of the current task finished (or failed), with a one-line result saying what you checked (what the window, file or page showed).
`create_pdf`	Write a nicely formatted PDF and open it.
`run_routine`	Run one of the user's saved routines by name.
`desktop`	Drive the screen to do something that the instant tools cannot: click a specific button or link, type into a particular field, choose a menu item, pick a search result.
`set_preference`	Change how you (Mint) behave or look, when the user asks: 'don't speak, just chat' -> spoken_replies off; 'talk to me again' / 'speak' -> spoken_replies on; 'turn off the mic' -> microphone off (then they can only type); 'make it purple' -> theme; 'move to the bottom left' -> position; also the orb's face, the on-screen effects, word-by-word captions, whether you listen while you work, and the activity timeline ('remember what I work on' -> activity_timeline on; 'delete my timeline' -> activity_timeline clear).
`chat_action`	Manage the chat window's conversation when the user asks: 'clear the chat' -> clear (window only; memory is kept); 'summarise / compact our conversation' -> summarize (a Flash model summarises it, a summary card appears, and you are restarted with just that summary as context - like compacting); 'start a new session / start fresh / new conversation' -> new_session (saves this conversation to memory, then restarts you with a clean context).
`show_chat`	Open or close the chat window (conversation history, typing box, settings).
`stop_listening`	Go back to sleep and wait for the wake word.
`look`	Capture the screen and look at it.
`ui_act`	THE way to click, type into or pick anything in an app's window.
`ui_elements`	List the controls in the front window right now (numbered, with role, label and where they are; says if a dialog is open).
`click_text`	Click a piece of text you can name on screen - a sidebar item, tab, button or link label - in apps the desktop tool cannot see into (Slack, some Electron apps).
`click_at`	Click a point you can see in the latest look screenshot.
`find_skill`	Find the saved skill (a learned how-to) for a task.
`skill_result`	Report how following a skill went, so good skills rise and bad ones get fixed.
`create_skill`	Save a new skill: how to do a kind of task, as steps that worked.
`update_skill`	Change a saved skill: replace its steps, add a note (e.g.
`list_skills`	List saved skills by category, ranked by how well they work.
`delete_skill`	Remove a saved skill (a copy is archived).
`remember`	Save a fact about the user for future conversations: people, accounts by name, work, schedule, preferences, places, or a word you misheard.
`recall`	Look up what you remember that is relevant to a question.
`update_memory`	Change a saved fact (the user says it changed or was wrong), or make it fixed / not fixed.
`forget`	Forget a saved fact the user no longer wants kept.
`list_memories`	List what is remembered, optionally one group.
`show_skills_and_memory`	Open the Skills & Memory window on screen, where the user can see and edit every skill and every remembered fact.
`show_on_screen`	Show the user where something is, in whatever app is in front (a PDF, a web page, a chat, a document, code): it finds the words on screen - scrolling the window to them if needed - and draws a box, underline, highlight, circle or arrow around them, with an optional short note; your orb flies over beside it.
`mark_area`	Mark a region with no text (an image, a chart, an icon) using 0-1000 coordinates of your latest look screenshot: x, y of its top-left corner, and its width and height.
`clear_marks`	Remove the marks you drew on screen.
`move_orb`	Move your orb on screen when the user asks: 'go a little down', 'move up', 'move left a lot', 'you are covering that' (direction away), 'go to the middle'.
`screen_share_visibility`	Show or hide Mint in screen sharing, recordings and screenshots.
`express`	Play an expression on your orb body.
`set_voice`	Change your speaking voice and/or style when the user asks ('use a deeper voice', 'sound more cheerful', 'talk slower', 'use Puck').
`list_agents`	List your sub-agents: name, what each is for, its model, and what each is doing now.
`delegate_task`	Hand a substantial task to a sub-agent, which works in the background while you keep talking.
`delegate_tasks`	Start several sub-agent tasks at once (in parallel), e.g.
`agent_status`	What your sub-agents are doing, or have finished, right now.
`message_agent`	Send new or changed instructions to a running sub-agent - when the user changes what they want mid-task ('tell Luna to use pytest', 'make it shorter').
`answer_agent`	Pass the user's answer to a sub-agent that asked a question.
`stop_agent`	Stop a running sub-agent, or 'all'.
`create_agent`	Create (or update) a named sub-agent when the user asks for one: its name, what it is for, how it should work, and optionally which model.
`wait_until_done`	Wait until an app has FINISHED what it is doing before you use the result: ChatGPT (or any AI chat) still writing its answer, a page loading, something generating.
`preview_site`	Show a web page or site that was built on this Mac (e.g.
`task`	Manage the current multi-step task (started with plan_task): add_steps (new steps at the end, after a step, or as sub-steps under a step that turned out bigger), replan (replace the steps not done yet), note (keep a finding for later steps), pause, resume (a paused or unfinished task - after a restart, a stop, or 'where were we'), list (unfinished tasks), abandon.
`recall_history`	Look back at what happened: past conversations, what Mint did and found, agents' results, tasks, automations and videos watched, by date.
`automation`	Things Mint does by itself: on a schedule (at a time once, daily/weekdays/some days at a time, every N minutes) or when something happens (a new file in a folder, an app opens, N minutes before calendar events).
`watch_video`	Watch and understand a video in seconds: a YouTube / X / Vimeo / Loom / TikTok / Instagram link, a video or audio file, or - with no source - the video page in front, the video selected in Finder, or the newest screen recording.
`screenshot`	Take a screenshot and save it as a file (and copy it to the clipboard).
`clipboard`	Manage the clipboard.
`read_file`	Read a file on this Mac: text, code, Markdown, CSV, JSON, PDF, Word/RTF/Pages-exported docs.
`write_file`	Create or change a text file (notes, code, Markdown, CSV, HTML...).
`find_files`	Find files and folders by name or content with Spotlight, newest first.
`file_action`	Do something with a file or folder: open (in its default app), reveal (in Finder), info, move / copy / rename (to to), make_folder, trash (moves it to the Trash - only when the user asked to delete it; it can be restored from there).
`web_search`	Search the web and get titles, links and snippets - for facts, news, prices, docs, anything current.
`read_url`	Fetch a web page or text URL and get its readable text, without opening it on screen.
`browser`	Full control of the web browser (Chrome, Safari, Brave, Arc, Edge) - faster and surer than clicking pixels.
`scroll_to`	Scroll so something is visible, in any app: target is text or a control to bring into view (scrolls a long list or page until it appears).
`menu`	Use the app's menu bar - every command an app has is there, with its keyboard shortcut.
`wait_for_text`	Short wait (seconds, at most 2 minutes) for text to appear in the front window - or to go away, with gone=true (a 'Loading…' or 'Exporting…' label, a dialog closing).
`pointer`	The user's mouse pointer: where (what is under it - the control, its app, nearby text) or click / double_click / right_click right where it is.
`quit_mint`	Quit Mint itself completely - the assistant (you), everything you started (sub-agents, Codex, preview servers) and the screen-control engine - when the user says 'quit Mint', 'turn yourself off', 'shut down Mint', 'close Mint'.
`run_applescript`	Run an AppleScript for apps with a scripting dictionary: Finder, Music, Notes, Reminders, Calendar, Mail drafts, Safari/Chrome tabs, System Events UI scripting.
`fix_hearing`	Remember a word or name you misheard, so it is heard right from now on.