Google has revealed a new feature for the Gemini AI app for Mac computers that makes natural voice interactions easier than ever, while also responding in context to the window you’re working within.
Google is commandeering the Fn key on Mac’s keyboard and a long press of that button, typically used to access secondary system shortcuts for the function keys, will summon Gemini into action. The primary function will be advanced dictation, Google says.
In a blog post, Google writes: “By long-pressing the Fn key, you can speak naturally into any window on your desktop. By default, this enables intelligent dictation, letting you easily transcribe your spoken words into clean, polished text — automatically removing the “ums” and “ahs,” catching mid-sentence corrections, and dropping the formatted text instantly at your cursor.”
The functionality gets more powerful when you opt into Gemini reasoning. From there, you can ask the assistant to perform app window-dependent tasks after pressing that Fn button.
Google says these include requests to extract and summarise information from a file, photo or document open on your desktop. Users could also highlight text and say “turn these notes into an executive summary with a TL:DR at the top.” They could also use the command to generate or tweak imagery, such as creating a dark mode version of an illustration.
In a video posted by Google on Wednesday, the company showcased a workflow where users could ask Gemini to check out restaurant options for a company dinner that sit within the budgeting policy outlined in company documentation. It can be summoned to find dishes that work with dietary restrictions of certain team members, then send out an email reminding users of the upcoming event asking them to vote on a restaurant. It can also include an invite for a sweet treat afterwards. It’s quite an impressive process if it works well in real life context.
Google taking over the Mac, eh? Who’d have thunk it.
Read the full article here
