Microsoft's AI-Powered Copy and Paste Can Now Use On-Device AI (theverge.com) 43
An anonymous reader shares a report: Microsoft is upgrading its Advanced Paste tool in PowerToys for Windows 11, allowing you to use an on-device AI model to power some of its features. With the 0.96 update, you can route requests through Microsoft's Foundry Local tool or the open-source Ollama, both of which run AI models on your device's neural processing unit (NPU) instead of connecting to the cloud.
That means you won't need to purchase API credits to perform certain actions, like having AI translate or summarize the text copied to your clipboard. Plus, you can keep your data on your device.
That means you won't need to purchase API credits to perform certain actions, like having AI translate or summarize the text copied to your clipboard. Plus, you can keep your data on your device.
Better data leaks! (Score:5, Insightful)
They will likely still listen to and record everything. The main difference is that you pay the power used. But now you have the illusion of the data staying on your device. Nice!
See also: https://thehackernews.com/2025... [thehackernews.com]
Re: Better data leaks! (Score:1)
"you pay the power used."
Will the difference be too insignificant to notice?
Re: (Score:2)
Additionally on a laptop you pay for local prompt execution by sucking the life out of the battery, users will notice.
Re: Better data leaks! (Score:1)
Why do you think decoupling polices explicitly separate supply and demand? Was it motivated by vast electricity oversupply such that utilities would have to sell you less efficient machines to make profits?
Re: (Score:2)
NPUs were developed so you can do AI efficiently, such that you won't, for example, drain your laptop battery with it.
Re: (Score:2)
Efficiency and power are two sides of the same coin. A GPU can accelerate graphics rendering by offloading some of the rendering logic to hardware. Google Maps can run fast with barely any battery drain. But a AAA game is still going to drain your battery. It can just look prettier while doing so.
Re: (Score:2)
Efficiency is the ratio between work done and power used. A GPU has a very good ratio for graphics and a good radio for AI, a NPU has a very good ratio for AI (but is currently more limited in total capacity). Both allow to perform certain operations with less power than a CPU would require, and for a NPU even less than the GPU would require. So I wouldn't say it is two sides, but one is the derivative of the other.
Re: (Score:2)
Difficult to say. I have no idea how much power an "NPU" draws.
Re: Better data leaks! (Score:1)
Ahh, the optimism of n00bs - I kind of miss it.
Zuboff states that surveillance capitalism "unilaterally claims human experience as free raw material for translation into behavioural data [which] are declared as a proprietary behavioural surplus, fed into advanced manufacturing processes known as 'machine intelligence', and f [wikipedia.org]
Re: Better data leaks! (Score:1)
Are they serving me meat ads because they predict I'll give up vegetarianism, and how hallucinatory is that?
Re: (Score:2)
You just do not know yet that you _will_ give up vegetarianism! They already know.
Re: (Score:2)
They will likely still listen to and record everything. The main difference is that you pay the power used. But now you have the illusion of the data staying on your device. Nice!
Powertoys, Microsoft Foundry Local, and Ollama are all open source, so show us the code where they are 'still listening to and record [sic] everything':
https://github.com/microsoft/P... [github.com]
https://github.com/ollama/olla... [github.com]
https://github.com/microsoft/F... [github.com]
See also: https://thehackernews.com/2025... [thehackernews.com]
That is exactly the issue this is solving for.
Re: (Score:2)
They will likely still listen to and record everything. The main difference is that you pay the power used. But now you have the illusion of the data staying on your device. Nice!
Powertoys, Microsoft Foundry Local, and Ollama are all open source, so show us the code where they are 'still listening to and record [sic] everything': ...
That is exactly the issue this is solving for.
WTH are you trying to say? OP said, "They will likely still listen to and record everything." That doesn't state anything about ex-filtrating that data. So if the processing now has an option to be done locally, that means it's doing it one way or the other, does it not?
We're already aware of troves of data that does get sent to them, and I suspect you've heard of MS Recall? If they can justify regularly saving full screenshots, then could easily justify saving a transcript of all the audio they record.
Re: (Score:2)
WTH are you trying to say? OP said, "They will likely still listen to and record everything." That doesn't state anything about ex-filtrating that data. So if the processing now has an option to be done locally, that means it's doing it one way or the other, does it not?
No, it doesn't mean they are going it "one way or another". This is an feature you can turn on or off in powertoys. And since it's open source, you can audit it yourself if you have doubts.
We're already aware of troves of data that does get sent to them, and I suspect you've heard of MS Recall?
You mean the MS Recall that processes data 100% on device and doesn't send any of it to Microsoft? Now I have to ask WTF are you trying to say?
If they can justify regularly saving full screenshots, then could easily justify saving a transcript of all the audio they record.
You just making shit up to strawman now?
Going back to this
That doesn't state anything about ex-filtrating that data.
The last line of OP's post is to a article titled "New Research: AI Is Already the #1 Data Exfiltration Channel i
Summarize the text copied to your clipboard (Score:2)
I think they stopped listening to themselves long ago at this point.
An Obvious Development (Score:5, Insightful)
They want everyone to use their AI tools.
Certain government, health care, and financial organizations are legally restricted from sharing data. Having worked in such a place before, they'll blanket ban AI rather than risk the unauthorized transmission of protected data. This functionality offers an alternative that they can accept.
If Microsoft wants their AI features to be adopted widely, they basically had no choice but to implement a local model. The fact that it's cheaper to use the customer's electricity is probably meaningless to them; they want to capture the userbase.
Re: (Score:2)
I get the driver behind this. Also trade-secrets, etc.
But is this really a local model and is no data transferred? There is some fuzzy language in "use an on-device AI model to power some of its features". What bout other features? What about if it "powers" the features, bit the data is still sent home? The whole thing looks like a lie by misdirection to me.
Re: (Score:2)
What about if it "powers" the features, bit the data is still sent home? The whole thing looks like a lie by misdirection to me.
Agreed. My first thought was that likely meant they'd do local transcription, then use that for other things. That's how most audio stuff gets processed anyway, and that would skip the bandwidth heavy step of transferring the full audio (when needed, they can just transfer the transcription text). If that sounds like I'm gung ho about it, I'm not. But it is still an improvement in user control; Maybe we could probably swap the local transcription engine with a no op?
Re: (Score:2)
Re: (Score:2)
No, they want everyone to be used and tracked.
Re: (Score:2)
Having worked in such a place before, they'll blanket ban AI rather than risk the unauthorized transmission of protected data
Now back in reality, what they'll do is blank ban all AI other than CoPilot and force users to use CoPilot, since Microsoft provides AI tools that specific follow data classification systems in place in the corporate Windows world, and that includes those used by their cloud services ... services that very much approved for use in government, healthcare and financial organisations, and explicitly compliant with laws such as HIPPA.
Now repeat after me: What you as a pleb can get is not the same thing a corpor
AI powered copy-paste? (Score:5, Insightful)
I see they mention translation, but that feels like a weird thing to want to do with a copy-paste. If I turn this on and copy some code from an older project and paste it into a new project, will it automatically refactor it into something that doesn't function but looks prettier? It just seems like copy-paste is one of the simplest, dumbest things a computer helps us with in a way that's unobtrusive. Is anyone asking for "smart" copy-paste beyond "paste as plain text?"
Re: (Score:3)
The article has a screenshot:
https://devblogs.microsoft.com... [microsoft.com]
Re:AI powered copy-paste? (Score:4, Insightful)
Is anyone asking for "smart" copy-paste beyond "paste as plain text?"
In case you're unaware, copy/paste, even on Linux, is context aware and does smart things. No LLM integration I'm aware of, but it's not just plain text. You can try it. In a (possibly Linux) web browser, copy a big chunk of some webpage, then paste that into a new gmail message (or somewhere that accepts context-type: text/html). Bullet points, tables, font sizes, etc.. will be retained. Those aren't plain text.
Even more simple of an example: you can copy/paste images.
Given that there's already some hooks in there to do things with the clipboard data based on context, it's not *too* big of a stretch to see them try to hook in other stuff... not that I want it. I frequently find myself pasting to a temporary vim text file just to ensure I'm removing all the unseen markup before pasting a message (email, comments, IMs, etc..). I should probably setup a shortcut to paste as plain text someday.
Wot? (Score:4, Funny)
US: Hey MS, we just wanna type in a text file, which app do I use?
MS: Well, you can use our AI Powered PowerText-to-SuperFile app. It does it all, writes your text for you.
US: No, no, no. We want to write the text.
MS: That's not the AI-Way, you really want to use our AI Powered app, it will answer any questions you have while it writes your text.
US: Errrrmmmm.....cannot I just write text file on my own?
MS: Uh-oh, I guess not. We do not have an AI Powered app to do that.
US: Nevermind, Linux and MacOS will do just fine. Now go away.
MS: Ooops, too late, our AI Powered Agent has already accessed you bank account and transferred the required sum into our coffers for wanting to use an un-AI Powered app.
US: Eat death!
MS: So you want to commit suicide. May we suggest our AI Powered Schedule-Your-Death app?
Re: (Score:2)
Nano works fine though
Re: (Score:2)
US: Hey MS, we just wanna type in a text file, which app do I use?
You do. "We" (meaning the normal general users of the world) don't. Install a special purpose tool for the job like Notepad++ for your strange edge case of needing to manually write just text to a file.
In all seriousness though your post gave me a chuckle. Reminded me of the suicide booths in Futurama.
Great (Score:2, Insightful)
Re: Great (Score:1)
Remember when GUIs were bloat, then everyone got used to it?
Any opportunity to fuck things up (Score:5, Insightful)
The Copy-Paste mechanism has been a well-defined, widely-understood basic action for personal computers since at least 1984, arguably before.
The defining characteristics are either duplicating or moving one or more digital objects to a new position in a document.
Notice those verbs. Nothing about rewriting/translating/making different. The closest the mechanism gets to that is, for inter-application transfer, there can be content negotiation to deal with format issues.
Just waiting for the first time this "fixes" something in a mortgage or a sentencing memo...
Re: (Score:2)
Working technology doesn't sell subscriptions to services that fix the things they broke.
Re: (Score:2)
In what way has copy-past been well defined since 1984. What is it you're copying and what is it you're pasting. In 1984 it was easy, that was text. It's not 1984. It's 2025. What is your text? Does it have formatting? Does it contain unicode characters? How does the target app handle the discrepancy. Are you copying the words, the underlying character codes which make them, the look of them, in what format is that define?
Precisely none of this is well-defined, in any OS from any vendor, largely because in
I see you don't read posts you respond to. (Score:2)
Fascinating (Score:2)
Microsoft is all in (Score:2)
on hallucination.
Re: (Score:2)
The always have been. Unfortunately, most of their customers are too. It just has gotten more concrete with LLMs now.
As if rich copy paste wasn't enough... (Score:2)
You know this situation well. You try to do a simple copy paste from a website. You highlight the text, ctrl+c, switch windows, ctrl+v... BOOM! It copies over the text, the font, the formatting, the background, captions, urls, alt texts and half the source code.
Welcome to rich/context-aware copy-paste. We all know it and hate it. ...and now you're telling us that things are about to get much, much worse?!
Better not copy-paste your passwords then... (Score:2)
Those still copy-pasting their passwords will be in for a hell of a surprise one day...
"Help me CortanaPilot, I forgot my password!" (Score:2)
"Don't worry Dave, just press CTRL-V in the password box"