Security professional Bruce Schneier argues that large language models have the same vulnerability as phones in the 1970s exploited by John Draper.

"Data and control used the same channel," Schneier writes in Communications of the ACM. "That is, the commands that told the phone switch what to do were sent along the same path as voices." Other forms of prompt injection involve the LLM receiving malicious instructions in its training data. Another example hides secret commands in Web pages. Any LLM application that processes emails or Web pages is vulnerable. Attackers can embed malicious commands in images and videos, so any system that processes those is vulnerable. Any LLM application that interacts with untrusted users — think of a chatbot embedded in a website — will be vulnerable to attack. It's hard to think of an LLM application that isn't vulnerable in some way.

Individual attacks are easy to prevent once discovered and publicized, but there are an infinite number of them and no way to block them as a class. The real problem here is the same one that plagued the pre-SS7 phone network: the commingling of data and commands. As long as the data — whether it be training data, text prompts, or other input into the LLM — is mixed up with the commands that tell the LLM what to do, the system will be vulnerable. But unlike the phone system, we can't separate an LLM's data from its commands. One of the enormously powerful features of an LLM is that the data affects the code. We want the system to modify its operation when it gets new training data. We want it to change the way it works based on the commands we give it. The fact that LLMs self-modify based on their input data is a feature, not a bug. And it's the very thing that enables prompt injection.

Like the old phone system, defenses are likely to be piecemeal. We're getting better at creating LLMs that are resistant to these attacks. We're building systems that clean up inputs, both by recognizing known prompt-injection attacks and training other LLMs to try to recognize what those attacks look like. (Although now you have to secure that other LLM from prompt-injection attacks.) In some cases, we can use access-control mechanisms and other Internet security systems to limit who can access the LLM and what the LLM can do. This will limit how much we can trust them. Can you ever trust an LLM email assistant if it can be tricked into doing something it shouldn't do? Can you ever trust a generative-AI traffic-detection video system if someone can hold up a carefully worded sign and convince it to not notice a particular license plate — and then forget that it ever saw the sign...?

Someday, some AI researcher will figure out how to separate the data and control paths. Until then, though, we're going to have to think carefully about using LLMs in potentially adversarial situations...like, say, on the Internet.
Schneier urges engineers to balance the risks of generative AI with the powers it brings. "Using them for everything is easier than taking the time to figure out what sort of specialized AI is optimized for the task.

"But generative AI comes with a lot of security baggage — in the form of prompt-injection attacks and other security risks. We need to take a more nuanced view of AI systems, their uses, their own particular risks, and their costs vs. benefits."

A blind worker for the National Federation of the Blind said Sonos had a reputation for making products usable for people with disabilities, but that "Overnight they broke that trust," according to the Washington Post.

They're not the only angry customers about the latest update to Sonos's wireless speaker system. The newspaper notes that nonprofit worker Charles Knight is "among the Sonos die-hards who are furious at the new app that crippled their options to stream music, listen to an album all the way through or set a morning alarm clock." After Sonos updated its app last week, Knight could no longer set or change his wake-up music alarm. Timers to turn off music were also missing. "Something as basic as an alarm is part of the feature set that users have had for 15 years," said Knight, who has spent thousands of dollars on six Sonos speakers for his bedroom, home office and kitchen. "It was just really badly thought out from start to finish." Some people who are blind also complained that the app omitted voice-control features they need.

What's happening to Sonos speaker owners is a cautionary tale. As more of your possessions rely on software — including your car, phone, TV, home thermostat or tractor — the manufacturer can ruin them with one shoddy update... Sonos now says it's fixing problems and adding back missing features within days or weeks. Sonos CEO Patrick Spence acknowledged the company made some mistakes and said Sonos plans to earn back people's trust. "There are clearly people who are having an experience that is subpar," Spence said. "I would ask them to give us a chance to deliver the actions to address the concerns they've raised." Spence said that for years, customers' top complaint was the Sonos app was clunky and slow to connect to their speakers. Spence said the new app is zippier and easier for Sonos to update. (Some customers disputed that the new app is faster.)

He said some problems like Knight's missing alarms were flaws that Sonos found only once the app was about to roll out. (Sonos updated the alarm feature this week.) Sonos did remove but planned to add back some lesser-used features. Spence said the company should have told people upfront about the planned timeline to return any missing functions.
In a blog post Sonos thanked customers for "valuable feedback," saying they're "working to address them as quickly as possible" and promising to reintroduce features, fix bugs, and address performance issues. ("Adding and editing alarms" is available now, as well as VoiceOver fixes for the home screen on iOS.)

The Washington Post adds that Sonos "said it initially missed some software flaws and will restore more voice-reader functions next week."

An anonymous reader quotes a report from BleepingComputer: The Securities and Exchange Commission (SEC) has adopted amendments to Regulation S-P that require certain financial institutions to disclose data breach incidents to impacted individuals within 30 days of discovery. Regulation S-P was introduced in 2000 and controls how some financial entities must treat nonpublic personal information belonging to consumers. These rules include developing and implementing data protection policies, confidentiality and security assurances, and protecting against anticipated threats.

The new amendments (PDF) adopted earlier this week impact financial firms, such as broker-dealers (funding portals included), investment firms, registered investment advisers, and transfer agents. The modifications were initially proposed in March of last year to modernize and improve the protection of individual financial information from data breaches and exposure to non-affiliated parties. Below is a summary of the introduced changes:

- Notify affected individuals within 30 days if their sensitive information is, or is likely to be, accessed or used without authorization, detailing the incident, breached data, and protective measures taken. Exemption applies if the information isn't expected to cause substantial harm or inconvenience to the exposed individuals.
- Develop, implement, and maintain written policies and procedures for an incident response program to detect, respond to, and recover from unauthorized access or use of customer information. This should include procedures to assess and contain security incidents, enforce policies, and oversee service providers.
- Expand safeguards and disposal rules to cover all nonpublic personal information, including that received from other financial institutions.
- Require documentation of compliance with safeguards and disposal rules, excluding funding portals.
- Align annual privacy notice delivery with the FAST Act, exempting certain conditions.
- Extend safeguards and disposal rules to transfer agents registered with the SEC or other regulatory agencies.

Two university students discovered a security flaw in over a million internet-connected laundry machines operated by CSC ServiceWorks, allowing users to avoid payment and add unlimited funds to their accounts. The students, Alexander Sherbrooke and Iakov Taranenko from UC Santa Cruz, reported the vulnerability to the company, a major laundry service provider, in January but claim it remains unpatched. TechCrunch adds: Sherbrooke said he was sitting on the floor of his basement laundry room in the early hours one January morning with his laptop in hand, and "suddenly having an 'oh s-' moment." From his laptop, Sherbrooke ran a script of code with instructions telling the machine in front of him to start a cycle despite having $0 in his laundry account. The machine immediately woke up with a loud beep and flashed "PUSH START" on its display, indicating the machine was ready to wash a free load of laundry.

In another case, the students added an ostensible balance of several million dollars into one of their laundry accounts, which reflected in their CSC Go mobile app as though it were an entirely normal amount of money for a student to spend on laundry.

An anonymous reader quotes a report from Ars Technica: An Arizona woman has been accused of helping generate millions of dollars for North Korea's ballistic missile program by helping citizens of that country land IT jobs at US-based Fortune 500 companies. Christina Marie Chapman, 49, of Litchfield Park, Arizona, raised $6.8 million in the scheme, federal prosecutors said in an indictment unsealed Thursday. Chapman allegedly funneled the money to North Korea's Munitions Industry Department, which is involved in key aspects of North Korea's weapons program, including its development of ballistic missiles. Part of the alleged scheme involved Chapman and co-conspirators compromising the identities of more than 60 people living in the US and using their personal information to get North Koreans IT jobs across more than 300 US companies.

As another part of the alleged conspiracy, Chapman operated a "laptop farm" at one of her residences to give the employers the impression the North Korean IT staffers were working from within the US; the laptops were issued by the employers. By using proxies and VPNs, the overseas workers appeared to be connecting from US-based IP addresses. Chapman also received employees' paychecks at her home, prosecutors said. Federal prosecutors said that Chapman and three North Korean IT workers -- using the aliases of Jiho Han, Chunji Jin, Haoran Xu, and others -- had been working since at least 2020 to plan a remote-work scheme. In March of that year, prosecutors said, an individual messaged Chapman on LinkedIn and invited her to "be the US face" of their company. From August to November of 2022, the North Korean IT workers allegedly amassed guides and other information online designed to coach North Koreans on how to write effective cover letters and resumes and falsify US Permanent Resident Cards.

Under the alleged scheme, the foreign workers developed "fictitious personas and online profiles to match the job requirements" and submitted fake documents to the Homeland Security Department as part of an employment eligibility check. Chapman also allegedly discussed with co-conspirators about transferring the money earned from their work. Chapman was arrested Wednesday. It wasn't immediately known when she or Didenko were scheduled to make their first appearance in court. If convicted, Chapman faces 97.5 years in prison, and Didenko faces up to 67.5 years.

Palo Alto Networks is acquiring IBM's QRadar cloud software and migrating customers to its Cortex Xsiam platform as part of a broader partnership aimed at expanding its consulting capabilities and customer base. The sum of the deal was not disclosed. CNBC reports: The move normally takes one to three months, Nikesh Arora, Palo Alto's CEO, told CNBC. Also, IBM will train more than 1,000 of its consulting employees on Palo Alto's products. [...] For IBM, a more robust lineup of contemporary security tools for consulting might help the company deliver on its stated goal of revenue growth in the mid-single digits for 2024. In the first quarter, revenue increased 3%, with a 2% bump in the consulting segment.

Palo Alto is growing much faster than IBM. In the January quarter, revenue jumped 19%. The company will report results for the latest quarter on Monday. Palo Alto more than doubled in value last year and its stock is up 6% year to date, lifting the company's market cap past $100 billion. The stock rose more than 1% in extended trading. IBM is up close to 5% this year and is now valued at $154 billion. The companies said the transaction should close by the end of September, subject to regulatory approval and other conditions. [...] IBM will continue to sell its QRadar software for use in on-premises data centers. At the same time, IBM will suggest that clients using it consider switching to Palo Alto's Cortex Xsiam.

Japan's Toshiba said on Thursday it will cut up to 4,000 jobs domestically as the industrial conglomerate accelerates restructuring under new ownership. From a report: Toshiba delisted in December due to a $13 billion takeover by a consortium led by private equity firm Japan Industrial Partners, capping a decade of scandal and upheaval. The consortium's efforts to engineer a turnaround at Toshiba are seen as a test for private equity in Japan, which used to be seen as "hagetaka" or vultures due to its rapacious reputation. The restructuring amounts to up to 6% of Toshiba's domestic workforce. The company also said it would relocate office functions from central Tokyo to Kawasaki, west of the capital, and target an operating profit margin of 10% in three years.

In an op-ed on Windows Central, the site's co-managing editor Jez Corden laments Microsoft's "short-sighted" decision-making and "inconsistent" investment in its products and services, which he argues has led to a loss of trust among customers and missed opportunities in the tech industry. Despite Microsoft's advancements in AI and cloud computing, the company has made "baffling" decisions such as shutting down Windows Phone, under-investing in Xbox, and canceling promising Surface products.

The author argues that Microsoft's lack of commitment to security, customer support, and long-term quality has "damaged" its reputation and hindered its potential for growth. Examples include recent hacking scandals, poor customer service experiences, and the aggressive promotion of Microsoft Edge at the expense of user choice. The author also expresses concern over Microsoft's handling of the Xbox brand, particularly the decision to release exclusive games on PlayStation, which could undermine the reasons for customers to choose Xbox. The op-ed concludes that while Microsoft has the potential to be a leader in the tech industry, its pattern of short-sighted decisions and failure to learn from past mistakes has led to a growing sense of doubt among its customers and observers.

The recent surge in bitcoin prices has the phones at crypto wallet recovery firms ringing off the hook, as retail investors locked out of their digital vaults make frantic calls to regain access to their accounts. From a report: Cryptocurrencies exist on a decentralized digital ledger known as blockchain and investors may opt to access their holdings either through a locally stored software wallet or a hardware wallet, to avoid risks related to owning crypto with an exchange, as in the case of the former FTX. Losing access to a crypto wallet is a well-known problem. Investors forgetting their intricate passwords is a primary reason, but loss of access to two-factor authentication devices, unexpected shutdowns of cryptocurrency exchanges and cyberattacks are also common.

Wallet passwords are usually alphanumeric and the wallet provider also offers a set of randomized words, known as "seed phrases," for additional security - both these are known only to the user. If investors lose the passwords and phrases, access to their wallets is cut off. With bitcoin prices regaining traction since last October and hitting a record high of $73,803.25 in March, investors seem to be suffering from a classic case of FOMO, or the fear of missing out. Reuters spoke to nearly a dozen retail investors who had lost access to their crypto wallets. Six of them contacted a recovery services firm and managed to regain access to their holdings.

An anonymous reader shares a report: There are concerning reports on Reddit that Apple's latest iOS 17.5 update has introduced a bug that causes old photos that were deleted -- in some cases years ago -- to reappear in users' photo libraries. After updating their iPhone, one user said they were shocked to find old NSFW photos that they deleted in 2021 suddenly showing up in photos marked as recently uploaded to iCloud. Other users have also chimed in with similar stories. "Same here," said one Redditor. "I have four pics from 2010 that keep reappearing as the latest pics uploaded to iCloud. I have deleted them repeatedly." "Same thing happened to me," replied another user. "Six photos from different times, all I have deleted. Some I had deleted in 2023." More reports have been trickling in overnight. One said: "I had a random photo from a concert taken on my Canon camera reappear in my phone library, and it showed up as if it was added today."

The FBI has seized the notorious BreachForums hacking forum that leaked and sold stolen corporate data to other cybercriminals. From a report: The seizure occurred on Wednesday morning, soon after the site was used last week to leak data stolen from a Europol law enforcement portal. The website is now displaying a message stating that the FBI has taken control over it and the backend data, indicating that law enforcement seized both the site's servers and domains. [...] The seizure message also shows the two forum profile pictures of the site's administrators, Baphomet and ShinyHunters, overlaid with prison bars.

At Google I/O on Tuesday, Google previewed a feature that will alert users to potential scams during a phone call. TechCrunch reports: The feature, which will be built into a future version of Android, uses Gemini Nano, the smallest version of Google's generative AI offering, which can be run entirely on-device. The system effectively listens for "conversation patterns commonly associated with scams" in real time. Google gives the example of someone pretending to be a "bank representative." Common scammer tactics like password requests and gift cards will also trigger the system. These are all pretty well understood to be ways of extracting your money from you, but plenty of people in the world are still vulnerable to these sorts of scams. Once set off, it will pop up a notification that the user may be falling prey to unsavory characters.

No specific release date has been set for the feature. Like many of these things, Google is previewing how much Gemini Nano will be able to do down the road sometime. We do know, however, that the feature will be opt-in.

A portal linking New York City to Dublin via a livestream has been temporarily shut down after inappropriate behavior ensued, according to the Dublin City Council. From a report: Less than a week after the 24/7 visual art installation was put in place, officials have opted to close it down temporarily after people began to flash each other, grind on the portal, and one person even shared pictures of the twin tower attack to people in New York City. Alternatively, the portal had also been the site of reunions with old friends and even a proposal, with many documenting their experience with the installation online.

The Dublin City Council said that although those engaged in the inappropriate behavior were few and far between, videos of said behavior went viral online. "While we cannot control all of these actions, we are implementing some technical solutions to address this and these will go live in the next 24 hours," the council said in a Monday statement. "We will continue to monitor the situation over the coming days with our partners in New York to ensure that portals continue to deliver a positive experience for both cities and the world."

Marc Whitten, Unity Create's chief product and technology officer, is stepping down on June 1, 2024, following the company's contentious Runtime Fee policy. Whitten will assist with the transition until December 31, 2024. The now-discarded Runtime Fee, announced in September 2023, faced severe backlash from developers who viewed it as a punitive per-install tariff. Unity reworked the fee and acknowledged its lack of communication with developers. CEO John Riccitiello also departed in October 2023, succeeded by Matthew Bromberg. Upon resignation, Whitten will receive a total of $814,801 in various payouts and benefits.

A panel of judges in the Netherlands has found Alexey Pertsev, one of the developers behind crypto anonymizing tool Tornado Cash, guilty of money laundering. Wired: Over the course of two days in March, the Russian national was tried on the allegation that the tool he developed had allowed criminals -- among them hackers with ties to North Korea -- to freely launder $1.2 billion in stolen cryptocurrency. "The management of Tornado Cash welcomed the bank robbers with open arms," the prosecutors wrote in a March court filing.

Dutch judges sentenced Pertsev to five years and four months in prison on Tuesday, which was the term requested by prosecutors in the case. "With Tornado Cash, the defendant created a shortcut for financing crimes and terrorism," said the court in a statement, translated from Dutch. "He chose to look away from the abuse and did not take any responsibility." The purpose of tools like Tornado Cash, known as crypto mixers or tumblers, is to mask the origin and destination of users' coins. Funds belonging to many parties are pooled, jumbled up, and spat out into brand-new wallets, by which time it is no longer clear whose crypto is whose. These services are promoted as a way to improve the level of privacy available to crypto users, but have been readily co-opted for the purpose of money laundering.

On August 8, 2022, Tornado Cash was sanctioned in the United States, making it illegal for US citizens to use the service. Any product that "indiscriminately facilitates anonymous transactions," wrote the US Treasury's Office of Foreign Assets Control, represents a "threat to US national security." Two days later, Pertsev was arrested in the Netherlands, where he resided. Money laundering activity, the Dutch prosecutors claim, accounted for more than 30 percent of the funds that passed through Tornado Cash between 2019 and 2022. [...] Pertsev built his defense on the argument that Tornado Cash, which remains in operation, is under nobody's control -- including his own -- as a piece of software that runs on the Ethereum blockchain, a distributed network of computers. Further reading: Coinbase Employees and Ethereum Backers Sue US Treasury Over Tornado Cash Sanctions (September 2022).