Microsoft Copilot now reads, thinks, and speaks

In a new update to their Copilot AI collection, Microsoft is launching a range of improvements expected to make interacting with devices easier. Among them, Copilot Vision, Think Deeper, Copilot Voice are further designed to offer user-focused, timely and context-aware experience on Windows, iOS, Android, and Web.

All these features are in one way or the other belonging to the broader approach that Microsoft is taking to have Copilot in the middle of people’s daily activities alloting AI solutions merged with the protection of the users’ privacy.

Copilot Vision: A fresh view as well as approach to the use of the World Wide Web

Among new features, there is Copilot Vision, which lets users ask all sorts of questions regarding the content on their screen – be it the text or images on a web page. For example, it is possible to ask for a particular recipe that is depicted in a photo or for furniture advice while using the site. Copilot Vision then operates in manners where it scans through the contents that the users are consuming and responds to the user in real-time, based on the content.

Accessible via Microsoft Edge by typing “@copilot” in the address bar, Copilot Vision is available as part of Copilot Labs, an exclusive opt-in program that offers experimental features to a select group of users. While this feature is not entirely groundbreaking—Google’s Android platform offers similar capabilities—Microsoft claims that Copilot Vision takes things a step further by ensuring that no data is stored or used to train AI models.

Microsoft’s complicated Copilots explained in detail

Microsoft has been keen to emphasize its commitment to privacy with this tool. In a recent blog post, the company highlighted that Copilot Vision deletes all processed data immediately after a conversation, with no audio, images, or text being stored. This is likely in response to growing concerns about data misuse in AI systems, a point Microsoft has been careful to address following legal scrutiny over past incidents involving its AI products.

However, Copilot Vision comes with certain limitations. Currently, the tool cannot analyze content from paywalled or sensitive websites, restricting its functionality to a pre-approved list of web properties. This restriction seems designed to avoid legal challenges similar to the one Microsoft faces with The New York Times, which accused the company of allowing its AI tools to bypass paywalls and access premium content.

Think Deeper: Tackling complex problems with AI reasoning

Another significant addition to the Copilot suite is Think Deeper, a feature that leverages advanced reasoning models to help users navigate complex tasks. Whether it’s solving intricate math problems or comparing multiple home renovation options, Think Deeper is designed to break down and analyze complicated scenarios.

Microsoft has described Think Deeper as a tool that provides step-by-step guidance, making it useful for both everyday challenges and more specialized problems. The company noted that the feature is powered by “reasoning models” that take additional time to respond, ensuring more thorough and accurate answers. Though Microsoft has remained tight-lipped about the specific models used, it is likely a customized version of OpenAI’s o1 model, which Microsoft has fine-tuned for this purpose.

Currently, Think Deeper is available only to a select group of Copilot Labs users in countries like the U.S., U.K., Canada, Australia, and New Zealand, with plans for broader availability in the future.

Microsoft Copilot now reads, thinks, and speaks (Image credit) Copilot Voice: Conversational AI at your command

The last in the line of Microsoft’s AI features is Copilot Voice, the ability to address Copilot and issue commands. Launched in the USA, UK, Canada, Australia, and New Zealand first, Copilot Voice extends the ethos of Copilot with answers verbalized.

Like OpenAI’s voice in ChatGPT, Copilot Voice is capable of sensing voice tone, and respond accordingly. Some features of the application make it even more engaging: for example, users can pause Copilot during the conversation or interrupt its answers in the middle of the sentence. The feature of the application is the four synthetic voice which are very natural and realistic to the touch.

However, there are limits on how much users can engage with Copilot Voice. Microsoft has implemented a time-based usage limit, with Copilot Pro subscribers gaining access to additional minutes depending on overall demand.

Personalization: Tailoring Copilot to your preferences

In addition to the major feature updates, Microsoft is also introducing personalization options to Copilot. These settings allow the AI to tailor its suggestions based on the user’s past interactions, drawing insights from activity across other Microsoft apps and services. According to Microsoft, this will help Copilot offer more relevant recommendations and faster access to useful features.

However, the rollout of personalization is being delayed in certain regions. For now this feature is not going to be available in EU or UK anymore as Microsoft continues to weigh options on how to adapt to the tight data protection laws in these regions. It follows the strategy that leading IT enterprises like Apple and Meta as well as other firms have executed when rolling out AI solutions in the European Union.

Microsoft has expressed that the firm is looking forward to providing personalization incorporated into the users in these areas and at the same time satisfying the legal provisions such as the Digital Services Act, which regulates the use of AI and data protection in EU. In the past, the company has received certain regulatory issues, particularly in the assortment EEA’s recent admonition to Microsoft that it might be penalized for not giving enough details about its generative AI goods.Privacy and responsibility at the core

Throughout the rollout of these new features, Microsoft has maintained that privacy and user feedback are at the heart of its development process. The company has stressed that none of the data processed by Copilot Vision, Think Deeper, or Copilot Voice is used for AI training or stored beyond the immediate session.

Featured image credit: Microsoft