[Ally is available exclusively for Envision Glasses users with a 2024 Pass, 2025 Pass, or Professional Edition]

This version brings the much awaited Ally to Envision Glasses. While this beta version is an early step in Ally’s journey, it already delivers remarkable capabilities. Releasing Ally at this stage allows early access to its powerful features, as we continue refining and expanding its potential. Here’s a detailed overview of Ally’s current capabilities and what’s planned for the future.

What is Ally?

Ally is your personal, conversational assistant. It intuitively understands your questions and uses advanced AI tools and integrations to deliver the best answers within seconds. Ally is designed to be intuitive, snappy, and the most accessible assistant available.

How to Use Ally

1. Navigate to the Ally feature in the Main Menu.
2. Double-tap to open it and hear Ally’s startup sound and greeting.
3. Ask your question, and Ally will answer.
4. Interrupt Ally at any time by talking—whether for follow-up questions or corrections.
5. Use a one-finger double-tap to mute the mic in noisy environments and double-tap again to unmute.
6. Adjust Ally’s volume by swiping forwards or backwards during the call.

What You Can Do with Ally (Beta) on Envision Glasses Version 2.9.0

General Information:

• “What’s the capital of France?”
• “What’s the square root of 625?”

Current Information:

• “What movies are releasing this weekend?”
• “Tell me the top news stories from last night.”

Weather:

• “How cold is it going to get tomorrow in San Francisco?”
• “Do I need to take my umbrella to Amsterdam tomorrow?”

Vision Assistance:

• “Describe what I am holding.”
• “What does it say on the notice board ahead?”
• “Do you spot an empty chair?”

Work Assistance:

• “Can you read this manual and tell me what the contact email is?”
• “Can you tell me if there are any boxes blocking the aisle?”

Study Help:

• “Can you help me summarize this chapter?”
• “Can you quiz me with trivia about Orientation and Mobility exams?”

Cooking:

• “What can I replace an egg with while baking a cake?”
• “How do I make a shakshuka?”

Translations:

• “Can you tell me what this road sign says in English?”
• “Teach me how to say ‘Good Morning’ in German.”

Creativity:

• “Can you help me write a poem about sunsets?”
• “Help me come up with a new slogan for my business.”

These examples highlight Ally’s capabilities as a conversational assistant. The possibilities are vast, so feel free to ask away!

What’s Next: Ally’s Roadmap

Account Linking:

• Connect Ally on the glasses with the Ally app on your phone or the web for seamless access across platforms.

Personalization:

• Customize Ally’s personality, voice, and answer style.
• Provide personal information to give Ally context for more tailored answers.

Multilingual Support:

• Ally will soon support multiple languages, responding to queries in your preferred language.

Calendar Integration:

• Link your calendar to Ally to ask about upcoming events and appointments.

Scan Text:

• Leverage advanced document detection and text recognition for efficient document reading and browsing.

Shortcuts:

• Create custom shortcuts for frequently asked questions or prompts, such as “Give me a short description of what you see” or “Do you see any obstacles in front of me?”

These updates will roll out through future software updates, and we’ll keep you informed as they become available. Ally is poised to become the primary interface for accessing the full potential of Envision Glasses, simplifying the experience for all users, including those less familiar with technology.

Share Your Feedback

We value your insights! Join our beta Slack community to share feedback and ideas that can shape Ally’s development.

We’re thrilled for you to experience Ally on Envision Glasses—a live, hands-free, personal, and conversational assistant designed to redefine accessibility.

‍

• Fix TTS issue for Icelandic and Slovenian.

We’re excited to introduce Smart Detection in this release, bringing even more functionality to your Envision Glasses.
Smart Detection [2024 pass users only]: Replacing the old Layout Detection, Smart Detection enhances document reading by adding alt-text descriptions for visuals like images, graphs, charts, and tables. For tables, it identifies rows and columns to ensure a clear understanding of the data and provides a brief summary of the content whenever encountered.
Smart Detection also maintains the correct reading order for columns and detects headings, giving you much richer context when reading textbooks, magazines, or reports. To enable Smart Detection, simply activate it in the context menu of the Scan Text feature.

• We have updated all recent changes in the Spanish language string-specific translations.
  

‍

In this release, we’ve made several improvements to enhance your experience with the Envision Glasses:

• Improved Text-to-Speech Engine [Pass users only]: The voice now sounds smoother and more natural, providing a better listening experience.
• Manual Voice Command Input: You can now manually send voice commands by pressing the hinge button again as soon as you finish speaking. This eliminates the wait time for automatic submission and is especially useful in noisy environments.
• Enhanced Cash Recognition: The Recognize Cash feature has been updated with improvements and bug fixes for more accurate results.
• Battery Percentage Announcements: Based on your feedback, the glasses now announce battery levels more frequently to keep you informed and avoid unexpected shutdowns.

Thank you for your continued support and feedback!

• An issue with Describe Scene (Beta) and Instant Text Offline in Japanse has been fixed.

‍

Introducing ‘Instant Results’: achieve much faster scene descriptions with the ‘Describe Scene’ feature and get faster replies with Ask Envision inside the Reader and and after doing a scene description.

‘Instant Results’ is available, and turned on by default, for everyone with the 2024 Pass and the Unlimited Pass.

‍

• This version includes a correction of the ‘Describe Scene’ feature.

• New user interface for language selection.
• Bulgarian language support.
• An issue has been fixed where the steps to connect an Aira account to Envision Glasses were not indicated.
• An issue with instant text language selection has been fixed.

‍

• This update fixes the issue that prevented the activation of Multilingual Voice Commands.
  

Introducing a generational leap of Describe Scene Feature in Version 2.5.0

This release is for everyone that has the 2024 Software Updates Pass.

‍We are thrilled to announce the release of version 2.5.0 for Envision Glasses, featuring a major upgrade to the ‘Describe Scene’ capability. This update marks a significant leap forward from the beta version, delivering unparalleled accuracy and richness in image description.

‍Key Enhancements:

1. Improved Accuracy: Drastic reduction in hallucinations, ensuring highly accurate descriptions.
2. Richer Descriptions: The level of detail in the descriptions now closely matches, and sometimes surpasses, human-generated descriptions.

How to Use the New Describe Scene:

• Ensure your Envision Glasses are updated to version 2.5.0.
• Navigate to ‘Describe Scene’ in the Main Menu. The ‘Beta’ tag is now removed, indicating its full-fledged functionality.
• Double-tap to activate. The glasses will take a picture after 3 beeps and then process the image to provide a concise, 'Alt-text' sized description of the captured scene.
• You can then press and hold the hinge-button for 2 seconds and Ask Envision interactive follow-up questions. Here are a few examples of what you can do.

Interactive Follow-Up Questions:

• Richer Descriptions: Request a more detailed description by saying, 'Describe this image in as much detail as possible.'
• Specific Elements: Ask for detailed descriptions of particular aspects, like 'Describe the clothing of the man in more detail?' or 'What items are on the table?'
• Detail Inquiries: Get specific details about objects by asking, 'What kind of a plant is that?' or 'What does the label on the can say?'

Limitless Possibilities with Describe Scene:
The new 'Describe Scene' feature is designed to understand the world around you with incredible depth and precision. We encourage you to explore its capabilities, test its limits, and share your feedback with us. Your experiences are invaluable in our continuous effort to improve and innovate.

‍Enjoy the new level of visual interpretation with Envision Glasses 2.5.0 – Where Technology Meets Perception.

We are excited to announce the release of the Envision Glasses software update version 2.4, a significant enhancement focusing on multilingual capabilities.

‍

Multilingual Voice Commands Now Available:‍

Building on the multilingual support introduced in Ask Envision (version 2.3), this latest update brings the convenience of Multilingual Voice Commands to you. In version 2.4, you can now use voice commands in your preferred language. Simply select your language, press the hinge button on your Envision Glasses, and command your device to activate the desired feature.

‍

Expanded Language Support:‍

We are broadening your experience with the addition of two new languages:

• Hebrew: You can now use Envision Glasses in Hebrew, choosing from five voice options.
• Arabic: Envision Glasses are also available in Arabic, with five unique voice options for your convenience.

‍

Fixed an issue regarding the 'Describe Scene (beta)

Introducing Envision Glasses 2.3: Discover a world of enhanced detail and seamless interaction with our latest update, featuring an enriched 'Describe Scene' experience and multilingual support for 'Ask Envision'. Your vision, reimagined.

‍

Supercharged Scene Description (Beta)

We're thrilled to announce that the latest Envision Glasses 2.3 update brings an even richer experience to your 'Describe Scene' feature. Just in the latest release, we elevated its capabilities to paint a clearer picture of your surroundings. Now, we've turbocharged it to deliver hyper-detailed narratives of everything within your view. As this is a beta feature, please allow for some room for refinement. We're eager for you to explore its potential!

‍

Ask Envision within Describe Scene

Prepare for an added layer of interaction with your surroundings! The 'Describe Scene' feature is now integrated with 'Ask Envision', our intuitive powerful visual assistant. This enhancement, already a favorite within the 'Scan Text' Reader, allows you to inquire further into the scenes you capture. Imagine being told there are flowers before you; now you can query the number, color, and even the species of these blooms. Dive into a world of interactive possibilities!

‍

Speak your language with Ask Envision

Embrace the world in your language with the new multilingual voice input for 'Ask Envision'. Query your device in the language you're most comfortable with, whether within the Reader or the 'Describe Scene' feature. Simply select your preferred language in the 'Settings', and you're all set—provided you're connected to the internet.

‍

And yes, we are working on bringing multilingual voice input for all key features and navigation soon. More about that in the next release.

Welcome to a new exciting update for the Envision Glasses.

In this update you will find Offline Scan Text and Offline Batch Scan, Improved Find Object, Find People and Explore and an improved Describe Scene.

Let’s dive into it:

• Offline Scan Text and Batch Scan: you can now use the features Scan Text and Batch Scan even when you’re not connected to the internet with the Offline Mode. You can enable Offline Mode in the context menu of Scan Text and Batch Scan. Offline Mode brings the same great experience you are used to, but processing the text will be much faster. Do keep in mind that Layout Detection is not available yet when using Offline Scan Text and Offline Batch Scan.
• Improved Find Object, Find People and Explore: we’ve added more objects to these features and also more relevant and much requested objects like a door, sink and stairs. More importantly, we have added clock-notation position to all of these three features. So instead of just hearing a notification, you will now hear whether the water bottle you’re looking for is located at 9 o’clock or 2 o’clock in front of you. Same for the Explore feature, this will now announce many more objects than before and also provide you a clock-notation position of the object.
• Improved Describe Scene: the feature to describe what is in front of you has been improved and you will not only get a more accurate, but also much more detailed description of what is in front of you.

In addition to these new features, we’ve now also added support for the Korean 🇰🇷 language.

‍

We can’t wait for you to try these new features, so please do share your experiences by tagging us on social media: @LetsEnvision

‍

We’re excited to announce the latest software update for Envision Glasses, version 2.1.0! This update introduces three new features designed to enhance your experience and make your life even easier: Ask Envision, Wi-Fi Wizard, and Scan Wi-Fi QR Code:‍

‍
Ask Envision

The new 'Ask Envision' feature allows customers to ask questions about a piece of text or document they have just scanned, such as a menu card, electricity bill, or cooking instructions.

‍

For example, after scanning a menu, you could ask Envision, 'What are the vegetarian options available?' and receive a helpful response. This feature is powered by GPT-4 by OpenAI, one of the most advanced language models in the world, to understand and answer natural language questions.

‍
To use 'Ask Envision', access the text 'Reader' after scanning your document with the Scan Text feature. Press and hold the 'Hinge Button' on your Envision Glasses until you hear a chime, then ask your question and wait for the virtual assistant to answer.

‍

Currently, voice prompts are only supported in English, however, the 'Ask Envision' output can be in any of the languages supported by Envision Glasses.

‍

Please note that since this feature uses ChatGPT, it may occasionally produce inaccurate information about people, places, or facts. For more detailed instructions and examples, check out the video tutorial below.

‍
Wi-Fi Wizard

The Wi-Fi Wizard provides step-by-step guidance to help you connect your Envision Glasses to the internet, whether it’s an iPhone hotspot, an Android hotspot, or a regular router.

‍

This feature aims to make it easier for new and existing users to stay connected anywhere, anytime. To access the Wi-Fi Wizard, navigate to the sub-menu Wi-Fi, in the Settings menu or the sub-menu Trainings, in the Help menu, and perform a one-finger double tap to start the guide. Simply follow the instructions to get connected.

‍
Scan Wi-Fi QR Code

Based on user feedback, we’ve integrated Wi-Fi QR Codes into the existing Scan QR Code feature. Now, if your router has a QR Code or you plan to buy one with a QR Code, you can easily connect your Envision Glasses to the router by opening the Scan QR Code feature and pointing the camera at the QR Code.

‍

Your Envision Glasses will automatically and immediately connect to the router, making the process seamless and hassle-free.

‍

In addition to these new features, we’ve made numerous under-the-hood changes and performance improvements to provide you with an even better experience. We’re grateful for your continued support, and we can’t wait to share more exciting updates with you in the future. Happy Envisioning!

• In this version no changes were made. It served for testing purposes.

Introducing Editions, Updates and Upgrades.

‍With the latest update, 2.0.0, we are introducing Envision Glasses Editions, which offers customer more choices based on their individual needs. You can read all about it in our blog post by tapping here.

‍‍This update also fixes to small bugs regarding:

1. Forgetting known Wi-Fi Networks.
2. Face recognition within Find People feature.

They should both now perform optimally. There are more minor fixes and improvements that make for a smoother experience of using Envision Glasses. We hope this marks the beginning of many more great updates we have in the pipeline for the year 2023!

• An issue with going to the context menu from a feature has been resolved.

• New menu design
• New Wifi UX
• Quick switch

‍

• Fixes a bug that was causing issues in switching between known Wi-Fi networks.

• UX Updates
• Fixes for minor bugs
• In this version, the forgetting of passwords for previously connected Wi-Fi networks has been corrected.
• This version includes device information such as Serial Number, Version Number and MAC address. This has been implemented for the Envision App.

‍

Call an Aira Agent

‍
There 3 exciting elements in this updates!

• Today we are excited to announce that Aira, the live, professional visual interpreting service, is now available on Envision Glasses.
  
  For Envision Glasses customers, the integration with the Aira service provides access to 24/7, human-to-human, professional visual interpreting for any task at home, work, or when on the move. This addition of Aira to the Envision Glasses substantially strengthens and compliments our Ally video call feature which enables customers to make video calls to chosen family members and friends.
  
  Here's a quick video. that explain how this works:

• This version also bring improvemets to Voice Commands, with a voice command being also added for "Home". So saying "Home" or "Take me home" from anywhere in the glasses will bring you back to the home screen.
• This version also introuduces the Scan QR function, which enable you to scan any text based QR code around you and have the contents of it be spoken out.

Just say the word.

‍With this new version, we are introducing one of the most requested additions to Envision Glasses – Voice Commands!

You can now press and hold the Hinge Button on the Envision Glasses and simply speak which function you would like it open, and voila, it just opens. Sounds too good to be true? Try it for yourself!

Currently, Envision Glasses will be able to recognise the following commands (only English for now):

• Instant Text
• Scan Text
• Batch Scan
• Call an Ally
• Describe Scene
• Detect Light
• Recognise Cash
• Detect Colors
• Find Object
• Find People
• Explore
• Change Volume

Here's a quick introductory video about it: 

We will continue to add more features and languages to Voice Commands and improve its ability to understand what you need. We eagerly look forward to your feedback on how you would like to see this feature evolve in the future! Until then, your voice is our command.

• Fixed issue with auto sleep not working in the main menu
• Switching from production to beta has been fixed
• Updated translations and strings

After months of research, development testing and refinement, Envision team is proud to announce our latest software updates, 1.6.0. Here's everything you need to know about the new update:

‍

1. It's a big one

This update is significantly bigger in size than the updates that you're used to and that's because we are packing some significant firmware changes in this. This greatly boosts the performance and battery life. This allows us to have our AI do a lot more on the small processors of the glasses. Watch this video as Maria from Envision, given you an overview of how this update works:

‍

2. Smart Guidance : Make sure you get the whole document, quickly and reliably.

One of the great feature this update brings is called Smart Guidance. It's a simple, yet effective way, to make sure you capture your complete document while using the Scan Text feature. Developing this feature on smartglasses was no easy task, but our team of engineers and group of beta testers channeled their energies to make this a reality.

‍

To get this feature to work, go to Scan Text and do a two-finger double tap to make sure that Smart Guidance is Enabled. Then when you double tap on Scan Text, you will hear a voice that will offer you simple instructions like "Move document left" or "Move document down". Once the entire document is in frame, it will automatically capture a picture for you.
‍

In addition to making sure the entire document is captured, this method also ensures that only the text in the document in your hand is read, ignoring any text that might be in the backgroud. So no more picking up stray texts from your surroundings.

Here's Ferkan from Envision, explaining this in a quick video:
‍

‍

3. Layout Detection : Read every document, the way you were meant to

The other great addition to this feature is Layout Detection, which analyses the text captures and identifies elements such as columns, paragraphs and headers. This makes it easier to read text that appears in columns (like those in magazines or newspapers) in the correct reading order. It will also speak out Headings as 'Headings' so you know. Layout Detection can be enabled or disabled from the context menu (two-finger-single-tap) of Scan Text.

Going forward, we will also start identifying elements in a document, to make sure you read everything the way you were supposed to.

Here's Ferkan from Envision, with a quick explainer video about Layot Detection.

‍

Those are all the major development with this update. There are several under the hood improvments that ensures that your Envision experience stays buttery smooth. We would love for you to play around with these new features and share your experiences with us. Happy Envisioning

‍

Reading text instantly just got a whole lot better!
The offline version of our instant text just received a major upgrade. It is now much more accurate and faster than ever before, making it the best companion to have for all the short pieces of text you wish to read on the go!
We have also worked on adding more scripts to our offline instant text recognition. You can now read Chinese, Japanese, Korean and  Devanagari (Hindi) scripts, completely offline with the Instant Text feature.
We hope you enjoy these improvements. Our team is hard at work to bring more major upgrades to you shortly, so stay tuned!

‍

Your favorite feature now at your fingertip
We are introducing a new feature called 'Favorites'. This enables you to set the features that you use more frequently as 'Favorites'. You can then quickly access these selected features by doing a swipe back from the 'Home' on the Envision Glasses. Here’s Marinda explaining it in more detail:

‍

A brief introduction to 'Favorites':

‍

How to add, delete and rearrange your 'Favorites':

‍

‍As usual, this update also contains under the hood bug fixes and usability improvements.

Too dark to recognise something? We will let you know
In the previous update we added the much requested feature - Detect Light. This feature provides information about how bright or dim your surrounding light is. We have now integrated this aspect within all features of Envision Glasses. So whether you’re trying to recognise cash or finding objects, the glasses will automatically check and alert you if the surrounding is too dark. This will highly improve your experience on the Envision Glasses and we can’t wait for you to try it out!‍

Under the hood improvements
In this update we also improved the performance of our functions within the Identify and Find tab. The new Detect Light implementation also makes it faster and provides more detailed variation in lower light settings. Overall, the Envision Glasses are now even smoother and faster than ever!

Language Selector for Instant Text
Update your Envision Glasses to find out about the new language selector in Instant Text. This allows you not only to use Instant Text in your system’s language, but in any language that you prefer. Perfect for us multilingual people who read a lot in a foreign languages, or when you can finally go on a holiday again and order your sangría in Spanish. It currently supports up to 65 languages.

‍Improved Instant Text
Instant Text is the easiest and fastest way of quickly reading what is in front of you. And with this latest update we have improved the accuracy of the Instant Text feature. The results should be even more accurate, whether you’re trying to read what is on that packaging with small letters or to see if you are waiting at the correct platform on the train station.

‍Translation improvements
We have made several improvements to the text and translations in the glasses thanks to our global group of Envisioners.

Introducing Detect Light - This feature provides information about how bright or dark your environment is, through sound cues. You can choose to be notified with varying pitch of tone or frequency of beeps, by doing a two-finger single tap within the feature. Lower pitch or frequency means darker environment. You can use this to detect whether the lights in your room are turned on or whether the the computer screen is working. This feature located in the Identify category.
‍

‍

There are more new features:

‍
Time format
This feature is located in the Device Setting and allows you to switch between 12 hours or 24 hours format.
Change voices
Have you ever wanted to have different person voice for your Envision Glasses? Now you can do that by just going to Change Voices in the Audio Setting and choose between different voices.
Hei Norge
From this update we’re supporting Norwegian language too.

‍

‍

‍

• This update includes a very handy volume shortcut! Anywhere in the menu, do a two-finger swipe up and swipe forward or back with one finger to change the volume. Besides that, this update also supports more languages and local translations can be done through envision.crowdin.com

• This update now has the very first app added to it: the Cash Reader app! This means you can now recognise banknotes of nearly 100 different currencies with your Envision Glasses. You can find this new app inside the Identify category

• New and improved object detection (Find Object, Find People, Explore)
• More objects for the Find Object-feature
• New languages added: Czech and Romanian
• Added the possibility to check the MAC address of the Envision Glasses (Help > About)
• UX improvements and bug fixes

• Improved user experience for double-tapping
• The Pairing Mode menu has been moved to Device Settings
• New languages added: Polish, Swedish and Traditional Chinese
• Smart sleep mode: as of now the glasses will go in sleep mode depending on where you are in the menu. This will eventually allow for a better battery life
• Overall improvements and bug fixes

• Updated Strings for all languages.
• Bug fixes.