Disambiguation Keyboard for Visually Impaired

Hi!

I am back with a new blog post in which I’ll be discussing my experience at IIT B while interning under Prof Anirudha Joshi, Industrial Design Centre.

After the selection procedure, all the selected interns were asked to give 2 topics of their choice out of the list of 7.

Going through all the topics and researching a bit about each. I decide to give chorded keyboard for the blind as my first priority and Making android phones tangible as my second priority. Luckily enough, I was assigned the topic of my choice. The internship was to start in the month of May but we were encouraged to read about the topic beforehand. weekly Skype calls were scheduled to get a sense of familiarity among project partners and the topic. we were further encouraged to participate in discussions and present our doubts and queries if any. The internship was of a 3-month duration from May till July. but due to my semester exam ending on 24th May. I was able to join from the 25th.

There, We used to have a thrice a week meet where all the interns, Professor, and the research assistants would meet up and discuss the progress of each one’s project, give constructive feedback among each other. Apart from updates, one intern was supposed to present a paper either related to their topic or any paper which they found interesting. After the paper presentations, fellow interns would ask doubts if any or discuss anything particular they found interesting. Then there used to be a one to one interaction of the presenter with Manjiri — Senior project engineer at IDC, where she would draw conclusions and discuss the key findings of the paper.

The reason why I chose this project was that firstly it was something which hasn’t been done in India before: i.e development of a chorded keyboard that too for Indian languages with special focus on blind community.

I’ll be further talking about my Research Project: Disambiguation Keyboard for Blind in this Blog Post.

Talking about the current scenario as to where we stand as of now, we have around 12 million blinds in India which is roughly equivalent to 30% of the global blind population. And out of those 12 million, majorities are Devanagari speakers not knowing braille at all. So the question that arises is that, how would a blind person who has no knowledge of Braille, would type on a Hindi virtual android keyboard.

Since we all are familiar that Devanagari has a large character set — having 30+ consonants, vowels, vowel modifiers, diacritic symbols, etc. which adds up to more than 60+ characters and makes it extremely difficult to locate and input the desired characters further making it difficult for the blinds and sighted users alike to input using the existing inputting methods.

In Devanagari characters are grouped on the basis of their phonetic similarity

We devised a novel way to make use of this already existing logical based grouping of characters to make our keyboard layout which is explained in the video below.

So in simpler terms what the keyboard does is that: A set of characters is getting replaced by a single character. So the user just has to remember what all characters are present in what keys and then he just needs to select the correct groups in order to input the desired characters. This is followed by a corpus lookup i.e. A corpus search is performed looking for all those words which are having that particular input swipe sequence and return those in order of their frequency of occurrences in the prediction bar.

The most frequently used word is returned first in the prediction bar, followed by the second most frequently used word and so on. The user then slowly moves his hand away from the key while he’s getting constant voice feedback and once he reaches the desired word he simple lifts his finger and the desired word is inputted.

The user can always go back and undo his mistakes by making a double tap on the last key of the keyboard marked as J in the layout and a single tap on the same key is for space.

Since the corpus plays a pivotal role in our keyboard model. An error-free corpus prediction system work much more efficiently. We used Swarachakra’s Hindi corpus which had a total of 51,61,445 words recorded from 34,745 users for a period of about two years. Since it had a lot of errors, we used Wikipedia corpus and carried out the corpus cleaning task to get a final set of 56,000 unique words with their corresponding frequencies.

There wasn’t much increase in the corpus coverage when we move from Corpus size of 10,000 to 56,000, So we have used the effective corpus size to be the one having 10,000 words giving 78.20% corpus coverage.

Happy learning!

Related posts:

A negotiation can be defined quite quickly from looking at the balance of power during the two parties that are engaged in the negotiation. How you are perceived and the power you hold within those…