Shopping online is settling as a norm in the retail industry across many countries. Every day, millions of people casually search and purchase their wanted items on e-commerce websites, such as Amazon.

The case of South Korea is nothing different. According to the Korea Consumer Agency’s report last November, 82 per cent of Koreans over 20 years old said that they use online shopping platforms to buy something. The report noted that this is a rapid increase from the result in 2019, which was 44 per cent. 

It is no surprise that many people with visual disabilities use the internet every day. Statistics Korea, a government agency responsible for statistics in South Korea, reported that 78 per cent of people with disability have access to the internet.

However, shopping malls are not so accessible to use for people with visual impairments. A survey report from the Korea Foundation for Persons with Disabilities revealed that over 40 per cent of visually impaired people replied that they need another person’s help to buy products from online shopping malls.

Texts are often too small to read unless the user zooms the screen. While many visually impaired people use voice-assistant applications installed on smartphones, such as Apple’s VoiceOver and Android’s TalkBack, these applications cannot recognise texts imprinted in products’ imagery. This has been a problem for many people with visual impairments as images often contain more information than basic introductions of a product.

This pain point came to the attention of seven students at Korea Advanced Institute of Science and Technology (KAIST), a research university based in Daejeon, South Korea, in 2018. The students decided to make an online shopping mall for people with visual impairments a year after.

Their idea became Sori Market today, the only Korean online shopping application that can read every registered item on its platform aloud.

“Reading Out” Product Details

Waddle, the developer of this market, introduces Sori Market as a “speech-based mobile shopping mall for people with visual impairments”. 

Waddle’s CEO Park Ji-hyuk, who became aware of how the digital divide affects people with disabilities during his university years, says he wanted to make a platform that can be helpful to those people.

“We tried to give a solution to the accessibility issues of many other shopping malls with our barrier-free platform Sori Market,” Park and Waddle COO Cho Yong-won told 4i Magazine. 

The platform works like any other shopping mall application, from choosing an item to the final checkout stage. The main feature of this application, however, is reading product details aloud, even printed, handwritten texts superimposed in the imagery of registered products.

“Sori Market uses AI as a leverage to provide an accessible shopping experience for digitally-excluded people.” Park and Cho said. “The platform can ‘read’ the details of products for users with visual impairments, by using image-to-speech technology.” 

“Sori” in Sori Market is a Korean word that translates to “sound”. As the name implies, the platform’s main focus is on its text-to-speech, image-to-speech service. However, the name also includes the big ambitions of young developers.

“Waddle decided to call the platform ‘Sori Market’ because it provides a speech-based shopping experience,” Park and Cho said. “Furthermore, this name includes our hope to create a shopping platform where users can have natural interaction (with the system).”

Making Machines Read Texts

Before developing Sori Market, Waddle members visited care centres for people with visual impairments to know more about their struggles with using online shopping malls. Through this process, they learned that screen readers — software that can convert in-text messages into speech data — cannot read text superimposed on imagery.

After their discovery, the developers adopted the optical character recognition technology that can change typed and hand-written texts in images into coded messages for machines. The platform’s artificial intelligence system then reads the converted data aloud.

“Waddle’s Sori Market can extract text data from imagery,” Park and Cho explained. “It reads out texts imprinted on detailed shots of products (for example, information about ingredients and properties) for the customers.”

Sori Market’s system also groups and recategorises collected text data based on structures of sentences and context. For example, while reading through product details, the system will identify whether the collected text data is a product’s name or a description. It also filters out irrelevant information that might not be useful to customers.

Waddle also tried to make the checkout stage more accessible. Most shopping malls’ checkout software includes third-party security programmes, which do not accommodate voice-over functions that can read the written text.

“To solve this problem, we installed an in-application payment system where customers can make their payment (without using third-party security programmes),” Park and Cho said.

The platform also provides an online community called “Review Village” for customers to share their honest reviews about products they bought.

“We are giving technical support for people with visual impairments to browse and shop freely without others’ help,” Park and Cho reiterated.

Young Pioneers on a Rocky Road

Making a shopping platform with a concept that no one has ever tried was not an easy challenge for young entrepreneurs. 

“We jumped into the business at a young age,” said Park and Cho, who first built on this idea when they were only university students. “The company faced a wide range of events, including investment, human resources, and development. We had many trials and errors in the progress and made countless mistakes.”

Still, they tried to overcome those difficulties by working harder than their competitors. Some experts who reviewed the platform’s proposal also gave help to make the platform up and running.

“In our case, having less experience (than our competitors) boosted our courage to challenge any issue that comes towards us,” the young entrepreneurs said. 

“Founding a new start-up is not an easy journey. However, we decided to make it through new challenges because we did not fear failure.”

With their fearless and bold approach to the business, Sori Market is now more successful than ever. Since its launch in July 2020, they garnered over 2,000 registered customers, and its repurchase rate is estimated to be over 65 per cent. Waddle also had a chance to participate in the Consumer Electronics Show this year to introduce its platform.

Customers seem to love the application, too. “Most of our customers say they are happy that they can shop without another person’s help,” Park and Cho said. “We remember one user saying that our platform changed his life forever. We feel delighted when we receive such positive feedback.”

Waddle’s efforts, however, do not stop there. Park and Cho say that they have bigger plans for this year to improve the platform and make the business bigger. Not only do they seek a new target audience, such as seniors, but they also are considering making technological advancements for a better user experience soon.

“We are planning to work more on stabilising and improving the system,” they said. “Waddle also plans to develop speech-based AI technology that can do natural communications with the customers.”

Waddle’s grand goal is to improve the lives of digitally-excluded people, who don’t have skills or confidence in using the internet, by using AI and other relevant technologies.

“Waddle is now opening up the era of the speech-based interactive platform with the help of AI,” Park and Cho said. “Sori Market is a start — our dream is to make a convenient digital world where everyone can enjoy.”