Evaluation of Amazon Shopping’s Accessibility with Screen Readers

Participants and Methods

Participants

We recruited three visually impaired participants through personal networks, ensuring they were at least 18 years old. The study was conducted virtually using Zoom, with participants performing tasks related to browsing, adding items to their cart, checking out, and reviewing orders. Data was collected through recordings, notes, and affinity diagrams to identify patterns and themes.

Procedures

All usability testing sessions were conducted virtually using Zoom. Each session included one researcher acting as both the moderator and observer, following a standardized procedure script. Participants were given an informed consent document to review and verbally consented to the session, which was recorded. Each participant received a $15 Amazon gift card for use during the testing.

To establish participants' online shopping habits, we asked pre-test questions about their experience with screen readers and online shopping. Participants then shared their screens to complete the following tasks:

1. Search for an item within given parameters (under $13 with Prime/under $7 without Prime, 4-star user rating, eligible for Prime shipping) and add it to their cart.
2. Visit their cart and purchase the item using the provided gift card.
3. Imagine a few days have passed, navigate to their recently placed order and find the expected delivery date.

Participants rated their experiences and shared thoughts after each task. We followed up with final questions about their overall experience on Amazon, additional comments about online shopping, and demographic questions.

Data Collection and Analysis

We collected data through handwritten and digital notes, further detailed in follow-up team sessions reviewing recorded Zoom meetings. We created an affinity diagram using Figma’s FigJam feature to identify patterns/themes, informing our recommendations for Amazon Shopping.

Findings

Task #1: Browsing Amazon

Participants found browsing challenging due to clutter and time spent ensuring the accuracy of chosen products.

• Product Description: Participants needed sighted assistance for items requiring more insight into texture or “look and feel”.
• Product Recommendation: Recommended products caused confusion and increased navigation time, as participants sifted through unwanted items before finding their search results.

Task #2: Checkout

Checkout was less challenging but included familiar processes like navigating the cart and completing payment.

• Information Overload: Participants experienced frustration with advertisements and related item suggestions on the checkout page.
• Adding a Gift Card: Challenges arose when participants tried to add gift cards via email or code, with difficulties finding the code field or understanding the gift card email structure.
• Payment Process: Participants struggled with default payment options prioritizing credit cards over gift cards, causing mistrust and hesitation.
• Pricing Changes: One participant encountered an unexpected price change, not informed until the checkout page.

Task #3: Order Review

Participants found reviewing orders the easiest task.

• Efficient Navigation: Navigation to the "Orders" page was intuitive and quick.
• Mobile Preference: Participants typically preferred the mobile version of Amazon Shopping, finding it more accessible than the desktop version.

Discussion

Our evaluation highlights the need for improved accessibility on Amazon Shopping for screen reader users. Recommendations include enhancing product descriptions, optimizing product recommendations, simplifying gift card usage, and implementing continuous accessibility improvements. These findings underscore the importance of web accessibility for visually impaired users, advocating for more inclusive digital experiences.

Limitations

The study had a limited number of participants, all with similar backgrounds. A more diverse participant pool would be preferable for generalization. Additionally, the freedom given to participants in selecting items for purchase led to substantial variation in task completion times.

Thank you for checking out my study, check the top of the page for the link to the full report!

References

1. Patricia Acosta-Vargas, Belén Salvador-Acosta, Luis Salvador-Ullauri, and Janio Jadán-Guerrero. 2022. Accessibility challenges of e-commerce websites. PeerJ Computer Science 8, e891 (2022). https://doi.org/10.7717/peerj-cs.891
2. Amnah Alluqmani, Morgan A Harvey, and Ziqi Zhang. 2023. The Barriers to Online Clothing Websites for Visually Impaired People: An Interview and Observation Approach to Understanding Needs. In Proceedings of the 2023 ACM Designing Interactive Systems Conference (DIS '23). Association for Computing Machinery, New York, NY, USA, 753–764. https://doi-org.ezproxy.depaul.edu/10.1145/3563657.3595978
3. Amazon. 2023. Who We Are | About Amazon. Retrieved from https://www.aboutamazon.com/about-us.
4. Natã M. Barbosa, Jordan Hayes, Smirity Kaushik, and Yang Wang. 2022. “Every Website Is a Puzzle!”: Facilitating Access to Common Website Features for People with Visual Impairments. ACM Trans. Access. Comput. 15, 3, Article 19 (September 2022), 35 pages. https://doi-org.ezproxy.depaul.edu/10.1145/3519032
5. Alex H. Cohen, Jorge E. Fresneda, and Rolph E. Anderson. 2020. What retailers need to understand about website inaccessibility and disabled consumers: Challenges and opportunities. Journal of Consumer Affairs 54, 3 (2020), 854-889. https://doi-org.ezproxy.depaul.edu/10.1111/joca.12307
6. Blake Droesch. 2023. Target, Carvana, and Lowe’s are moving up the US ecommerce sales rank. (April 2023). Retrieved from https://www.insiderintelligence.com/content/target-carvana-lowe-s-moving-up-us-ecommerce-sales-ranks.
7. Patricia Lanford and Roland Hübscher. 2004. Trustworthiness in e-commerce. In Proceedings of the 42nd annual Southeast regional conference (ACM-SE 42). Association for Computing Machinery, New York, NY, USA, 315–319. https://doi-org.ezproxy.depaul.edu/10.1145/986537.986614
8. Guanhong Liu, Xianghua Ding, Chun Yu, Lan Gao, Xingyu Chi, and Yuanchun Shi. 2019. "I Bought This for Me to Look More Ordinary": A Study of Blind People Doing Online Shopping. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, Paper 372, 1–11. https://doi-org.ezproxy.depaul.edu/10.1145/3290605.3300602
9. Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, and Koki Kusano. 2022. Investigating Accessibility Challenges and Opportunities for Users with Low Vision Disabilities in Customer-to-Customer (C2C) Marketplaces. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22). Association for Computing Machinery, New York, NY, USA, Article 79, 1–4. https://doi-org.ezproxy.depaul.edu/10.1145/3517428.3550390
10. Hironobu Takagi, Shin Saito, Kentarou Fukuda, and Chieko Asakawa. 2007. Analysis of navigability of Web applications for improving blind usability. ACM Trans. Comput. -Hum. Interact. 14, 3 (September 2007), 13–es. https://doi-org.ezproxy.depaul.edu/10.1145/1279700.1279703
11. Ruolin Wang, Zixuan Chen, Mingrui Ray Zhang, Zhaoheng Li, Zhixiu Liu, Zihan Dang, Chun Yu, and Xiang 'Anthony' Chen. 2021. Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision Users. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 494, 1–14. https://doi-org.ezproxy.depaul.edu/10.1145/3411764.3445547