3

Boost Ajaib’s UX Performance 72% Faster with A/B Testing & System Usability...

 1 year ago
source link: https://uxplanet.org/boost-ajaibs-ux-performance-72-faster-with-a-b-testing-system-usability-scale-sus-scoring-f708272deff7
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Boost Ajaib’s UX Performance 72% Faster with A/B Testing & System Usability Scale (SUS) Scoring

UX Case Study on Ajaib — Indonesian Investment Platform

ilustrasi analisa saham, illustration of stock analyzing, reksa dana
Illustration: Stock analyzing

Blessed with more than 270 million people who live in diversity, Indonesia is the fourth-largest country in terms of population size, the world’s 10th largest economy in terms of purchasing power parity, and a member of the G-20. However, 66% of population are “unbanked” (don’t own a bank account).

Disclaimer: This article is translation from my paper in the journal of Informatics and Software Engineering that has been published by the Faculty of Engineering, Wahid Hasyim University, Semarang.

Read the original full paper on Bahasa Indonesia here: https://www.publikasiilmiah.unwahas.ac.id/index.php/JINRPL/article/view/4762

According to The Jakarta Post, many Indonesians, especially those living in rural areas, remain unbanked or have yet to become part of financial inclusion. Being unbanked means not having access to financial products and services such as transactions, payments, savings accounts, credit and insurance.¹

The number of domestic investors in the Indonesian capital market increased by a half to close to 2.5 million in 2019, reflecting growing interests among locals to invest in financial instruments like stocks, mutual funds and bonds according to a report by The Jakarta Globe.²

However, that number is still very small considering the total population of 270 million people. Meanwhile, the number of productive age (working-age population) in Indonesia is 189 million and the number of domestic investors is only 4.16 million people or around 2.2 percent.³

About Ajaib

logo ajaib indonesia online investment platform, what is ajaib investment
Logo Ajaib Group

Ajaib is an online investment platform for millennials in Indonesia. Seeing the current low level of financial inclusion & financial literacy in Indonesia, Ajaib Group seeks to take this opportunity by increasing the financial inclusion rate of the Indonesian people through investment. Ajaib allows their users to trade stocks and invest in mutual funds through the Ajaib mobile app. In this study, we will learn about who Ajaib’s target market is and what kind of core problems they face in investing.

Design Process (Approach)

Design Thinking was chosen as the perfect approach to solving problems. A creative approach that helps individuals or teams to identify real problems and solve them. Apart from being an affordable approach, design thinking doesn’t require a lot of time and resources.

design thinking stages illustration by darmawan r dipo, what is design thinking, 5 stages of design thingking process, apa itu design thinking
The five stages of Design Thinking

Keep in mind that this stage does not have to be sequential from stage 1 to stage 5, but the facilitator/researcher can start from any stage as needed.

Empathise

Please leave your assumptions at the door. If you have ones, challenge them! Assumption will only keep you away from your customers. You have to really know your users, and the great way to get to know your customers or users is to ask them. Conducting interviews can give you tons of useful insights into what your customers think about your product and what your customers really need.

Conduct Research

The very first step to do was research with real target users. Ajaib Group is targeting millennial users for their products, so the sample population in this study had to be between 24–39 years old and are first-time investors (or enthusiastic about investing).

In-depth Interview (Qualitative)

Go out and ask the right questions. Start asking simple, clear and open-ended questions to find out more information about their behavior, their views on the product and to the core problems they face in their daily lives.

post-it notes, stiky notes, stiky ux design, interview key finding
Interview key finding

Insight directly from participants:

  • Many unfamiliar words in the investment terms.
  • Hard to recognize end-products from the issuer (publicly listed company).
  • Assume that stock investment requires millions of rupiah (around USD $100) for one transaction.

Survey (Quantitative)

Distributing surveys will support research quantitatively. Aside from cost-effective and easy, surveys can reach remote respondents by simply sending them an online form. Google Forms was chosen as the survey tool for creating, distributing and gathering responses at free cost.

Survey result with Google Forms
Partial survey results with Google Forms

Data collection was carried out by distributing surveys to non-randomly selected respondents (non-random sampling).

Conclusion of survey results:

  • BCA Sekuritas, Mandiri Sekuritas and BNI sekuritas, the investment management companies (stock brokerage) with a big cap banks-backed have succeeded in gaining top-of-mind awareness. On the other hand, startups-based on financial technology (investment platform for smart mutual fund investments and state bonds) like Bareksa and Bibit are getting to be recognized by people.
  • 2 out of 3 respondents know and learn about the stock market from online video sharing and social media platforms (YouTube) and will approach their friends to understand more about investing.
  • Almost half of the respondents will spend IDR200k — 500k (US$15–35) to invest/month. Meanwhile, the GDP per capita in Indonesia was last recorded at US$3.911,7 in 2020.⁴
  • 26.7% of respondents consider the halal/haram as a serious concern in stock investment.

Define

After conducting interviews, emphasizing the insights, distributing and analyzing the survey results, it’s time to define the real problem.

Persona

All the information from the research is now converted into Persona, a useful design tool to reflect the targeted user in the interview session.

persona design thinking, persona sample, persona finance and investment
Persona

Creating a persona can help researcher in identifying users’ needs more deeply, knowing their behavior and the goals they want to achieve. Moreover, persona can be used to guide design decisions and prioritize user’s problems.

Problem Statements

Based on the research results, here are the problems decided to focus on:

  • Users need to know what end-products (product brands) of the issuer (publicly listed company) is producing.
  • Users need guidance on the app for choosing stocks to invest in.
  • Most novice (first time investors) find investing confusing because of the confusing investing terms.

Business Goals

  • Increase users’ awareness of stocks that appeal to them.
  • Encourage users to make simple stock choices to invest in.
  • Educate and inform users about terms in investing.

Ideate

Start generating ideas and solutions. Based on the research results, we have sharpened the core problem and are ready to come up with solutions.

There are hundreds of ideation techniques to help you in your ideation sessions. You want an ideation technique that combines your conscious and unconscious mind — fusing the rational with the creative.⁵

The proposed solution is to group the stocks list into a theme. Stock theme is a feature that easily understandable groups stocks based on what users are exposed to in their daily lives and they can easily understand.

Examples of stock themes are 100 most popular stocks, a group of Indonesia’s State-owned Enterprises, a group of product brands, and a group of basic categories such as Food, Drink, Telecommunication, etc. The category is meant to help users discover stocks more easily based on their preferences or values.

Prototype

Before getting into craft work, there’s one small thing to do to clarify where the proposed solution will be placed. User flow is another great design tool to do that job and for analyzing the user’s steps in completing a task.

According to Technopedia, user flow is a term for the description of a set of tasks that a user must do to complete some process. In IT and on the Web, professionals might analyze user flows to try to make websites or technologies more user-friendly, and to understand the goals of the user as well as the goals of the company.⁶

1*TIQsagXkDdWrXFXblzoZ5Q.png
User flow/app flow

Sketches

Prepare the craft tools. Having some paper and a pencil is a good start before drawing sketches. Remember, you don’t have to be a genius to draw a square, so, be confident and put all your creativity and ideas on that paper.

sketch, wireframe, mobile app
Sketching wireframes

Visualizing ideas into a rough sketch will help us to explore the possibilities and work on it immediately and quickly.

Hi Fi Mockup

After tons of sketches and possibilities, let’s move forward into high fidelity mockups. Converted the sketches into beautiful design, gave it a nice color palette and we are ready to jump into Usability Testing.

1*pwCEkJk8vqEhPY8TIxNZVg.jpeg
Existing design
1*znMqYmhrM8ueHa8Jr09grg.jpeg
New design (proposed solution)

Are you ready to face the reality? Usability testing was conducted to validate the proposed solution. Don’t lose hope if you find the solution doesn’t work out, you can still go back to the ideate phase to generate more ideas.

The usability testing was applied with five participants. In line with Nilsen, elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.⁷

A/B Testing

The A/B testing was conducted to verify the proposed solution can produce better performance. In this scenario, we will find out how good users are at exploring a feature, so we will assign some tasks to the users and track how fast they can complete the tasks between the two designs.

First scenario: Indonesians know Indomie as a famous instant noodle brand produced by the Indonesian company, but only few people know the name of the company (issuer). In this test, we will challenge participants to find the ICBP ticker code (issuer/stock symbol) in the stock list. We prevent participants from using the search box feature, because our goal is to encourage users to discover stocks based on what they are exposed to in their daily lives and to make stock choices.

usability testing, prototype figma
Prototype demo
1*-WJR_iZxUF7fzr1F5lKj0g.jpeg
Usability testing

Second scenario: PT Telkom Indonesia is a multinational company from Indonesia. Telkom is known as a communications company. In this test, the participants will be challenged to find the TLKM ticker code in the stock list.

A/B Testing Results

The results of these experiments show that the majority of users were able to complete these 2 scenarios faster through the new design, and had recorded that the average user performance increased 72% faster for the new design than the existing design.

Prototyping result
Usability test result — Heatmap
a/b testing result chart column
A/B test results: Task completion rate per unit of time (min.sec — lower is better)
1*SpmZVLNCZAS0LlN2qc96LA.png
A/B test results details

System Usability Scale

In order to strengthen validation on A/B testing, several metrics have been defined to measure the experiment. By following the scoring of System Usability Scale (SUS) methodology, the findings clearly indicate that the new design gains an “A” (average SUS score 86) that describes Best Imaginable for the user experience.

System usability scale average chart
System usability scale average
1*zCEcbIaKYQ1efRHtsKl7Mw.png
System usability scale results
1*9L5kGqaG3JPp7wJtQlSGRg.png
Grading scale of SUS scores

Results

Based on the tests and analysis that has been done, these experiments clearly confirmed that:

  • In the existing design, users spent more time in completing the tasks, this is due to the user’s difficulty in identifying Telkom company which is included in the infrastructure industry sector and difficulty in identifying some companies in the consumer sector.
  • With the new design, users have accomplished the task very well, because the stock categories are grouped according to the user’s knowledge and based on what they are exposed to in their daily lives.

Design Is Not Just What It Looks Like And Feels Like. Design Is How It Works — Steve Jobs

To have a beautiful interface in your product is indeed an important thing. However, the interface isn’t the only product’s success metric. If we consider it as a whole thing, it’s about users understanding and finding it easily, and always wanting to use it again which makes the design look great.

Conclusion

This study has found that Indonesia has the most valuable potential for the digital economy and the demographic bonus should be able to boost the next level of economic growth. The collaborative partnership between the public and private sector can unleash the business potential of the digital economy. For instance, the government can provide assistance in the form of infrastructure development and invent a legal foundation breakthrough to help startup companies. On the other hand, the private sector can help by educating Indonesians about the benefits of investing and how to manage finances wisely.

In the design section, it is crucial to understand the users’ point of view, but the good design doesn’t only speak users’ language yet meet the business goals as well, whilst the best product is about balancing users needs with business goals and meets technical feasibility.

Thank you for reading and congratulations on completing what you’ve started!

View my experience details on LinkedIn | View my portfolio on Dribbble

References

[1] Wiradji, Sudibyo. ( April 15, 2021). Grassroot strategy to realize financial inclusion in Indonesia. The Jakarta Post. Retrieved from https://www.thejakartapost.com/life/2021/04/15/grassroot-strategy-to-realize-financial-inclusion-in-indonesia.html

[2] Prasetyo, Wisnu Bagus. ( December 30, 2019). IDX Registers 860.000 New Investors in 2019. Jakarta Globe. Retrieved from https://jakartaglobe.id/business/idx-registers-860000-new-investors-in-2019

[3] Uly, Yohana Artha. (October 22, 2020). Jumlah Investor Pasar Modal Indonesia Masih Tertinggal dari Singapura dan Malaysia. Kompas.com. Retrieved from https://money.kompas.com/read/2020/10/22/103000226/jumlah-investor-pasar-modal-indonesia-masih-tertinggal-dari-singapura-dan?page=all

[4] Badan Pusat Statistik. (February 05, 2021). Ekonomi Indonesia 2020 Turun sebesar 2,07 Persen (c-to-c). BPS — Statistics Indonesia. Retrieved from https://www.bps.go.id/pressrelease/2021/02/05/1811/ekonomi-indonesia-2020-turun-sebesar-2-07-persen--c-to-c-.html

[5] Ideation. Interaction Design Foundation. Retrieved from https://www.interaction-design.org/literature/topics/ideation

[6] User Flow. Technopedia. Retrieved from https://www.techopedia.com/definition/31852/user-flow

[7] Nielsen, Jakob. ( March 18, 2000). Why You Only Need to Test with 5 Users. Nielsen Norman Group. Retrieved from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

p.s. I was not directly-affiliated with Ajaib Group at the point of publication.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK