×

Bosch Automotive | Grade X: Vehicle Diagnostic Software

Project Kick Off: Settting the Stage

Introduction

Grade X, a Bosch vehicle diagnostic tool is a browser-based software used by a technician to check the status of a vehicle and detect possible problems.

Disclaimer: This case study is for the purpose of the portfolio presentation, so some data or events may not be entirely accurate.

Mission

Improving technicians experience by creating easy navigate interface, pleasant and intuitive design and efficient organisation of a large a number of additional complex tools. The software should reflect Bosch’s high standards of creating tools to help technicians with their day-to-day work.

Discovery Goals: What Business Needs vs User Goals

Business Objectives

During the meeting with senior stakeholders, the following business objectives were established:

  • Expanding the current market
  • Getting positive reviews by worldwide serving stations
  • Reflect the companies branding
  • Build the product to reflect Bosch high-quality standards

User Goals

We interviewed various technicians, showroom managers, stakehoders on what ideal product should reflect:

  • Easy to use
  • Easy to learn and intuitive
  • Easy for the older generation of technicians who are not computer savvy to migrate to the new platform
  • Ability to configure the application to exclude (make less visible) tools not used

user-business venn diagram

Evaluation of existing software: Can we learn anything from a current product?

Usability testing

After an introductory meeting with stakeholders, I asked to evaluate the current application. After a glance at the application, it was clear why clients were asking that the product was long due to a major update. It was hectic, confusing, some parts were not functioning etc. I relied on experienced engineers to help me navigate the application, while I was trying to find a specific diagnostic tool.

I asked 18 various age and experience people in the organisation, (8 who were used before the diagnostic tool,and 10 younger generation of potential technicians who have not used this particular product before), to accomplish few specific tasks, such as connecting with the vehicle, finding specific tools or find information about the vehicle, while the vehicle was connected to the application. I measured effectiveness, efficiency and satisfaction.

View of Vehicle Specification on an old product
View of Vehicle Specification on an old product (click on the image to enlarge)
View of Symptom Selector on an old product
View of Symptom Selector on an old product (click on the image to enlarge)

Testing effectiveness

Testing experienced technicians I found that, on average, the success rate of accomplishing a task was just above 60%. Interestingly, when a specific tool was on the screen, the participants found fewer problems. I compared the product with other similar products and found that, while the interface of the application basis was different, there were some commonalities with specific tools.

While experienced technicians had a success rate that was not too bad, the younger participants, who never used the tool before, struggled to complete specific tasks. They were almost 3 times less effective than experienced compatriots.

Testing effectiveness with experienced technicians
Testing effectiveness with experienced technicians
Testing effectiveness with students and apprentices
Testing effectiveness with students and apprentices

Testing efficiency

In the same test, I also measured how quickly participants have accomplished tasks. Of course, there was no point in measuring tasks that were not completed. I noticed that one younger participant had a much better score on both tests. I asked him how come that he was reasonably good at accomplishing tasks and turned out that, prior to the test, he tried to use not knowing that would contaminate the result hence we decided to ignore his data in the study.

Measuring efficiency with experienced technicians
Measuring efficiency with experienced technicians
Measuring efficiency with students and apprentices
Measuring efficiency with students and apprentices

Outcome

While it was clear that experienced technicians were more effective using the application, we had not enough information to make a sound judgement to compare the effectiveness between the two groups. But the main aim of the study was to compare existing product later on with the newly designed application.

Who are the users: Context of Use

Why?

It was important to learn about users, their behaviour and demographics and to find measurable ways to test forthcoming hypotheses about them. Contextual enquiry is about people who are, in reality, using the system or are about to use it. My organisation quickly provided me with a few vehicle service centres, who were willing to participate and I obtained names of technicians/apprentices who I am going to interview and observe in their work environment. Age and experience as technicians played an important part in the study.

Diagram showing the age of participants
Diagram showing the age of participants
Diagram showing experience of technicians
Diagram showing experience of technicians

Field Visit

The next day I visited the Jaguar Land Rover service station and met with technicians. I started the interview with Adrian, a 52-year-old technician. It was important that the following were satisfied:

  • A clear focus on enquiry - (the focus questions were obtained during the meeting with stakeholders)
  • In order for technicians to feel natural during the interview and not intimidated by my presence, they should feel like experts (I asked them to imagine that I am an apprentice and they are teaching me how to use the application)
  • Focus on observing (“technician have searched for help in Google browser”) and NOT on interpretation (“The software didn’t offer help to the technician”) or solution (“The software should provide a user with clear direction")

Observation

Adrian guided me through the whole process of vehicle diagnostic while I was observing. After, I asked the participant question to learn about their behaviours such as why he was using Google chrome during the vehicle examination, why he turned up a volume although no sound was produced from the application.

After the interview, I quickly made some notes on observations while I was waiting for the next participants. A similar procedure was carried out while interviewing the rest of the participants in the following few four days but there was some notable difference observed in the use of the software. Young participants were mainly helping and sometimes observing the more experienced technicians. They also regularly asked for assistance. Two store managers also participated although their day-to-day involvement was mainly to observe the work of the other technicians or to assist them if necessary.

Contextual enquiry in Jaguar Land Rover service station
Contextual enquiry in Jaguar Land Rover service station

Familiarity with technology

At the end of each observation, I asked participants to answer five questions to test their knowledge of a digital environment. I used my laptop because I reasoned it was better to use a neutral digital environment and not what they are using on daily basis such as devices used to service vehicle or their own digital devices. The questions I was asking was to:

  1. Start diagnostic application
  2. Put the system volume up
  3. Search “Jaguar” in the browser of choice
  4. Check internet connection through the Windows Control Panel
  5. Change screen resolution

Other important factors

On a few occasions technicians mentioned that many times quick access to various tools and application functionality was important. They think that the curent version attemped to address that hence the hectic layout. One of our aim will be to design interface with quick access to tools specific to technican job

Although the focus of the interview was to learn about the software I observed the use of the hardware as well. During the vehicle inspection, I noted that one participant leaned through the vehicle window to access a laptop that was inside a vehicle on a passenger seat. Later on, on two occasions participants were laying down with a laptop inside the vehicle while testing a car. I exchanged few words with technicians and they said that it was important to them that the device is physically stable and rigid but also that the device itself would not scratch a car if it gets in a contact with a vehicle (I have seen some technicians put the laptop on the front bonnet of the car). I suggested to my organisation to continuing the use of laptops instead of tablets (we got Samsung tablets used for testing during the development of the application). They are stable while free-standing and have a keyboard right in front of them.

Who are the users: Analising the outcome

Back at the office

I evaluated the recordings and plotted the participants on a diagram based on their experience as technicians and familiarity with technology to get a graphical view of the results. Based on the field visit and the interpretations of quantitative and qualitative data I got the idea of personas - an archetypical representation of a particular group of users.

Experience as technician vs digital technology experience
Experience as technician vs digital technology experience
Personas as representative of a group of users
Personas as representative of a group of users

The main findings

(Research conducted with me and the internal Bosch team. The in-depth report is not available. All interviewed participants were males 22 to 64-year-old)

  • The software is hard to use – as expected and a major complaint
  • The very steep learning curve for new users
  • Unintuitive interface – inconsistency in labelling and naming convention
  • Broken links
  • Younger users were far more skilled in the new digital environment
  • Senior users were more efficient using existing software
  • Senior users success on specific tasks vary depending on expertise

Their fears and frustrations

  • The older generation don't like small interface
  • Anxiety over laptop battery life during vehicle repair
  • Network connection trouble leading to some techicans not knowing the status of a current operation
  • Broken links
  • "Where am I?" - question asked often by new technicians
  • Confusion about what parts of the interface interact (some technicians forcefully pressed some interface icons in frustration while expecting the application feedback)
  • Some technicians feared what would happen if they stop the specific process

Creating personas

Using all the data available during the study, three main personas were given additional personality attributes to appear just like real users hence giving us a more emphatic approach while developing various stages of the design process

Three main personas were created
Three main personas were created (click on the image to enlarge)

Information Architecture: Flow to facilitate user tasks

Creating red route

Now that we have personas and know who we are designing for it was time to find out what kind of goals our users will have, what specific tasks they are trying to achieve or problems to solve. Since we have personas, we don't have to think about all possible users. We will focus on the persona's needs to create a user journey. The core features will be available at all times while some tools, not used by specific users, will be hidden. We are not trying to create a flexible application, that is a trade-off for usability.

One way of identifying red routes is to find out critical and frequent tasks. First, I quickly created a journey of obvious tasks which every user was completing. After, I categorised tasks based on how often technicians are using and how many of them were using specific tasks.

What are the barebones of user's tasks?
What are the barebones of user's tasks?
Categorising red routes
Categorising red routes

Affinity diagraming

Once we have classified the importance of tasks, it was time to map the content of the application. I organised a meeting and we categorised all ingredients of the application, including all diagnostic tools. The purpose of the workshop activity was to understand how various parts of the software are going to interact and to further analyse the most important path technician takes during the vehicle diagnosis.

Nature
Card sorting exersize
Items categorised in Excell
Items categorised in Excell (click on the image to enlarge)

Flow with red route

Finally, based on previous activities, I create a full user journey with a critical journey, a red route, identified on the map.

A user flow completed
A user flow completed (click on the image to enlarge)

User Interface Design: Connecting the dots of a structure

Wireframing and paper prototype

It was a time to zoom in into the system and based on what we created previosywith e knew the ingredients and the flow of the system cleat, it was time to organise on-screen elements. I created over 40 wireframe layouts to accommodate all important elements of the layout. I printed them and created a paper prototype of the application to test red routes and various other tasks. Since some stakeholders objected to investing time in another usability test, we quickly tested internally with five people which gave us reasonable results, although obviously not enough to make safe assumptions. The major test would be carried after the full visual interface design is completed.

Connecting vehicle wireframe
Connecting vehicle wireframe
Symptom selector wireframe
Symptom selector wireframe
Network view wireframe
Network view wireframe
Diagnostic tool wireframe
Diagnostic tool wireframe

User interface design paterns

Losely based on outdated styling guidelines I created visual design paterns. In fact, only colours and fonts were inherited from the guidelines

Design paterns
Design paterns

User interface menu design

Also, a menu design was created including icons, which should metaphorically reflect the tools as much as possible and visually guide a user through the product. One important feature we addressed here is the ability to quickly access various tools. I designed a facility so a user can pin the menu and have quick access. Since the menu is taking a lot of screen estate, we have decided to create a pinned many with icons visible without description.

Full and icons-only menu
Full and icons-only menu

User interface Visual design

Finaly, I have put togheter all ingridients and here is the final design outcome , including a mobile version

Languague selection
Languague selection (click on the image to enlarge)
Vehicle connection
Vehicle connection (click on the image to enlarge)
Symptom selector
Symptom selector
Symptom selector different view with open menu
Symptom selector different view with open menu (click on the image to enlarge)
obile version with limited capabilities
Mobile version with limited capabilities (click on the image to enlarge)

Evaluation of new software: What we have learned and how does it compere with old product?

Usability testing

We have used an identical methodology to test as we did with the old version of the software. This time we had the same 18 participants as in the previous test, and five more younger technicians. As you can see from the following pictures, the improvement was noticeable, especially with younger technicians.

Testing effectiveness

The previous test success rate with all participants was 47% and while testing the new application the score was 84%, suggesting that effectiveness was largely improved.

NatTesting effectiveness with experienced techniciansure
Testing effectiveness with experienced technicians
Testing effectiveness with students and apprentices
Testing effectiveness with students and apprentices

Testing efficiency

  • Experienced technicians: previous test 82 seconds, while test with the new interface was 107*
  • Younger technicians: previous test 99 seconds, while test with the new interface was 91
*We expected that experienced technicians would be more efficient with original software suggesting that time to adapt to new software will be needed.

Measuring efficiency with experienced technicians
Measuring efficiency with experienced technicians
Measuring efficiency with students and apprentices
Measuring efficiency with students and apprentices

Final Word:Outcome

Outcome

The application was developed with Angular JS which I had not much experience with but in terms of styling I felt I would be useful. I was with the team for only a further two weeks during the application development process to help developers understand designs and whatever was not clear to them at that point. During my stay, the application was already sold to Mazda, Mercedes Benz, and Jaguar Land Rover.

I have reflected on my assignment on what I thought would perhaps be even more beneficial during the design process:

  • System Usability Scale (SUS) quality test to measure users satisfaction (the senior stakeholders thought that wasn't necessary). I thought the test would give another dimension of users feelings while using the product but stakeholders felt that it was obvious that users are not satisfied with the current product.
  • Time spent with developers - I think it is important to collaborate closely with people who are actually making the product alive. Often they will need an explanation from a design point of view of how to solve a problem. A lot of small issues during the development is not easy to predict during a design process hence I wish I have spent a little longer with the team
  • Warmer clothes during the field visit :)

Finished product
Finished product

Aftermath

The project was judged as a success by senior stakeholders and the product was already on the market even before it was developed. In the meantime, I was approached by the Ford team to work on a similar project. I don't have the intention to write another case study for the Ford project but here are a few images:

Ford diagnostic tool, vehicle connection
Ford diagnostic tool, vehicle connection (click on the image to enlarge)
Ford diagnostic tool, modules
Ford diagnostic tool, modules (click on the image to enlarge)
Ford diagnostic tool, system selector
Ford diagnostic tool, system selector (click on the image to enlarge)
Ford diagnostic tool, measuring tool
Ford diagnostic tool, measuring tool (click on the image to enlarge)