Bosch Automotive | Grade X: Vehicle Diagnostic Software
Project Kick Off: Settting the Stage
Grade X, a Bosch vehicle diagnostic tool is a browser-based software used by a technician to check the status of a vehicle and detect possible problems.
Disclaimer: This case study is for the purpose of the portfolio presentation, so some data or events may not be entirely accurate.
Improving technicians experience by creating easy navigate interface, pleasant and intuitive design and efficient organisation of a large a number of additional complex tools. The software should reflect Bosch’s high standards of creating tools to help technicians with their day-to-day work.
Discovery Goals: What Business Needs vs User Goals
During the meeting with senior stakeholders, the following business objectives were established:
- Expanding the current market
- Getting positive reviews by worldwide serving stations
- Reflect the companies branding
- Build the product to reflect Bosch high-quality standards
We interviewed various technicians, showroom managers, stakehoders on what ideal product should reflect:
- Easy to use
- Easy to learn and intuitive
- Easy for the older generation of technicians who are not computer savvy to migrate to the new platform
- Ability to configure the application to exclude (make less visible) tools not used
Evaluation of existing software: Can we learn anything from a current product?
After an introductory meeting with stakeholders, I asked to evaluate the current application. After a glance at the application, it was clear why clients were asking that the product was long due to a major update. It was hectic, confusing, some parts were not functioning etc. I relied on experienced engineers to help me navigate the application, while I was trying to find a specific diagnostic tool.
I asked 18 various age and experience people in the organisation, (8 who were used before the diagnostic tool,and 10 younger generation of potential technicians who have not used this particular product before), to accomplish few specific tasks, such as connecting with the vehicle, finding specific tools or find information about the vehicle, while the vehicle was connected to the application. I measured effectiveness, efficiency and satisfaction.
Testing experienced technicians I found that, on average, the success rate of accomplishing a task was just above 60%. Interestingly, when a specific tool was on the screen, the participants found fewer problems. I compared the product with other similar products and found that, while the interface of the application basis was different, there were some commonalities with specific tools.
While experienced technicians had a success rate that was not too bad, the younger participants, who never used the tool before, struggled to complete specific tasks. They were almost 3 times less effective than experienced compatriots.
In the same test, I also measured how quickly participants have accomplished tasks. Of course, there was no point in measuring tasks that were not completed. I noticed that one younger participant had a much better score on both tests. I asked him how come that he was reasonably good at accomplishing tasks and turned out that, prior to the test, he tried to use not knowing that would contaminate the result hence we decided to ignore his data in the study.
While it was clear that experienced technicians were more effective using the application, we had not enough information to make a sound judgement to compare the effectiveness between the two groups. But the main aim of the study was to compare existing product later on with the newly designed application.
Who are the users: Context of Use
It was important to learn about users, their behaviour and demographics and to find measurable ways to test forthcoming hypotheses about them. Contextual enquiry is about people who are, in reality, using the system or are about to use it. My organisation quickly provided me with a few vehicle service centres, who were willing to participate and I obtained names of technicians/apprentices who I am going to interview and observe in their work environment. Age and experience as technicians played an important part in the study.
The next day I visited the Jaguar Land Rover service station and met with technicians. I started the interview with Adrian, a 52-year-old technician. It was important that the following were satisfied:
- A clear focus on enquiry - (the focus questions were obtained during the meeting with stakeholders)
- In order for technicians to feel natural during the interview and not intimidated by my presence, they should feel like experts (I asked them to imagine that I am an apprentice and they are teaching me how to use the application)
- Focus on observing (“technician have searched for help in Google browser”) and NOT on interpretation (“The software didn’t offer help to the technician”) or solution (“The software should provide a user with clear direction")
Adrian guided me through the whole process of vehicle diagnostic while I was observing. After, I asked the participant question to learn about their behaviours such as why he was using Google chrome during the vehicle examination, why he turned up a volume although no sound was produced from the application.
After the interview, I quickly made some notes on observations while I was waiting for the next participants. A similar procedure was carried out while interviewing the rest of the participants in the following few four days but there was some notable difference observed in the use of the software. Young participants were mainly helping and sometimes observing the more experienced technicians. They also regularly asked for assistance. Two store managers also participated although their day-to-day involvement was mainly to observe the work of the other technicians or to assist them if necessary.
Familiarity with technology
At the end of each observation, I asked participants to answer five questions to test their knowledge of a digital environment. I used my laptop because I reasoned it was better to use a neutral digital environment and not what they are using on daily basis such as devices used to service vehicle or their own digital devices. The questions I was asking was to:
- Start diagnostic application
- Put the system volume up
- Search “Jaguar” in the browser of choice
- Check internet connection through the Windows Control Panel
- Change screen resolution
Other important factors
On a few occasions technicians mentioned that many times quick access to various tools and application functionality was important. They think that the curent version attemped to address that hence the hectic layout. One of our aim will be to design interface with quick access to tools specific to technican job
Although the focus of the interview was to learn about the software I observed the use of the hardware as well. During the vehicle inspection, I noted that one participant leaned through the vehicle window to access a laptop that was inside a vehicle on a passenger seat. Later on, on two occasions participants were laying down with a laptop inside the vehicle while testing a car. I exchanged few words with technicians and they said that it was important to them that the device is physically stable and rigid but also that the device itself would not scratch a car if it gets in a contact with a vehicle (I have seen some technicians put the laptop on the front bonnet of the car). I suggested to my organisation to continuing the use of laptops instead of tablets (we got Samsung tablets used for testing during the development of the application). They are stable while free-standing and have a keyboard right in front of them.
Who are the users: Analising the outcome
Back at the office
I evaluated the recordings and plotted the participants on a diagram based on their experience as technicians and familiarity with technology to get a graphical view of the results. Based on the field visit and the interpretations of quantitative and qualitative data I got the idea of personas - an archetypical representation of a particular group of users.
The main findings
(Research conducted with me and the internal Bosch team. The in-depth report is not available. All interviewed participants were males 22 to 64-year-old)
- The software is hard to use – as expected and a major complaint
- The very steep learning curve for new users
- Unintuitive interface – inconsistency in labelling and naming convention
- Broken links
- Younger users were far more skilled in the new digital environment
- Senior users were more efﬁcient using existing software
- Senior users success on specific tasks vary depending on expertise
Their fears and frustrations
- The older generation don't like small interface
- Anxiety over laptop battery life during vehicle repair
- Network connection trouble leading to some techicans not knowing the status of a current operation
- Broken links
- "Where am I?" - question asked often by new technicians
- Confusion about what parts of the interface interact (some technicians forcefully pressed some interface icons in frustration while expecting the application feedback)
- Some technicians feared what would happen if they stop the specific process
Using all the data available during the study, three main personas were given additional personality attributes to appear just like real users hence giving us a more emphatic approach while developing various stages of the design process
Information Architecture: Flow to facilitate user tasks
Creating red route
Now that we have personas and know who we are designing for it was time to find out what kind of goals our users will have, what specific tasks they are trying to achieve or problems to solve. Since we have personas, we don't have to think about all possible users. We will focus on the persona's needs to create a user journey. The core features will be available at all times while some tools, not used by specific users, will be hidden. We are not trying to create a flexible application, that is a trade-off for usability.
One way of identifying red routes is to find out critical and frequent tasks. First, I quickly created a journey of obvious tasks which every user was completing. After, I categorised tasks based on how often technicians are using and how many of them were using specific tasks.
Once we have classified the importance of tasks, it was time to map the content of the application. I organised a meeting and we categorised all ingredients of the application, including all diagnostic tools. The purpose of the workshop activity was to understand how various parts of the software are going to interact and to further analyse the most important path technician takes during the vehicle diagnosis.
Flow with red route
Finally, based on previous activities, I create a full user journey with a critical journey, a red route, identified on the map.
User Interface Design: Connecting the dots of a structure
Wireframing and paper prototype
It was a time to zoom in into the system and based on what we created previosywith e knew the ingredients and the flow of the system cleat, it was time to organise on-screen elements. I created over 40 wireframe layouts to accommodate all important elements of the layout. I printed them and created a paper prototype of the application to test red routes and various other tasks. Since some stakeholders objected to investing time in another usability test, we quickly tested internally with five people which gave us reasonable results, although obviously not enough to make safe assumptions. The major test would be carried after the full visual interface design is completed.
User interface design paterns
Losely based on outdated styling guidelines I created visual design paterns. In fact, only colours and fonts were inherited from the guidelines
User interface menu design
Also, a menu design was created including icons, which should metaphorically reflect the tools as much as possible and visually guide a user through the product. One important feature we addressed here is the ability to quickly access various tools. I designed a facility so a user can pin the menu and have quick access. Since the menu is taking a lot of screen estate, we have decided to create a pinned many with icons visible without description.
User interface Visual design
Finaly, I have put togheter all ingridients and here is the final design outcome , including a mobile version
Evaluation of new software: What we have learned and how does it compere with old product?
We have used an identical methodology to test as we did with the old version of the software. This time we had the same 18 participants as in the previous test, and five more younger technicians. As you can see from the following pictures, the improvement was noticeable, especially with younger technicians.
The previous test success rate with all participants was 47% and while testing the new application the score was 84%, suggesting that effectiveness was largely improved.
- Experienced technicians: previous test 82 seconds, while test with the new interface was 107*
- Younger technicians: previous test 99 seconds, while test with the new interface was 91
The application was developed with Angular JS which I had not much experience with but in terms of styling I felt I would be useful. I was with the team for only a further two weeks during the application development process to help developers understand designs and whatever was not clear to them at that point. During my stay, the application was already sold to Mazda, Mercedes Benz, and Jaguar Land Rover.
I have reflected on my assignment on what I thought would perhaps be even more beneficial during the design process:
- System Usability Scale (SUS) quality test to measure users satisfaction (the senior stakeholders thought that wasn't necessary). I thought the test would give another dimension of users feelings while using the product but stakeholders felt that it was obvious that users are not satisfied with the current product.
- Time spent with developers - I think it is important to collaborate closely with people who are actually making the product alive. Often they will need an explanation from a design point of view of how to solve a problem. A lot of small issues during the development is not easy to predict during a design process hence I wish I have spent a little longer with the team
- Warmer clothes during the field visit :)
The project was judged as a success by senior stakeholders and the product was already on the market even before it was developed. In the meantime, I was approached by the Ford team to work on a similar project. I don't have the intention to write another case study for the Ford project but here are a few images: