How to use Free-listing to explore a domain

Author: Rashmi Sinha

On Nov 12th, 05

	Beginner’s guide to free-listing
	Brief Description:		Elicitation method used to understand domain contents and structure
	Uses:		Precursor to card-sorting, or developing user profiles. Understanding user language, creating a started controlled vocabulary. Analyzing cross-cultural differences.
Expertise level:			Beginner to expert
Research medium:			In person, phone, online and paper
Time taken:			3-4 minutes each person
Free-listing Visualizations:			Tag Clouds to show list of items, Onion Maps of domain structure, Association Maps of relationships, Consensus Maps between groups

Freelisting is an elicitation method for identifying whats salient in a domain, and for defining the boundaries of the domain. A simple and versatile method, it can answer many questions that product design and marketing professionals want to ask of their users. For example, a designer working on a personal financial management software could use freelisting to identify what people think is important in this domain. Or an information architect could use it to create a list of comic book genres for an online catalog.

 

How does freelisting work:

Conducting a free-listing exercise is very simple. All that is required is to ask someone: “Name all the X’s you can think of”. For example, name all the domestic animals you can think of. You can let people list as many items as they want to, or you can give them a specified amount of time. The average number of items people list depends on the domain, but 30-40 items are not uncommon. We use our MindCanvas platform to conduct free-listing exercises, but you can also conduct such exercises as part of an in-person interview, or on the phone.

When you ask multiple people about the same domain, you get two types of responses. For example, consider a freelisting exercise about the “movie watching experience.” Some of the listed items such as eating popcorn, going on a date, renting a movie, buying tickers online will come up in multiple lists, as they reflect cultural consensus around the movie watching experience. Some of these items will also tend to occur earlier in the list. Through a systematic analysis of these common terms, you can understand what items are culturally salient and map the boundaries of a domain. Other items represent more idiosyncratic associations with the topic. Although the stated goal of free-listing is to map semantic/cultural consensus, we find that these idiosyncratic associations also provide insight into the individual perspective. For example someone might list “arrange for babysitter” as part of the movie-watching free-list. Another person might list, "bad date".

 

Uses of freelisting

Free-listing can be used in a variety of ways to understand a domain. We have often used free-listing at the beginning of a design or research project, with users, stakeholders and design teams. In one instance we used it to map how three groups (users, designers and business stakeholders) differ in their thinking about a domain. This lead to insights about the business model, user experience and pricing starategy.

We have also used free-listing as a consensus building technique - to explore whats common in the perspectives of a group of people, and reflect that information back to the group. The scenarios below will illustrate different uses of free-listing.

• As the first step in a design/redesign project, prior to a card-sorting exercise. Example scenario - you are conducting a card sorting exercise to understand user mental models for a digital photo application. A free-listing exercise can help you gather items for use in the card-sorting exercise.
• As the first step in developing user profiles. Example scenario - you are conducting a survey to learn about users and develop user profiles and want to touch upon all relevant issues in the survey. A free-listing exercise could help you identify all the salient issues in user’s minds.
• To explore cross-cultural or other types of group differences. Example scenario - an online bank is extending its banking facilities to Asia, and wants to understand how the concept of online banking differs in Asia from America.
• As the first step in creating a controlled vocabulary or taxonomy. Example scenario - The online bank above wants to start developing a controlled vocabulary for use on the online Asia system. Free-listing can serve as the first step in this exercise.
• To compare two domains. Example scenario: A company that offers personal tax software is moving into small business tax software, and wants to understand similarities and differences in the way that consumers view the two domains.

How can a relatively simple technique like free-listing help provide answers to these questions? Free-listing tap into cultural consensus about an issue. Even though each person’s contribution takes relatively little time, across people you can start identifying trends and patterns.

 

 

Visualizing results of a Free-listing exercise

 

 

We deliver our findings from a free-listing exercise through a number of deliverables that allow a designer / researcher to get insight into the domain. The

 

 

Onion Map shows the contents in a domain and delineates the domain boundaries. Its called an Onion Map since it shows domain structure as concentric circles, like layers of an onion. We divide the domain into three levels of importance. Items in the core are considered the most important in the domain, the middle are somewhat important, while those in the periphery

 

 

Consensus map shows consensus and and differences in perceptives of multiple groups about the same domain. For example, a consensus map could depict the differences between two cultures using Venn diagrams. Each area shows the percentage of items that fall in it, allowing easy comparision of whats overlapping and whats unique for the groups.

Not every relationship can fit into a hierarchy although information architects have so far almost exclusively relied on a hierarchical representation of domain structure.

 

 

An association map shows a non hierarchical view of domain relationships. It is created through a statistical analysis of relationships across all respondents. The lines shows the distance between items in the domain. Core items occurr in the center while less important ones occur in the periphery.nted and The core items are in the center while the less important items a

 

 

APPENDIX

Number of users: Free-listing relies on identifying the common patterns in the lists of multiple people to identify the cultural conseseus. Depending on the nature of the domain (whether it is culturally well defined or not) the number of people needed can vary. Researchers suggest a minimum of 30. Our own exercises have varied from 5 to over 1000 (conducted online). 5 was used primarily for item generation mostly for a small, homogenous, hard-to-reach group whereas 1000-user free-listing exercise allowed exploration of subtleties, such as differences between four groups, identifying the core, middle and periphery of the concept, providing input for a basic controlled vocabulary.

Sampling: Free-listing helps identify cultural consensus regarding a semantic domain for a given group of people. If your study participants are students from Stanford, then your free-listing results represent exactly that: cultural consensus among Stanford students. Whether you can extrapolate from the Stanford students to all US students or any other group is a judgement call that is beyond the scope of the technique itself.

 

Tips for Data Analysis of Free-listing exercises

Like the exercise itself, free-listing analysis can range from simple Excel based analysis to sophisticated statistics and visualizations. Excel-based analysis will help you make some broad conclusions regarding the trends. Preliminary analysis of small datasets (upto 10-12 users) can be done using Excel. The Excel analysis is quite simple. Put all the words from all the lists into Excel, along with the associated User ID on each row. Column 1 should contain the word itself, while Column 2 should contain the User ID (look at Figure 1a for an example). Now select both the columns and sort by “word” so that it looks like Figure 1b. This will give you some rudimentary counts for each term. For example, wireless and gps occurred three times, while ringtones occurred two times

 

.

Deeper Insight requires statistical knowhow. But even such simple Excel based analysis can be worth the effort. If you have done the Freelisting exercise with different user groups, then do the above analysis separately for the different groups to understand group differences.