Final Report for CUTGroup #28 - City of Chicago Open Data Portal Homepage

CUTGroup #28
Open Data Portal
Report by
Sonja Marziano, Project Coordinator
Christopher Gumienny, CUTGroup Proctor
Smart Chicago Collaborative

CUTGroup 28 - Chicago Open Data Portal
In-person test on September 28, 2016
Introduction
For the 28th Civic User Testing Group (CUTGroup) test, we tested the redesigned homepage of
the City of Chicago’s Open Data Portal. The Open Data Portal allows users to find resources
and various datasets regarding the city of Chicago. The City of Chicago Department of
Innovation and Technology is working with Socrata to redesign the Open Data Portal, focused
currently on the homepage, to be more user-friendly while representing multiple data and
technology initiatives and applications created with open data.
Target Group
On September 22, 2016, we sent an email to 1,172 CUTGroup testers who lived in Chicago,
and notification via text message to our testers who prefer to receive notification via text. We
wanted to know if they would be available for an in-person test on September 28. When
segmenting our testers, we were interested in testing with:
● Testers who use a variety of devices: Android, iOS, and laptops. We wanted to have an
almost equal representation on all these devices types to better understand the
homepage’s functionality on different devices.
● Testers who have used the Open Data Portal in the past and those who have never
used the Open Data Portal before. We wanted to have people compare both data portals
while including new, first impressions.
● Testers who have all levels of data experience to see how user-friendly the search
functionality is.
Test type
● In-person test proctored on both mobile and laptop devices. We included 6 testers on
laptop devices, 6 testers on Android mobile devices, and 5 testers on iOS mobile
devices (total of 17 testers).
● Testers were paired with proctors to record their interactions and comments or opinions
about the City of Chicago Open Data Portal homepage.
Goals of Test
The main goal is to understand how testers who have some familiarity with the data portal (even
minimum) respond to the changes made to the homepage. We want to capture how residents
search and what is the best structure for the homepage to make searching easier. Lastly, we
want to see how responsive testers are to the other content that relates to the programs and
tech initiatives of the City of Chicago.
1

Responses
17 testers completed the in-person CUTGroup test of the City of Chicago’s Open Data Portal
homepage at Woodson Regional Library, 9525 S Halsted St, Chicago, IL 60628, on September
28, 2016.
Background questions
What successes or challenges have you had finding data that you were looking
for or using the Open Data Portal? Tell us about any of your past experiences.
6 out of 17 testers had no previous experience with datasets or using the Open Data Portal.
8 testers expressed positive experiences using Chicago’s Open Data Portal. For example,
CommunityResearcher 201601 (#11) said,“Somewhat successful. Used to research a
paper a number of years ago, [and] was able to find what I needed on the elderly
population.”
3 testers shared challenges using Chicago’s Open Data Portal.
Syliva (#3) said,“It’s not that data is difficult to access, but the results are not relevant. If
I'm looking up restaurant data I don't want school data for instance. Results are not
always relevant.”
Blessed (#6) said,“I usually look for crime in the immediate block. Theft/robbery, person
or house. [It] wasn’t accurate.”
We learned from this CUTGroup test that there was some confusion around whether or not
people had used the Open Data Portal before or the City of Chicago website. Some testers very
clearly used the Open Data Portal, while a few testers shared experiences finding city services,
which led us to believe that they were speaking about the official City of Chicago website.
What are your expectations of what the City of Chicago Open Data Portal would
be useful for?
Here are some responses from testers about what they expect the City of Chicago Open Data
Portal would be useful for (number indicates number of times a response was given as testers
sometimes had multiple answers):
6 – Search for specific data
3 – Analyze data
2

3 – Find City services
2 – Creates transparency
2 – Find jobs
1 – Find doctors
1 – Mapping
1 – Develop apps using the data
1 – Pay parking tickets
1 – Communicate with Aldermen
Gabireads (#2) said that the Open Data Portal is useful for “Mapping. Creating thematic
maps. Census, crime, restaurant, neighborhood data to make maps and analyze data.”
Batman (#14) said,“ To have data/info on various municipal systems, licenses, historical
data on various going-ons.”
M (#15) mentioned,“Great if everything that is not confidential was available. Would
enable better social science and community involvement, services, security, etc.”
Initial Homepage Review
Review the homepage and tell us about your first impressions. While you
review, try not to click on the next page just yet, but feel free to scroll up and
down the page.
When reviewing the redesigned homepage for the first time, 13 testers had initial positive
impressions, 2 had critical impressions, and 2 had neutral impressions.
Mobile users
The majority of mobile testers had a positive reaction to the homepage with 5 testers mentioning
that it seemed straight-forward. 3 testers had negative responses when viewing the homepage
on their mobile device.
4 out of 11 testers specifically mentioned the categories, whereas only 1 tester mentioned the
search bar. Other things testers mentioned: Divvy (3), “Service Tracker” (2), clear/readable font
(2), specific datasets (2).
Funky Burrito (#10) says, “Well laid out, straightforward. Right amount of pictures
versus text content. Feels easy to navigate based on criteria of information.”
VP (#12) thought that the homepage is “Well organized. Preview of what is to come is on
the banner. I assume that I would be able to swipe but you cannot. Also lots of scrolling.
Visually seems very cramped, not aligned.”
3

GPBlight (#4) believe that “Flashing promotions are unnecessary. It should focus on the
top three items people search for, not what they are pushing" in relation to the Divvy
advertisement.
Laptop users
4 out of the 6 testers who viewed the homepage mentioned the categories in their initial review,
while no one mentioned the search bar.
Most testers had a positive reaction to the homepage, while 1 tester specifically said that there
was too much information. Other things testers mentioned: Divvy (1), “Chicago apps” (1), “How
to video” (1).
Batman (#14) says, “Looks clean and modern. The old one wasn't easy to navigate and
this page has broken up categories to find stuff more easily. Info is more visual.”
TJ (#5) thought there is “Too much information and stuff on the homepage. ‘Digital
Chicago’ section should be bigger.”
Lhibbard (#7) says, “Not exactly what I expected. Didn't expect to see a map or
information about Divvy. Thought it would highlight city services. Noticed the Map first.
Looks more fresh and updated.”
What are three actions that you know you can do just from reviewing this first
page?
Here are responses we have heard from testers about actions they can do just from reviewing
the homepage (number indicates number of times a response was given as testers sometimes
had multiple answers):
6 – Browse data catalog by choosing category
6 – Tutorials/Videos
4 – Learn about community
4 – Use Chicago apps
4 – Request city services (street sweeping, garbage pick-up, 311, etc)
2 – Find events
2 – Navigate crime data
2 – Click on menu
2 – Type information in search box
2 – Transportation
2 – Find health and human services
1 – Environmental data
4

1 – Review free maps
Thrones (#1) mentioned,“Look at different events, coordinate routes, and find out what's
going on in your community.”
Syliva (#3) said,“Click on menu, can type in info in search box, or I can choose
categories from below.”
Lhibbard (#7) said,“Look up community, health and human services, look up information
about transportation.”
Who do you think this website is designed for? Why?
14 testers mentioned that this Open Data Portal is designed for the general public/residents to
help find city services/resources (3), government transparency (1), and access data (1). Testers
who said it was for residents said so because it is “easy to use and not laden with jargon”
(Gabireads #2) and “to help people find resources in the City of Chicago” (Thrones #1).
3 testers specifically said the Open Data Portal is designed with people with moderate to high
technical savviness. 1 tester mentioned it was for people interested in data analysis. 1 tester
said it was built for business owners, while another said it was built for developers.
After reviewing the homepage, what is the first thing you want to do or click?
Clicked on a data catalog category 59% (10)
Search to find a specific dataset 18% (3)
Clicked on a "Digital Chicago" topic 6% (1)
Clicked on a "Chicago App" 6% (1)
Other: Clicked on menu on the upper right 6% (1)
Other: Clicked on Divvy banner 6% (1)
What did you search? Or what was the category you chose?
Category: FOIA (3), Education (3), Events (2), Transportation (1), Health & Human Services (1)
Search terms: Food (1), Building safety data (1), Community (1)
Tell us more about the first thing you did, and why you did it. What were you
expecting to happen?
5 testers expected to find more general information about what the website does or the type of
information listed under a category. 4 testers had specific topics to query. 1 tester expected city
service options.
5

Clicked on a data catalog category
Mobile users (6)
FOIA 50% (3)
Education 33% (2)
Events 17% (1)
Laptop users (6)
Human Health Services 32% (2)
Transportation 17% (1)
Community 17% (1)
Education 17% (1)
Events 17% (1)
TJ (#5) chose the transportation category because he “wanted to find CTA maps dealing
with train transportation” and it “met the expectation.”
“Search to find a specific dataset…”
Mobile users (1)
“Food” 100% (1)
Syliva (#3) searched the term “food” and said,“Food, because I enjoy fine dining. Was
expecting to see a list of restaurants.”
Laptop users (1)
“Building safety data” 100% (1)
“Other”
Mobile users (4)
Clicked on a "Chicago App": OpenGrid 25% (1)
Clicked on a "Digital Chicago" topic 25% (1)
Other: Clicked on menu on the upper right 25% (1)
Other: Clicked on Divvy banner 25% (1)
VP (#12) clicked on Divvy banner because the user “thought it would take me to a list of
subcategories but it didn’t.”
6

Next, search for a dataset you're interested in learning more about. How did
you search?
Chose a data catalog category 59% (10)
"Search to find a specific dataset..." box 35% (6)
Used search in navigation 6% (1)
Search term (if used)
Animal control (1), CPS (1), Crime statistics for Lake View (1), Government (1), Lead
testing schools (1), Nursing (1), Potholes (1)
Category (if used)
Buildings (2), Community (1), Events (1), FOIA (1), Historic Preservation (1), Sanitation
(1), Service Requests (1), Transportation (1)
Tell us more about your search experience. Did you find what you were looking
for?
8 testers had difficulty finding datasets they wanted or said the “language” made it hard to find
desired results. 2 testers expected city services such as employment or 311 service requests. 3
testers were still exploring the site generally trying to learn more about what is listed in certain
categories. 2 testers mentioned that the data appeared outdated, while 2 other testers could not
distinguish a rational order to the results.
Miko (#13) typed in “public transportation and received “Open Spaces” results which
were not relevant to her. She mentioned how this site uses “A certain type of lingo for
certain type of people… not geared towards a regular Jane or Joe.”
Did the search meet your expectations?
Out of the 7 testers who searched:
Yes 29% (2)
No 71% (5)
Out of the 10 testers who chose a category:
Yes 30% (3)
No 70% (7)
Why or why not?
When expectations were not met, a lot of the time it was because it took too long or too many
steps for the tester to find their specific query. Some testers, again, expected city services or
7

employment information that could be found on the general City of Chicago website and not
necessarily an Open Data Portal.
Lhibbard (#7) did not find what they were looking for and chose the service requests
category and said,“I found the news about 311 service requests. I thought I might be
able to make a personal request directly to the alderman.”
Thrones (#1) searched “nursing” and it did not meet expectations as tester was looking
to review human resources or employment information.
For those that had expectations that were met, testers were pleased to find and that the
homepage provided more options and details.
TJ (#5) found what they were looking for and chose a data category. They said they
were “Looking for libraries and found locations/hours.”
Batman (#14) searched for “potholes” and found what he expected, “Curious that they
list recent pothole fixes. Wanted to see potholes patched recently and got what tester
expected.”
Search for the building permits dataset. How did you search?
"Buildings" Category > "Building Permits" dataset 70% (12)
"Search to find a specific dataset…." box 24% (4)
"Buildings" Category > "Building Permits" data lens 6% (1)
One tester, Miko (#13), first searched, but the results did not load on their mobile device, so the
tester then clicked on the “Buildings” category.
Building permits (4)
Anything else to note about this search experience?
Most testers agreed this search was simple and straightforward but GPBlight (#4) suggested
that in the data results there “should be an option for ward which is not on the map” and
Batman (#14) thought it was “Interesting that some things go back 2 years, others 10 years.
Might be useful if data can be accessed further back.”
Next, search for a map of crime data. How did you search?
"Public Safety" Category > "Crimes" map 35% (6)
"Public Safety" Category > "Crimes - 2001 to present" dataset 23% (4)
"Public Safety" Category > "Crimes - 2001 to present" data lens 18% (3)
8

“Search to find a specific dataset…” box 18% (3)
Chose "Crimes - 2001 to present" link on header image 6% (1)
News (1), crime map (1), crime data (1)
6 of 17 testers had little to no difficulty finding the map, however, a couple of testers commented
on the map functions and legend. For example, Gabireads (#2) zoomed in near her home and
said that it’s not clear what it is a map of, nor the time period.
7 testers knew that crime data would be listed under the “Public Safety” category but chose
either the dataset or data lens result, which appeared higher in the results list. These testers,
therefore, did not find the crime data map as tasked.
One tester, Miko (#13), reviewed the categories before typing in “News” into the search box.
This did not come up with relevant results, and so she typed in “Crime data” in the search box
and then chose the data lens option.
Next, find the potholes that have been recently fixed. How did you search?
“Service Requests” Category > Potholes Patched Last 7 Days 76% (13)
“Search to find a specific dataset…” 18% (3)
Other: “Sanitation Category” then “Search to find a specific dataset…” 6% (1)
6 testers out of the 13 who clicked the “Service Requests” category attempted to find this data
under another category first.
Potholes (1), Potholes fixed (1), Pot holes (1)
While almost all of the testers were able to find this dataset, a lot of testers had difficulty
choosing the correct category at first. 6 testers expected that potholes dataset would be either in
the Sanitation, Environment, Public safety, or Transportation categories.
One tester, Thornes (#1), tried to find this information first in the “Sanitation” category, then in
the “Public Safety” category, before finally switching to the search functionality and searching for
“potholes.”
9

Next, search for active business licenses. How did you search?
“Community Category > “Business Licenses - Active” dataset 65%(11)
“Search to find a specific dataset…” 35%(6)
10 out of 11 testers who eventually found the active business licenses dataset explored other
categories first. 5 out of the 6 testers who used the search functionality had difficulties with other
search terms.
Active business licenses (5), business licenses (1)
15 out of 17 testers had some level of difficulty in finding active business licenses either by
choosing a category or finding search. 6 testers specifically tried the “Administration & Finance”
category first, while 4 other testers suggested the “Business” category.
Brittsky (#9) explored the “Buildings” category before searching “active business license.”
Testers who searched had some difficulties with misspellings in their search terms. M (#15) first
clicked on the “Administration & Finance” category and then searched “license” and could not
find the correct data. Searching “Business licenses” led to a successful search.
Find the City of Chicago's community boundaries map. How did you search?
"Facilities & Geo. Boundaries" Category >
"Boundaries - Community Areas (current)" map 65% (11)
“Search to find a specific dataset…” 29% (5)
Other: Chose a Digital Chicago article 6% (1)
4 out of the 11 testers who selected a category first chose a different category before choosing
the “Facilities & Geo. Boundaries” one.
City of Chicago community boundaries map (1),community boundaries map (2),
community boundaries (1), community areas (1).
4 testers expected the community boundaries map to be located under the “Community”
category, and not only the “Facilities & Geo. Boundaries” category. TRobin312 (#17) even
pointed out that the hover over help text of the “Community” said “boundaries” would be found in
this category. The correct result, however, was not apparent in the many search results.
10

One tester, Miko (#13), was viewing the site on her mobile device and clicked the “Digital
Chicago” map image for the article “Building Safety Data on the Open Data Portal” because this
image appeared to contain a boundary map. She tried to search, but the magnifying glass
button was not working and therefore she assumed her results were invalid and became
frustrated with this experience. She also attempted searching in the “Parks and Recreation”
category.
The majority of testers had an easier time finding the correct results for this task than the last
task of finding active business licenses.
Overall, how easy was it for you to find these datasets?
5 – Very easy 35% (6)
4 – Easy 35% (6)
3 – Neutral 24% (4)
2 – Difficult 6% (1)
1 – Very difficult 0%
Tell us more about your search experiences and how you would improve the
site.
While the majority of testers said it was “very easy” or “easy” to find datasets, there were clear
improvements that should be made. There is also evidence in how testers completed the tasks
that indicate that categories and search terms were not always intuitive and it sometimes took
testers multiple tries to complete the task.
2 testers specifically told us improvements that were about city services and not open data.
Mobile
Testers on mobile devices discovered a bug with using the magnifying glass button that did not
function. On mobile devices, we saw that filters were rarely used, and some testers found it
easier to find a dataset using other options such as going back to the homepage and trying
another category or using the “Search for a Specific Dataset…” box function.
Laptop
We learned that the search bar was a bit lost on the homepage especially on laptop devices. A
couple of testers suggested to make the search box bigger and a higher contrast.
11

Next, explore the "Digital Chicago" section on the homepage. Tell us about the
information you find.
10 out of 17 testers clicked on the Array of Things article, which is the first article under the
“Digital Chicago” section. 4 testers exclaimed that they did not know what “Array of Things”
means and did not find the descriptive text helpful. When reviewing the “Digital Chicago section,
2 testers mentioned that that the “Month in Review: July 2016” article was out of date.
2 testers tried to click on the “Digital Chicago” header, but it was not hyperlinked as expected. 2
testers also mentioned that they do not know what “Digital Chicago” means.
One tester, who was viewing this on her mobile device, did not think that “Digital Chicago” was a
section different from the rest of the page.
Lhibbard (#7) mentioned that this section “seemed more friendly to city employees than
city residents.”
How relevant do you think the "Digital Chicago" section is to you?
5 – Very relevant 24% (4)
4 – Relevant 24% (4)
3 – Neutral 24% (4)
2 – Not very relevant 12% (2)
1 – Not at all relevant 12% (2)
Not applicable 6% (1)
Africa (#16) chose a dataset versus reviewing the “Digital Chicago,” and, therefore, is listed as
“not applicable” in this question.
Why or why not?
Here is what some testers said when describing “Digital Chicago” as “relevant” or “very relevant”
(8/17 testers):
TJ (#5) felt this section,“Provides valuable info for the public dealing with buildings,etc.”
Thrones (#1) was content and that the section was relevant but,“Could have more
details. Kind of short.”
For those saying it was not relevant (4/17 testers):
Lacked descriptive and concise content. One user felt it might be more meaningful to city
employees than residents. Some also felt it might be an interesting read from time to time but
not necessary to know.
12

TRobin312 (#17) claimed, “[I] don’t understand what information I'm supposed to get in
this section.”
Next, explore the "How to Videos" section on the homepage. Tell us about the
Mobile device testers experienced issues in viewing this section because it was not aligned
properly on the page and cut off the video and the description.
4 testers mentioned that they thought “How to Videos” are a good way to provide instructions or
information, another tester would not get information in this way. 2 testers mentioned that they
liked the specific information found on the City of Chicago Department of Innovation &
Technology’s YouTube page.
Upon viewing the videos in the playlist, 3 testers mentioned that the videos were too long.
Funky Burrito (#10) suggested breaking down the large videos into smaller modules. Both
Blessed (#6) and Funky Burrito (#10) were surprised to be directed to YouTube without any
notifications after clicking on “More videos.”
How valuable do you think the "How to Videos" section is to you?
5 – Very valuable 41% (7)
4 – Valuable 24% (4)
3 – Neutral 18% (3)
2 – Not very valuable 12% (2)
1 – Not at all valuable 6% (1)
Why or why not?
Valuable
Testers that found this section valuable thought that it was eye-catching and the videos
provided nice overviews/tutorials of different subjects. Gabireads (#2) said,“ Easy way to get an
overview of something… good way to provide instruction and make robust use.”
Not valuable
Testers that found this section not valuable felt that the videos should be shortened. 2 testers
said there should have been a notification when leaving the main site for YouTube. Blessed
(#6) also shared that they are “Not interested in [viewing] a video to find one piece of
information.”
13

Next, review the "Chicago Apps" section on the homepage. Tell us about the
Many testers (11) found out they can find info on when their streets would be plowed/swept and
could request 311 services which they felt was useful. 1 tester on mobile felt “Chicago Apps”
was not distinctly separated from rest of page. 2 testers tried to click on “Chicago apps” header.
Were you familiar with any of these apps? If so, which one(s)?
15 testers were not familiar with the apps.1 tester was familiar with 311 service tracker app
while another had heard of snow plow tracker app but never used it.
How likely are you to use these "Chicago apps" in the future?
5 – Very likely 59% (10)
4 – Likely 17.5% (3)
3 – Neutral 17.5% (3)
2 – Not very likely 6% (1)
1 – Not at all likely 0%
Why or why not?
Most testers (13) said they would be likely to use these apps in the future as they are
convenient, faster than calling a city employee, and, overall, useful to have as residents.
VP (#12) said he was unlikely to use this “Chicago apps” section in the future. He
mentioned that there’s “No need. Just do a Google search.”
GPBlight (#4) was neutral because“In my neighborhood, the data is seriously flawed,
which makes it inaccurate."
Overall, how easy or difficult did you think it is to use the new redesigned Data
Portal homepage?
5 – Very easy 41% (7)
4 – Easy 47% (8)
3 – Neutral 6%(1)
2 – Difficult 6%(1)
1 – Very difficult 0%
What makes you say this?
The majority of testers (15) found the new, redesigned homepage easy to use since they found
what they were looking for even if it took testers multiple tries to complete tasks. Some did,
14

however, agree that the search “language” and function could be improved to make it give
better results when trying to search. Categorization could also be improved as many users
expected to find one thing in one area and not another.
After reviewing the site, do you think you are the target audience for the Data
Portal?
Yes 76% (13)
No 12% (2)
Not sure 12% (2)
If not, who is the target audience?
Blessed (#6) also did not feel they were the target audience and claimed it was for “rich
people.
M (#15) was unsure and said,“Broad target audience. At least some parts of the site are
intuitive and do not require data science background but if you do have it, easy to find
data.”
How likely are you to use the City of Chicago's Open Data Portal in your
personal life?
5 – Very likely 58% (10)
4 – Likely 18% (3)
3 – Neutral 18% (3)
2 – Not very likely 6% (1)
Why or why not?
Out of the 13 testers who are likely to use it in their personal life, only 3 testers said they would
use it specifically for data purposes. 2 out of these 13 testers mentioned that they would use if
because of the apps. Additionally 2 testers said they would use it in their personal life because
they are curious. 5 of the 13 testers would use it because they believe it will give them
information about their community or connect them to City services.
Those who are neutral or unlikely (4) to use in personal life claimed they don’t use city services
or can just use Google to access what they need.
professional life?
5 – Very likely 53%(9)
15

4 – Likely 29%(5)
3 – Neutral 12%(2)
2 – Not very likely 6%(1)
Why or why not?
Many testers (14) are likely to use the data portal in their professional lives as they have a
business, work a career requiring use of this data such as a librarian or human resources
coordinator, or will use for educational/research purposes.
Gabireads (#2) is an academic librarian and said,“I work a lot with students who need
access to data in a range of areas.”
Those who are neutral/unlikely to use it professionally said the info and data available is nice
but don’t have a real need for them in their lives or careers.
Funky Burrito (#10) said,“Perhaps not the datasets but the apps I am very likely to use.”
[Question only for testers who used the data portal in the past]
How does this redesigned data portal homepage compare to what you
remember of the current portal?
From our background question, we learned that 11 testers used the Open Data Portal in the
past. As we previously mentioned, we understand that there might be some confusion around
whether or not people had used the Open Data Portal before or the City of Chicago website.
8 testers (73%) reacted positively and said it is more user friendly due to the addition of
categories and making search easier. Testers also shared that the new homepage looks
modern and updated.
3 testers (27%) felt neutral or indifferent about the redesigned portal, and mentioned
they only care about retrieving data and not design, or that it only targets specific people
such as people who recently move here or residential/business developers.
What improvements would you make to City of Chicago Open Data portal
homepage to make this site easier to use, more valuable/important to you, and
more likely for you to use it again?
In this question, 8 testers did not mention any additional improvements. 3 testers mentioned to
make the search bar more visible, 2 testers wanted to see simpler language used, and 2 testers
reiterated that better categorization was needed. 1 tester wanted to see the icon for the
“Community” changed from a shopping cart to something more relevant. Lastly, another tester
16

mentioned using a breadcrumb menu system as there is too much scrolling up and down and
each section is not distinctly separated and it seems like a continuous newsfeed.
All other improvements were shared when completing previous tasks or reviewing content.
CUTGroup Review
Was this your first CUTGroup test?
Yes 41% (7)
No 59% (10)
Do you like the CUTGroup?
Yes 100% (17)
No 0%
Did you like this CUTGroup test?
Yes 94% (16)
No 6% (1)
Anything else to add about CUTGroup?
“Love it, look forward to more surveys that are better to the city. No matter what age you
are.” -Blessed (#6)
“Learned more with this test. Thanks for coming to the Southside.” -Brittsky (#9)
GPBlight (#4) asked, “When are they going to make the data itself user-friendly, not for
those who use data professionally?”
Gabireads (#2) says,“It was fantastic. Has to remember to keep looking at the site to get
to play with these products and sharing them with other people.
17

Results
Testers believed that the redesigned Chicago Open Data Portal
homepage is designed for the general public and residents, but many
expected access to city services.
At the end of this CUTGroup test, we asked testers if they felt that they are the target audience
for the new, redesigned Open Data Portal and 13 testers (76%) said “yes.” Our testers included
individuals who had no, little, or some familiarity with the current data portal, and 14 testers
mentioned that this site appeared to be designed for the general public and residents.
3 testers specifically said the Open Data Portal is designed for people with moderate to high
technical savviness. 1 tester mentioned it was for people interested in data analysis. Another
tester said it was built for business owners, while another said it was built for developers.
One concern is that testers, even after review of the website, still expected more access to city
services and resources that could be found on the official City of Chicago website. There are
some possible solutions that would either provide this general service information to residents or
better define the purpose of the website.
Identify access points to city services or action steps for residents.
There are many different levels of creating these access points to city services that residents
expected when first viewing the Open Data Portal. A very simple option is to add a link to the
City of Chicago website in the navigation bar or footer of the homepage.
A second, more complex level is at the category level. For instance, if a user clicks on the
“Events” category there could be a link to the City of Chicago events’ page before users even
begin to explore datasets. This level of access would require reviewing all of the categories and
understanding if users associate city services with those category types.
The most complex level is at the dataset level. Since we conducted usability testing primarily on
the homepage and the action of searching for datasets and reviewing information, this would
require additional user feedback. The suggestion is to connect relevant datasets to the
appropriate city services. Therefore, if a user is reviewing data about potholes, as one example,
there could be a link to make a pothole service request.
Utilize the header image to clarify purpose about the site.
“What’s this sales thing? It makes it look like it’s advertising stuff,” GPBlight (#4) said in
response to reviewing the header images. When the homepage first loaded, a bug was
18

immediately noticeable because the header images would stack and cause the page to jump
upon load. While that is a detectable fix, we identified a larger opportunity to share information
about what the website does. Very few testers reacted to the initial “data sale” image and those
who did react had negative responses. The first header image, before rotating to either the
crime or Divvy data, should define the purpose to the site while being as clear and direct as
possible. The current “data sale” language and images confuse some users.
Searching for datasets was not always intuitive when using search or
categories and testers often had to try multiple times to complete a
task.
We asked testers to search for six different datasets; one was open-ended, while the rest
proctors requested that they find specific datasets: building permits, map of crime data, recently
fixed potholes, active business licenses, and Chicago’s community boundaries map. We were
interested to see how testers would search and find the datasets and what issues they came
across during the process. While 12 testers (70%) indicated that finding these datasets was
“easy” or “very easy” we witnessed that it often took testers multiple tries to find the correct
dataset and some testers could not find the proper result or chose a relevant, but not requested
dataset.
Categories were often chosen over search bar, but technical bugs and design
influenced those choices.
Only 1 tester out of 17 (6%) used the search bar consistently for all of the dataset search tasks,
whereas 7 testers (41%) used categories to complete all dataset search tasks. The remainder of
testers used both search and categories to find what they were looking for depending on the
type of dataset we requested them to find.
On mobile devices, the magnifying glass of the search bar was not working properly. Therefore,
if testers attempted a search nothing would appear and testers thought the search was broken
or that there were no search results.
When doing the initial review of the page, only 1 tester mentioned the search bar, which
indicates that it is not a prominent part of the page. When asked about what actions they knew
they could do from reviewing their page, only 2 testers mentioned the search bar. When asked
to take their first action, 3 testers (18%) searched whereas 10 (59%) clicked on a data catalog
category.
We understand that a lot of these testers did not have much experience with the data portal and
the data catalog categories are useful for exploration purposes. Nonetheless, we realized that
the search bar was not very prominent for testers. Suggested improvements included larger
font, higher contrast, and a more prominent location on the page.
19

Categorization needs to be more intuitive and filtering a higher priority in the
user experience or search needs to be more flexible.
While testers knew where to find building permit and crime data, finding recently patched
potholes, active business licenses, and community boundaries was more challenging. In one
example, 10 out of 11 testers who found the active business licenses dataset had explored
other categories first.
Testers expected to find the potholes data in these categories: Public Safety, Environment, and
Sanitation. Testers expected to find active business licenses in Administration and Finance.
Lastly, testers looking for community boundaries looked in the Community category.
Some testers did use the help text on the categories to make decisions about which category to
choose, but it was difficult to decipher the order of results and whether a category contained the
relevant dataset after choosing a category. Testers did not use the filters on the search results
page, and upon reviewing a few datasets without identifying a relevant one, would often rely on
search or finding another category. Providing users with an understanding of an order of the
results and then the ability to filter easily to find the most appropriate results would be beneficial
to this experience. Improving the filter functionality would make it easier to have datasets in
multiple relevant categories and give users ways to find the data more quickly.
We also witnessed that testers did not always distinguish between the type of data– whether it
was a dataset, data lens, filtered view, or map they were choosing. This was evident when we
asked testers to find a map of crime data and testers chose the dataset or filtered view results
that were higher on the results list.
A primary search method should be defined for the user. Currently, users can choose a
category or search, but both have their challenges. Outlined above are results and
improvements for the categories since that was a prominent way testers searched. For each
task, 11 or more testers used the categories in completing their search. If search was more
prominent on the page and functioned better on mobile, we could identify if search was a better
way for testers to complete these tasks. We did learn that testers had difficulties with the search
terms that they used and spelling errors caused stopping points in the experience. The bug on
mobile also caused a stopping point that forced testers to use categories. If search becomes the
primary method of finding datasets it should be flexible to account for spelling errors and find
relevant resources.
20

Testers liked the “Digital Chicago” & “How to Videos” resources on the
homepage but the “Chicago apps” were seen as being the most
relevant.
Testers reviewed the other sections of the homepage including the “Digital Chicago” section that
showcases recent articles by the City of Chicago’s Department of Innovation and Technology,
“How to Videos” that showcase how to use the data portal and other tools, and “Chicago apps”
tools that use open data that let people better visualize data (OpenGrid) or is related to city
services (311 service tracker).
48% of testers thought the “Digital Chicago” section was “relevant” or “very relevant,” but testers
felt that “Digital Chicago” did not mean much to them (calling this section “news” could be an
improvement), they had questions about the articles and what they meant, and this section
lacked descriptive content for residents.
66% of testers thought the “How to Videos” section was “valuable” or “very valuable” and a lot of
testers liked receiving information via video, although there was a consensus that they should
be shortened and users should know that they will be directed to YouTube.
76.5% of testers were likely to use the apps under the “Chicago apps” section because they
saw them as useful to have for residents. 15 of our 17 (88%) testers were not familiar with these
apps prior to visiting the website. The “Chicago apps” section met expectations from testers who
at the beginning searched for or were interested in city services. There was positive reaction to
these tools and increased the likelihood of using the Open Data Portal in the future.
For the resident user, we would suggest placing the “Chicago apps” section higher on the page
because it relates with their needs. To avoid the confusion in distinguishing between the Open
Data Portal and the City of Chicago website, adding a descriptive sentence about what
“Chicago apps” is and why it’s on this page would be valuable. Sharing that these tools are
using open data would describe the importance of the open data initiative while giving residents
tools that they could use in their daily lives.
Conclusion
Most testers found the new, redesigned Open Data Portal as an accessible resource in finding
information about Chicago. With different levels of familiarity to the current data portal, testers
were able to explore datasets and provide feedback to improve the user experience of
searching for data. Resources like the “Digital Chicago,” “How to Videos,” and “Chicago apps” is
a welcomed addition for residents who thought these resources are valuable and relevant to
their needs. Implementing the recommendations we learned from this CUTGroup test and
21

continuing to include residents in the process of building the Open Data Portal, will ensure a
continued user-centric website.
22

Make $20 at a test of the open data portal!
Hi <<First Name>>,

We've got a new opportunity for you to make money in the Civic User Testing Group
(CUTGroup) by testing a new redesigned version of the City of Chicago's Open Data
Portal homepage.

Are you available for a 30 45 minute inperson test in Chicago on Wednesday,
September 28 between 4:00 7:30 PM?

If so, please complete this form.

We are in the process of gathering responses, so we will be in touch in a couple of
days to check availability, confirm the location, and confirm that you are right for this
test. We are looking for about 25 testers.

For your participation, you will receive a $20 VISA gift card. You'll also help make better
software for Cook County. Thanks for being a member of the CUTGroup! As usual, call
or write with questions.

Remember: if you want to be a part of this test, please complete this form.

Sonja Marziano
(312) 5653274
smarziano@cct.org
Subscribe Share Past Issues Translate

Facebook Twitter Website
Creative Commons Attribution 3.0 Unported License 2016 Smart Chicago Collaborative. unsubscribe from this list
update subscription preferences

Subscribe Share Past Issues Translate

CUTGroup Test: Data Portal Redesign
If you would like to participate in the next CUTGroup test, please complete this form. We will follow-up to check your
availability and let you know if you qualify for this test!
Smart Chicago values your online privacy and is committed to safeguarding the data you share with us and your
online privacy. Our On-Line Privacy Policy applies to data collected at www.smartchicago2012.wufoo.com domain.
Email *
Are you available for an in-person test in Chicago on Wednesday,
September 28 for 30-45 minutes between 4:00 and 7:30 PM? *
Yes
No
How familiar are you with the City of Chicago's open data portal? *
5 - Very familiar
4 - Familiar
3 - Neutral
2 - Not very familiar
1 - Not at all familiar
Have you ever used the City of Chicago's open data portal before? *
Yes
No
Do you work with or analyze datasets in your personal or professional
life? *
Yes
No
How familiar are you with the City of Chicago's data and technology
programs and initiatives? *
5 - Very familiar
4 - Familiar
3 - Neutral
2 - Not very familiar
1 - Not at all familiar
Tell us more about the programs you are familiar with.

CUTGroup 28: Data Portal Redesign
Thanks for participating in our test of the City of Chicago Data Portal redesigned homepage.
About you
First, some preliminaries. Keep in mind that we are going to publish the results of this test, but will not associate your name or
email address with the results.
Background
First Last
Name *
Email *
Give yourself a tester profile name. *
Proctored by
What device are you using for this test? *
What browser are you using right now? *
Chrome
Firefox
Internet Explorer
Safari
Other
What successes or challenges have you had finding data that you were
looking for or using the open data portal?
Tell us about any of your past experiences.
What are your expectations of what the City of Chicago Open Data Portal
would be useful for?

Initial homepage review
Review the homepage and tell us about your first impressions.
While you review, try not to click on the next page just yet, but feel free to
scroll up and down the page. *
What are three actions that you know you can do just from reviewing this
first page? *
Who do you think this website is designed for? Why? *
After reviewing the homepage, what is the first thing you want to do or
click? *
Search to find a specific dataset
Clicked on a data catalog category
Clicked on a "Digital Chicago" topic
Clicked on "How to Videos" link
Clicked on a "Chicago App"
Clicked "Developer Docs" in navigation bar
Clicked "Feedback" in navigation bar
Clicked "Data Catalog" in navigation bar
Other
What did you search? Or what was the category you chose?
Tell us more about the first thing you did, and why you did it. What were
you expecting to happen? *

Website review
Next, search for a dataset you're interested in learning more about. How
did you search? *
"Search to find a specific dataset..." box
Chose a data catalog category
Used search in navigation bar
Other
Tell us more about your search experience. Did you find what you were
looking for? *
Did the search meet your expectations? *
Yes
No
Why or why not?
Search for the building permits dataset. How did you search? *
"Buildings" Category > "Building Permits" dataset
"Buildings" Category > "Building Permits" data lens
"Data Catalog" in navigation bar > Search
"Data Catalog" in navigation bar > "Buildings" Category
Search box in navigation bar
Other

Next, search for a map of crime data. How did you search? *
"Public Safety" Category > "Crimes - 2001 to present" data lens
"Public Safety" Category > "Crimes - 2001 to present" dataset
"Public Safety" Category > "Crimes" map
Chose "Crimes - 2001 to present" link on header image
Search in navigation bar
Other
Next, find the potholes that have been recently fixed. How did you search?
*
"Service Requests" Category > "Potholes Patched Last Seven
Days"
"Service Requests" Category > "311 Service Requests - Pot Holes
Requested" dataset
Other
Next, search for active business licenses. How did you search? *
"Community" Category > "Business Licenses - Active" Filtered
View
"Community" Category > "Business Licenses" Dataset

"Community" Category > "Business Licenses" Dataset
Other
Find the City of Chicago's community boundaries map. How did you
search? *
"Facilities & Geo. Boundaries" Category > "Boundaries -
Community Areas (current)" map
Other
Overall, how easy was it for you to find these datasets? *
5 - Very Easy
4 - Easy
3 - Neutral
2 - Difficult
1 - Very Difficult
Tell us more about your search experiences and how you would improve
the site. *
Overall, how easy was it for you to find these datasets? *
5 - Very Easy
4 - Easy

3 - Neutral
2 - Difficult
1 - Very Difficult
Next explore the "Digital Chicago" section on the homepage. Tell us about
the information you find. *
How relevant do you think the "Digital Chicago" section is to you? *
5 - Very relevant
4 - Relevant
3 - Neutral
2 - Not very relevant
1 - Not at all relevant
Why or why not? *
Next explore the "How to Videos" section on the homepage. Tell us about
How valuable do you think the "How to Videos" section is to you? *
5 - Very valuable
4 - Valuable
3 - Neutral
2 - Not very valuable
1 - Not at all valuable
Why or why not? *
Next review the "Chicago Apps" section on the homepage. Tell us about

Overall Review
Were you familiar with any of these apps? If so, which one(s)?
How likely are you to use these "Chicago apps" in the future? *
5 - Very likely
4 - Likely
3 - Neutral
2 - Not very likely
1 - Not at all likely
Why or why not?
Overall, how easy or difficult did you think it is to use the new redesigned
Data Portal homepage? *
5 - Very easy
4 - Easy
3 - Neutral
2 - Difficult
1 - Very difficult
What makes you say this?
After reviewing the site, do you think you are the target audience for the
Data Portal? *
Yes
No
Not sure
If not, who is the target audience?
personal life? *
5 - Very likely
4 - Likely

CUTGroup Review
4 - Likely
3 - Neutral
2 - Not very likely
Why or why not?
professional life? *
5 - Very likely
4 - Likely
3 - Neutral
2 - Not very likely
Why or why not?
[Only for testers who used the data portal in the past]
How does this redesigned data portal homepage compare to what you
remember of the current portal?
What improvements would you make to City of Chicago Open Data portal
homepage to make this site easier to use, more valuable/important to
you, and more likely for you to use it again? *
Was this your first CUTGroup test? *
Yes
No
Do you like the CUTGroup? *
Yes
No
Did you like this CUTGroup test? *
Yes

Yes
No
Anything else to add about CUTGroup?

Final Report for CUTGroup #28 - City of Chicago Open Data Portal Homepage

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (18)

Andere mochten auch

Andere mochten auch (18)

Ähnlich wie Final Report for CUTGroup #28 - City of Chicago Open Data Portal Homepage

Ähnlich wie Final Report for CUTGroup #28 - City of Chicago Open Data Portal Homepage (20)

Mehr von Smart Chicago Collaborative

Mehr von Smart Chicago Collaborative (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Final Report for CUTGroup #28 - City of Chicago Open Data Portal Homepage