MS PowerPoint, 144 KB, 49 slides, July 2004

1. ICGI Content
Standards
Usability Test Results
Content Managers’ Forum
July 15, 2004

2. The Tests:
 Tested eleven content standards
 Terminology
 Provide description—”What would you call this?”
 Rank list of options by preference
 Placement on the web page
 Identify placement on generic web page
 Categorization
 Card sort exercise
 Reaction to “typical” federal page

3. Generic Federal Homepage

4. The Test Participants:
 Eleven participants
 Mix of races/ethnicities
 Variety of web experience
 Wide age spectrum
 Both genders
 Range of formal education

5. Sponsorship of Gov Web Sites
 Participants shown a homepage (HHS/HUD)
What tells you it's an official govt. site?
.gov in URL 54.5%
"U.S. Dept .of…" visible 45.5%
"Dept. of/agency name" visible 18.2%
graphic logo/seal 18.2%
other graphics 9.1%

6. Comments
 “ “Mainly checks for the .gov URL such as
dot.gov and trusts it more than other sites like
.com.”
 “Definitely goes to .gov for government
information and checks for the .gov URL.”

7. Link to Gov’t-wide Portal
Which label most accurately depicts:
 An official gateway to all government information
in the United States
 A portal containing links to federal and state
government sites

8. Ranking:
Unadjusted Adjusted
www.us.gov 1.90 2.05
www.fedinfo.gov 1.91 1.91
www.usa.gov 2.10 2.23
www.firstgov.gov 3.89 3.82
 Participants thought that “U.S.” and “fed” meant
federal, and therefore, were looking for a more
encompassing term that did not seem to exclude
state governments.

9. Location
Users expected the link to be at the top of the
page:
 3 participants- top left menu bar
 2 participants- top right menu bar
 2 participants- top nav bar
 2 participants- bottom of page/
footer

10. Comments
 “Regarding the site name www.firstgov.gov –
doesn’t know what this means at all.”
 “Regarding the site name www.fedinfo.gov – said
this doesn’t apply to states because of the fed term.”
 “Suggested govinfo.gov”
 “Said firstgov.gov is ridiculous and looks like a typo”
 “Said to be consistent and put all links on the left. I
don’t want to look all over the page for it. There
should be one spot where I can expect to see
everything.”

11. About Us
Which label most accurately depicts a place
where you can:
 Learn who operates this web site
 Learn what this organization does
 Find the organizational structure of the group
that runs this site

12. Ranking
Unadjusted Adjusted
About Us 2.25 2.50
About [Organization] 2.31 2.56
Learn About Us 2.50 2.50
Who We Are 2.94 2.94
For More Information 4.50 4.50

13. Location
Users were split as to expecting the About Us
link to be at the top or the bottom of the page.
 3 participants- top left menu bar
 1 participant- bottom of page
 1 participant- bottom/ middle of
page
 1 participant- bottom/ right

14. Comments
 “Expects basic stuff near the top of the page and
lower links to be more explanatory.”
 “Said the test page looked as expected- it has links
to specific organizations where you can find more.
Expects to find jobs and then sees employment.”
 “Said the test page looked as expected. Expects to
find email not the mailing address.”
 “A guide to the organization structure is great”
 “Would like to see contact us closer to the top. At
the bottom it makes me think they don’t want me to
contact them.”

15. Laws and Regulations
Card Labels
Guidance
 Policy
 Current Legislation in
Congress
 Major Laws & Statutes
 Executive Orders
 Proposed Rules
 Regulatory Actions
 U.S. Code
 Federal Register Documents
 Preambles to Final Rules
 Dockets
 Testimony
Tester Recommendations
 Legislative Rulings
 Legal/ Legislation
 Building a Law from Start to
Finish
 Steps to Creating a Law
 Information Reference/
Resources
 Rules and Regulations
 Governmental Procedures

16. Ranking
Laws and Regulations 1.71
Policies and Regulations 2.14
Policy 3.00
Laws and Regulatory Assistance 3.14
The majority of the participants (90%) thought “passed
and signed laws are different than proposals, so
they would not be grouped together” or “these terms
should be arranged in a developmental way.”

17. Location
The majority though the Laws and Regs link
belonged at the top of the page
 4 participants- top left nav bar
 2 participants- top right nav bar
 2 participants- middle left menu bar
 2 participants- bottom of page/
footer

18. Comments
 “Divided cards into two groups. From the two groups in the card
sort would call them Preamble (for future laws) and Statutes (for
current laws).”
 “Said they should be arranged in the order of the process of
becoming a law.”
 “Thought these looked like the steps to creating a law.”
 “Sees the list in a developmental sense such that proposed rules
come before laws. Would arrange the list into three categories
and didn’t think “Guidance” fit in the group.”
 “Said terms were related but didn’t all fit together. Saw a
difference between the concrete laws and other terms.”
 “Said the information did not fit together and should be broken
into categories: 1. Agency Legislation in Process, 2. Historical
Source of Information for Legislation, 3. Executive Orders, 4.
Policy & Guidance.”

19. Search
Which label most accurately depicts a place where you can:
 Enter terms in order to find content most relevant to your needs
 Scan a website using keywords to find matching information
 Use single words or phrases to quickly locate information within
an agency's web site
6 out of 10 (60%) of the participants came up with at least one label
suggestion that included the word “Search” (search, keyword
search, quick search, etc.).

20. Ranking
Users were provided the following options:
 Find It
 Ask
 Search
 Where do I…
 Select a Topic
 Clear preference here: 80% of the participants tested ranked the
label “Search” #1.
 For most participants "Ask" ranked very low, and some
commented it sounded like Ask Jeeves or like one could actually
ask questions online. Many participants would like a "Select a
Topic" feature and would expect to see a drop-down menu, but
said this was obviously different than Search.

21. Location
 90% expected Search to be somewhere in
the top menu bar
 Right hand side of the bar 4
 Left hand side of the bar 2
 Center of the bar 0
 No real preference within bar 3

22. Search Preferences
Participants were given color printouts of three home
pages:
 example with search box empty (ED.gov, box in left hand
menu)
 example with search box filled in (DOT.gov, filled box in
header graphic)
 example with no search box, just a link to "Search" (DOD,
link in menu bar)
During this test, 60% of the participants expressed a
definite liking for a “search box” (2 with text inside, 2
empty, 2 no preference or didn’t matter).

23. Contact Us
Which label most accurately depicts a place where you
can:
 Email the owners of the site
 Get the phone number of the site owners
 Get the address of the site owners
 Report a technical issue or a question about the site
Suggestions for labels included Public Information (3),
Request… (3), For More Info (3), FOIA (2 – one
user had worked for govt. and recognized acronym).

24. Ranking
Assistance
Comments
Help
Contact Us
For More Information
There was a strong preference for Contact Us-
eight of the nine participants listed it as #1.

25. Location
70% expected to find the Contact Us link at the
bottom of the page
 3 participants- top right menu bar
 3 participants- bottom left menu bar
 2 participants- bottom of page
 1 participant- bottom right menu bar
 1 participant- bottom center of page

26. Comments
 “Said this should be a link to a phone
directory for agencies, divisions, and bureaus
. . . you can never find anybody anymore!”
 “Thinks agencies should send at least a
boiler plate response in an email.”
 “Contact Us should be a way to email an
agency.”

27. FOIA
Which label most accurately depicts a place
where you can:
 Learn about your rights to request from a
Government Agency public information that in
the past had not been available to the public.
 Request information – such as books, charts,
emails, and records - on a specific
Government Agency’s activities.

28. Ranking
Unadjusted Adjusted
Public Rights to Gov Info 2.43 3.18
Public Access to Gov Info 2.50 2.82
Freedom to Request 2.86 3.59
Request Gov Info 3.44 3.55
Freedom of Information Act 3.67 3.77
FOIA 4.07 4.23

29. Location
Participants were mixed on expected location:
 5 participants left menu bar
 3 participants page center
 4 participants bottom of page
 2 participants right menu bar

30. Comments
 “Recognized "Freedom of Info" Thinks media/big wigs have this,
not people like her… Would not recognize FOIA, Definitely
prefers the term "public" Tells her she, a citizen, has access to
it.”
 “[says] If she didn't know FOIA, she would pick public
rights/access.”
 “"Rights" suggests these may be problems or roadblocks in
getting the info.”
 “… people want to go right to submitting a request. The example
page [USDA FOIA page] is what's expected but info about law is
not useful…. Links on the page should be moved up.”
 “Example page: does not like to have to go to another page to
find out more about filing a request.”

31. Current Information
Test participants were shown two pages:
 Today’s Date at top of page (DOL)
 Last Modified Date at bottom of page (BLS)
 Is this page up to date?
 What does the date indicate?
 Is this date useful?

32. Ranking
Participants were then asked to rank, in terms
of usefulness, the following types of dates:
 Today’s Date
 Posted Date
 Last Modified Date
 Last Reviewed Date
 Last Certified Date
 Last Updated Date
 Creation Date

33. Results
Participants either loved or hated Today’s Date:
 “Today's date doesn't tell whether info is current.”
 “Likes both Today's Date and Last Modified.. Depends on
info he's looking for.”
 “Date up on top left corner is not updated date, not useful.”
 “Date by itself could mean today. Many people may not
know what "modified" means - but there has to be a label
with a date.”
 “Today's date is important. Indicates that page is kept
current all day. Stats should be current!”

34. Results
Participants seemed to like either “Last
Modified” and “Last Updated” to indicate the
currency of content on a particular page.
Several ranked them equally or close to
equal.
 “Stats and Statements of fact- wants a last
modified date. If no date no way of knowing how
accurate or up to date is the info.”
 “Last modified-content updated. TP likes the last
modified date info.”

35. Location
Most participants expect the date to be at the
top of the page:
 5 participants- top left of page
 2 participants gave an expected location vs.
actual (desired) location--“prefer top of page,
expect upper right”

36. FAQs or Common Questions
Which label most accurately depicts a place where you
can find information requested most often from the
agency?
 4 out of 11 (36%)- Frequently Asked Questions
 3 out of 11 (27%)- FAQs
 Other suggestions: Frequently Requested…, Most Often
Requested….
Strong preference for “standard” labels of Frequently Asked Questions
or FAQs. Note: Throughout testing, users often expressed dislike
for, or confusion from, acronyms such as FAQ, FOIA, etc.

37. Location
Participants were mixed on expected location:
 3 participants- top left of page
 2 participants- bottom of page
 3 participants- right menu bar (1
bottom, 2 unspecified)
 2 participants- middle of left menu
bar

38. Comments
 “Doesn't like to hunt. Uses it as a last resort.
Looking for questions/answers. On test page [EPA
FAQ on Radon]…a lot on one page.”
 “Want to reach a human being. Not machine.”
 “Usually get the answers I want”
 “Would like contact info or help if you can't find what
you need.
 “Does use FAQs depending on what he is looking
for, if he doesn't see info on first page and he thinks
it is common information.”

39. No Fear Act
Which label most accurately depicts a place where you can:
 See a summary of any grievances related to unfair treatment of a
person or group on the basis of prejudice filed against an agency
 Find statistical information about any acts of reprisal against
government workers who reported unfair treatment at an agency
Suggestions for labels included: Grievances, Government Workers
Disputes, Report on Grievances, Discrimination Against
Government Employees, Employee Actions & Grievances,
Reprisal, Discriminatory Site, Unfair Employment…

40. Ranking
 Discrimination and Retaliation Complaints 2.20 2.41
 Equal Employment Opportunity Complaints 2.44 2.82
 Equal Employment Opportunity Data 3.25 4.45
 Employee Complaint Data 3.56 3.73
 Federal Employee Antidiscrimination 4.29 4.64
and Retaliation Act
 EEO Data 4.75 5.05
 Antidiscrimination & Whistleblower 4.88 4.91
Protection Reporting

41. Location
Top locations selected on generic page layout:
 5 of 11 (45%) of participants- look for it at
bottom of the page, (no predominance for
left/right/center).
 4 of 11 (36%) of participants- look for it in
top nav bar, (2 left side, 2 right side)

42. Comments
 “Didn't like Federal Employee
Antidiscrimination & Retaliation Act and
Antidiscrimination & Whistleblower Protection
Reporting because they were "very wordy"”
 “Employee Complaint Data is too generic and
vague, not very familiar with EEO or EEOC”

43. Web Site Policies and
Important Notices
Card Sort Items
 Privacy & Security Statement
 Freedom of Information Act (FOIA)
 Information Quality Guidelines
 Portable Document Format (PDF)
 Accessibility Policy
 Linking to Outside Websites
 Disclaimer of Endorsement
 Copyright Information
 No Fear Act
 Photo Credits
 Schedule for Publishing Information
 Nondiscrimination Statement
 Legal Notices
 Free Commercial Downloads
Participant Suggestions
 Policies
 Legal
 Overall Info
 Site Info
 Additional Info
 Information Guidelines
 Information Statement

44. Card Sort Results
 Users had a lot of difficulty creating one group
that covered all topics. Some users suggested
two groupings of “legal” and “information”.
 The following content items were singled out by
multiple (3+) users as either irrelevant or
confusing with relationship to grouping the other
items: PDF, Linking To Outside Websites, Free
Commercial Downloads, and Photo Credits.
 Participant comment: “stuff you don’t even know
what it is. PDF? Photo Credits?”

45. Ranking
Unadjusted Adjusted
Notices and Disclaimers 1.50 1.55
Site Notices and Links 2.44 2.50
Site Policies and Links 2.71 2.85
Important Notices 3.00 3.09
Note: 54% of the participants ranked “Important
Notices” at or near the bottom.

46. Location
Whatever we call it, users expect to find it at
the bottom of the page:
 7 of 11 (63%) of participants- look for it at
bottom of the page (0 left side, 2 right side, 1
center, 4 not specified)
 No other location got more than 1 suggestion.

47. Comments
 “Important Notices (this doesn't say anything)”
 “Does not know what "Accessibility Policy" is. Accessibility does
not fit.”
 “Keep policies/legal info separate from free stuff .”
 “Card Sort: PDF, Linking to Outside Websites, Free Commercial
Downloads -these three don't fit with others”
 “Free Commercial Downloads - What does that mean! Nothing to
do with govt.”
 “"I like policies more than notices. " "They are legal policies not
all are site policies." Why is "Photo Credits" here.”
 “No to [suggested label] Important Notices (Not important)”

48. What testing could be done
next
 Two tests were already designed by the
testing subcommittee but were dropped from
this round of testing because of time
constraints
 Label, location and expected content for “Jobs”,
“Privacy”, “Site Index” and “Site Map”.
 Best practices for Linking to Outside Websites.

49. Possible additional tests
 Label, location and content for “Information
Quality Guidelines”
 Appropriate content and best practices for
cross-government portal implementation
 Plain language (writing content for the web)
 Limited English Proficiency (translation
services / cross-language content delivery)