Considerations for Citizen Surveys
•Download as PPTX, PDF•
0 likes•580 views
Citizen surveys are an important tool for governments to engage their constituents and gauge citizen needs. This presentation provides guidelines for planning and implementing this tool.
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (9)
Similar to Considerations for Citizen Surveys
Similar to Considerations for Citizen Surveys (20)
Recently uploaded
Recently uploaded (20)
Considerations for Citizen Surveys
- 1. Considerations for Citizen Surveys Presenter: Morgan Drdak Digital Communications Manager, Forest Preserve District of Will County For Gonzaga University COML 511 Communication Consulting & Training
- 2. Citizen Surveys: An Overview • Introduction • Motivation and Goals • Sample • Instrument • Delivery • Results Analysis • Application of Findings • Conclusion
- 3. Introduction • Device to measure public need or opinion. • Means for assessing organizational performance. • Used by government agencies nationwide. • Frequently performed with consultant assistance. What is a Citizen Survey? (Miller & Kobayashi, 2001, p. 8)
- 4. Motivation and Goals • Assess organizational performance; i.e., productivity, effectiveness, responsiveness. • Seek input for future planning and budgeting. • Promote public participation and organizational responsiveness. • Measure changes in public need over time. Why Perform a Citizen Survey? (DiGrino & Balling, 1984; Miller & Miller, 1991, p. 503)
- 5. Sample • Represent entire constituency. • Consider demographics. • Determine sample size. – Population – Demographics – Survey budget – Acceptable level of error Whose Opinions Are Important? (Daneke & Klobus-Edwards, 1979, p. 424; DiGrino & Balling, 1984)
- 6. Instrument • Consider goals and sample. • Keep it concise. • Be careful with wording and sequencing. • Use measurement criteria appropriate for statistical analysis. • Offer language translations. • Pre-test instrument with small sample. What Questions Should Be Asked? (Daneke & Klobus-Edwards, 1979, pp. 423-425; DiGrino & Balling, 1984; Miller & Kobayashi, 2001, p. 7)
- 7. Delivery • Direct Interview – High response rate. – Costly and time consuming. – Subject to interviewer bias. • Telephone – Adequate response rate. – Less costly than direct interview. – Subject to interviewer bias. How Will the Survey Reach Constituents? (Daneke & Klobus-Edwards, 1979, p. 424; DiGrino & Balling, 1984; Miller & Kobayashi, 2001, pp. 7-8)
- 8. Delivery • Mail – Similar response rate as telephone. – Less expensive than telephone. – Maximize response rate with multiple contacts. • Internet – Not a proven technique. – Excludes demographic groups without Internet access. How Will the Survey Reach Constituents? (Daneke & Klobus-Edwards, 1979, p. 424; Miller & Kobayashi, 2001, pp. 7-8 ; Zickuhr & Smith, 2012, p. 2)
- 9. Results Analysis • Perform statistical analysis. – Identify frequency of responses. – Calculate average ratings. – Look for trends. – Consider demographics. • Evaluate results. – Keep survey goals in mind. What Does the Feedback Mean? (Daneke & Klobus-Edwards, 1979, p. 424)
- 10. Results Analysis • Compare results to past survey data. – Have constituent needs changed over time? – Are organizational changes made since previous surveys effective? • Compare results to similar organizations. – Is the agency measuring up to its peers? – Are there industry standards to be met? What Does the Feedback Mean? (Kelly & Swindell, 2002, p. 614)
- 11. Results Analysis Make some graphs. They can be fun. What Does the Feedback Mean?
- 12. Application of Results • Share results publicly to promote transparency. • Remember survey goals. – Demonstrate responsiveness to needs? – Make changes to current services or policies? – Consider for future planning or budgeting? – Evaluate success of changes since last survey? • Do not use survey results as only measurement tool. How Will the Feedback be Used? (DiGrino & Balling, 1984; Miller & Kobayashi, 2001, p. 7; Stipak, 1979)
- 13. Conclusion • Citizen surveys: – Assess government performance and identify needs. – Require careful planning and execution. – Provide valuable information about constituents. – Can measure current success. – May help an organization plan for the future. Should We Perform a Citizen Survey?
- 14. References Daneke, G. A. & Klobus-Edwards, P. (1979). Survey research for public administrators. Public Administration Review, 39(5), 421-421. DiGrino, N. & Balling, C. (1984). Who are all those people, anyway? Illinois Parks and Recreation, 15(6). Retrieved from http://www.lib.niu.edu/1984/ip841111.html Kelly, J. M. & Swindell, D. (2002). A multiple-indicator approach to municipal service evaluation: Correlating performance measurement and citizen satisfaction across jurisdictions. Public Administration Review, 62(5), 610- 621. Miller, T. I. & Kobayashi, M. (2001). The voice of the public: Why citizen surveys work. Public Management, 83(4), 6-9. Miller, T. I. & Miller, M. A. (1991). Standards of excellence: U.S. residents' evaluations of local government services. Public Administration Review, 51(6), 503-503. Stipak, B. (1979). Citizen satisfaction with urban services: Potential misuse as a performance indicator. Public Administration Review, 39(1), 46-46. Zickuhr, K. & Smith, A. (2012). Digital differences. Pew Internet & American Life Project. Retrieved from http://www.pewinternet.org/Reports/2012/Digital-differences.aspx
- 15. Image Credits Slide 1: amovita.com.au Slide 2: karikko.wordpress.com Slide 3: buffalomountaincoop.org Slide 5: priyanathan.imagineteachers.com Slide 7: aisb.hu Slide 9: forbes.com Slide 11: graphjam.com Slide 13: surveygizmo.com Slide 15: wanelo.com
Editor's Notes
- [Begin.]Greetings. My name is Morgan Drdak, and today we will be discussing considerations for citizen surveys.[Click]
- Before we get started, I’ll provide an overview of what we’ll be covering in this presentation.[Click] First, I will introduce what citizen surveys are, and how they are used.[Click] Next, I will discuss common motivations for performing a citizen survey, and how you can select goals for your survey.[Click] Then, I will cover the selection of a survey sample, or the population to be surveyed.[Click] Next, I will discuss the survey instrument itself, which is commonly a questionnaire.[Click] Then, I will share options for delivering the survey to your selected sample.[Click] Next, I will address how to analyze your survey results.[Click] Then, I will discuss ways to apply the findings from your survey, and[Click] We’ll wrap it up. [Click]
- First, I would like to introduce you to the idea of citizen surveys. What are they?[Click] Citizen surveys are devices used by government agencies to measure public need or opinion.[Click] They are also means for assessing organizational performance. As you know, governments don’t operate like other organizations, such as businesses, so you don’t have things like sales figures or stock values to tell you when you’re doing things right. You need another way to determine if you’re meeting the needs of your constituency – and not just the ones who reach out to you to compliment or, of course, complain. This is why citizen surveys exist.[Click] Citizen surveys are used by government agencies nationwide, and[Click] They are frequently performed with the assistance of a consultant. These consultants may include major research centers, local survey firms, or universities. A survey may seem like a simple thing, but by the time I’m done with my presentation, you’ll understand that it is actually a major undertaking that requires a commitment of substantial planning and resources to properly execute. Few agencies are able to go it alone.[Click]
- Now, let’s get started with planning your organization’s citizen survey. The first step is to identify your motivation for this undertaking, and only you can do that. Maybe you’re seeing a lack of support for your agency, and you want to know why. Or maybe you’re doing some strategic planning and want to find out what’s important to the public before moving forward. You know why you’re here, so we’ll move on to determining your goals for this project’s outcome.[Click] One possible goal is to assess public perception of your organization’s performance in terms of things like productivity, effectiveness, or responsiveness. You might just want to check in with your constituency to ask, “We doing alright here?”[Click] Or, you may be seeking input for future planning and budgeting. You want to know what your constituents’ needs are, so you can fulfill them.[Click] Maybe you just want to show that you’re open to public participation, and you’re responsive to your constituency. They do like to know they’re being paid attention to.[Click] Or, if you’ve performed a survey in the past, you may want to see how things have changed since then. Have constituents’ needs evolved? Do they see your recent work as successful? Performing regular citizen surveys can tell you. Keep your goals nearby. You’ll be referring to them constantly while planning and executing your survey. During the process, you may also need to clarify, simplify, or otherwise tweak your goals, so remain flexible but keep a clear idea of what you want to learn from your constituency in the end.[Click]
- Once you’ve set the goals for your survey, it’s time to determine your sample (or portion of the population) to survey. Ask yourself, “Whose opinions are important?”[Click] Well, the answer is, of course, everyone’s. The sample you select should represent your entire constituency. [Click] In order to achieve this, you will need to consider your constituency’s demographics. If there are small minority populations that could be missed by a simple sampling, you may need to divide constituents into subgroups (this is called a stratified sample) to make sure that all of your population’s demographic groups are represented in your survey results.[Click] You’ll also need to determine your sample size, or how many people will be asked to take your survey. This will depend on a number of variables, including[Click] Your jurisdiction’s population,[Click] Your constituency’s demographics,[Click] Your survey budget – more surveys means more money, and[Click] What you will accept in terms of error level. More surveys also means a more accurate representation of your full constituency.[Click]
- Next, you will need to develop your survey instrument or, in other words, decide what questions you should ask.[Click] The goals you’ve set should be the basis for this decision. The responses you receive from your survey should answer the questions you’re dying to know about your constituency. If you’re trying to figure out whether your services need improvement, ask for an evaluation of them. If you want to know how effective your marketing is, focus questions on that. If you’re looking for an idea of what the public wants from your future work, ask that. Your goals need to guide this discussion.[Click] But your goals can’t be too far-reaching, because the survey needs to be concise. Longer surveys result in fewer responses. People don’t think they have time to answer 100 questions. Stay focused.[Click] Once you’ve determined the information you’d like to know, you have to figure out how to ask for it. There are lots of books and articles and experts on how to ask a good question. A few starter tips: Don’t make your respondent feel stupid – ask questions that most of your constituents should understand, and allow them to answer, “I don’t know.” Don’t make your respondent feel judged – offer categorical responses to personal questions, like age groups or income groups. Don’t lead your respondent to an answer, like asking “Should we clean the dirty, disease-infested creek?” (What? Disease? Kill it with fire!!!) Constituents may be passionate about certain topics, but don’t cause them to be passionate with the wording of your questions. And don’t ask questions that could be confusing – remember that jargon that may be familiar to you may not be understood by your constituents. Offer screening questions, if necessary, like “Do you know what a prescribed burn is?” before asking, “Should we do more prescribed burns?” Finally, be sure the sequencing of your questions doesn’t also lead to confusion. Previous questions can influence future answers. Design the survey with separate sections, if necessary.[Click] While figuring out what to ask and how to ask it, you’ll also need to think about how you want the answers. The measurement criteria you select, like scales or rankings, should be useful for the statistical analysis you plan to perform with the responses. Remember that the responses will need to be converted into data for a computer program, so have a plan for their design. Also keep in mind that although open-ended questions can provide new ideas or more specific input, they are more difficult to process, understand, or sometimes, even read. Use them carefully.[Click] Don’t forget your demographics when designing your survey. Be prepared to offer the survey in translation for different languages. This offering can be communicated in a cover letter sent along with the survey if done by mail.[Click] Finally, pre-test your survey with a small, actual sample from your population. If you receive no responses or a see lot of “I don’t know,” you may need to rework the survey. Be prepared to take the time to get the survey itself right, because it will have a major impact on your response rate and how accurate your results are.[Click]
- So now you’ve got your goals set, your sample selected, and your questions ready. Now, how do you get your survey out to your constituents? There are several options.[Click] One option is direct interview, where someone from your organization or your consultant’s firm asks questions of your constituents in an in-person, one-on-one environment.[Click] This method has a high response rate, because people are less likely to ignore a person talking to them in public than a phone call or a piece of mail.[Click] However, direct interviews are costly and time consuming, because you need to pay for someone to go out there to find and talk to your constituents.[Click] That “someone” also needs to be highly trained, because direct interviews are subject to interviewer bias. That means that the person asking your survey questions can influence the responses with how he asks the questions, even with something as simple as his tone of voice. You may receive very different results from different interviewers if they are not adequately trained in performing surveys. For most organizations, the high response rate of direct interviews is not worth the cost, time, and risk associated with them.[Click] The second option is telephone interviews.[Click] Telephone interviews used to be a preferred survey method due to their high response rates. But over the past twenty years, with the growing aversion toward telemarketing and popularity of caller ID, response rates for telephone interviews have dropped to levels similar to those of mailed surveys. Honestly, how many of you are willing to spend ten minutes on the phone with a stranger?[Click] Telephone interviews are attractive because of their lower cost and time requirements than direct interviews, but they are still subject to interviewer bias. There is, however, another option that is even less expensive and does not invite the risk of interviewer bias.[Click]
- [Click]That option is a mailed survey.[Click] Now, response rates for mailed surveys are similar to those of telephone interviews – between 35 and 50 percent.[Click] Mailing surveys is also less expensive than direct or telephone surveys.[Click] If you choose to mail surveys, keep in mind that you may need to remind your sample about the survey several times to maximize your response rate. You’ll also need to carefully consider how your survey is presented to recipients – you don’t want them to think it’s “junk mail.”[Click] A quick note about the Internet. Sometimes people think the Internet can do everything.[Click] But it is not a proven technique for citizen surveys, and[Click] It excludes demographic groups who do not have access to or utilize the Internet. These groups include senior citizens, individuals with lower income or educational attainment, and individuals with disabilities. These members of your constituency should be equally able to provide their feedback as other groups, so although the Internet is great, it’s not great for this.[Click]
- “And the survey says!” What? Now that you’ve established your goals, found your sample, written your survey, and delivered it… you’ve got your responses. Now comes the exciting part! It’s time to find out what your constituents have to say.[Click] Okay, maybe we’re not excited yet. Statistical analysis? (Aren’t you glad you hired a consultant now?) There are lots of computer programs that will perform these calculations for you and produce nice, clean numbers that will help you[Click] Identify frequent responses. For example, you might learn that 99% of constituents don’t want flies in their soup. Or that only 10% of constituents think that having more slumber parties is an important goal for your organization. (I know, they’re being killjoys.)[Click] You can also calculate average ratings. For example, you might learn that the average rating for the helpfulness of your staff is a two out of ten. (Wow, you people are rude. No wonder you’re not getting slumber parties.)[Click] What you’ll want to do is look for trends in the data. Are questions pertaining to your staff being consistently rated highly? Are questions pertaining to your marketing being consistently rated low?[Click] With some additional analysis, you’ll also be able to identify trends with regard to your demographics. Are constituents over the age of 50 using your services more than others? Do Black constituents find your Web site more useful than Latino constituents?[Click] Take all of the data into account when evaluating the survey results. Depending on what questions you asked based on your goals, you may be able to assess your organization’s public performance, identify problems or successes, or learn about what your constituents really expect from you.[Click] Remember your survey goals during your evaluation, but keep an open mind. You may find answers to questions you didn’t even ask.[Click]
- Once you’ve reviewed your survey responses independently, there are more ways to evaluate your results.[Click] If you’ve conducted a constituent survey in the past, you can compare your current results to that data.[Click] This is great for determining if constituent needs have changed over time, or[Click] If you’ve made good (or bad) decisions since your last survey.[Click] Thanks to the growing trend of public participation and government transparency, you can also compare your survey’s results to those of similar organizations. Many agencies post their citizen survey reports publicly on their Web sites, or you can access meta-analyses of citizen survey data like that produced by the International City/County Management Association’s Center for Performance Measurement.[Click] Comparing your results to those of like agencies can tell you whether you’re measuring up to your peers (or if your constituents just have crazy expectations of you).[Click] It can also help you determine if you are meeting industry standards or even best practices for organizations like yours. All of this comparison data will be useful when it’s time to apply your survey results.[Click]
- But in the meantime, [Click] Make some graphs. They can be fun, and they really add some pizzazz to your results report.[Click]
- Once you’re done with charts-and-graphs-fun-time, think about how you will put your citizen survey results to good use.[Click] First, share your results publicly, like on your Web site, to promote that public participation and transparency I talked about a minute ago. For better or worse, this is what your constituents think of you. Own up to it.[Click] Again, again, and again, remember your survey goals. What did you want to accomplish when you started this whole thing? Keep those in mind as you consider how to apply your results.[Click] For example, did you just want to demonstrate a responsiveness to your constituents’ needs by performing a survey? Well, you’ve accomplished that, but you may want to actually be responsive by doing something with it.[Click] Did you want the survey to make a case for changes to current services or policies? Look at the results with an eye on what constituents are dissatisfied with.[Click] Did you want the survey to provide input for future planning or budgeting? Take note of what constituents think is important and worthwhile. [Click] Did you want the survey to show whether your new-and-improved services are new-and-unappreciated? This information can also be valuable in future planning.[Click] Whatever great ideas the survey results give you, do not rely on them as your only tool for measurement. They demonstrate public perception of your organization, not actual performance of your organization, so be sure to consider all possible factors for negative responses before making sweeping changes. This means reaching out to staff, too – they probably have some insights to share as well.[Click]
- We’re almost done! I’ve given you a lot to think about, and by this time, you’re probably asking yourself, “So, should we do a citizen survey?” Let’s do a quick recap.[Click] Citizen surveys[Click] Assess government agencies’ performance and identify constituents’ needs.[Click] They require careful planning and execution. (And there are people out there to help you.)[Click] They provide valuable information about constituents…[Click] And can help you measure your organization’s current success.[Click] They can also help you plan for the future. Remember, it’s your constituents’ future too.[Click]
- Thank you for your time. Are there any questions?[Click]
- [Answer questions.][End.]