Estrategias de Marketing

• Gauge cultural appropriateness. Pretesting and revising were critical to developing the “¡Mírame! Look at Me!” curriculum to help South Texas Hispanic youth ages 9–13 avoid alcohol, tobacco, and drug use. The research found that this intended audience learned best with active learning strategies and interpersonal interaction, which influenced curriculum format revision. Testing also found that the language used (typical of South Texas conversations, 85 percent English, 15 percent Spanish) was appropriate and easily understood and that using intended audience representatives telling their own stories ensured credibility and close audience attention.

• Identify strong and weak points. Pretests of an NCI booklet, “Cancer Research Studies with Patients: What You Need to Know,” found that patients and family members were enthusiastic about the format, comprehensibility, approach (enabling patients to take an active role in decisions about their cancer care), and relevance of the book to patients’ needs and situations. Participants’ comments led to suggestions for strengthening sections on treatment costs and insurance and adding specific questions to the list patients should ask before agreeing to take part in a research study. In addition, pretests provided useful feedback that allowed NCI to improve graphics portraying complex concepts such as randomization and control groups.

• Identify sensitive or controversial elements. Would using vernacular language to discuss diarrhea in a booklet on chemotherapy for a low-literacy intended audience be preferable to a more technical approach? Pretest results showed that the intended audience found the vernacular offensive and preferred the technical discussion, even though it had a higher reading level.

Limitations

• A pretest is only as objective as the person designing and interpreting the study.

• Pretesting cannot absolutely predict or guarantee learning, persuasion, behavior change, or other measures of communication effectiveness.

• Pretesting is not statistically precise. It will not reveal that booklet A is 2.5 percent better than booklet B. (Presumably, pretests of such precision could be applied, but the cost of obtaining such data would be high, and the findings may be no more useful than those from more affordable approaches.)

• Pretesting is not a substitute for experienced judgment. Rather, it can provide additional information to help guide sound decisions.

• Pretesting does not guarantee success. Good planning, thorough concept exploration, and sound pretesting can be negated by mistakes in final production or in program implementation. The message in a television PSA on cancer treatment, for instance, may pretest well, but then be flawed by an execution that uses an actress who seems too happy to be awaiting the results of a biopsy report. Similarly, leaflet copy that pretests well may be rendered ineffective by a poor layout, hard-to-read type, or inappropriate illustrations.

ST

AGE 2

chronic disease know too much about the condition to objectively assess materials designed for patients recently diagnosed. Don’t rely on these participants for creative guidance. They are expert consumers, and it is important to learn their preferences. They are not, however, communication or creative professionals, and their ideas for substantive message or materials changes are likely to be off the mark.

Choose Methods

A variety of research methods can and should be used to explore message

concepts and test messages and materials. Which you choose depends on the research questions you want to have answered, the nature of the materials, the intended audience, and the amount of time and resources available for pretesting.

The following is a list of methods to use to test materials:

• Concept testing with intended- audience members

— Focus groups (face-to-face or telephone)

— In-depth interviews • Pretesting with intended-

audience members — Self-administered

surveys/questionnaires (by mail, handout, or computer)

— Interviewer-administered

surveys/questionnaires (by telephone, through central-location intercepts, or other face-to-face scenarios)

— Theater testing (large groups respond to messages via questionnaire

or electronic)

— Observational studies (e.g., observing behaviors of shoppers in a store or patients waiting in a clinic)

• Other assessment methods — Readability assessments — Expert/gatekeeper review Sometimes, using several methods in combination will help overcome the limitations of individual methods. For example, readability testing should be used as a first step in assessing draft

manuscripts, followed by self-administered questionnaires or interviews or, for some long documents, a combination of the two with intended audience respondents. Central-location intercept interviews for short audio, video, or print materials permit contact with larger numbers of intended audience respondents, which is especially useful prior to final production of materials. See the Communication Research Methods section for definitions of commonly used pretesting methods and descriptions of the pros and cons and common uses of each.

Secure Vendors, Facilities, and Moderators or Interviewers

Some research methods require securing appropriate facilities. If you are using commercial facilities (e.g., for focus groups, central-location intercept interviews, or theater tests), the American Marketing Association’s New York Chapter’s Green Book provides an extensive directory of vendors, indexed by geographic locations and services offered. The Communication Research Methods section provides tips for working with vendors and for securing alternative facilities if location or budget precludes using a vendor. The following are vendor services your program may need: • Teleconference services to set up

telephone focus groups. Most allow observers to listen without being heard, and some provide remote viewing programs to allow the moderator to see a list of participants (with a symbol next to the

ST

AGE 2

name of the one currently speaking) or Draft Test Instruments (Discussion

notes sent by a technician from observers Guides, Questionnaires)

listening to the call. Some teleconference services can recruit participants; with others, recruit or contract with a recruiter separately. • Focus group facilities to recruit participants

that also offer audio and video recording equipment as well as one-way mirrors with observation rooms for viewing both focus groups and in-depth interviews. These are usually available only in larger

metropolitan areas.

• Commercial facilities to conduct central- location intercepts at shopping malls. You can also use other facilities or public locations that attract a large number of pedestrians (e.g., other stores, train stations, university campuses, large medical centers); usually permission is required to interview people in these types of locations.

If you are conducting focus groups or in- depth interviews, identify trained,

experienced moderators or interviewers. If your organization has no experience in such studies, consider hiring a good, experienced moderator or interviewer to conduct the project or to train internal staff to develop in­ house skills. Local advertising agencies, the American Marketing Association’s Focus Group Directory, the Qualitative Research Consultants Association, or other health communication program managers may be of assistance in identifying a good moderator. Before choosing a moderator or interviewer, ask for and check references.

Identify, Screen, and Recruit Respondents

Use the intended audience description developed in Stage 1 to draft questions for recruiters to use to identify participants. See Appendix A for a sample form to use to screen potential participants.

Involve the creative team in developing the discussion guides or questionnaires. The team will often have specific issues or questions. Assess how much the intended audience likes each set of material, because research has found this to be a leading indicator of success. Other specific questions should be used to identify strengths and weaknesses in rough messages and materials. See Appendix A for a sample focus group moderator’s guide and an intercept questionnaire.

Conduct Testing

If possible, have other team members, such as creative professionals, content experts, partners, and gatekeepers (including

decision-makers who control your program), observe at least some of the focus groups or interviews. Creative professionals’

observations are particularly important during concept exploration, because they often spot comments or trends important for creative development. Hearing the

responses improves team members’ understanding of intended audience reactions and can illustrate, more vividly than any report, the need for simple language or the power of a particular creative approach.

Analyze Results

To analyze and communicate the results of materials testing, write a report outlining the process and the findings. The report should include the following sections:

• Background—Who was tested? Why? How? What did you hope to learn? (Describe each in detail.)

• Highlights—Summarize the main points that emerged from testing that answer the research questions.

KEEPING PRETEST COSTS DOWN