Logging Data

There are two forms of data logged (there could be others): field notes and interview transcriptions

It is usually best to write field notes by hand at the site, then type them into the computer at the end of each day or at least by the next morning. In some cases field notes can be done at intervals (if writing them openly arouses suspicion or there are other reasons they cannot be done at the moment of observation). One researcher used the restroom to write notes every hour--which he reported worked well, except that some may have wondered why he had to go there so often! People usually write between five and ten handwritten pages an hour if they are observing carefully.

When observing, write very concretely. Quantitative research speaks of operationalizing concepts--stating them in observable, countable terms. This is how you write your descriptions; avoid inferences, generalizations, vague terms. Avoid sophisticated terms that will obscure what actually occurred ("they interacted" could mean many possible terms--even mud wrestling!). Get down the details, even if they seem irrelevant at first. Describe the obvious, because it may be less obvious (and less likely to be remembered) once you leave the site. Also what is obvious to you may not be obvious to outsiders. Push yourself to describe actions without evaluating (evaluating, generalizing, inferring can all occur in the other kinds of notes, but not the field notes section of notes). Students must often push themselves to get details. If you begin to generalize too early, you may be recording more your bias that what actually occurs (although you might put something in the margins of your notes to be recorded separately when you type them up IF the idea is absolutely overwhelming or if you think you'll forget an important aspect). You need data from which to generalize, otherwise the results cannot be trusted any more than folk tales or generalized impressions (but do record impressions later in personal notes or theoretical notes). One important distinction between research and general experience is that research relies upon carefully documented data from which conclusions are formed.

Alternative methods of making field notes include making recordings at intervals or, if it won't be too distracting, talking quietly into a cassette recorder. I even used a camcorder with special amplified microphone that hung next to my mouth for making verbal field notes. It worked well, and the tiny compression system in the microphone made it possible to hear what was happening in the environment as well. You may need to take period breaks to say your field notes into the cassette recorder. The drawback for mechancial recordings is that they will need some kind of transcription later (see comments on transcribing interviews, below). Some are able to take a small laptop with them to the field site and type in field notes that way. However, many people find typing on a laptop to be distracting to people at the site, and a single computer crash can destroy the whole day of data (or more if you didn't back up regularly!).

If nothing--absolutely nothing--is happening, then describe the physical context in excruciating detail. Look around carefully, even get down on the floor and look at the floors and walls carefully during "dead times" when absolutely nothing is happening.

Handwritten notes are then typed into the computer when not observing. During the typing process, some details will probably be recalled that were not written down at the site--include these. While typing, separate the personal notes, theoretical notes, and methodology notes from the standard field notes. This can be done by using separate computer files for each of these, or simply denoted by some code within a single text. You will probably find that you will add much more to personal notes, theoretical notes, and methodology notes at this time--good! Time at the site should emphasize events and descriptions, while the typing time will tend to be more reflective. This reflection is actually the beginnings of analysis which is a reflective process (a less formal analysis than the formal approaches to be used near the completion of the study).

Field notes should be typed in a column on one half of the paper (set your margins accordingly in your word processor). The other half of the paper is for coding and comments. If you are right handed, the notes should be on the left side of the paper so you can write in the right column (and vice versa if you are left handed).

Good interviews have lots of open-ended questions, most of which are formed prior to the interview. I personally like questions that come out of observations better than those created out of the student's imagination. Sometimes, though, good questions emerge during an interview because of what has been said by the one interviewed. Usually I'd go with the flow and ask the emergent questions, if it's appropriate. Transcribing interviews can be done several ways. Word-for-word transcriptions are probably best, but they are laborious. If you are well funded, this can be hired out. But there is value in the researcher listening to interviews, as the researcher may be able to figure out a muffled word that a transcriptionist cannot. Also, the interviewer may learn how to better interview by listening to his or her mistakes. It is also possible to use word-for-word transcriptions of some sections and summarize others when typing up the interviews. It is also possible to listen to the tape of the interview several times in order to better discover what sections are important enough to transcribe, which sections need to be summarized, and which sections should be ignored. But keep the tape--later in the research you may find that what was not typed was indeed more important than you thought! It is even possible to code directly from the tape--there are computer programs that allow you to connect a cassette player or even a videotape player directly to the computer, so that time markers and even your transcription can be added to the audio or video data.

As with field notes, transcriptions should be typed in a single column with a wide margin for coding and comments. You can't code well if your field notes are not concrete and tangible (think "operationalized"). Coding includes categorizations, classifications, and other kinds of comments about the field notes (you could code your other notes as well, but people usually do not--generally what you are most concerned about is what happened at the site). There are many possible things you can code (see Lofland and Lofland). As you code, think about possible linkages and relationships between the different codes you use. This thinking will tend to produce more sophisticated codes, broader or more precise codes. Be sure to record what you are thinking during the coding process and the thinking that produced new, more sophisticated codes. This is also a form of analysis which should be carefully described in your theoretical notes. Push yourself to develop deeper and more revealing/descriptive/accurate categories and codes.

As you code, keep in mind that possible linkages and relationships may also include confounds to causation. This is a valuable aspect of quantitative research that should be considered in qualitative reflection. Don't get too bogged down in thinking about confounds at first--let the ideas flow in your theoretical notes, but especially give this some thought as you move to more formal analysis near the end of the study. Robert Rosenthal's writings help in exploring this topic in greater detail.

[See Ratcliff, 1995, pp. 20-31 for an extensive discussion of qualitative validity and reliability]

Can find validity in qualitative research by:

• Divergence from initial expectations--see personal notes kept from the beginning to see how the data has pushed you from initial assumptions
• Convergence with other sources of data--using variation kinds of triangulation and comparisons with the literature
• Extensive quotations--from field notes, transcripts of interviews, other notes
• Other research data--such as archival data, recordings (video or audio)
• Independent checks/multiple researchers--more than one person involved in the research of those studied; team research approach or other sources of verification.
• Member check--where you go back to those researched, at the completion of the study, and ask them if you are accurate or need correction/elaboration on constructs, hypotheses, etc. Some take this to the point of the researcher and those researched working together in the planning, conducting, and analysis of results.

Can find reliability in qualitative research by:

• Multiple viewings of videotape--by same person or different people
• Multiple listenings of audio tape--by same person or different people
• Multiple transcriptions of audio tape--by same person or different people

These would probably come out quite high in most cases, but at least they would be an attempt.

It is important to note that high reliability may suggest a systematic bias at work in data, a bias shared by multiple researchers or across observations by the same researcher. This is why many qualitative researchers emphasize validity rather than reliability; documenting what occurs in an accurate manner may reveal inconsistencies. As qualitative researchers are fond of saying, "You never cross the same river twice" (because it's never the exact same water, the banks of the river are never exactly the same because of erosion, etc.). Reality is dynamic; it changes constantly. Of course, this overlooks important continuities of the real world when taken to extremes, most of us recognize. But it is important to realize that low reliability could be consistent with high validity if the social situation is constantly in flux, or people might see things differently because they are seeing different aspects, different levels, different perspectives, of the whole which is far more complex than any single perspective/person might see. Putting two different accounts together might result in a better understanding of the whole than either one separately, even though the consistency between those accounts might be rather low. Together, the two very different accounts--reflecting low reliability--could produce even higher validity. It's something to consider.