Chapter One: Structured observation - the nature of the investigation
Project INISS was set up in October 1975 to investigate information needs and information services in local authority social services departments, with support from the British Library Research and Development Department. The principal aims of the project were:
A mixed strategy was developed to satisfy these aims:
The remainder of this account concentrates on the observation phase of the work and on the conclusions which we drew. Information needWe are not primarily concerned here with the debate over the concept of 'information need'(2); but it is necessary to express our point of view. We assume that only the subject of an investigation can truly experience his own information need. The investigator may be able to share that experience to a greater or lesser extent, depending how closely he is able to share in the circumstances that give rise to the need, and on the degree of mutual understanding of those experiences. Our assumption is that information needs are created as the result of an individual's performance of a social role.(3) In performing a given role an individual will primarily experience psychological (or emotional) and cognitive needs. Information may help to satisfy these needs, either in the narrow sense indicated by the idea of "anomalous states of knowledge" which clearly identifies a class of cognitive needs (Belkin, 1977), or in a more complex sense which takes into account underlying psychological needs such as the need to dominate, to achieve, or to demonstrate competence. Whether or not steps are taken by the individual to satisfy either cognitive or psychological needs by information-seeking will depend, at least partially, upon the relevance(4) of the information for the performance of his role. Note, however, that psychological needs in particular may be satisfied (at least partially) by the process of communication rather than by the information communicated. For example, the upward-striving executive may make a point of communicating information to superiors not because of the intrinsic worth of the information but because of his psychological need to demonstrate his competence. It will be clear that we regard information need as a subjective, relative concept with no existence outside the mind of the experiencing individual. Depending upon how relevant the need is for present or planned action, or for satisfying psychological needs, information need may lead to information-seeking acts. The direct, objective perception of the subject's information need by another party (research worker, information officer), is not possible. What is possible is that potential information needs may be imputed to an individual on the basis of:
The first three of these are variations upon the last: all methods of investigation are ultimately based upon observation. Either the individual observes his own behaviour and his own perceptual states and reports upon them within more or less artificially constrained frameworks (interview schedules, questionnaires, diary record sheets(5) or records of his information-seeking behaviour are kept (e.g., book issue records) from which deductions about the underlying needs are made. This account is concerned with the observation of individuals in their ordinary, everyday work roles as they communicate with others and as they perform information-production or information-seeking acts. Reasons for the choice of observation as a research methodIn any social survey research it is difficult to envisage circumstances in which interviews or self-completed questionnaires should be used in the field before adequate information has been obtained on the circumstances surrounding the survey subjects. To the survey researcher this is the stage of gaining 'site-specific' knowledge. Obtaining this kind of knowledge was the prime purpose of the observation phase of Project INISS. Before carrying out any interviews it was felt necessary to discover certain things about social services departments and those who work in them, specifically:
Clearly, information of this kind can be obtained by other means, particularly through reading research reports, monographs and internally-produced documents of the departments as well as by visits and informal, semi-structured discussions. These techniques were employed by the research team but further reasons suggested observation as an additional method:
The choice of 'structured observation' as a specific form of observation was influenced by Mintzberg's 'The nature of managerial work' (1973) which showed that observation could result not only in increased site-specific knowledge, but also in hard data on various aspects of work. Structured observation in action - development and training phaseStructured observation is easy to describe but difficult to appreciate without actually engaging in the process. Very simply, it involves placing an observer in a social setting to observe all activities defined as of interest to the research. In essence, the method is derived from participant observation in social anthropology and the distinction which is sometimes made between 'participant' and 'non-participant' observation does not fully hold in practice: some degree of participation is inevitable. The 'structure' of structured observation is imposed by the aims of the research in the same way as such aims impose structure upon any method of data-collection. Just as is the case when open questions are used in interviews or self-completed questionnaires the researcher using structured observation recognizes that not all of the structure can be determined in advance and that some structure must be imposed on the data after they have been collected. This can be illustrated by reference to our own work. The research instrument was an edge-notched card which had printed upon it certain categories according to which data were to be collected (Figure 3). These pre-determined categories were:
Categories derived from the data after the observation period were:
The printed edge-notched cards were used for preliminary manual analysis of the pre-coded categories, four-ring binders were used to carry the cards in two packs, one above the other, and this provided a firm support for writing. The cards were numbered to preserve the time sequence and to allow a rough count of events. Meetings were treated rather differently: they were recorded on ordinary paper, much as notes might be taken for minutes, and then the events which made up the meetings were transferred to the edge-notched cards when time allowed. The word 'events' has been used above and needs definition: a communication event was defined as a subject pursued over a period of time. Thus, a telephone call might be a single-event or multiple-event occurrence depending upon how many topics were raised during the conversation. Events could be linked over time as telephone calls were returned, or messages passed on, or as issues arose in more than one situation. Such events were physically linked on the cards by recording the numbers of the linked-event cards on all of the cards in the sequence. Clearly, these definitions and categorizations could only be arrived at as a result of sane preliminary planning work. Most of these resulted from a number of training exercises of two kinds:
Structured observation in action - field phaseThe basic recording process involved brief indication of relevant information under the pre-determined headings, leaving most of the recording time free for as full as possible a transcription of the content of the information event. Limitations on the observer's ability to record adequately were provided by:
Variations in the normal recording process were introduced to anticipate sane of the difficulties listed above. During the introduction procedure, described below, subjects were asked to pass on the contents of telephone discussions. (In practice it was often relatively easy to follow such a discussion by listening to the subject, particularly if the subject had initiated the call and announced the intended contact). Activities involving writing, reading or scanning imposed minor difficulties in that it was often impractical for the observer to read and describe documents as they were being scanned. Arrangements were made with subjects to ensure that observation records were adequate. These varied according to the work habits of the subject but included scanning incoming or outgoing post at pre-arranged times, usually before the subject started work; making minor adjustments to the positioning of out-trays so that they were within reach of observers; or occasionally, the subject reading written comments aloud. When incoming mail was scanned the observer concentrated upon identifying the sequence in which items were dealt with and the time taken to do so, using later pauses between information events to complete the observation records. When the subject was at his own desk the observer sat in a pre-arranged place, between one and two metres from the subject, giving an adequate view of the desk. It proved possible to sit only one metre from the subject without causing undue distraction and when a larger distance was involved this appeared to be caused by the general distribution of the furniture within the room rather than any deliberate attempt to maintain or bestow status. (These distances approximated closely to the "social distance, near phase" of Hall (1969) who noted that "Impersonal business occurs at this distance ... people who work together tend to use this distance.") The 'client chair' was always avoided. The process of observation was viewed by the Project INISS research team as a series of linked stages. Once entry to a particular department had been gained, background information about the department had been acquired and access to volunteers in appropriate posts obtained, the subsequent stages involved:
the observation period, during which the observer attempted to record all work-related information events; The 'ice-breaking' function of the introductory meet ings is illustrated by these extracts from a researcher's notes:
Something of the flavour of doing observation is conveyed in this segment from an observer's-eye view of a day in a department:
Structured observation in retrospectA number of key questions present themselves in reviewing structured observation as a working tool.
An attempt is made to answer these questions below. The effects of observationMost subjects reported feeling an initial period of awareness of being observed, lasting between a few minutes and the whole of the first morning of observation, but they all became accustomed to the experience and in many cases appeared to become largely oblivious to the fact that they were being watched. The ideal preparation activity for the observation week seemed to be one or more meetings during the opening morning, because subjects who were engaged in early meetings appeared to relax more quickly under observation presumably because their attention was focused on the other participants). After the opening day of observation only one subject, a research officer, appeared to be uncomfortable under observation, when he spent periods of up to four hours in assimilating documents and preparing reports. When not engaged in this sort of activity he appeared to behave in a 'normal' manner, suggesting that structured observation is not an appropriate working tool for use with deskbound staff, unless they have substantial supervisory or reporting responsibilities requiring contact with others. Several subjects expressed a preference for observation as a means of data gathering over being interview respondents or being asked to complete questionnaires. A number of subjects also compared the experience to that of having staff in training 'at their elbows'. Their familiarity with this type of training activity may have contributed to the success of this method of gathering information. Observation subjects were encouraged to comment about the experience of being observed both at the end of the week and in subsequent interviews. Typical comments included:
In retrospect it can be seen that 'having an observer along' was rapidly accepted by all concerned and it also emerged that it could be turned to the advantage of the subject in various ways:
The basic strategy adopted by observers when 'used' in the ways described was to make minimal acknowledgement and concentrate upon the job in hand. This task was made easier by the fact that recording usually required a 'head down' stance, eliminating most eye contact with subjects and others involved in the discussion. This was borne out during an observation period when, because of an arm injury, the observer relied on tape recording supplemented by minimal written records. It was during this period that the subject attempted to enlist the observer's support, as described above. One observation subject raised the possibility that she might have scheduled her activities for the observation week in the knowledge that she would be under scrutiny. She then dismissed the suggestion and, with the exception of one staff meeting which may have been held in a particular week because the chairman was being observed, no evidence of programme 'juggling' was detected. Comments were occasionally made by departmental colleagues to observers about the normal behaviour of subjects before, during and after observation took place. None of these led the observers to doubt the typicality of the behaviour of their subjects under observation. Indeed, the daily work pressure, the high degree of fragmentation in the work of all individuals observed and the frequent interaction with other people would have made any attempt to maintain a 'false' pattern of behaviour extremely difficult. Although subjects were shown sample record cards before observation began they did not and could not have known in any detail what aspects of their behaviour were being studied. It was, no doubt, possible for subjects to start new tasks when, without the presence of their observers, they might have taken a rest, but again the pressure of work and the fact that 'natural breaks' were taken from time to time by subjects, suggests that this was not a major influence on the pattern of work observed. Only one subject suggested that, after the initial acclimatization period, being observed had made any difference in his behaviour. He said that "I tended to cut out behaviour which would have been incomprehensible to an observer, like sitting and just thinking, or talking to myself." Other subjects showed no apparent inhibitions in indulging in either activity! Exclusion from activitiesSubjects were given the option of excluding their observers from delicate encounters but this prerogative was only rarely exercised. In each case the subject provided sufficient information for most of the events to be recorded adequately. Exclusions were occasioned by a delicate staff interview (before which the veto was exercised by the member of staff concerned), a staff appointment interview and occasional client interviews which subjects expected to be 'difficult'. The level of cooperation throughout was surprisingly high, with subjects obtaining special clearance for their observers to attend meetings and with apparently uninhibited discussion taking place on all manner of managerial, staffing and client-related topics, The independent position of the research staff was an important factor here; it is quite certain that nobody employed by the local authority would have been allowed to observe same of the activities and it is possible that the status of the research team as attached to a Postgraduate School of Librarianship rather than a School of Social Work may have, in some instances, alleviated a 'professional' threat. The effectiveness of structured observationWhether any research tool is effective can be assessed using a number of criteria. In the case of Project INISS several approaches were taken to this problem. At various times during observation, more than one subject (and attendant observer) came together for discussions and ail the records of each observer in such situations were subsequently reviewed. Very few variations were detected and these could usually be attributed to the differing roles of subjects in meetings rather than to any difference in recording style or ability (for example a chairman was likely to take at least passing interest in each agenda item, and other meeting participants might engage in non-meeting discussions). However, this test was only of limited value because, as has already been stated, observers compared notes whenever possible at the end of each working day. An attempt was made at the end of the final observation period to compare observer performance by assigning two observers to one subject and 'ringing the changes' on these pairs. In retrospect it seems fair to suggest that this exercise only demonstrated the extreme tolerance of the observation subjects under ludicrous conditions. Two subjects sitting in a smallish office accompanied by their two observers presented no particular problems, doubling the number of observers and waiving the essential continuity in observation resulted in what one subject described as "a bit of a circus." The observers were rapidly convinced of the value of structured observation in a social services setting. None of the anticipated problems in acceptance, intrusiveness or recording provided any real problem and the position of observer proved to be highly rewarding. All the observers felt that the experience provided real insights into the way in which the work was done and the contribution that information made to that work. The activity was valuable as a learning experience for the observers and, in particular, proved stimulating in terms of generating ideas about communication in these organizational settings. Much of the information gathered during observation did not lend itself readily to statistical analysis but was seen as potentially important in contributing to the design of effective information systems. Regrouping of linked events in series provided valuable examples of observed information behaviour on specific topics. The way in which information was used to reach decisions could be retraced and patterns of information behaviour were identifiable. It was realised that a great deal of background information on the working environments of subjects, their comments on sundry topics, the organization of departments and services and other themes could be retrieved and, more important, presented to the subjects for their comments if narrative reports were prepared. Accordingly reports were prepared for each subject by the observer concerned (using an agreed framework and subject to some editing by the Principal Investigator) and the subjects were later questioned about these reports in the manner described above. All of the observation subjects recognised the picture presented of themselves and no major errors in reporting were discovered. Some comments included:
One incident may serve to illustrate how these reports were used by sane respondents. The subject had questioned a statement in the narrative about his 'passive' behaviour in certain discussions (minimal use of 'supportive' noises, little eye contact, etc). He had read this section to his senior social workers and had asked "Is this really me?" They had discussed the section and two things emerged:
The subject had found this section useful in provoking the discussion and felt that he had learned sonething from it. A footnote to this illustration may be added. When at a subsequent stage in the development of Project INISS the Principal Investigator (who had read this narrative in his editing role) found himself negotiating with the same subject, knowledge of this particular idiosyncracy was very helpful when occasional 'pregnant pauses' occurred during the discussion. A further method of assessing the results of observation was to show the narratives to other people in the field. Although permission has been received from most subjects for this process to take place this has not yet been attempted systematically. However, the comments received from the social services staff who have seen one or more narratives confirm that the roles described are recognisable and that the behaviour as described is in no way alien. When is structured observation appropriate?Experience in using this technique in social services departments suggests that it is a highly appropriate method for gathering basic information about how, when and why information is used in organizations which are characterized by considerable face-to-face or telephone communication and heavy fragnentation of work. To sunmarize a number of points already made, the method is probably inappropriate:
Structured observation is likely to provide very rich data about staff, if employed in an acceptable organizational context. The data also allow generalization about organizations of that type in the form of theoretical propositions. Use of structured observation to help design appropriate information systems within an organization also has much to recommend it. The value of structured observation as part of a systematic process of investigation which leads on to structured interviews with a larger number of randomly selected staff, has, we feel, been amply demonstrated by Project INISS. NOTES
However, the positivist-oriented attempts to measure relevance have failed to take account of the inter-related nature of relevancies. In this context Schutz's work on 'zones' of relevance and intrinsic relevance systems is of major importance. (See:
References
|