The FEPL specifically tests the characteristics of skill and expertise associated with human perceptual and cognitive processes. Forensic handwriting science is a discipline which uses these processes almost exclusively to provide evidence as to the authorship, or process of production, of questioned writings. In spite of the common use of forensic handwriting evidence in courts of law for over a century the expertise itself and the theoretical claims made by the community of Document Examiners has, until recently, remained almost devoid of appropriately designed validation studies. Criticism of the field, particularly in the United States, has been widely reported and discussed in the forensic literature (Risinger, Denbeaux & Sacs, 1989; Risinger & Sacs, 1996; US v. Starzecpyzel, 1995).

Since our collaborative research interest in validation testing of forensic handwriting examiners' skills has commenced, a number of relevant events have occurred and important studies have been published on the topic. Judge McKenna in the Starzecpyzel decision (United States v. Starzecpyzel, 1995) stated that ‘The Daubert hearing established that forensic document examination which clothes itself with the trappings of science, does not rest on carefully articulated postulates, does not employ rigorous methodology, and has not convincingly documented the accuracy of its determinations’. Forensic handwriting identification was in spite of this statement recognised as a practical skill and therefore held to be admissible in evidence. Risinger and Saks (1996) argued that an implication of this decision was the potential for plummeting validation standards for admissibility, which may result in the burden falling on the opponent ‘to prove affirmatively that the skilled witness cannot do what they claim they can do’. Clearly this is not a position that forensic examiners and legal specialists would be comfortable with. These authors then open the window to allowing some resolution of the concerns by stating that ‘science can examine the dependability of such a process (handwriting identification) even when the process is not a science’. Science has commenced to do so.

Forensic validation studies have been reported by Kam, Fielding and Conn (1997), Kam, Wetstein and Conn (1994), Found, Sita and Rogers (1999) Kam, Gummadidala, Fielding and Conn (2001) and Sita, Found, Rogers (2002). These studies have provided support for the expertise claimed by practitioners, or at least those that have been tested within the trials, in terms of it being real and demonstrable. In each case an error score has been reported. It is this error score that is relevant to document examiner client groups, particularly the judicial system. It is the magnitude of the error score that best dictates the probative value of the evidence being presented.

In order to assess the magnitude of any error the Forensic Expertise Profiling Laboratory has adopted a philosophy of testing, largely based on the criticisms of this field historically. Specifically our philosophy attempts to address the following guiding statements: "The level of correctness of the assertions made by examiners from day to day casework is not likely to prove to be a credible source for the (validation) data needed" (Huber & Headrick, 1999). "A process such as handwriting identification presents a number of potential subtasks dealing with variables such as writing instruments, forgery of various sorts, age, health and so forth. No single test can map the abilities of any one practitioner, or any group of practitioners" (Risinger & Saks, 1996). "A great many tests… would be necessary to know what, if anything, (examiners) can do accurately, and under what conditions"(Risinger & Saks, 1996). "A complete testing regime would have tests which covered the entire spectrum of conditions and difficulties" (Risinger & Saks, 1996).

The FEPL designed validation trials, often containing nested experiments on theoretical aspects of the discipline, aim to provide both the forensic and legal communities with data that may assist them to assess the validity and error rate of routine forensic handwriting examination practices. The following are examples of how information generated by the FEPL may be used;

• Laboratories may assess the skill of their specialists according to a comparison between individual results and grouped results
• Laboratories may assess the skill development profile of their trainees according to a comparison between individual results and grouped results
• The Document examination community can assess the validity of their theoretical beliefs according to the results of trials designed to test such beliefs
• Document examiners can assess shortfalls in their own skill according to revision of trials where errors were made
• A numerical profile of individual/grouped expertise can be documented over time
• The data can provide information to the judicial system as to the validity and potential error rates associated with the type of evidence being presented.

Trial types

The FEPL has experience in the development and administration of the following trial types:

• Proficiency trials. These trials are designed to imitate typical cases routinely examined at forensic laboratories. The trials comprise a small number of documents and generate only a small number of opinions. To date trials of this type have been administered covering expertise areas such as typewriting examination and identification, obliterations and erasures, handwriting and signature examinations and photocopy examinations.
• Individual skill validation trials (barrage trials). These trials are designed to investigate the characteristics of the skill claimed by individuals and laboratories. The trials imitate case structure, where specimen and questioned documents are submitted for comparison, however comprise a large number of questioned writings (typically 200 opinions expressed per person). To date trials of this type have been administered covering aspects of forensic handwriting and signature expertise such as identification, elimination, and the identification of simulation, auto-simulation and disguise behaviour. These trials provide information as to the potential error rates associated with each of the writing types.
• Laboratory skill validation trials (barrage trials). These trials have the same format as the individual skill validation trials however all opinions expressed are the product of full laboratory quality assurance programs.
• Expert / Lay group validation trials (barrage trials). These trials are designed to investigate the group expertise claimed by Document examiners in handwriting and signature examinations. These trials have the same format as the individual expertise validation trials however all opinions are statistically compared to a lay group.

Phases in the validation trial process

1. Design and construction phase

Each trial is specifically designed to be realistic, produce results that are useful to the users and which test specific aspects of the skill that is applied to casework. Skill associated with examiners ability to identify handwriting written by specimen writers, exclude handwriting not by specimen writers, and identify simulation, auto-simulation and disguise processes has been tested for samples of both original and non-original writing.

Cases are constructed using writers from the La Trobe university and the wider community. In addition, over the past five years, a pool of practised ‘forgers’, exhibiting various skill levels, is also used. Handwriting populations are constructed, according to the aims of the trial. From the population of handwriting collected a suitably sized sample is constructed into the questioned and specimen groups.

Trials are produced in non-original (high resolution ink jet printed) form. Each questioned writing is individually coded. Accompanying the trial is:

• A population of specimen (exemplar) writings.
• An instruction booklet (clear instructions are provided outlining the examinations that should or should not be undertaken).
• An answer sheet (usually a tick sheet) which reduces the amount of time examiners spend recording results.
• A CD containing high-resolution scans of all images.

At the commencement of the program each individual and laboratory is provided with a unique identifying code. All results are referenced using these codes. A timetable is set by the experimenters.

2. Data collection phase

Results are either faxed or mailed back to the laboratory for processing.

3. Report preparation phase

All results are compiled and prepared into a report for release (usually by email) to the participants. This usually takes four to six weeks from the closing date.

4. Support phase

Experience has shown that for a number of months after each trial result has been released, laboratories seek further information regarding the trial and the implication of results. Laboratories can seek assistance from the FEPL in the form of detailed presentations regarding the trial and advice regarding their current practices. This function has been conducted on a user pays basis.

 

Return to FEPL homepage

 




Content Approved by: Head of School
Page maintained by: Administration Manager
Last Updated: 18 September, 2003