The Investigation Process Research Resource Site
A Pro Bono site with hundreds of resources for Investigation Investigators
Home Page Site Guidance FAQs Old News Site inputs Forums
. . . .. . . . . . . . last updated 8/8/09

INVESTIGATING
INVESTIGATIONS

to advance the
State-of-the-Art of
investigations, through
investigation process
research.



Research Resources:

Search site for::


Launched Aug 26 1996.

 

Posted 1 April 97

 

THE USE OF THE EVIDENCE EVALUATION MATRIX IN ACCIDENT INVESTIGATION

R. Leon Horman

System Safety Development Center

Lockheed Martin Idaho Technologies Company

June 1995

Idaho National Engineering Laboratory

Idaho Falls, Idaho 83415

SUMMARY

Evidence Evaluation Matrix Analysis is a new method of using an old tool, a matrix, to prove and substantiate the findings of an investigation. The technique lists findings, evidence, sources of evidence and other information in a matrix. Properly used, it clearly establishes the logic used to reach conclusions and communicates that logic to others. It is a valuable tool for resolving conflicting evidence. It also provides a record of the analysis used to examine the facts to ensure the validity and repeatability of tracking all the facts through analysis and on to conclusions.

INTRODUCTION

A persistent problem in accident investigation has been the failure to verify and validate all findings and conclusions reached by the investigators. What may have been obvious and clear to the investigators is not so obvious and clear to those reading the report of the investigation. This failure comes from a lack of correlating the investigative analysis to the facts given in the report, not providing all the necessary facts to justify the analysis, not correlating findings and conclusions back to the analysis, and other similar oversights. Evidence matrixing is a tool developed to alleviate these problems.

Although no analytical tool will absolutely validate and prove all of the findings and conclusions of an investigation, Evidence Matrixing can prove most and clearly show the logic used to establish the burden of proof for the rest. It can also show where evidence was obtained and, if desired, show the disposition of the evidence.

Evidence Validation Matrices can furnish a permanent record to be included as part of the formal report or in the investigative files. Since it is analysis that communicates significant information in comparatively little space, a logical place for it would be in the analysis section of the report.

ANALYSIS USING MATRICES

Evidence Evaluation Analysis Matrixing is not a difficult analysis process. A form similar to that shown in figure I will work very well. The investigator should feel free to modify the form to meet the needs of the investigation.

EVIDENCE EVALUATION ANALYSIS MATRIX

(Suppositions)
Findings

Evidence

Sources of
Evidence

Comments

















* Underlined evidence is contrary  to findings

Figure 1

The evidence matrix is designed to be an active analytical tool to use in the course of the investigation. Therefore, the first column can be used to list suppositions early in the investigation process. Some of these suppositions may be proven to be incorrect and never turn into findings. If they are disproven the investigator may want to remove them from the form or leave them in place. If they are left in place, they should be clearly identified as things that were disproven so no confusion will arise in mistaking these for proven conclusions or findings. On occasion, the investigator may find that a disproven supposition will result in a new finding. For example, there could be a supposition that a fire caused damage to electrical insulation causing an electrical system to short out. In the course of the investigation it may be proven that the electrical short, presumed to be have been caused by the fire, was actually the precursor event causing the fire.

Following are a few basic rules to make Evidence Evaluation Analysis Matrixing more effective:

I. First list suppositions on the matrix worksheet. These will either be disproven or will change to findings and conclusions as information is gathered.

2. List all important findings or conclusions, including positive ones. Part of this is obvious but the part about listing positive findings may not be. If the investigator lists positive findings as well as the negative ones, it adds credibility to the investigation. It will help assure that good things in the system are not changed while correcting problem areas. Those in the system will also appreciate anything positive identified in an otherwise bad situation, this helps furnish the little bit of sugar that helps the medicine go down.

3. List all evidence both supporting and contrary. If this is not done, those in a defensive position may point out that the investigation was biased in that it only looked at supporting evidence for the findings. Listing contrary evidence adds to the credibility of the report since it shows that all evidence was looked at.

4. Show where each piece of evidence originated. This also adds to the credibility of the report. Be very specific in doing this. Do not use catch all phrases, such as, "Operation's personnel knew the valve was faulty." The reader will want to know how they knew this and how the investigators got the information and who gave it to them. There may be 200 operation's personnel. The above quote does not say whether all 200 knew the valve was faulty or if only 2 people said it was faulty. It is also important to identify specifically which people said they knew it was faulty. Who they are and their positions and responsibilities as operation's personnel is vital information to the investigation. Be specific or the report is suspect and its credibility will be questioned.

5. Include any comments that would be beneficial to the board or others, e.g., the location of the evidence. Add any other information that may be beneficial.

6. Include all types of evidence, i.e., physical, paper and software, and people. Be thorough in documenting each finding with solid evidence.

7. Give enough detail that you will not have to depend on memory. There is always the possibility that the analysis may have to be justified at a later date. e.g., in litigation. Memory tends to fade and to become distorted with time.

In listing the evidence, remember that you are trying to build your case as to why you reached the conclusions given in the report. To be effective, a person may need to think like a lawyer and remember that there will be those who will try to discredit the report. Effectively using evidence matrixing will eliminate most opposing arguments to the investigators' findings and conclusions.

EVIDENCE MATRIXING EXAMPLE

Following (fig. 2) is an example of how to perform evidence matrixing. The example is based on a real event but has been modified considerably for illustration purposes. In this example, only one finding is listed. In an investigation many findings (or conclusions) may be reached. Each should be validated in a similar manner. Note, in the example, that evidence contrary to the finding is clearly stated.

In the event, upon which this example is based, a miner fell to his death about 1000 feet down an access shaft. The D hooks, meant to be used to attach the lifelines to the miners' safety belts, were so encrusted with salt that opening them by the usual hand pressure was not possible. Much of the other information given in the example has been added to show how to use evidence matrixing.

The example illustrates how, even with contrary evidence, Evidence Validation Matrixing Analysis dynamically justifies findings. Using this technique, the logic used to reach conclusions is obvious even if when the contrary evidence is strong enough to allow divergent viewpoints.

EVIDENCE EVALUATION ANALYSIS MATRIX

(Suppositions) Findings

Evidence

Sources of
Evidence

Comments

D hooks were not being used
Salt encrustation made the hooks very difficult to open
D hooks

Photographs

D hooks are in locked safe in Mr. McGruder's office. Photos in the investigation files

Worker unable to open D hook taken from victim's lifeline
Mr. Smith, the co- worker, was unable to open the D hook during interview
Witnessed by Board Members, R.L. Smith and J.P Swanson. Statements in Al files

Oral testimony that the D hooks were being used
Mr. Smith's, the co- worker, testimony
Taped testimony and transcript in Al files

Mr. Miner's (the victim) hook took 486 lbs. of torque to open
D Jones Testing, and independent lab.
Test results are in the investigation files
* Underlined evidence is contrary to findings

Figure 2

USING EVIDENCE MATRIXING WITH OTHER TOOLS

Evidence matrixing can be used as a stand alone tool in small investigations. However, it becomes much more effective when used with other accident investigation analytical tools, e.g., MORT Analysis, Barrier and Control Analysis, Change Analysis, and Events and Causal Factors Analysis. The experienced investigator will recognize the relationships between these other tools and evidence matrixing. For example, findings in an accident investigation typically relate to failures in barriers and controls or to changes in the system. If Evidence Evaluation Matrixing Analysis is conscientiously used, it will force proper logic in the analysis section of the accident investigation report.

End