The Investigation Process Research Resource Site
A Pro Bono site with hundreds of resources for Investigation Investigators
Home Page Site Guidance FAQs Old News Site inputs Forums
. . . .. . . . . . . . last updated 8/8/09

INVESTIGATING
INVESTIGATIONS

to advance the
State-of-the-Art of
investigations, through
investigation process
research.



Research Resources:

Search site for::


Launched Aug 26 1996.

 

  

View more comments about Rand report.

(Rebuttals/opinions/invited. Go to Researcher's Forum.) I am also willing to post additional views here if anyone cares to submit them to me.

Comments on RAND -ICJ Report about NTSB

Pro bono comments from a former head of the NTSB's aviation investigation bureau about the study "Safety in the Skies: Personnel and Parties in NTSB Aviation Accident Investigations" by the RAND Institute for Civil Justice. 12/99.


PRELIMINARY COMMENTARY ON THE RAND CORPORATION STUDY:

"Safety in the Skies - Personnel and Parties in NTSB Aviation Accident Investigations"



FOREWORD

The National Transportation Safety Board (NTSB) is a good organization; at times, a great one. As a concept devoted to public safety in transportation or viewed from the perspective of their personnel, the agency has contributed immensely to the safety and therefore the success of air travel throughout the world. It has been very largely a distinct personal and professional pleasure for me to be directly or indirectly associated with them for over thirty years. However...


For all entities, government or otherwise, a periodic, introspective look can be beneficial. Not only do technological and social factors change with time but also a constant trap exists to keep doing things like they have always been done. Respond to the current disaster, solve the existing crisis, keep the public happy, retain their confidence as well as that of their representatives who authorize budgets, etc., etc., etc. All too often it is easier to defer or deny longer range improvements because of these exigencies. Besides, admit to the need for change and some people - uninformed people - will interpret that as the job not being done reasonably in the first place. Thus, the RAND report presents a challenge to the Safety Board. Reduced to a one word question, it would be "modernization?".


In any event, the study Chairman Jim Hall requested is a significant milestone in the in the saga of the NTSB. He asked for comments. I hope these are beneficial.


C. O. Miller

Posted 6 April 2000

Table of Contents

INTRODUCTION


In June 1998, The National Transportation Safety Board (NTSB) commissioned the RAND Corporation to conduct "a self critical examination of the (NTSB's) capability to carry out...the investigation of major commercial aviation accidents". "Preserving and enhancing the NTSB's ability to fulfill (this) crucial mission" was the central motivation for the research.


A seventy-seven page executive summary, "Safety in the Skies - Personnel and Parties in NTSB Aviation Accident Investigations", was released to the public December 9, 1999. A longer "Technical Report" was to follow in early 2000 but has not yet been received. Hence, this commentary is, in a sense, preliminary.


Another dimension of these comments is what my role was in the RAND research, albeit a limited one. Several past professional papers and documents related to key issues were sent to RAND during the summer of 1998, including numerous suggestions made to the Safety Board over the years. Then in late November and early December 1998, I was interviewed by telephone for a total of one and one-half hours by one of the research staff. Finally, I met with the project director and an assistant briefly during a family visit to Washington, DC in the Spring of 1999; however, this did not involve any substantive discussion about the study.


The discussion below is in four parts; (1) overview impressions, (2) what seemed particularly valuable about the study albeit with reservations in some instances, (3) RAND findings with which disagreement prevails and/or issues that were not covered at all and (4) brief concluding remarks. A few personal suggestions as to what the Safety Board should consider doing now are included throughout the text. Others are certainly implied. A comprehensive specific list of recommendations could follow in another format should that be deemed worthwhile by recipients of this commentary.


References to RAND report page numbers relevant to the point just discussed are shown in parentheses at the end of given paragraphs. Relevant previous professional papers of mine are identified in footnotes should the reader care to examine previously expressed views on the subject in question. Countless similar documents by other authors on issues described herein are in my personal library; available as the need arises.



THE REPORT IN GENERAL


The study was structured initially around two major issues, the NTSB "Party" process as used in major aviation accident inquiries and the adequacy of Board resources in terms of personnel (including training), equipment, funds, etc. To the everlasting credit of the RAND team - and to NTSB Chairman Hall for his support - many other issues were also examined; issues that clearly affected the operational capability and efficiency of the entire NTSB. The scope of these issues can be deduced from the eight objectives used as the basis for RAND's conclusions and ultimate recommendations as quoted below:


  1. Strengthen the party process
  2. Create a more expansive statement of causation
  3. Modernize investigative procedures
  4. Streamline internal operating procedures
  5. Better manage resources
  6. Maintain a strategic view of staffing
  7. Streamline training practices
  8. Improve facilities for engineering and training

(p. xv)

Little or no exception is taken with these generalities. Fortunately, they are discussed encompassing more detailed recommendations on pp. 47-57 of the report. The questions arise there. Indeed, the report could be summarized using the "Bikini" analogy (placed into air safety lore by Jerry Lederer, the "Father of Aviation Safety", years ago). Like a bikini bathing suit, what the report reveals is indeed interesting and meaningful; however, what it fails to disclose is vital!


The report is well indexed and quite thoughtful in acknowledging sources of information including a wide variety of air safety experts and other knowledgeable personnel.



PARTICULARLY VALUABLE FINDINGS BY RAND


This section describes briefly the most cogent findings although occasional limitations are also cited.


The Party System.

The report is a primer in understanding the required "party" system for the Board to use during the complex, real world investigation process. Enhancing the concept by consideration of technical assistance from other than the usual, current organizations is especially noteworthy since those current organizations cannot be expected to always be objective.

(pp. xiv, 20-21, 29-32, 47-48)


Project Management Reform

This includes a broader vision of the investigation process than the concentration on the field phase which dominates attention today. It also includes creating an effective cost accounting system that seemed to be shockingly absent based on RAND's assessment of that issue. Interestingly, a systems approach to NTSB aviation investigation tasks was first introduced about thirty years ago as the "Accident Inquiry Manager" (AIM) program. [1] It included precisely the same objectives and systems approach suggested by RAND in 1999 to improve investigation project management. It got lost somewhere along the internal politics way.

(pp. xv, 9, 21-22, 50, 52-53)


The Accident PREVENTION Mission

The report effectively emphasized the need to shift from the fundamentally reactive nature of accident investigations more into prevention activities such as incident investigations. After all, accident investigation is part of accident prevention but prevention includes far more than investigation. In this regard, it is unfortunate that RAND did not reference the "system safety" discipline as a needed part of NTSB's modus operandi . This concept entails application of engineering, operations and management skills and practices to prevent accidents considering the entire system and life cycle of that system. [2],[3] This will be discussed more later.

(pp. 5, 49)


Acquisition and Control of Safety Information

Innovations such as "Knowledge Management", "Office of Information Management" and a "Knowledge Agent" demonstrated a willingness to attack one of the most difficult problems in air safety today: the failure to fully appreciate the lessons of the past and implement those lessons in a timely, effective and efficient manner. [4] Expected use of sources external to the Board will also aid in improving the quality of Board products. Unfortunately, this topic did not recognize the need for some national, or even international, center for aviation accident prevention knowledge as has been sought many times in the past. Furthermore, the issue of the proper release of factual findings in the weeks immediately following an accident was not addressed.

(pp. xiv, 21, 38, 40, 52)


An "Insular" NTSB

RAND perceptively identified staff isolation from critical members of the aviation community. While not cited specifically, this has probably occurred because of budget and time induced limitations regarding participation on professional society technical committees and attendance at seminars, let alone deficiencies in more conventional training programs. A much welcomed change in this respect was Chairman Hall's advocacy and approval for significant staff attendance at the August 1999 annual conference of the International Society of Air Safety Investigators (ISASI).

(pp. 8, 47)


Investigations and Family Assistance

Necessary - specific - delineation of responsibilities for these two Board functions was noted by RAND so as to not dilute the already time critical efforts of field investigators yet carry out the Board's legislated direction to aid and comfort victims' relatives and close friends.

(p. 18)


Training Requirements

Management skills, professional capabilities and investigative competence were rightly identified as the major subsets of the training mission. These constituted a fundamental learning objective: to be able to ask the right questions at the right time and recognize the validity of the answers. Specific areas of learning, not necessarily available at the present time, included automation, complex system hazard diagnosis and use of an interdisciplinary approach. Long term staffing and training programs were also emphasized. What was not covered was a need to train and/or indoctrinate other than Office of Aviation Safety (OAS) personnel who become involved in investigations; e.g., the Board Members, personnel from the Office of Public Affairs, etc.

(pp. 21, 35, 53-56)


Justification for Additional Staff

This position was supported very well by RAND, not only in terms of needed increase in quantity and capability of investigative staff but also in terms of factors which can negate the effectiveness of those persons; e.g., stresses involved in sustained, long hours and other physical and psychological aspects of field investigations. Morale problems among the investigative staff have been reported in recent years as being severe. The totality of this problem is not known.

(p. 23)


Need for Improved Investigation Facilities

This is one of the more critical items needed to ensure maintenance of independent investigations; however, it does not negate the expected continuing need to have work performed at manufacturers, government test facilities, etc. where NTSB oversight must be maintained. Procedures related to such activity need to be clarified.

(p. 37)


Concern for the Tardiness of Reports

Obviously, staff and other resources affect this issue. Nevertheless, RAND pointed out clearly that overall efficiency of management has at least an equivalent impact. Improved project management, noted earlier as being required, is a major factor in this issue. Not covered was the influence of "probable cause" as will be discussed shortly. Suffice for now, the mindset to establish the cause of an accident or prioritized versions of causes invariably delays the entire inquiry process.

(pp. 24, 42, 53)


Investigation Organization Deficiencies

Listed among these was the absence of a "Coordinator" position on teams where the Investigator-In-Charge (IIC) has too many things to do to be able to effect adequate coordination among the various disciplines contained in the investigation groups. In the field this could be a Deputy IIC. At headquarters, this could be accomplished via the Accident Inquiry Manager (AIM) during the entire investigation life cycle as discussed earlier.


A specific investigation segment, absent at the NTSB and not addressed by RAND, is the need for "accident prevention program management" investigation, (nee "safety program management"). Finally, in recent years, such programs have become requirements of airlines. Does it not follow that this should become a defined area of inquiry in most cases? The programs are supposed to be a major factor in prevention - and they really are - thus, if an accident does occur where did the program break down? What else could have been done from a program viewpoint. [5],[6]

(pp. 40-41, 50)


Enhanced Report Quality Control

Improving product quality is a continuing objective of every organization. Use of peer review teams was one of the excellent suggestions by RAND in this regard. Another effective method would be to provide samples of reports selected randomly for review by qualified outside investigators, particularly retirees. This possibility has been brought to the attention of the International Society of Air Safety Investigators (ISASI) for possible endorsement and coordination; however, nothing has resulted thus far.

(pp. 41, 48, 51)



ABSENT OR INADEQUATE TREATMENT OF ISSUES


Notwithstanding the many meaningful issues examined and numerous excellent suggestions for resolution thereof (not all have been covered in this text) the RAND report contains inadequate treatment of some points if, indeed, they were addressed at all. This could readily have been caused by the broad scope of NTSB operations. RAND had a difficult issue selection task at hand. However, non appreciation for the issues that follow could also have occurred because the research team did not have anyone directly associated therewith who could be deemed an air safety or investigation professional. This is a phenomenon witnessed in many air safety studies in recent years and is an issue in itself. Finally, many of the criticisms to follow could probably have been mitigated by callback to interviewees or other forms of review of tentative findings. Some of the deficiencies are critical on their surface. Others fall victim of "the devil is in the details" phenomenon.


Needed Upgrade of NTSB's Enabling Legislation

NTSB has had only one major revision to its statutory authority since its founding in 1967 (The Independent Act of 1974). Actually, most of NTSB procedures can be traced back well before that; as early as 1938 and throughout the days of the Civil Aeronautics Board until the CAB "Bureau of Safety" became the "Bureau of Aviation Safety" of the NTSB
the organizational pattern on which the entire NTSB developed. Of course, historical precedent is fine, not something that should be superseded frivolously. However, a symptomatic example of what happens if modernization does not occur can be found in the introductory remarks still heard these days from NTSB public hearing Chairpersons. They will state something like, "These investigations are intended as fact finding with no formal issues and no adverse parties."
with lawyers and other advocates all over the place including being party spokespersons! Other traditional but meaningless legal or quasi-legal procedures are followed, frequently by persons not having requisite prior background to question or speak in those kinds of proceedings.


Thus, a major omission in the RAND report is the absence of recognition that NTSB's legislated responsibilities and authorities need objective review and probable changes. Some are critical now. Others are longer range, particularly in the safety policy area; for example, the degree of involvement of state/local authorities involvement in G. A. investigations. Other situations abound.


The qualifications for and the precise role of Board Members needs clarification. Are they to be individual, independent jurists, jury members, safety experts in a particular mode, or just what? Indeed, what is the underlying mission of the agency? Does it include oversight responsibilities for the entire aviation system. Some members of congress seemed to have endorsed this despite no specific charge for NTSB to do so in the legislation. Collectively, are Board Members to be, as was once stated as a Board public relations gimmick, "The Supreme Court of Aviation"?
the final determiner of safety decisions? Confusion reigns between investigation for "cause", meaning blame, or for prevention purposes; that is, accountability or recommendations. Current legislative authority of the Board is ambiguous as to criteria for and choice of which accidents should be investigated and to what degree. As another detailed example, the Board has no leverage to preclude parties to an investigation screaming "propriety interest" and thus justify not releasing vital evidence that could lead to meaningful recommendations. This issue came up - quietly - during a relatively recent, very major case. Fortunately, and to their credit, the NTSB Office of General Counsel has this matter on the agenda of a public meeting to be held in the Washington, D. C. area, April 25-26, 2000. Many of us in the accident prevention business are watching with extreme interest for the outcome of this latest test of accident prevention and competing social justice interests.


Absence of a National Air Safety Policy

RAND was careful to accurately quote many rules and procedures for NTSB investigations. What they did not do was to identify many upstream factors bearing upon NTSB's performance which could be considered policy matters. For example, What about uniform definitions for "safety", "safety cost benefit analyses", or "risk management", Does anyone really believe "We never compromise safety" as has been voiced by numerous FAA Administrators, airline CEOs, etc. What is the role of enforcement and punishment in accident prevention? These and many other similar subjects have been discussed for decades across all modes of transportation among other endeavors; yet no group has taken the initiative to explore these issues on a national basis. [7] A role for the NTSB???


Probable Cause" et al

This primary deficiency in the specifics of the RAND report relates to their acceptance of the NTSB's continual emphasis on "probable cause" and/or other indicators of prioritized causes. RAND recommended "Create a More Expansive Statement of Causation" based, correctly, on multiple factors involved; however, then they backtracked. They also stated, "The probable cause statement should clearly state the principal event or failure that led to the accident. The probable cause statement should then include all the related causal factors. Then causal factors should be ranked in terms of their contribution to the event, according to methods to be outlined in investigative procedures". To anyone with significant investigation experience, this is simply illogical and fraught with pitfalls.

(p. 49)


The "links of a chain" analogy explains this readily (even though in itself is an oversimplification). If seeking accident prevention rather than blame, all accidents involve a sequence of events - links of a chain which en toto , produce the mishap. Thus, accident prevention can be aimed at any of the links that can reasonably be eliminated
or at least controlled. Therein lies the fatal defect about single or prioritized cause. Attention becomes focused upon only one link, even when other links are identified. That is the real world of oversimplified valued judgments, media headlines, and thought processes which do not know the difference between accident prevention and looking for scapegoats.


A simple question: if it is agreed accidents are comprised of a chain of hazards without any one of which the accident would not have occurred, then which link is most important?


That this concept was not appreciated by RAND is evidenced by at least six references in their text wherein "cause" was treated in the singular. Safety Board publications continue to refer to singular "cause"; e.g., "This effort is being undertaken (to determine) the cause of the...accident" [8] (Italics added.)

(pp. 1, 7, 15-16, 41)


This concentration on single cause factors has other ramifications as well. For example, the RAND report cites the TWA Flight 800 and the USAir Flight 427 cases as being very lengthy and "yielding a conclusion that was technically controversial and circumstantial". Still, the Safety Board provided numerous meaningful recommendations which were accepted by the FAA and others long before the 427 report was published. (The 800 report is yet to arrive as of this writing.) It follows that the "controversial" and "circumstantial" apply to which donkey the tail will be pinned on, not accident prevention. That sort of decision process is what contributes markedly to the Board's inquiry process becoming very time consuming.

(p. 6)


A bit of personal history teaching accident prevention might be worth mentioning here. One of the my most effective classes during the Advanced Safety Management course at the University of Southern California Institute of Aerospace Safety in the mid 60s involved providing students, as homework, a factual report of a relatively complex accident. The next day, the students were broken into four groups. Group #1 was charged to provide a sequence of events of the accident listing the events in chronological order. Group #2 was told to also provide a sequence of events; however, they were to use any one of several accident occurrence models we had discussed in class. Group #3 was given the task to simply present the probable cause of the accident. Finally, Group #4 was told to be ready to share with us their recommendations for remedial actions.


After a suitable break for discussions within each group, a "volunteer" from Group 1 was asked to list on a blackboard the sequence in chronological order. Similarly, Group 2 provided their sequence but with more sophistication than just the chronological approach. Then, Group 4 (not Group 3) provided their recommendations for preventive action. Next, the class was dismissed.


The howls from Group 3 erupted. After all, they had done their work conscientiously and wanted to be heard. I replied that based upon how the discussions went with Groups 1, 2 and 4 what could Group 3 add to the prevention mission? No one really ever argued. Lesson learned! Facts, analysis and remedial recommendations are all that is really needed if the objective is only accident prevention.


"Cause" may be meaningful for other purposes (i.e., civil or criminal law or to satisfy public curiosity) but that is not what the role of NTSB is - or should be - when functioning in the public interest.


Knowing this and having brought it to RAND's attention, I was quite curious as to why, on one hand, they came down on the side of multiple cause factors yet they recommended continuing NTSB's use of the "Probable Cause" and prioritized causes. The answer to this question can be found in the last paragraph on page 42 of the report:


  • "...the statement of probable cause carries considerable
    weight in the aviation community. Lacking regulatory or
    enforcement authority, the NTSB's influential and highly
    public pronouncement of probable cause is one way the agency
    can play a central role in aviation safety.


I read that as "necessary for attention and public relations", the need for an NTSB icon, an image of professionalism. All this does have some significance, for example, at budget approval time; however, the price they pay for dilution of effort towards prevention as described above is not logical in the minds of the vast majority of accident prevention professionals.


For further discussion on this issue including its history, pros and cons, suggestions, etc. see the articles cited below (among many others available). [9],[10],[11]


NTSB's "Customers"

Any producer of a product or offerer of a service will be effective only to the extent his or her "customer" is recognized and understood. That is an elementary management precept. NTSB's problem is that were they to try to define their "customer, the result would be very generalized and include words like "public interest". Neither the legislation, RAND nor the NTSB have addressed that fundamental issue. Unfortunately, the result is that much of what NTSB does has no substantiation of need. For example, what "data" should be attained? Who uses the NTSB generated statistics and for what purposes? Should it not be more properly oriented towards "knowledge" gained from investigations on how to prevent accidents. Literally speaking, "data" in itself means little.


Through their web site, NTSB has done a highly creditable job making available materials that should be in the public domain (analyses of G.A. factual information being an exception). However, does their "data" system really serve the needs of safety researchers, safety officers or even their own investigators, etc. That question is open at this time based upon the RAND report and other sources.


Limited Attention to Past Lessons

This problem was not addressed by RAND in connection with the NTSB - perhaps because it is not unique to the Board in matters of public safety. The issue was described in a recent paper of mine as "...the continuing failure by persons in the (aviation) system to effectively and efficiently apply lessons of past mishaps in order to prevent future accidents". [12] In the final analysis, RAND apparently did not go back far enough in time with their research into what investigative procedural remedies had been offered and why these might have been denied, thus depriving the Board of earlier views meaningful to issues described by RAND. In particular, NTSB's role in "action failure" did not come through.


Success Dependency

Whereas, according to RAND, the staff plays a major role in NTSB's success, it is a gross oversimplification to observe the agency,"...is wholly dependent on the professionalism of its staff". Management style and people factors at all levels throughout the NTSB also are obviously vital ingredients to the efficacy of end products. Fiscal and project management were quite rightly questioned by RAND. personnel management was not. Morale among the staff is always a problem in high energy work as conducted by NTSB investigators among others there. Nevertheless, morale at the Board is at a very low level; however, is that only from the stress situations reported by RAND?

(p. 19)


Personnel Selection and Political Influence

Beginning with the process of Board Member selection and extending into the upper levels of NTSB management, "political" influences (if that is the right term) were not addressed by RAND. For example, the premier aviation trade publication observed early this year, "A five-year appointment to the NTSB is a high-visibility...handout..." [13] The enabling legislation also raises questions when it states "no more than three Members are to be appointed from the same political party". This projects a questionable and generally unfair portrayal of the capabilities of appointees; that is, politics before substance. Some appointees in the past have left observers justifiably disappointed (laughable in a few cases). From my experience, however, the vast majority of Members have taken their role seriously with the only criticism being perhaps the two year learning process they usually must go through to really become effective. The so-called qualifications required for Board Members as contained in NTSB enabling legislation are virtually meaningless. Objectivity, intelligence and willingness to put in many working hours are criteria superior to experience in some mode of transportation safety or a modicum of knowledge in, say, human factors. Indeed, it can be argued that Board Members should not be a paragon in some transportation or safety field. In such cases, he or she might unduly influence colleagues on the Board, a value judgment, at the expense of objective review of products from the staff.

(13)


A related problem that never has been addressed by RAND or anyone else to my knowledge has been the effect of the Senior Executive Service (SES) and antiquated Civil Service position descriptions and requirements. SES allows the vulnerability of senior staff members to whims of the Chairman and/or other Members let alone the original placement of senior managers based more on views "compatible" with the Board than with needed administrative and technical managerial capability. The Civil Service placement system has yet to recognize the existence of a safety profession beyond the hard hat and steel toed shoes concept. A research background Ph.D. in some non technical field will be given more credit than decades of getting hands (and everything else) dirty in field investigations or other aspects of accident prevention. This is a national disgrace and certainly is not limited to the NTSB.


Background and Duties of "Professional Staff Members".

Numbers of personnel and other budget figures are one thing; precisely what are people's qualifications, assignments and access to budgets for particular endeavors could be quite another thing. Whereas RAND presented statistical data regarding professional staff members, they did not get into task descriptions and the sometimes unique technical and managerial skills needed within an organization such as the Office of Aviation Safety. An example of this is the absence of NTSB's recognition of skills implicit in the system safety engineering, operations and management process including extensive attention to hazard analyses on a life cycle basis. The DoD has utilized this concept for decades. (The FAA exemplifying civil aviation has not understood it even to this day.) "System Safety" contains valuable principles affecting investigation, and not just for design, test and operations per se .[14] RAND failed to include system safety in its analysis of NTSB's processes.

(p. 19-21)


Safety/Accident Prevention Professionals.

The RAND report clearly identified the need for a "...broad set of technical skills and combine them with an acquired set of skills unique to the examination of technical failures on a massive scale." Unfortunately, they did not identify this as a profession in itself, the safety/accident prevention profession. Nor did they include the need for these professionals to also be conversant with broad-based operational and management technology. Accident prevention is not limited to application of "technical" knowledge. This may be the perception of the lay person but only goes to prove that preventing accidents is not for amateurs. As shown in the safety education field, it takes more than a pilot's certificate, a degree in engineering or whatever, to learn and apply the immense, interdisciplinary remedial knowledge available from past incidents and accidents.

(p. 33)


Party Representatives Qualifications.

Closely allied with the foregoing is the issue of qualification of party participants, especially spokespersons. These persons theoretically are accepted or rejected by the IIC; however, in the immediate aftermath of an accident, this falls well down the list of IIC priorities. As a result, accidents tend to be on-the-job-training to an exorbitant number of persons. This, in turn, adversely affects the Board's investigation efficiency but was not part of RAND's assessment of the party process. As recommended to the NTSB in the past, this problem could be at least partially resolved by establishing specific training and/or experience requirements for party participation in NTSB investigations. This could well entail some sort of a certification process analogous to the FAA's Designated Engineering Representative (DER) program.

(p. 31)


Information Dissemination During the Investigation

This has become a particularly sensitive issue in view of the recent EgyptAir and Alaska Airlines tragedies. Numerous charges and countercharges have surfaced concerning media allegedly misrepresenting what the Board has provided as "facts". This dimension of the problem was not addressed by RAND but is a first cousin to something they did speak to...sort of. They left unchallenged the following naive statement obtained from NTSB procedures:

"Party representatives are not permitted to relay information

back to corporate headquarters without the consent of the

IIC, and then only when necessary for accident prevention

purposes. Information is not to be used for litigation or

public relations."

Who is kidding whom? This rule has never been enforced or at least such action has never become a matter of public knowledge, even when party spokespersons at hearings have been members of a company's legal staff. In other cases in general aviation, it was common knowledge that company sponsored investigators reported, directly to their general counsel's office in some cases.


This information dissemination problem is a difficult one. It deserves someone's attention to bring together the experts from the accident investigation/prevention community, the media and the Bar in an attempt to improve things. Just writing unenforceable, thus unenforced, rules, is not the answer. My personal thoughts on this subject include required training/indoctrination and certification for investigators and spokespersons participating in NTSB proceedings.

(pp. 15-16)


Board Member Participation in Report Development

RAND implies that Board Members do not really become involved in major report preparation until the document is presented at a "Sunshine Meeting". That is simply untrue. Granted, some Board Members over the years have adopted a hands-off approach until the staff has completed their work. Others, however, have acted like and were perceived as an Investigator-In-Charge. Indeed, a frequent comment has been heard on this issue from frustrated investigators when dealing with some Members, "Well, just tell me what you want and I'll write it that way", long before the report reaches its "Notation" stage.

(p. 16)


Compounding this fact is RAND's suggestion that, "...the Board Members should be afforded greater opportunity to monitor the progress of the report." What, indeed, is "monitor"? NTSB has never disclosed any policy in this regard; that is, the precise role of the Members vis-a-vis the investigating staff in substantive matters in the report. As stated earlier, do Members constitute a collective investigating group, jury, a panel of judges or just what? To my knowledge, this has never been the subject of any NTSB policy made available to the public.

(p. 41)


Case Reconsideration Deficiencies.

The Board's procedures state, "Parties to the investigation may petition the Safety Board to reconsider and modify its findings and/or probable cause statement...". The basic test used has been the introduction of new evidence. Of course, gross errors in analysis should also count but rarely, if ever, has this occurred The procedures really do not account for this. Nor are the reconsideration procedures available for persons other than "parties", despite the fact that frequently, subsequent investigations (not just during litigation) reveal facts exculpatory to persons or groups whose reputations have been damaged by Board findings. The Board also has lacked a defined tracking and followup system to "petitions" or equivalent submissions by persons not used to formal legal procedures. RAND did not reflect any of these problems in their report.

(p. 16)


Delegation of Authority With International Accidents

.As exemplfied by the EgyptAir accident Oct. 31, 1999, numerous problems can arise when an air carrier is lost over a foreign land, in another nation's territorial waters, or even in international waters but near another country. When this entails "delegation" of investigative tasks, access to and control of wreckage and other evidence, communications regarding the investigation, fiscal accountability for search, rescue and salvage and other jurisdictional disputes are almost a given. RAND seemed to dismiss this problem area only with reference to practices suggested by the International Civil Aviation Organization (ICAO). Unfortunately, ICAO tells everyone they have no authority to become involved in a given case, nor will they seek such authority. Of particular detriment to accident prevention is the process by which NTSB "observer" investigators are precluded from returning to the U. S. with any written notes, reports or other hard material reflecting what they learned. On numerous occasions, evidence such as recorders are sent to the NTSB for analysis. Still, communicating the results of such examinations is usually strictly prohibited by agreement between the governments involved.

(p. 17)


Budget Uncertainties.

It is intuitively obvious that some kinds of accidents (e.g., TWA Flight 800) can make a mockery of traditional budget forecasting. This would have occurred even with zero based budgeting, the absence of which having hampered the NTSB for decades in many ways. It seems inefficient for the Board to have to ad lib during arguments as to which government agency or "party" will be accountable for various costs. A standing joke among general aviation investigators has been the need to go back to the classroom so as to take a course in "Negotiation" (or "Begging"). Another dimension to this problem is related to the thoughtful suggestion by RAND that outside experts be utilized to augment technical and other staff personnel. However, how do you integrate a $200 per hour consultant with a civil servant at the GS 13-15 range, especially when the consultant's objectivity might be questioned by various parties? A policy in these areas needs to be explored before committing to some hurried, ad hoc procedure.

(pp. 20-21, 32)


Influence of Investigation Complexity

RAND stressed the "increasing technological complexity" and the growth in air transport activity. They imply that makes investigation in the future much more difficult. That is not necessarily accurate. They forgot two things. First, "complexity" has increased because of non-safety driven inputs corollary to the "technical" tasks; the news media frenzies, proliferation of litigation, politician grandstanding, excessive demands by families of victims, etc. Thus the investigation team needs to be expanded beyond the traditional "air crash detectives", albeit all of these persons must not lose sight of the primary mission - accident prevention.


Second, technical complexity of aircraft and ground systems can be handled on a par with past challenges with technical (and managerial) advances in investigation methodology; for example, modern hazard analysis techniques, sneak circuit analysis, advanced modeling and simulation, new materials analysis procedures, improved safety information storage and retrieval systems, accident prevention program evaluation protocols and, of course, state of the art Digital Flight Data and Cockpit Voice (and Video) Recorders. Add to this real time data link - a technology which has been around for a half century - and the investigative "complexity" factor becomes moot. However, the powers that be must authorize and be willing pay for these investigation improvements even if, indeed, a cost benefit does not show a saving by becoming modern.

(p. 6, 25-26, 39)


Report Preparation and Timeliness

RAND suggests the life cycle of accident through report release should be targeted at one year and possibly an additional six months for unique cases. This may be realistic given current resources and policies; however, it need not be. It has been a rule of thumb to experienced investigators that at least 90% of the ultimate level of facts learned are available in the first two weeks after the crash. Obviously, that last 10% might be critical if the attitude is to concentrate on cause as distinguished from accident prevention recommendations. However, to the credit of the Safety Board, justified recommendations can and usually are promulgated long before any one year cycle. This then becomes a matter of tradeoff between dotting every "i" and releasing Board resources to more productive events, such as more incident investigations. Attention should be directed to the Military which puts time requirements on their investigations which may not be as complete as some purists might prefer but their prevention programs do not seem to be suffering from excessive expediency. At the base of this issue is a combination of accident inquiry management effectiveness and the preeminent emphasis on cause as discussed earlier.

(p. 31)


NTSB Recommendation Process.

The RAND report paid limited attention to the Safety Board's recommendation process. Among other things, they failed to explain the role of the Board compared to the FAA (or other target recipients of recommendations). Over the years, NTSB/FAA conflicts have arisen frequently concerning this issue with the public being caught in the middle of the confusion. In reality, this is simply characteristic of a classic staff-line relationship in total management; NTSB being the staff advisory group and the FAA carrying out the difficult line, decision-making function. Both have their place in the National Aviation System and should remain separate from one another. The legislative history of the DOT Act of 1966 and the Independent Safety Board Act of 1974 are textbooks for this situation.


RAND did make an excellent point when they said, NTSB recommendations should be "structured around a statement of expected performance". The Board has limited expertise and time to structure details of often complex issues albeit they have a unique perspective on such issues, institutional knowledge, so to speak. They may suggest as RAND said, "operational or design solutions" for consideration; however, the Board and the public should acknowledge making a recommendation is one thing, implementing it within the confines of a viable transportation system may well be something else. And that is usually the FAA's difficult job. In any event, the NTSB should exercise a much greater role regarding their recommendations; namely, develop a system that tracks not just what someone says they will do but also what actually was done and when, plus the effectiveness of the action taken.

(p. 41)


General Aviation Investigations

Whereas this RAND study was not aimed at general aviation (G.A.), it quite correctly identified investigation quality problems therein today and more so in the future brought about the growth of G.A. operations, the proliferation of more complex aircraft and the ever present resources problem. As an avenue towards this problem, RAND suggested a new look at the possible roles of state and local agencies as well as that of the FAA - a concept that has been discussed on and off for at least 30 years. Unfortunately, RAND's proposed solution to this problem which entails possible delegation of cases to local authorities may just exacerbate the situation. Many of those "local" authorities may well be police officials inserting criminal law complications by persons not trained in effective investigation for prevention purposes. Action on this issue, however, is needed.


Unfortunately, RAND did not address at all a basic issue that also exists now; namely, the acceptance of only the FAA and manufacturers as parties to virtually all G.A. investigations. (by law for the FAA). This has led to gross unfairness to pilots, aircraft owners and victim family members among others, let alone meaningless or skewed available accident prevention information. G.A. investigation quality control has been a problem for decades given the minimal resources applied to investigations applicable to this vital segment of air travel. This is another one of those problems that has been brought to the Board's attention for decades without resolution. [15]

(7, 18, 28-29, 49-50)


Continuing Assessment of NTSB Performance.

The RAND study showed clearly the importance of an "outside" review of NTSB operations. Unfortunately, nothing was said about continuing this well known procedure, be it through periodic review by consultants, or an ongoing advisory group. Admittedly, administrative restrictions placed on advisory groups by the Office of Management and Budget are discouraging to organizations which genuinely seek periodic advice. Nevertheless this or some similar process would seem to be supported by RAND's findings in their report.



CONCLUDING REMARKS


Certainly, followup to the RAND report is essential. Let us hope RAND's fine effort does not suffer the same fate as countless past well meaning and important Washington, D.C. generated studies. Actions - or non actions - have been withheld from the public for reasons ranging from bureaucratic sensitivity over the study's findings to simple procrastination until people have forgotten about it. To this end, it would behoove the NTSB to act one way or another on recommendations offered by RAND and others such as are implicit in this paper. Assignment of a specific staff person to head such an effort would seem reasonable plus a possible seminar open to public discussion. This would even further enhance the Safety Board's justified image of openness, professionalism and objectivity.


[1] Miller, C. O., "Systems Approach to Accident Investigation", presented at the Flight Safety Foundation Annual Seminar, Montreaux, Switzerland, Oct. 23, 1969.

[2] Miller, C. O., Why 'System Safety'?, MIT Technology Review , Feb. 1971.

[3] Miller, C. O. "System Safety", Chapter 3, E. L. Weiner and D.C. Nagel (Eds) Human Factors in Aviation, Academic Press, Spring 1988.

[4] Miller, C.O., "The Most Significant Human Error in the Aviation System", presented at the Canadian Aviation Seminar, Vancouver, B.C., May 11, 1999. Published in Journal of System Safety , Q1, 2000.

[5] Miller, C. O., Aviation Accident Prevention Management: Needed Requirements and Investigations", ISASI FORUM , Nov. 1994.

[6] Miller, C. O., "Accident Prevention Management...", ISASI FORUM, April-June 1997.

[7] Miller, C. O., "Candidate Issues for a Federal Safety Policy", published in the Proceedings of the 11th International System Safety Society Conference, Cincinnatti, OH, July 30, 1993. Revised as "Candidate Issues and Answers for a Federal Safety Policy", Jan. 9, 1997.

[8] NTSB Press Release, "SAFETY BOARD TO RECOVER MORE EGYPTAIR FLIGHT 990 WRECKAGE, Mar. 3, 2000.

[9] Miller, C. O., "Down With Probable Cause...", presented at the ISASI Seminar, Canberra, Australia, Nov. 7, 1991. Published in the Proceedings, Jan. 1992

[10] Miller, C. O., ":'Probable Cause': The Correct Legal Test in Civil Aviation Accident Investigations?", LPBA Journal , Spring, 1992.

[11] Miller, C. O.,"Trapped by Probable Cause", Air Line Pilot , (in two parts), Jan./Feb,1998

]

12. Miller, C. O. "The Most Significant Human Error in the Aviation System", presented at the Canadian Aviation Safety Seminar, Vancouver, B.C., May 9, 1999. Published in Journal of System Safety , Q1, 2000.

[13] Aviation Week and Space Technology , "Washington Outlook", Jan. 1, 2000.

[14] Department of Defense, "Standard Practice For System Safety", MIL-STD-882D,

10 Feb. 00.

[15] "Remarks Re the Petition to NTSB", (to broaden participation in NTSB aviation accident investigations), presented at the annual ISASI seminar, Washington, D.C.,

Oct 1, 1981 and the AOPA annual seminar, Oct. 9, 1981 in Orlando, FL.