The Investigation Process Research Resource Site
A Pro Bono site with hundreds of resources for Investigation Investigators
Home Page Site Guidance FAQs Old News Site inputs Forums
. . . .. . . . . . . . last updated 8/8/09

INVESTIGATING
INVESTIGATIONS

to advance the
State-of-the-Art of
investigations, through
investigation process
research.



Research Resources:

Search site for::


Launched Aug 26 1996.

 

This section provides a discussion of criteria for evaluating methodologies, a discussion of why evaluations are needed, and dentities of currently available ethodologies. Then evaluations of those investigation methdologies are presented or referenced.

Evaluation criteria reflect underlying concepts about both the investigation process and the phenomena being investgated.

Comments are invited, and cn be posted on the AIPRE, or submitted to me for posting here.


What is an Investigation Methodology?

First, we need to decide what a Methodology is. Like the term scientific method, it is perceived in differently ways in different organizations. Appendix A shows the diversity of perceptions with a list of 24 different thoughts about what a methodology is from a variety of academic, government, corporate and privates sources. For investigation process research and evaluation purposes, I have highlighted what I believe are useful elements for defining a methodology.

  1. research procedures used (the "why" for a methodology - should have a methodology for research, which is what an accident investigation really is)
  2. approach, strategy, and methods (the intellectual foundation for a methodology - conceptually, it should be based on an integrating approach and strategy)
  3. system of principles, practices, and procedures (what a methodology is - this is one I have used for prior research; the principles provide the intellectual foundation for development of a methodology and are an essential component )
  4. system of methods followed in a particular discipline (more of what it is - not a method but system of methods)
  5. documented approach for performing activities in a coherent, consistent, accountable, and repeatable manner (more of what it is - a methodology is "formalized" by documenting it, so other users can achieve verifiable results when they use it)
  6. organised, documented (more of what it is - a methodology can't just be an unorganized, unintegrated jumble of methods or procedures,
  7. A set of procedures or methods used to conduct research (more of what it is - it incorporrates a set of procedures or methods rather than just a single procedure or method)
  8. Methodology is sometimes used synonymously with "method", particularly a complex method or body of methods, rules, and postulates employed by a discipline. Some usage arbiters regard this usage as pretentious and questionable. (more of what it is - but don't confuse method with methodology)

  9. In software engineering and project management, a methodology is a codified set of practices (sometimes accompanied by training materials, formal educational programs, worksheets, and diagramming tools) that may be repeatably carried out to produce software. (more of what its elements are)

  10. methods, procedures, and techniques used to collect and analyze information (what imethodology does)
  11. collect, store, analyse and present information; a research process. (more of what it has to do - methodology has to provide for presentation of informate it generates)

  12. Proven (it has to have been shown to work)
  13. knowledge organisation methodology (more of what it has to do -provide for the organization of information which it uses)

These snippets from Appendix A can be consolidated into criteria for deciding whether something presented as an investigation methodology actually is a methodology. A methodology must

  • be based on a intellecual framework that ties all the practices and procedures of the methodology into an internally consistent and mutually supportive whole.
  • be documented or "formalized" so all users can achieve coherent, consistent, accountable and repeatable results.

  • address the acquisition, documentaiton, < en.wikipedia.org/wiki/Methodology_(project_management)

    Investigation Methodology Evaluations: Why bother?

    I want to run a "good" incident investigation program. What investigation methodology should I specify for my program. What choices are available? What would be my best choice? How can I tell which is "the best" for me?

    The answers depend on, among other considerations,

    • your policy for investigations
    • what you want to have investigated
    • what results you would like your investigation program achieve
    • what resources you are willing to devote to support the program
    • what constraints you want to impose on your program?
    • how it ranks among other demands on your resources
      • Investigation problems involving the objectives, scope, methodology, output specifications and uses of work products were first reported in 1975[1], . and expanded in 1980 [2] For example, in 1974 and 1975, I personally took classroom polls of individuals involved in investigations to learn the methodology they used. Almost without exception, they were unable to name their methodology, and hardly any realized they had any methodological choices. Further probing disclosed significant differences among their methods, which were summarized and published. (Benner 1980) From 1982-1985 additional research into Rating accident models and investigation methodologies was published, describing the models at 17 government agencies and organizations. In 1981 Ted Ferry, a colleague of mine at USC, published the first book providing a list and snapshot of investigation methods. In 1992, the AIChE's Center for Chemical Process Safety published a list of 18 investigation "techniques." Today, the growing list of "investigation methodologies" and my observations of how they evolved further suggest the need for more research into the comparative merits and value of investigations of the growing list of "methodologies" and their supporting materials.

        What methodologies are available?

        The following is a list of proposed investigation methodologies that I am aware of.

        How do they compare?


        Several works offer evaluations of investigation methodologies. I am working on a summary of these evaluations
      • .

Footnotes


    [1] Benner, L., ACCIDENT THEORY AND ACCIDENT INVESTIGATION, Proceedings of the Annual Seminar of the Society of Air Investigators, Ottawa, Canada, 1975 existing theories, process charting and application and expectations of theory.

-->

Appendix A

Examples of perceptions of "methodology" including highlighted elements referenced in Section I.

  • A system of principles, practices, and procedures applied to a specific branch of knowledge.
    www.dmreview.com/resources/glossary.cfm.

    1. A system of principles, practices, and procedures applied to a specific branch of knowledge.
      www.dmreview.com/resources/glossary.cfm Tis is one I have used for prior research.

    2. The way in which information is found or something is done. The methodology includes the methods, procedures, and techniques used to collect and analyze information. www.epa.gov/evaluate/glossary/m-esd.htm

    3. A methodology represents a package of practical ideas and proven practices for a given area of activity, such as the planning, design development or management of IT-based systems. www.nao.org.uk/intosai/edp/directory/misc/glossary.html

    4. A documented approach for performing activities in a coherent, consistent, accountable, and repeatable manner. [Treasury Enterprise Architecture Framework] www.ichnet.org/glossary.htm

    5. type the portion of the phrase denoting the method, leaving off the ending: www.lib.duke.edu/lilly/artlibry/dah/screens/search.htm

    6. A methodology is a process where the activities are primarily intellectuall. Typically only the end goal of the process is manifested as a physical work product. In software the analysis and design activities are normally governed by a specific methodology. pathfinderpeople.blogs.com/hslahman/glossary_of_oomda_terms/

    7. A set of procedures or methods used to conduct research. www.usg.edu/galileo/skills/ollc_glossary.html

    8. the experience, expertise, processes, procedures and methodology relating to the Services and/or Deliverables developed by Clik during the course of providing the Services; www.cliksoftware.com/pages/terms.htm

    9. The procedures and techniques used to collect, store, analyse and present information; a research process. www.parliament.vic.gov.au/sarc/E-Democracy/Final_Report/Glossary.htm

    10. Proven processes followed in planning, defining, analyzing, designing, building, testing, and implementing a system. it.csumb.edu/departments/data/glossary.html

    11. The methods to be employed to accomplish the objectives and the goals of the grant project. The methodology is embodied in the "Work Plan." www.ciwmb.ca.gov/GreenBuilding/Grants/FY200405/definitions.htm

    12. A method with a set of rules for applying the method and a set of heuristics for judging when the different stages are complete. Incorporates a management process in addition to the technical process of the method. www.intelera.com/glossary.htm

    13. Refers to the system used to calculate the Expected Family Contribution (ie, the Federal Need Analysis Methodology). www.petersons.com/bcd/articles/glossary.asp

    14. "AVID Methodology" is not about changing curriculum, but is about allowing almost all students to have access to a rigorous college preparatory curriculum. The teaching methodologies which are most effective in this quest are writing, inquiry, and collaboration. hines.nn.k12.va.us/Pages/avid/glossary.html

    15. ongoing projects in the framework of the European Environment Agency (EEA) show that (traditional) methods of terminology, nomenclature, classification and thesaurus development should be further developed and integrated into a comprehensive knowledge organisation methodology. This approach also has to address the problem of thematic (vertical) fragmentation and specialisation, lacking concomitant cross-disciplinary (horizontal) re-integration of terminological knowledge; www.wsl.ch/forest/risks/mexft/budin.html

    16. A documented process for management of projects that contains procedures, definitions and roles and responsibilities. www.mccombs.utexas.edu/faculty/Linda.Bailey/glossary.htm

    17. The approach, strategy, and methods used in market research. www.glencoe.com/sec/busadmin/marketing/dp/mktg_resrch/gloss.shtml

    18. The research procedures used; the section of the final report in which the researcher outlines the approach used in the research, including the method of recruiting participants, the types of questions used, and so on. www.dmdsurveys.com/dmd_site3/terminology_pages/terminology_m.html

    19. The study of how we gain knowledge about the world. www.stile.coventry.ac.uk/cbs/staff/g_urwin/glossary.htm

    20. An organised, documented set of procedures and guidelines for one or more phases of the software life cycle, such as analysis or design. www.kelso.scotborders.sch.uk/departments/computing/resources/reference/glossary_of_terms.htm

    21. the branch of philosophy that analyzes the principles and procedures of inquiry in a particular discipline

    22. the system of methods followed in a particular discipline wordnet.princeton.edu/perl/webwn

    23. Methodology formally refers to the study of the methods involved in some field, endeavor, or in problem solving. Most sciences have their own specific methodology. Methodology is sometimes used synonymously with "method", particularly a complex method or body of methods, rules, and postulates employed by a discipline. Some usage arbiters regard this usage as pretentious and questionable. ... en.wikipedia.org/wiki/Methodology

    24. In software engineering and project management, a methodology is a codified set of practices (sometimes accompanied by training materials, formal educational programs, worksheets, and diagramming tools) that may be repeatably carried out to produce software. en.wikipedia.org/wiki/Methodology_(project_management)