Par Henri FANCHINI,
henri.fanchini@artis-facta.com

In this ultimate case, how much can we trust in the behaviors of operators layed in simulated situations ? We know that biases do exist. But, how can we identify and quantify them ?

Faced with this complex challenge, the main attitude consists in improving the simulator's fidelity, hoping it will at the same time increases the representativeness of operator's behaviors.

A review of the literature shows that most papers are focused on the engineering fidelity of simulators and only a few on psychological (or functionnal) fidelity. Engineering fidelity can be described as the degree to which the simulator duplicate the physical, functional and environmental conditions of the simulated system. Psychological fidelity corresponds to the perception that the operators have of the realism of the simulator.

With regard to the question of knowing if the behaviors of operators on simulators are more or less optimized, compared to the behaviors they might have in the real situation, the results don't match.
For example, in the nuclear field, Krois and Hass (85) carried out an evaluation of the performance, calculated as the delay before operators activate safety systems.
They show that experienced operators took 6 or 7 time less in the simulated situation as they used to do in the real situation. It appears that during the simulator session, those operators expected more for unforseen events and so they were able to distinguish annonciator signs.
Vice versa, in the aerospace field, Powers and Sarrafian compared two kinds of simulators, the first one, a traditionnal ground-based simulator, the other one, more up to date and air-borne.
They observed that the performance of experienced pilots was better on the traditionnal simulator, and conversely, novice pilots had better performance on the new simulator, where engineering fidelity was best. They also identified differences between the operative strategies carried out by the two parties.
They discovered out that the most experienced pilots had learnt to make up for the hiatus of the ground-based simulator.

The main conclusions of the literature are the following :

• psychological fidelity and engineering fidelity are not well correlated : more and more engineering fidelity doesn't ensure an increase of psychological fidelity. Conversely, it is possible to run a "good" simulation session on a defective simulator.

• the automatic data collecting tools on simulators are not very usefull as long we don't get a certified model of the optimum human performance.

• the performance level of the operator on simulator isn't a reliable indicator of the intrinsic value of the simulator.

If we had to establish a hierarchy in the way that different aspects of fidelity contribute to the behavior's representativeness, certainly, the psychological fidelity would have the most important influence, followed by the engineering fidelity.
As well, considering the engineering fidelity, we are almost able to appreciate the deviation with the reality, it is much more difficult when focusing on psychological fidelity.

In our quest for improving representativeness, two relevant points have to be taken into account :

• sometimes, especially during low probabilistic accidental simulated situations, there is no way for comparing the simulation to any real reference situation.
  It that case, representativeness is appreciate through a comparing of a simulated situation with an imaginary situation.

• in almost all simulated situations, operators are faced with human interface like instructors selecting defaults and environmental conditions, people acting as external workers

While managing the simulation session, the instructor mainly contribute to the psychological fidelity.
The instructor found his own landmarks which he confronted with the operator's behavior. Those landmarks are specific to each instructor who constructs his reality on his own and also by integrating the shared and normalized culture of the instructors people. While comparing his own vision of reality (the way as-it-might-be) to the observed behavior, the instructor evaluates in some degree the representativeness of the situation he is faced with.
Through the weighting of the gap between his own reality and what he is currently observing, he builds the evaluating criteria which will allow him to appreciate the operators.
Obviously, those evaluating criteria applied to the factual behaviors, the expected actions and the established omissions. But, they generate the following dilemma : On the one hand are the signs that are related to a more or less explicit model of humain performance, and on the other hand, his judgement is based on other indications.
All the instructor shared a kind of clinical vision of the operator's behavior. Even for a temporary observer, it is possible to identify some operator's attitudes that are related to what transactional analysts used to describe as a "T-shirt".For an experienced observer, a T-shirt indicate the favorite passtime, game and feeling of the subject, the kind of mental universe where he livesÉ
The most frequent T-shirt are :"I know that you are watching me", "You can't trust anyone (in this crew)", "Look at all the efforts I am making", "This time you have been caught in the act".

Our point of view is, that instructors are aware of those signs, because they are involved in a strong relationship with the trainees. Sometimes, during the simulation session, the instructor cunningly deteriorates the process conducted by the trainees, using repeated failures, with the intention of involving the operators in choosing and carrying out a specific procedure.
Doing this, he annihilates all the efforts of the trainees that were crowned with success. Without the knowledge of the instructor, the meaning of the failures change, they no longer originate from the process but appear as dirty tricks.
Through the remarks exchanged by the trainee, a discernible guessing game can be identified "guess the procedure or action he wants us to do now?".

A simulation session evolve in three dimensions : on one axis, representativeness, on the second, pedagogy, and on the third, evaluation of the trainee (and on fourth dimension, time). In this three dimensional space, repelling area and attraction area might exist. Each simulated session is located in this space.
Is it reasonable to believe that the instructors are able to manage those three dimensions on the go at once ?

The degree of psychological fidelity mostly depends on the quality of the relationship between the instructors and the trainees, and the psychological fidelity is the main constituent of representativeness.
But the paradox is, that in as much as this strong relationship is based on mutual trust it exists only in the simulated situation and is lacking in the real situation, it becomes an insurmountable obstacle if we want the simulated situation to match the real one.