Open Access Paper
12 April 2021 Digital twinning within a novel human-in-the-loop verification method for HUD safety-critical approach and landing
Author Affiliations +
Abstract
We have designed a visual flight guidance system that enables manual control of aircraft operations in degraded visual environments down to Cat III, both for take-off and landing. This has been achieved by means of visual guidance cues displayed on a head up display (HUD) and whereas this is not in itself novel, our development methods and approach to verifying its operation we believe are. In order to certify the system as airworthy, compliance with the relevant airworthiness standards defined in 14 CFR part 21 and other related guidance material, needs to be demonstrated. Demonstrating compliance by actually flying the aircraft a statistically appropriate number of times is prohibitively expensive. The challenge in this case was to harmonize existing flight guidance algorithms with a faithful aero model of a new airframe and to demonstrate their effectiveness in a manner that was economically viable. To achieve this, we constructed a Digital Simulation and Verification Environment (DSVE) that hosted a Digital Twin (DT) of the system such that its operation in real conditions could be accurately predicted. Paul Wisely's paper is being published without an associated oral presentation because he passed away on 18 March, 2021 after a long illness. Paul Wisely was a member of SPIE since 2006 and a senior member since 2011. He won SPIE's "Best Paper" award two years running, 2009 and 2010, for papers presented to the Display Technologies and Applications for Defense, Security, and Avionics III & IV conferences. Paul will be sadly missed by the many SPIE colleagues who knew him.

1.

Introduction

We have designed a visual flight guidance system that enables manual control of aircraft operations in degraded visual conditions both for take-off and landing in environments down to Cat III a.

This has been achieved by means of visual guidance cues, computed from aircraft sensor data, displayed on a head up display (HUD) and whereas this is not in itself novel, our development methods and approach described in this paper to verify its operation we believe in many respects are.

In order to certify the system as airworthy, compliance with the relevant airworthiness standards defined in 14 CFR part 21 and other related guidance material, needs to be demonstrated to the satisfaction of the authorities.

The challenge in this case was to harmonize existing flight guidance algorithms with a faithful aero model of a new airframe and to demonstrate their effectiveness in a manner that is economically viable.

To achieve this, we constructed a system called a Digital Simulation and Verification Environment (DSVE) that hosted a Digital Twin (DT) of the system such that its operation in real conditions could be accurately predicted and hence its fitness for purpose verified.

The design approach for the digital twin was to host the guidance algorithms, developed by Hoh Aeronautics Inc. of Lomita California and implemented using the model-based implementation techniques of Ansys SCADE Suite and SCADE Display, to replicate the flight guidance symbology, together with an aerodynamic model of the aircraft developed by Laminar Research Inc, using blade element theory.

2.

Purpose of the Flight Guidance System.

The guidance system in this application is constructed such that it provides a replication of the head down instrumentation information together with additional specific flight guidance information to the pilot by means of projection on a Head Up Display (HUD) Combiner located in the pilot’s forward field of view as shown in figure one. This enables the pilot to fly an instrument approach solely by reference to the HUD.

Figure one:

General view through the windshield

00445_PSISDG11759_117590H_page_2_1.jpg

Whereas the HUD provides similar information to that available head down, flying with a HUD is superior in several respects:

  • Firstly, the displayed information is conformal, that is to say objects depicted on the display overlay the real-world objects that they represent when appropriate.

  • Secondly, the display is collimated, that is to say effectively focussed at optical infinity. For technical and human factors reasons, the focus is of course short of true infinity to ensure that all points within the binocular disparity field are convergent.

  • Thirdly, the viewing angles and scale of the display are larger than those subtended head down which makes the display easier to use by the operator.

  • Finally, the pilot can maintain situational awareness more readily when flying “eyes out” for the approach and is thus focussed to acquire the real-world touchdown zone.

3.

The Requirements

In order to be certified for commercial airworthiness, systems must be shown to comply with the appropriate regulations. For low visibility operations to Cat III a, these are defined in FAA advisory circular AC 120 28D appendices 2 and 3. This document relates to the on-board and off-board equipment used during take-off and landing and the demonstration of the accuracy and integrity requirements of the combined equipments.

The type certification approval for the equipment, system installations and test methods should be based upon a consideration of factors such as the intended function of the installed system, its accuracy, reliability, and fail-safe features.

Landing System Performance for low visibility landings systems shall be demonstrated to achieve the accuracy of performance with the probabilities defined as follows:

  • Longitudinal touch down earlier than a point on the runway 200 ft. from the threshold to a probability of 10-6

  • Longitudinal touchdown beyond 2700 ft. from threshold to a probability of 10-6

  • Lateral touchdown with the outboard landing gear more than 70 ft. from runway centreline to a probability of 10-6

  • Structural limit load, to a probability of 10-6

  • An acceptable means of establishing that the structural limit load is not exceeded is to show separately and independently that the limit load that results from a sink rate at touchdown is not greater than 10 f.p.s.

  • Bank angle resulting in hazard to the airplane to a probability of 10-7

  • A hazard to the airplane is interpreted to mean a bank angle resulting in any part of the wing, high lift device or engine nacelle touching the ground

  • Also, specific precision must be met on approach as follows:

    • Glideslope tracking between 700 and 100 feet; the greater of ± 35μA

    • Localiser tracking between 700 and 300 ft; ± 35μA

    • Localiser tracking between 300 and 100 ft; ± 25μA

    • Airspeed tracking between 700 and 50 ft; ± 5 knots

4.

System Operation

The facility to provide manual flight guidance by means of a HUD in a commercial aircraft is not novel [1].

Guidance cues calculated by specialist proprietary algorithms created and developed by Hoh Aeronautics1 have been Certified by the FAA for use in Cat III a conditions for a number of years on commercial aircraft2.

Our particular use of digital twinning is, we think, fairly novel (and therefore interesting) because it includes an operator-in-the-loop control task interposed between the digital twin of the aero model and the digital twin of the control laws themselves.

The system provided a flight simulation at realistic airports worldwide by means of the Laminar Research X Plane simulation system that included adjustable weather conditions.

The twin recorded data during the approach and landing to facilitate subsequent statistical analysis such that comparisons with the regulations could be performed.

5.

Twin Test Station Arrangement

The basic arrangement of the station comprises a computer, display screen, and twin inceptors to control the simulation operation.

The computer hosts several functions as follows:

  • X Plane 10 professional aircraft simulator which includes an aero model of the aircraft calculated through the use of blade element theory.

    • With blade element theory, any surface, for example a wing, may be made up of several sections, one to four being typical, and each of these sections further divided into as many as ten separate subsections. The lift and drag of each section are then calculated, and the resulting effects applied to the whole aircraft.

    • When this process is applied to each component, the simulated aircraft will fly in a manner that is very similar to its real-life counterpart.

  • The Hoh Aeronautics control algorithms, implemented using a certified model-based software suite.

  • A comprehensive HUD symbol set, again implemented by means of a model-based software suite.3

  • An application providing a graphical user interface to enable the operator to enter and edit initial system parameters, manage the overlay of the HUD symbol set over the X Plane outside world scene, and record the necessary flight and touchdown data.

6.

The Approach Control Task

The approach symbology is illustrated in figure two

Figure two:

approach symbol set

00445_PSISDG11759_117590H_page_4_1.jpg

Taking the symbols indicated by the arrows from top left and working clockwise we have

SymbolNameComment
AIIIApproach modeMode currently active
VOR/LOCHorizontal guidance mode engaged 
BoresightAircraft Physical heading 
G/SVertical Guidance mode engaged 
05Pitch reference ladder 
6.6Angle of attack 
WormSideslip errorCrab error indicated against the vertical fin
ScaleGlideslope deviation scale 
1740Current altitude 
-3.0Desired glidepath 
-521Vertical speed 
Circle with wingsFlight path marker or velocity vector.Indicates the direction of the centre of mass of the aircraft
1700Radar Altitude 
ScaleLocaliser deviation 
CircleFlight Director 
WormSpeed error wormError between desired and actual
ChevronLongitudinal Acceleration CaretIndicates longitudinal state
127Groundspeed 
123Current airspeed 
130Selected AirspeedDesired approach speed

The task is basically to fly so as to minimize indicated errors as follows:

The velocity vector is purely driven by aircraft/environment state and shows where the centre of mass of the airframe is going.

The flight director is driven by the horizontal/vertical guidance algorithms and indicates where the aircraft ought to be going, hence the task is to keep the symbols overlaying which indicates the approach is on track.

The speed error worm indicates the error between desired airspeed and the actual. The task here is to adjust the power to minimize the error worm to zero.

The longitudinal acceleration caret responds to power and for example would be set above the Velocity Vector wingtip if the worm indicates the need to speed up. When the airspeed is steady and on value, the acceleration caret will sit on the wingtip.

The crab angle error worm indicates whether slip or skid is occurring.

7.

Performing an Approach and Landing

The operation is somewhat complex and is best appreciated through following an example of a test run with data capture and analysis as follows.

The airport selected for this example is London Heathrow, where an approach and landing is illustrated using runway 27L. HUD X plane switch-on initialisation pre MCDU data entry is shown in figure three.

Figure three:

X plane display with HUD Symbols on initialisation

00445_PSISDG11759_117590H_page_5_1.jpg

The approach data, desired glidepath, runway length and runway elevation were set as shown on the MCDU display in figure four, and airport details as shown in figure five.

Figure four:

Station Display containing autopilot control panel, flap settings, stabiliser trim, gear control and Multi Function Control and Display Unit (MCDU). This display is used to set up the desired approach

00445_PSISDG11759_117590H_page_6_1.jpg

Figure five:

Airport Control Panel

00445_PSISDG11759_117590H_page_6_2.jpg

As data is input, the display updates until typically figure six.

Figure six:

initialised display

00445_PSISDG11759_117590H_page_7_1.jpg

Figure seven:

ILS capture and AII mode active

00445_PSISDG11759_117590H_page_7_2.jpg

The system entered AIII mode automatically when conditions were right as controlled by the algorithms.

The approach was flown by adjusting the power to reduce the speed error worm on the wingtip to zero whilst keeping the Flight Path Marker (FPM) over the Flight Director (FD). This ensured correct ILS tracking and airspeed control.

The aircraft was flown down to the flare, power retarded and the flare cue followed in a smooth movement until touchdown. The system detected touchdown and captured the point on the runway where this occurred together with other data.

Approaching touchdown, the vertical guidance entered a pitch only mode termed Flare Guidance which ensured a gentle touch down.

Figure eight illustrates the aircraft tracking the Flare Guidance Cue and commands power reduction. The horizontal guidance enters a lateral only mode post touchdown to track the runway centreline.

Figure eight:

Flare and power retard

00445_PSISDG11759_117590H_page_8_1.jpg

8.

Data Sets

Approach runs were repeated many times to build a significant data set table as shown in figure nine.

Figure nine:

Sample of a data table. Approach speed, vertical speed, lateral and horizontal position at touchdown, roll angle Pitch angle Yaw and vertical acceleration were typically captured as shown. Overall, more than 1,000 runs were required at a variety of airports under a variety of conditions to capture a suitably diverse data set.

00445_PSISDG11759_117590H_page_8_2.jpg

9.

Data Analysis

Sequential analysis was carried out continuously during testing as the data was gathered to ensure that the system was behaving in a convergent manner and that there were no problems.

If there were, then the cause would need to be investigated before proceeding.

Figure ten shows how this process was applied in the first few hundred runs. To be acceptable the plots must lie below the orange line, which is conformed.

Figure ten:

X & Y std dev

00445_PSISDG11759_117590H_page_9_1.jpg

We will recall that performances required to be verified were as follows:

  • Longitudinal touch down earlier than a point on the runway 200 ft. from the threshold to a probability of 10-6

  • Longitudinal touchdown beyond 2700 ft. from threshold to a probability of 10-6

  • Lateral touchdown with the outboard landing gear more than 70 ft. from runway centreline to a probability of 10-6

  • Structural limit load, to a probability of 10-6.

    • An acceptable means of establishing that the structural limit load is not exceeded is to show separately and independently that the limit load that results from a sink rate at touchdown is not greater than 10 f.p.s.

  • Bank angle resulting in hazard to the airplane to a probability of 10-7.

    • A hazard to the airplane is interpreted to mean a bank angle resulting in any part of the wing, high lift device, or engine nacelle touching the ground

  • Also, specific precision must be met on approach as follows

  • Glideslope tracking between 700 and 100 feet; the greater of ± 35μA

  • Localiser tracking between 700 and 300 ft; ± 35μA

  • Localiser tracking between 300 and 100 ft; ± 25μA

  • Airspeed tracking between 700 and 50 ft; ± 5 knots

Examples of this are shown.

Figure eleven illustrates plots of Vertical Speed Indication deviation with 99% confidence Intervals which show that the results are within the limits expected.

Figure eleven:

Vertical speed indication deviation with 99% Confidence Intervals

00445_PSISDG11759_117590H_page_10_1.jpg

Figure twelve:

Touchdown footprint. In this example, a bias towards a longer landing was noted caused by a simulation error. This was subsequently corrected.

00445_PSISDG11759_117590H_page_10_2.jpg

Figure thirteen:

Sink rate at touchdown

00445_PSISDG11759_117590H_page_11_1.jpg

Figure fourteen:

Probability of landing less than 200ft from threshold at a probability of 10-6. As may be appreciated, the 10-6 value indicated is some 620 feet.

00445_PSISDG11759_117590H_page_11_2.jpg

Figure fifteen:

ILS tracking accuracy example

00445_PSISDG11759_117590H_page_12_1.jpg

The application of “twinning” is a very powerful technique to predict the performance of non-deterministic systems where a probability distribution is unknown and must be investigated by trial.

It should be noted that the system approach described is an engineering development tool to enable correct implementation of the desired system.

For certification purposes, more formal arrangements would need to be employed.

The system also includes extremely effective diagnostic capabilities which are beyond the scope of this paper to describe, both in terms of fault analysis and dynamic adjustment of performance “on the fly”.

Acknowledgements

Hoh Aeronautics

Ansys

Chris Taylor, Simon Bracken, and Darren Guscott of AAL whose invaluable work made all this possible.

Professor Ken Edwards for his valuable inputs

References

[1] 

Paul Wisely, “A digital head-up display system as part of an integrated autonomous landing system concept,” in Proc. SPIE. 6957, Enhanced and Synthetic Vision, (2008). https://doi.org/10.1117/12.793806 Google Scholar

Notes

[1] Hoh aeronautics Inc. Palos Verdes Drive North, Lomita California

[2] Such as the Boeing 737 NG of American Airlines which is fitted with a BAE Systems HUD

[3] The model-based software suites are produced by Ansys and comprise SCADE Suite and SCADE Display

© (2021) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Paul Wisely "Digital twinning within a novel human-in-the-loop verification method for HUD safety-critical approach and landing", Proc. SPIE 11759, Virtual, Augmented, and Mixed Reality (XR) Technology for Multi-Domain Operations II, 117590H (12 April 2021); https://doi.org/10.1117/12.2585318
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Visualization

Monte Carlo methods

Avionic systems

Complex systems

Algorithm development

Control systems

Heads up displays

RELATED CONTENT

Safely enhanced low visibility taxi
Proceedings of SPIE (May 05 2017)
New integrated backup flight data display in Gripen
Proceedings of SPIE (August 28 2000)
Bayesian sensor resource allocation
Proceedings of SPIE (September 03 1998)
F-22 cockpit display system
Proceedings of SPIE (June 17 1994)

Back to Top