Intelligent Landing System for Landing UAVs at Unsurveyed Airfields

19 Pages·2012·1.87 MB·English

28THINTERNATIONALCONGRESSOFTHEAERONAUTICALSCIENCES
INTELLIGENT LANDING SYSTEM FOR LANDING
UAVS AT UNSURVEYED AIRFIELDS
Paul Williams, Michael Crump
BAE Systems Australia
[email protected];[email protected]
Keywords:UAVs, autonomous landing,computer vision
Abstract addresses the need for a self-contained
autonomous recovery system that enables the
Autonomous UAV technology is currently
UAV to locate and land on suitable runways
limited in its ability to land safely at alternate
without ground support equipment such as
airfields that have not been accurately surveyed
differential GPS or ILS. Such systems can be
and where differential GPS is unavailable. This
used to recover UAVs in the event of unplanned
paper presents flight-test results of anew system
mission events such as component failure or
that is capable of autonomously recovering a
deteriorating weather conditions. The system
UAV to a previously unvisited airfield. The
can also be used to recover to ad hoc landing
system utilizes a priori location information of
sites, or to simply provide greater robustness to
airfields (for example ERSA, which provides the
navigationerrors duringlanding.
location of the airfield to approximately 100 m)
The system described in this paper was
to decide which airfield is the most feasible for
designed specifically for dealing with recovery
landing. When the aircraft reaches the vicinity
of fixed-wing aircraft to runways. However, its
of the airfield it uses an onboard gimbaled
applicability extends to wider areas such as
camera to provide a precise vision-aided
rotary-wing UAVs landing on moving decks.
landing.
Unlike recent work presented in [1][2][3] which
uses LIDAR to generate detailed maps of
landing zones, our system is capable of landing
1 Introduction
safely using only passive vision sensors.
UNMANNED air vehicles (UAVs) are Additional sensing can be used to augment our
widely used in defense environments to carry solution to handle issues such as obstacles.
out autonomous or remotely-piloted missions. However, the implementation presented here
Remotely-piloted vehicles require numerous focuses on the central problem of landing the
personnel to operate and maintain – in some UAVinarobust manner.
cases more than is required for a manned The system presented in this paper
aircraft. Autonomy removes much of the low- combines several key technologies that enable a
level decision making and therefore relieves the trulyautonomous recoverysystem for UAVs: 1)
operatingpersonnel of their strenuous workload. Onboard auto-routing system for generating
In addition, the operating footprint can be waypoints through complex no-fly regions [4],
significantly reduced for autonomous UAVs. 2) Gimbaled camera control and associated
UAVs are also expanding into the civilian image processing algorithms for runways [5], 3)
domain for a variety of purposes such as border Closed-loop image processing and flight control
patrol,minemonitoring, andbush firespotting. robust toocclusions.
Most UAVs typically rely on ground This paper is organized as follows: First, a
support infrastructure for asset recovery. Fixed- review of the Intelligent Landing concept is
wing UAVs require pre-surveyed runways and a presented; next, an overview of the capability of
differential GPS correction source to enable the system is given; the implementation of our
accurate autonomous landing. This paper approach in simulation and on real-time
1
PAULWILLIAMS,MICHAELCRUMP
hardware is presented; finally, simulation results use detracts from the willingness to establish
and flight-test results are presented that alternates. As mission requirements become
demonstrate the effectiveness of the system more complex, it may not be possible for the
whenimplementedwithlow-cost sensors. aircraft to reach one of its alternate landing
runways, and controlled ditching may result in
recovery of the aircraft by enemies, or could
2 IntelligentLandingSystem
result in injury or third party damage. To
UAV systems that are currently used enable truly autonomous UAV operations, a key
operationally do not have cognitive abilities. feature of the flight control computer would be
Human operators are required to plan and its ability to select and land at a suitable
undertake missions, with the UAV flight alternate not previously surveyed and without
computer performing the low-level task of any existing landing aids. This is the core idea
flying the aircraft. Some operational UAVs ofthe Intelligent LandingSystem.
have a fully automatic takeoff and landing The system presented in this paper differs
capability. However, the landing phase is from similar concepts of using vision for aiding
usually carried out using a combination of pre- the landing of UAVs. Tang et al. [7] presented
surveyed runway with known landing an algorithm for estimating the position and
waypoints, an accurate height sensor for attitude of a UAV relative to a runway by
determining height above ground, and projecting arbitrary points from the runway into
differential GPS. These requirements can the image frame. This assumes full knowledge
severely limit the operational case of modern of the runway position and orientation.
UAV technology. There are several examples Fitzgerald et al. [8] consider forced landing site
of unplanned mission events that can lead to a selection using machine vision. However, their
UAV operator needing to land the aircraft as approach considers general landing sites such as
soon as possible, such as: engine performance paddocks and does not consider the more
problems (temperature/oil pressure); desirable case of landing on available runways.
deteriorating weather conditions; bird strike or Their approach has not been used to land a
attack damage; flight control problems related UAV. Daquan and Hongyue [9] present
to malfunctioning hardware (single lane on simulation results of an algorithm that uses
multi-lane flight control systems, actuator runway landing lights observed through an
failure); and so on. In current systems, the onboard camera to estimate a UAV’s position
above situations can easily lead to the complete and attitude relative to the runway. The results
loss of the aircraft. The operator must either showed reasonably large errors. Their approach
attempt a recovery to a mission alternate does not consider locating the runway initially,
runway, or in the worst case, undertake a nor how to handle gaps in image data. Pan et al.
controlled ditching. Most modern UAV control [10] consider the estimation of approach angle
systems allow multiple alternate landing sites to and height of a UAV using a combination of
be specified as part of the mission plan. monocular and stereo vision. The approach has
However, the limit with these alternate landing not been used with imagery of real runways.
sites is that they require the same level of a The approach also requires continuous
priori knowledge (i.e., accurate survey) and observation of the runwayand does not consider
support infrastructure as the primary recovery initially locating the runway. Joo et al. [11]
site. Therefore, this generallylimits the number considered estimating the motion of a UAV
and location of the alternate landing sites. This during landing relative to a runway by using a
is due to the amount of time and manpower ground-based camera system. However, this
required to setup, maintain, and secure the sites. defeats the purpose of the Intelligent Landing
The combined cost/effort and low probability of System considered in this work. Pouya and
2
INTELLIGENTLANDINGSYSTEM FORLANDINGUAVSAT
UNSURVEYEDAIRFIELDS
Saghafi [12] simulated the use of a fuzzy logic System is: 1) location of the airfield reference
controller with runway relative position point (latitude = -38,05.5, longitude = 146,57.9),
measurements for controlling the landing of a height above mean sea-level (93 feet), magnetic
UAV. Their approach does not consider the field offset (+12 deg), number of runways (3),
actual image processing or its limitations. runway surface characteristics (2 grass, 1
bitumen), runway lengths (1527, 699, 500 m)
Meng et al. [13] considered the detection of
and width (30 m), and runwaymagnetic heading
runways within images based on template
(44, 87, and 133 deg). Note that the airfield
matching. They used actual images of runways
reference point gives the approximate location
in their work, but only considered the case
of the airfield to the nearest tenth of a minute in
where the aircraft is already on approach to the
latitude and longitude (±0.0083 deg). This
runway. Miller et al. [14] consider the use of
equates to an accuracy of approximately 100 m
image registration using a priori imagery of the
horizontally. Furthermore, the reference point
runway to enable vision-aided landing. They
in general does not lie on any of the runways
demonstrated their approach via Microsoft flight
and cannot be used by itself to land the aircraft.
simulator. To the best of our knowledge, all
It is suitable as a reference point for pilots to
previous approaches to vision-aided landing of
obtainavisual of theairfieldandland. The goal
UAVs on runways have been based on a visual- of the Intelligent Landing System is to perform
servoing approach. These approaches are a similar airfield and runway recognition and
limited in nature and do not address the plan a landing/approach path. Note that the
requirement for initially locating and planning minimal amount of information required is the
an approach to the runway. The Intelligent approximate location of the airfield. The
Landing System described in this paper remaining information is utilized, if available,
addresses all oftheselimitations. but not required.
3 IntelligentLandingOverview Airfieldreferencepoint
3.1 Assumptions
In order to design a robust and reliable
Intelligent Landing System capable of landing
the plane as quickly as possible, it is necessary
to make certain assumptions about the
availabilityof runwaylocations. It is unrealistic
to expect the UAV to perform a fast landing
without anyprior knowledge of nearbyairfields.
In fact, our system assumes a similar level of
FIG. 1. EXAMPLE ERSA AIRFIELD DATA FOR
information is available to the flight control
WESTSALEAERODROME[15].
computer as is available to a manned aircraft in
a similar situation. Our system utilizes an
onboard database of airfields compatible with
industry standard definitions such as Jeppesen
NavData. The results presented in this paper
were acquired using the En-Route Supplement
Australia (ERSA) [15], which contains
information about all airfields in Australia. An
example of the key airfield data provided from
ERSA is shown in Fig. 1. The important FIG. 2. EXAMPLE ERSA AIRFIELD REFERENCE
information used by the Intelligent Landing POINTS().
3
PAULWILLIAMS,MICHAELCRUMP
In order for the UAV to identify and algorithms. A recovery can also be initiated via
perform an autonomous landing on the desired an operator command. Once a recovery has
runway, the navigation solution must be commenced, the Intelligent Landing System is
accurate. In low-cost UAVs, a GPS-aided INS fully responsible for commanding the UAV
system is the norm. In fact, GPS is heavily until it has completed the landing. If there is an
relied upon due to the poor performance of low- active communications link, the Intelligent
cost inertial measurement units. GPS has very Landing System can be disabled. This is a
good long term stability, but can drift in the feature that is required for flight testing in
short term due to variations in satellite civilianairspace.
constellation and ionospheric delays. The When the Intelligent Landing System is in
amount of drift is variable, but could be in the control of the UAV, it finds the nearest suitable
order of 20 m. This is one of the main reasons airfield using a tailored search algorithm. It
why differential GPS is a requirement for must take into account runway surface
automatic landing of UAVs. Differential GPS characteristics, runway length, and flight time.
allows accuracies of the navigation solution on Flight time is obviously affected by prevailing
the order of approximately 1-2 m. The wind conditions, flyable regions, and current
Intelligent Landing System is assumed to have UAVpositionrelativeto neighboringairfields.
nodifferential GPS available. The Intelligent Landing System guides the
The intended operation of the Intelligent UAV to the vicinity of the airfield, where
Landing System is through the use of image onboard image processing algorithms locate the
processing to extract information about the runway from background clutter. The system is
airfield. The main reason for this is that able to distinguish the runway from other
virtually all UAVs are equipped with gimbaled similar looking features such as roads and
cameras as part of their mission system. taxiways. A localization algorithm is able to
Therefore, in an emergency situation, the convert image data into runway positioning
camera system can be re-tasked to enable a safe information. Armed with this data, the
landing of the system. Other sensors such as Intelligent Landing System creates an alternate
LIDAR cannot always be assumed to be return-to-base (RTB) landing waypoint set,
available. A core assumption is that additional whichincludes inbound,andcircuit waypoints.
sensors are not required to be installed on the In the final stage of the approach and
platform to enable the Intelligent Landing landing, the onboard camera provides closed-
System to work. Hence, only image processing loopinformationabout therelativetrack errorof
canbeused. the UAV from the desired landing path. The
system accounts for the potential occlusion of a
non-retractable undercarriage, as well as
3.2 IntelligentLandingSystemFeatures
variations in satellite constellation that could
The Intelligent Landing System is designed result in relative errors between the aircraft and
as a plug-in mission system to existing UAV runway at touchdown. This results in the
flight control computers. For the flight tests system typically landing close to the runway
described in this paper, the flight control centerline. For the purposes of flight
computer has been developed by BAE Systems demonstration, the system aims to land
Australia, and has been flown on the Mantis and longitudinally at roughly the runway
Herti UAVs. It is a highly capable autonomous designationmarking, as showninFigs. 3 and4.
system with existing mission system interfaces.
The Intelligent Landing System exploits many
of the features of the flight control computer,
such as routingandwaypoint control.
The primary flight control computer is
responsible for initiating an autonomous
recovery via its internal health monitoring
FIG.3. STANDARDRUNWAYMARKINGS[16].
4
INTELLIGENTLANDINGSYSTEM FORLANDINGUAVSAT
UNSURVEYEDAIRFIELDS
performing full 360 pan and -5 to 95 deg tilt.
For the Intelligent Landing System, we have
exclusively used the EO camera, which is a
Sony FCB-EX408C. The camera uses the
VISCA binary communication protocol, which
is transmitted over RS-232. Turret control
commands are transmitted to the Rubicon
device using an ASCII protocol, also over RS-
232.
Care must be taken when communicating
with the Rubicon device to avoid packet
collisions or sequential commands sent too
quickly. Message rate handling is dealt with via
algorithms implemented in Simulink. All turret
and gimbal control is handled via the Simulink
model, and translation of Simulink commands
into a byte-stream occurs via a software codec.
The Simulink implementation of the interfaces
matches the interface used on the actual device,
minimizingtheneedfor handwrittencode.
FIG. 4. RUNWAY THRESHOLD MARKING
GEOMETRY[16]. Rubiconturret
4 Camera Control and Calibration
One of the key features of our system is its
ability to work effectively even when low-cost
sensor packages are used. The rationale for
employing low-cost sensors is to demonstrate
the robustness of the algorithms. It is
reasonably straightforward to extend the system
to use other sensing packages depending on the
customer’s requirements. It is, in general, much
FIG. 5. RUBICON AHIR25D TURRET USED FOR
more difficult to go from using expensive
INTELLIGENTLANDINGTESTFLIGHTS.
sensors to using low-cost sensors due to the
large difference in accuracy and reliability. A
4.1 Camera Coordinates
large amount of effort is required to mitigate the
effects of sensor errors when using low-cost
A pinhole camera model is assumed which
sensors.
relates measurements in the sensor frame to the
The key requirement of the system is the
image frame ,as showninFig.6,
(u,v)
presence of a gimbaled camera on the UAV. A
gimbaled camera allows the Intelligent Landing
System to control the direction and zoom level 1 1
u  y /x u ,v  (z /x )v (1)
of the imagery it is analyzing. We have s s 0 s s 0
f f
u v
installed a turret manufactured by Rubicon with
model numberAHIR25D,showninFig.5. This
where and are the camera focal lengths,
f f
turret includes an electro-optical (EO) and u v
infrared (IR) camera, and is capable of and u and v arepixel offsets (principal point).
0 0
5
PAULWILLIAMS,MICHAELCRUMP
frames used by the system. The ECEF frame
is the reference frame used for
(X,Y,Z)
Objectinsensorframe
Imageframe navigation. The local navigation frame
is an intermediate frame used for the
u (N,E,D)
x definition of platform Euler angles and is
s
common across external interfaces for
navigation. The NED frame has its origin on
the WGS84 ellipsoid. The IMU/body frame is
v aligned with the vehicle body axes and has its
origin at the IMU. The installation frame
z
y s
s has its origin at a fixed point on the
(x ,y ,z )
i i i
FIG. 6. PINHOLE CAMERA MODEL USED TO camera mount. This allows some of the camera
CONVERT PIXEL MEASUREMENTS INTO extrinsics to be calibrated independently of the
MEASUREMENTSINMEASUREMENTFRAME. mounting on the airframe. The gimbal frame
has its origin at the center of the
(x ,y ,z )
g g g
The bearing and elevation measurements
gimbal axes. Finally, the measurement frame
are derived from the pixel information
has its origin at the focal point of the
accordingto
camera/sensorsystem.
  tan1 u u / f  (2)
 0 u
  tan1 v v cos/ f  (3)
 0 v
x
b
Note that distortion effects must be accounted y
b
for before using the raw pixel coordinates. This x
y i
step has been omitted here. The uncertaintyof a i z
i
measurement is specified in the image plane,
and must be converted into an equivalent zb yi xi
uncertainty in bearing/elevation. The
y
g
uncertainty in bearing/elevation takes into
E
account the fact that the intrinsic camera N zi zg xg
parameters involved in the computation given in
Eqs. (2) and (3) are not known precisely. The Z
D
uncertaintyis computed via
X x
g
x
BE  yxx xyT yppypT (4) Y yg z m
g y
m z
m
where p  [fu,fv,u0,v0]T , y  ,T , FGIIGM.BA7L.EDCCAOMOERRDAI.NATE FRAMES USED BY
x  [u,v]T,  is the uncertainty in the pixel
x
plane coordinates, and is the uncertainty in The position of the camera in the ECEF

p frameis givenby
thecameraparameters.
In order to compute the camera position in
the Earth-Centered-Earth-Fixed (ECEF) frame, pe  pe CeCn(pb Cb(pi Cipg ))(5)
m b n b i i g g m
the coordinate frames used by the system must
be first explained. Fig. 7 shows the coordinate
6
INTELLIGENTLANDINGSYSTEM FORLANDINGUAVSAT
UNSURVEYEDAIRFIELDS
the turret, camera system, and collecting and
where pe is the position of the aircraft/IMU in
b forwarding image data and associated meta-data
the ECEF frame, Ce is the direction cosine to the Intelligent Landing System. The velocity
n
matrix representing the rotation from the NED control loop uses pan and tilt commands and
frame to the ECEF frame, Cn is the direction closes the loop with measured pan and tilt
b values. Due to the delays in the turret
cosine matrix representing the rotation from the
communication protocol and the low rate of
body to the NED frame, pb is the position of feedback sampling, the control loop employs a
i
the installation origin in the body frame, Cb is predictive mechanism to provide for fine
i
angularcontrol.
the direction cosine matrix representing the
High-level pointing commands are
rotation from the installation from to the body
received by the turret controller from the
frame, pi is the position of the gimbal origin in
g Intelligent Landing System. The Intelligent
the installation frame,Ci is the direction cosine Landing System arbitrates to select from
g
commands issued from a ground controller, and
matrix representing the rotation from the gimbal
its own internally generated commands. In all
frame to the installation frame, and pmg is the cases, the ground controller has priority and can
origin of the measurement frame in the gimbal
manually command the turret to move to a
frame.
specified angle, angular rate, or point at a
The direction cosine matrix representing
selected latitude/longitude/height. The operator
the rotation from the measurement frame to the
must manually relinquish control for the
ECEFframeis givenby
autonomous system tobe incontrol.
During intelligent landing, the turret is
Ce CeCnCbCiCg (6) commanded to point in a variety of different
m n b i g m
modes controlled through a single interface.
The turret can be made to “look at” selected
where Cmg is the direction cosine matrix points specified in different reference frames
representing the rotation from the measurement
(camera frame, body frame, NED frame, ECEF
frametothe gimbal frame.
frame). The most useful mode is a bounding
In the case of a body-fixed camera, the
box mode that adaptively changes the pointing
matrix CbCiCg is a fixed quantity and can be position and zoom level to fit up to 8 points in
i g m
the camera field-of-view. The turret control
calibrated reasonably reliably. However, when
algorithm computes a line of sight and uses a
a turret is used, the gimbal frame is constantly
Newton algorithm to iteratively calculate the
changing with respect to the installation frame.
requiredpan/tilt angles.
This makes calibration of the sensor more
difficult, particularly where there are
uncertainties in the timing, and is addressed in a 4.3 ZoomControl
latersection.
Camera zoom is either controlled
independently of the pointing command, or
4.2 Gimbal Control
coupled to it. The zoom can be set via a direct
setting command as a ratio or rate, or can be
The commands sent to the turret are in the
specified as an equivalent field-of-view
form of rate commands about the pan and tilt
measured by its projection onto the ground
axes. This is required if the stabilization
plane (i.e., units are in meters). This type of
function is to be used on the turret. The
control maintains an area in the image quite
stabilization mode uses gyroscopes in the turret
well byadjustingthezoom level as afunctionof
to mitigate the effects of turbulence on the
the navigation position relative to the pointing
pointing direction of the camera. A velocity
location. The combined zoom mode uses up to
control loop is executed on our turret control
8 points in the ECEF frame to select a zoom
subsystem, which is responsible for control of
7
PAULWILLIAMS,MICHAELCRUMP
setting such that all 8 points lie within the while simultaneously capturing images. By
image. extracting the position of the LED, and by using
the inverse map of pan position, we were able to
estimate the image capture time and compare it
4.4 Imageand Gimbal Timestamp
with the time of arrival of the first image byte.
In addition to running the abstracted turret We found a constant offset of approximately 60
control loop, the turret control subsystem is ms between the image capture time and the
responsible for capturing still images from the arrival ofthe first byteat theturret controller.
camera video feed and timestamping the data.
The turret control subsystem obtains 100 Hz
4.5 IntrinsicCalibration
navigation data from the Intelligent Landing
System, and zoom and gimbal measurements Intrinsic calibration refers to estimating the
from the camera and turret, respectively. These properties of the camera lens system. The key
measurements are buffered and interpolated parameters that must be calibrated are: 1) focal
upon receipt of an image timestamp lengths, 2) principal point, 3) radial and
(timestamps are in UTC Time). Euler angles tangential distortion. These parameters can be
are interpolated by using a rotation vector estimated using standard calibration techniques
method, whereas other quantities can be involving a checkerboard of known geometry.
interpolatedinastraightforwardmanner. The lens parameters are determined bysolvinga
Navigation data is timestamped according least squares problem so as to minimize the
to the UTC Time of the IMU data packet used error in pixel coordinates for the corners of the
on the 100 Hz frame, which is kept in checkerboard tiles. This is done over a large
synchronization with GPS time via the PPS line number of images photographed at various
from the GPS unit. Gimbal data is timestamped orientations relative to the camera. The
on transmission of a trigger pulse sent by the parameters are stored as a function of zoom
turret controller to the turret. The trigger is used level and interpolated by the turret control
by the turret to sample the gimbal angles, which subsystem during operation. The interpolated
is transmitted to the turret controller after it intrinsic parameters are attached as meta-data,
receives a request for the triggered data. Zoom together with camera extrinsic parameters and
data is timestamped by the turret controller on navigation data, and passed to the Intelligent
transmission of the zoom request message. Landing System. This interface allows the
Note that the camera does not provide a means turret control subsystem to be swapped for an
for accurate timestamping of the zoom. We alternate one without impacting the
have assumed a constant delay time between configurationofothersubsystems.
writing the binary stream to the serial port and We note that our choice of low-cost turret
the sampling of the zoom position. This results in in-house development of camera and
assumption is reasonable and is alleviated gimbal control algorithms. Much more
somewhat through the use of discrete zoom expensiveturrets canbe usedoff-the-shelf.
levels by the turret controller. Images are
timestamped upon receipt of the first byte from
4.6 ExtrinsicCalibration
the video capture card, and is intercepted on a
device driver level. However, this does not Extrinsic calibration refers to the
provide the time that the image was actually estimation of the position and orientation of the
captured by the camera. We have computed the camera relative to the navigation solution’s
assumedconstant offset byplacingan LED light reference axes on the aircraft. In our case, we
in front of the camera in a dark room. A static are interested in estimating the installation
map of pixel location versus pan position was position and installation angles, using the
obtained by manually moving the turret to terminology given in Fig. 7. When the turret
various positions. The turret was then control subsystem is installed into the aircraft,
commanded to rotate at various angular rates the position and orientation is known coarsely,
8
INTELLIGENTLANDINGSYSTEM FORLANDINGUAVSAT
UNSURVEYEDAIRFIELDS
but not to the accuracy required for accurate on the image to extract the pixel
localization of targets. In general, the coordinates ofthetargets.
installation angles dominate the projection error 3) Using the known position of the targets,
compared to the installation position. A and the meta data associated with the
dedicated flight is performed to obtain data to image, together with the extrinsic
enable extrinsic calibration to be performed. estimates, the projected pixel
Themethodusedis describedinthis section. coordinates ofthetargets arecalculated.
Calibration is performed using an 4) A Mahalanobis gate is used to reject
expectation maximization algorithm. A set of 5 detected blobs that are too far awayfrom
white targets are placed in an accessible the predicted pixel coordinates. All
paddock near the airfield with the aid of a surviving coordinates are added to a list,
surveyor’s GPS (accuracy ~2 cm), as shown in togetherwiththeassociatedtarget index.
Fig. 8. The aircraft is flown in a manner to 5) A sequential quadratic programming
allow the turret to be pointed at the targets from problem is solved to minimize the cost
a variety of positions and attitudes. The flight function of the pixel error squared
path is designed to incorporate a large number divided by pixel uncertainty (uncertainty
of alternatingturns to give good observabilityof is the combined blob pixel uncertainty
the true aircraft heading by the navigation and the uncertainty in pixel coordinates
system. During the flight, the turret is manually arising from the use of the navigation
pointed at the targets with zoom settings solution in the calculation), summed
coveringtherangefrom maximumtominimum. overall images inthelist.
6) The process is repeated starting at (1)
until the estimated extrinsic parameters
convergetowithinatolerance.
We have found that the above iterative
procedure converges quickly with an accuracy
that depends on the combination of timestamp
accuracy and navigation solution drift during
the flight. We have also found that performing
the calibration makes a large difference in the
localizationaccuracyofthesystem.
5 Simulation Architecture
All components of the system are
implemented in the Simulink environment
(running MATLAB R2010b SP1). Simulink
FIG. 8. EXAMPLE OF CAPTURED GEO IMAGE
provides a powerful model-based design
SHOWINGCALIBRATIONTARGETS.
environment that can be linked to a set of
functional requirements described in a set of
The algorithm used to determine the
IBM Rational DOORS modules. In fact, all
extrinsic calibration parameters may be
system interfaces are described in DOORS
summarizedas follows:
modules, from which a complete set of interface
1) The current best estimate of the extrinsic
codeforC++andMATLABis autogenerated.
parameters is used to approximately
The complete flight vehicle is modeled
calculate if the targets are visible in any
using Simulink blocks (engine, undercarriage,
particularimage.
aerodynamics, actuators), together with a
2) If the targets are estimated to be visible,
representative model of the environment (wind,
a blob detection algorithm is performed
gust, turbulence, gravity, atmosphere, ground).
9
PAULWILLIAMS,MICHAELCRUMP
Flight physics are modeled in the Earth- set of software interfaces to the code is
Centered-Earth-Fixed (ECEF) coordinate frame. autogenerated from DOORS, minimizing the
In additional to the physics model of the amount of hand-code that is required to run the
vehicle, a complete sensor simulation with software as an executable on the target
interfaces and noise characteristics matching the hardware. The C code is wrapped in C++ with
real sensors is performed (Inertial Measurement appropriate data and message handling code. It
Unit, GPS, air data, etc). The sensor data is fed is scheduled using a real-time scheduler which
into a Simulink model of the flight control runs in the main thread at 100 Hz. The linux
computer to enable simulation of the closed- operating system running a real-time kernel is
loop control system. Note that because the used for the Intelligent Landing System. The
flight control computer code is generated from Green Hills Integrity operating system is used
the Simulink models via Real-Time Workshop, fortheactual flight control computer.
the simulation is virtually an exact match to the In addition to the core algorithms, an
implementation on the target hardware on the additional process is executed to log the input
real vehicle. The high fidelity simulation and output data of the algorithmic code. All the
environment allows the Intelligent Landing inputs and outputs are logged, which enables
algorithms to be tested completely without ever offline data replays to be conducted using a set
havinganaircraft inthesky. of support tools that we have developed. This
Two levels of simulation based testing are enables bugs to be found and removed in a rapid
performed. In the first level, the image fashion.
processing algorithms are not in the loop (as
they require a detailed synthetic environment to
7 HardwareOverview
be generated). This enables the full Intelligent
Landing System to be developed and tested
The flight vehicle platform that we have
without relying on a working image processing
used to conduct flight operations is the
system. In the second level of tests, the model
Kingfisher 2 vehicle, shown in Fig. 9.
of the image processing algorithms is replaced
Kingfisher 2 was designed and built by BAE
with a Simulink S-function that communicates
Systems Australia to allow rapid prototyping of
to an external vision processing module. The
payloads and is not a production system. The
vision processing module runs on a dedicated
key characteristics of the platform are given in
PC that renders the camera view from the turret.
Table1.
The captured view from the turret is fed into the
The flight control computer forms the core
image processing algorithm code to exercise its
product of the vehicle management system,
logic and returns messages matching the
which consists of an actuation unit, two GPS
software interface. During the rendering and
units, an IMU, an air data system, weight on
image processing testing, the simulation in
wheel sensors, an accurate height sensor, and a
Simulink is held and waits for the vision
C2 communications system. The FCC uses a
processing module to return. This ensures that
Radstone IMP2A as the hardware board. The
timing anomalies of the test system do not
FCC runs a varietyof core processes in separate
interfere with algorithmic development.
address spaces to maintain a high integrity
Communication with the vision processing
system. The FCC enclosure with IMU is shown
moduleis doneoverUDP.
in Fig. 10. The FCC mounting in the airframe,
together with the Intelligent Landing System
computerand turret,is showninFig.11.
6 Embedded Real-TimeImplementation
The algorithms that are designed,
implemented and tested in Simulink are
converted into C code by the Real-Time
Workshop Embedded Coder. As noted above, a
10