Onboard sensor electronics of satellites are hard real-time systems and exploit high performance of heterogeneous computing. This paper describes sensor electronics design framework with heterogeneous computing edge nodes based on onboard demonstration records of satellites. Dedicated functional processing elements (PEs) for specific purposes implemented on Field Programmable Gate Arrays (FPGAs) and Application Specific Integration Circuit (ASIC) are used in addition to conventional Micro-Processing Units (MPUs). Many core processors like General Purpose Graphics Processing Units (GPGPUs) are also used for signal processing of sensor electronics in these days. Semiconductor process shrink is accelerating this technological trend. Because it reduces power consumption, size and mass while maintaining high processing performance. The applications of artificial intelligence, such as image recognition, became common for onboard sensor electronics. Dedicated PEs for image recognition implemented on FPGAs enables wire rate processing. Sensor signals are processed without interrupting data flow, and in-situ measurement results can be used for other purposes such as optical guidance and navigation. Heterogeneous computing edge nodes are often realized with distributed memory system. In addition to that the semantic gap between hardware and application software is widening. Despite these complexities, changes to the operation scripts of onboard sensor electronics are often needed on orbit. We have found that the layered architecture of heterogeneous PEs and the middle-out approach of system integration design are practical enough for onboard operation to aim at realizing user-centric command operation scripts. The design scheme is explained in this paper.
KEYWORDS: Space operations, Satellites, Model-based design, Data processing, Asteroids, Field programmable gate arrays, Process modeling, Data modeling, Aerospace engineering, Signal processing
DESTINY⁺ is a science and technology demonstration mission to asteroid Phaethon, the parent body of the Geminids meteor shower. It will explore the asteroid during a flyby for observations of cosmic dust, which is a source of the organic matter on Earth. In-situ analysis of interplanetary and interstellar dust will be carried out. It is planned to be launched in fiscal year 2024. Model-based development process using a bread board model of an onboard mission data processor is employed prior to system level integration test to follow the tight development schedule. Hybrid and reconfigurable computers are exploited as ground-based models to pursue digital development process. Dynamically reconfigurable devices are used as the central processing unit, and extensive simulation is performed by a hardware-in-the-loop simulator.
HAYABUSA2 asteroid probe has completed its mission successfully in the vicinity of asteroid Ryugu on November 13, 2019. It is on its way to the Earth now. Digital Electronics and Optical Navigation Camera (DE-ONC) was developed for scientific observation and real-time image recognition for optical navigation. The development process and its highspeed wire rate signal processing architecture of onboard electronics are explained in this lecture. Highly efficient lossless and lossy image compression algorithm were developed to send observed images through within the limited capacity of communication channels between the asteroid Ryugu and the Earth for scientific purposes. Onboard sensitivity and distortion correction functions for image sensors were also developed to improve compression ratio of images. High level synthesis technology was employed to implement the image recognition functions for optical navigation functions into limited numbers of space grade field programmable gate arrays (FPGAs) and to achieve wire rate signal processing speed. It must also satisfy high reliability and safety requirements of HAYABUSA2 missions. Functional distribution mode, standby redundancy mode and hot redundancy mode were realized with the same device configuration. Model based design was performed to satisfy these requirements. The onboard image processing unit of DE-ONC adopts a unified language processing system and a distributed memory model with reference to a parallel inference machine developed for the Fifth Generation Computer Systems aiming at artificial intelligence technology development. Its image processing module integrates a radiation hardened micro-controller unit (MCU) and FPGAs with the unified language processing system and the distributed object model.
Digital Electronics and Optical Navigation Camera (DE-ONC) is an edge computing node of the asteroid probe HAYABUSA2. DE-ONC was developed to provide real-time image recognition performance for optical navigation. Lightweight, low power consumption and miniaturization are realized to overcome resource restrictions. It also satisfies high reliability and safety requirements of HAYABUSA2 missions. There are static and dynamic requirements for reliability and safety. The former increases reliability by adding redundancy combining the concept of functional distribution and time-division redundancy to meet resource constraints. Functional distribution mode, standby redundancy mode and hot redundancy mode were realized with the same device configuration. The real-time performance of optical navigation exploiting image recognition functions of the unit was demonstrated through the interplanetary cruising phase, as well as touch down to and taking off from the asteroid Ryugu. DE-ONC is always required to operate in the critical operation phase. In addition to that, it must always satisfy latency requirements to complete processing within a predetermined duration and to guarantee hard real-time performance. In order to satisfy these requirements, the image processing unit of DE-ONC adopts a unified language processing system and a distributed memory model with reference to a parallel inference machine, which is a so-called the second generation artificial intelligence technology. Its image processing module integrates a radiation hardened micro-controller unit (MCU) and field programmable gate arrays (FPGAs) with the language processing system and the distributed object model. We report the evaluation result of reliability and safety with real-time performance of the unit’s architecture.
Fast timing capability in x-ray observation of astrophysical objects is one of the key properties for the ASTRO-H (Hitomi) mission. Absolute timing accuracies of 350 or 35 μs are required to achieve nominal scientific goals or to study fast variabilities of specific sources. The satellite carries a GPS receiver to obtain accurate time information, which is distributed from the central onboard computer through the large and complex SpaceWire network. The details of the time system on the hardware and software design are described. In the distribution of the time information, the propagation delays and jitters affect the timing accuracy. Six other items identified within the timing system will also contribute to absolute time error. These error items have been measured and checked on ground to ensure the time error budgets meet the mission requirements. The overall timing performance in combination with hardware performance, software algorithm, and the orbital determination accuracies, etc. under nominal conditions satisfies the mission requirements of 35 μs. This work demonstrates key points for space-use instruments in hardware and software designs and calibration measurements for fine timing accuracy on the order of microseconds for midsized satellites using the SpaceWire (IEEE1355) network.
Infrared sensor system is a major concern for inter-planetary missions that investigate the nature and the formation processes of planets and asteroids. The infrared sensor system requires signal preprocessing functions that compensate for the intensity of infrared image sensors to get high quality data and high compression ratio through the limited capacity of transmission channels towards ground stations. For those implementations, combinations of Field Programmable Gate Arrays (FPGAs) and microprocessors are employed by AKATSUKI, the Venus Climate Orbiter, and HAYABUSA2, the asteroid probe. On the other hand, much smaller size and lower power consumption are demanded for future missions to accommodate more sensors. To fulfill this future demand, we developed a novel processor architecture which consists of reconfigurable cluster cores and programmable-logic cells with complementary atom switches. The complementary atom switches enable hardware programming without configuration memories, and thus soft-error on logic circuit connection is completely eliminated. This is a noteworthy advantage for space applications which cannot be found in conventional re-writable FPGAs. Almost one-tenth of lower power consumption is expected compared to conventional re-writable FPGAs because of the elimination of configuration memories. The proposed processor architecture can be reconfigured by behavioral synthesis with higher level language specification. Consequently, compensation functions are implemented in a single chip without accommodating program memories, which is accompanied with conventional microprocessors, while maintaining the comparable performance. This enables us to embed a processor element on each infrared signal detector output channel.
Infrared sensor system is a major concern for inter-planetary missions in order to investigate the nature and the formation processes of planets and asteroids. Since it takes long time for the communication of inter-planetary probes, automatic and autonomous functions are essential for provisioning observation sequence including the setup procedures of peripheral equipment. Robotics technology which has been adopted on HAYABUSA2 asteroid probe provides functions for setting up onboard equipment, sensor signal calibration, and post signal processing. HAYABUSA2 was launched successfully in 2014 for the exploration of C class near-Earth asteroid 162173 (1999JU3). An optical navigation camera with telephoto lens (ONC-T), a thermal-infrared imager (TIR), and a near infrared spectrometer (NIRS3) have been developed for the observation of geology, thermo-physical properties, and organic or hydrated materials on the asteroid. ONC-T and TIR are used for those scientific purposes as well as assessment of landing site selection and safe descent operation onto the asteroid surface for sample acquisition. NIRS3 is used to characterize the mineralogy of the asteroid surface by observing the 3-micron band, where the particular diagnostic absorption features due to hydrated minerals appear. Modifications were required in order to apply robotics technology for the probe due to the difference of operation on satellites from robot operation environment. The major difference is time line consideration, because the standardized robotics operation software development system is based on event driven framework. The consistency between the framework of time line and event driven scheme was established for the automatic and autonomous operation for HAYABUSA2.
Onboard signal processing system for infrared sensors has been developed for HAYABUSA2 for the exploration of C class near-Earth asteroid 162173 (1999JU3), which is planned to be launched in 2014. An optical navigation camera with telephoto lens (ONC-T), a thermal-infrared imager (TIR), and a near infrared spectrometer (NIRS3) have been developed for the observation of geology, thermo-physical properties, and organic or hydrated materials on the asteroid. ONC-T and TIR are used for those scientific purposes as well as assessment of landing site selection and safe descent operation onto the asteroid surface for sample acquisition. NIRS3 is used to characterize the mineralogy of the asteroid surface by observing the 3-micron band, where the particular diagnostic absorption features due to hydrated minerals appear. Since the processing cycle of these sensors are independent, data processing, formatting and recording are processed in parallel. In order to provide the functions within the resource limitation of deep space mission, automatic packet routing function is realized in one chip router with SpaceWire standard. Thanks to the SpaceWire upper layer protocol (remote memory access protocol: RMAP), the variable length file system operation function can be delegated to the data recorder from the CPU module of the digital electronics of the sensor system. In consequence the infrared spectrometer data from NIRS3 is recorded in parallel with the infrared image sensors. High speed image compression algorithm is also developed for both lossless and lossy image compression in order to eliminate additional hardware resource while maintaining the JPEG2000 equivalent image quality.
A thermal-infrared (TIR) imager system is developed for HAYABUSA2, which is planned to be launched in 2014 and aims at sample-return from a C-class near-Earth asteroid 162173 (1999JU3) considered to contain organic or hydrated materials. The system consists of a TIR imager and digital electronics, which are used not only for the scientific investigation of physical properties of the asteroid surface, but also for the assessment of landing site selection and safe descent operation onto the asteroid surface with in situ measurement. TIR adopts an uncooled bolometer. Image operations such as multiple images summation, dark image subtraction, and the compensation of dead pixels are processed onboard. A processing module is connected to sensor interfaces through SpaceWire in order to provide deterministic processing time. Data compression is also provided to reduce the restriction of transmission time, which provides the equivalent compression ratio as JPEG2000 in 1/30 processing time in average. A high-speed data recorder is connected through SpaceWire in order to record TIR data in parallel with other sensor data. The modularity of SpaceWire enables us to use these as built devices for TIR and inherits the same design as the long-wavelength infrared imager developed for the Venus climate orbiter Akatsuki.
Thermal infrared imager system is developed for HAYABUSA2, which is planned to be launched in 2014 and aims at sample-return from a C class near-Earth asteroid 1999JU3 considered to contain organic or hydrated materials. The system consists of a thermal-infrared imager (TIR) and a digital electronics, which is used not only for the scientific investigation of physical properties of the asteroid surface, but also for the assessment of landing site selection and safe descent operation onto the asteroid surface with in situ measurement. Since round trip communication time between the asteroid and the Earth is more than thirty minutes, onboard automatic data processing function and high speed data recording capability are provided to exploit the limited downlink capacity which is up to 32kbps. TIR adopts an uncooled bolometer with 320 x 240 effective pixels. Image operations as multiple images summation, dark image subtraction, and the compensation of dead pixels are processed onboard. A processing module is connected to sensor interfaces through SpaceWire in order to provide deterministic processing time. Data compression is also provided to reduce restriction on storage capacity and operation time, which provides the equivalent compression ratio as JPEG2000 in 1/30 processing time in average. A high speed data recorder is also connected through SpaceWire in 50Mbps in order to record TIR data in parallel with other sensor data. The modularity of SpaceWire enables to use as built devices for TIR and inherits the same design as the long-wavelength infrared imager developed for the Venus climate orbiter Akatsuki.
Fast and small-footprint lossless compressors for multi and hyper-spectral sensors have been developed. The compressors are employed for HISUI (Hyper-spectral Imager SUIte: the next Japanese earth observation project that will be on board ALOS-3).
By using spectral correlations, the compressor achieved the throughput of 30Mpel/sec for hyper-spectral images and 34Mpel/sec for multi-spectral images, which covers the data acquisition throughput of HISUI, on a radiation tolerant FPGA (field-programmable-gated-array). We also implemented the compressor on the evaluation model device of HISUI, and confirmed its feasibility and compression performance of actual hyper-spectral sensor data.
Fast and small foot print lossless image compressors aiming at hyper-spectral sensor for the earth observation satellite
have been developed.
Since more than one hundred channels are required for hyper-spectral sensors on optical observation satellites, fast
compression algorithm with small foot print implementation is essential for reducing encoder size and weight resulting in
realizing light-weight and small-size sensor system. The image compression method should have low complexity in
order to reduce size and weight of the sensor signal processing unit, power consumption and fabrication cost. Coding
efficiency and compression speed enables enlargement of the capacity of signal compression channels, which resulted in
reducing signal compression channels onboard by multiplexing sensor signal channels into reduced number of
compression channels.
The employed method is based on FELICS1, which is hierarchical predictive coding method with resolution scaling. To
improve FELICS's performance of image decorrelation and entropy coding, we applied two-dimensional interpolation
prediction and adaptive Golomb-Rice coding, which enables small footprint. It supports progressive decompression
using resolution scaling, whilst still delivering superior performance as measured by speed and complexity.
The small footprint circuitry is embedded into the hyper-spectral sensor data formatter. In consequence, lossless
compression function has been added without additional size and weight.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.