Timothy Aaron Stotts

126 Pages·2016·2.54 MB·English

by  Timothy Aaron Stotts

RRoocchheesstteerr IInnssttiittuuttee ooff TTeecchhnnoollooggyy
RRIITT SScchhoollaarr WWoorrkkss
Theses
8-2007
MMeetthhooddoollooggyy aanndd ooppttiimmiizziinngg ooff mmuullttiippllee ffrraammee ffoorrmmaatt bbuuffffeerriinngg
wwiitthhiinn FFPPGGAA HH..226644//AAVVCC ddeeccooddeerr wwiitthh FFRREExxtt..
Timothy Aaron Stotts
Follow this and additional works at: https://scholarworks.rit.edu/theses
RReeccoommmmeennddeedd CCiittaattiioonn
Stotts, Timothy Aaron, "Methodology and optimizing of multiple frame format buffering within FPGA
H.264/AVC decoder with FRExt." (2007). Thesis. Rochester Institute of Technology. Accessed from
This Thesis is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in
Theses by an authorized administrator of RIT Scholar Works. For more information, please contact
[email protected].
Methodology and optimizing of multiple frame format
buffering within FPGA H.264/AVC decoder with FRExt.
by
Timothy Aaron Stotts
A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of
Master of Science in Computer Engineering
Supervised by
Assistant Professor Dr. Marcin Lukowiak
Department of Computer Engineering
Kate Gleason College of Engineering
Rochester Institute of Technology
Rochester. New York
August 2007
Approved By:
Marcin tukowiak
Dr. Marcin Lukowiak
Assistant Professor, RIT, Department of Computer Engineering
Primary Adviser
Ken W. Hsu
Dr. Ken W. Hsu
Professor, RIT, Department of Computer Engineering
Secondary Adviser
Mark Grabosky
Mark Grabosky
Director of Engineering, Xelic, Inc.
Secondary Adviser
Thesis Release Permission Form
Rochester Institute of Technology
Kate Gleason College of Engineering
Title: Methodology and optimizing of multiple frame format buffering
within FPGA H.264/AVC decoder with FRExt.
I, Timothy Aaron Stotts, hereby grant permission to the Wallace Memorial Library to
reproduce my thesis in whole or in part.
Timothy Aaron Stotts
Timothy Aaron Stotts
:Joo f - 0& - I?-
Date
Dedication
To Christ Jesus, my one true source ofpeace.
"Peace I leave with you, my peace I give unto you: not as the world giveth,
give I unto you. Let not yourheart be troubled, neither let it be fearful."
in
Acknowledgments
A special thank you to each of my advisers for sharing their time and experience; and
especially to Dr. Lukowiak for his patient guidance, and Mark Grabosky at Xelic, Inc. for
encouraging andequiping metopull through. Thank you also toThomasWarsawformany
thoughtful discussions regarding his prior H.264/AVC work; and to family and friends for
theirloving support.
iv
Abstract
Digital representation ofvideo data is an inherently resource demandingproblemthat con
tinues to necessitate the development and refinement ofcoding methods. The H.264/AVC
standard, along with its recent Fidelity Range Extensions amendment (FRExt), is quickly
being adopted as the standard codec forbroadcast and distribution ofhigh definition video.
The FRExt amendment, while not necessarily affecting the overall decoder architecture,
presents an added complexity of providing efficient memory management for buffering
intermediate frames ofvarious pixel color samplings and depths.
This thesisevaluatedtheroleofdesigningtheframebufferofahardwarevideo decoder,
with integrated support for the H.264/AVC codec plus FRExt. With focus on organizing
external memory data access, the frame buffer was designed to provide intermediate data
storage for the decoder, while using an efficient store and load scheme that takes into con
sideration each frame pixel format ofthe video data.
VHDL was used to model the frame buffer. Exploitation ofreconfigurability andpost-
synthesis FPGA simulations were used to evaluate behavior, scalability and power con
sumption, while providing an analysis ofapproaches to adding FRExt to the memory man
agement. Real-time buffer performance was achieved for two common frame formats at
1080 HD resolution; and an innovative pipeline design provides dynamic switching offor
mats between video sequences. As an additional consequence of verifying the model, a
preexisting Baseline H.264/AVC decoder testbench was augmented to support testing of
multiple frame formats.
Contents
Dedication iii
...
Acknowledgments iv
...
Abstract
v
Glossary
xiv
1 Introduction 1
...
1.1 Background 1
1.2 Thesis objective 4
1.3 Thesis chapter overview 6
2 Video Coding 7
...
2.1 Y'CbCrcolor model 7
2.1.1 Y'CbCr sub-sampling 9
2.2 H.264/AVC overview 12
2.2.1 H.264/AVC coding summary 14
2.2.2 H.264/AVC Fidelity Range Extensions summary 17
2.3 Thesis relevance and specifics 19
2.3.1 H.264/AVC databuffering flow 19
2.3.2 H.264/AVC databuffering organization 20
2.3.3 Macroblock pixel types 23
3 H.264/AVC Research
.25
3.1 Decoder memory case studies and research 25
3.1.1 Identification ofmemory components 26
3.1.2 Optimization techniques 27
3.2 Analysis ofpublished results 30
4 Requirements and Modeling 32
4.1 Augmentation ofdecoder system 32
4.2 Algorithms used by the frame buffer. 32
4.2.1 Intraprediction buffering requirements 34
4.2.2 Deblocking filterbuffering requirements 36
4.2.3 Interprediction buffering requirements 36
vi
4.2.4 Combining ofbuffering mechanisms 37
4.2.5 Reference picture management 37
5 Synthesizable Implementation 40
5.1 External memory storage and control 40
5.1.1 DDR memory control 42
5.1.2 Block data controller 44
5.1.3 Implementation hiearchy 45
5.1.4 External DDR interface 46
5.2 Frame organization and addressing 46
5.2.1 Macroblock identification and frame slotting 49
5.2.2 Macroblock address mapping with FRExt 51
5.2.3 Frame storage marking 53
5.2.4 Sliding window implementation 54
5.3 Synthesis parameters 55
5.4 Frame buffer interface and pipelining 57
5.4.1 Frame buffer RTL interface 57
5.4.2 Pipeline semantics 62
5.5 Dual RAM frame buffer 63
5.5.1 Dual DDR SDRAM design 65
6 Verification - HDL Model Functionality . . .... 69
6.1 Unit testing 69
6.2 In-systemverification 71
6.2.1 Augmentation ofthe decoder system 71
6.2.2 Testbench redesign 74
6.2.3 Video sequences 76
6.2.4 Functional simulation 77
6.2.5 Post-synthesis simulation 79
7 Results and Analysis 81
...
7.1 Implementation analysis gl
7.2 Synthesis resource analysis 86
7.3 DDR timing analysis 37
7.4 H.264/AVC timing analysis 90
7.5 Powerconsumption analysis 92
7.6 Cost analysis 94
8 Conclusions
97
8.1 Synthesizable models 97
8.2 Proposed system interfacing 99
Bibliography 203
vn
Software Tools and Deliverables 106
A.l Software tools 106
A.1.1 Video processing and display 106
A.1.2 FPGA design and simulation 107
A.2 Thesis deliverables 107
vm
List Figures
of
1. 1 Digital representation ofapicture in terms ofdata size 1
1.2 Digital representation ofuncompressedvideo in terms ofdata size 2
1.3 The extra dimension ofpixel size upon total picture data size 3
1.4 Internal partitioning offrame buffer design 4
1.5 Video decoder system partitioning augmented fortesting frame buffer. ... 5
2.1 RGB vs. Y'CbCr decomposition ofa "foreman" test frame 8
2.2 Y'CbCr sub-sampling 4:4:4 10
2.3 Y'CbCr sub-sampling 4:2:2 11
2.4 Y'CbCr sub-sampling 4:2:0 11
2.5 Y'CbCr sub-sampling 4:0:0 12
2.6 Scope ofH.264/AVC Standard: only decoding [24] 14
2.7 The correlation between source (uncoded) picture frames and encoded slices. 15
2.8 Pixel sampling and depth, increasingly stacked by FRExt profile. [24] ... 18
2.9 Buffering within the H.264/AVC Hypothetical Reference Decoder. [9] 20
. . .
2.10 DPB operation: macroblock in, macroblock out 21
2.11 Buffer arow ofmacroblocks to retain neighbor MBs 21
2.12 Organization ofthe reference frame buffer 22
3.1 FPGA hybrid on-chip, off-chip decoder architecture proposed in [21]. 26
...
4.1 Video decodersystem architecture and data flow 33
4.2 Intra Prediction macroblock neighborpermutations 34
4.3 Main datapath "tap" locations for buffering 36
5.1 Vendor-supplied Xilinx Spartan 3E DDR SDRAM controller. 43
5.2 Customized Xilinx Spartan 3E DDR SDRAM controller 43
5.3 Block datacontroller state machines 44
5.4 Internal partitioning offrame buffer design 45
5.5 External DDR interface 47
5.6 Binary 16x8 sub-macroblock memory maps for each 8-bit sub-sampling. 49
. .
5.7 Single RAM frame buffer inner and outer interfaces 58
5.8 Dual RAM frame buffer inner and outer interfaces 65
6.1 Testbench flow with emphasis on dataprocessing 74
6.2 Testbench flow with emphasis on storage operations 75
6.3 Elephant's Dream Frames 11290, 11310 77
IX