Depth measurement apparatus, image pickup apparatus, and depth measurement program

18 Pages·2013·1.5 MB·English

US 20130307966A1
(19) United States
(12) Patent Application Publication (10) Pub. N0.: US 2013/0307966 A1
Komatsu (43) Pub. Date: NOV. 21, 2013
(54) DEPTH MEASUREMENT APPARATUS, Publication Classi?cation
IMAGE PICKUP APPARATUS, AND DEPTH
MEASUREMENT PROGRAM (51) Int- Cl
G06T 7/00 (2006.01)
(71) Applicant: CANON KABUSHIKI KAISHA, (52) US. Cl.
Tokyo (JP) CPC .................................. .. G06T 7/0069 (2013.01)
USPC ......................................... .. 348/135; 382/106
(72) Inventor: Satoru Komatsu, Yokohama-shi (JP) (57) ABSTRACT
A depth measurement apparatus calculates depth information
(73) Assigneer CANON KABUSHIKI KAISHA, on a subject in an image by using a plurality of images having
TOKYO (JP) different blurs taken under different imaging parameters, and
includes a region segmentation unit that segments at least one
of the images into regions based on an image feature amount,
(21) App1.No.: 13/869,232
Wherein in each of the regions pixels are presumed to be
substantially equal in depth to the subject, and a depth calcu
(22) Filed; APL 24, 2013 lation unit that calculates a depth for each region resulting
from the segmentation by the region segmentation unit and
(30) Foreign Application Priority Data serving as a processing target region for depth calculation,
and sets the calculated depth as the depth of the processing
May 17, 2012 (JP) ............................... .. 2012-113780 target region.
100 IMAGE PICKUP APPARATUS
/~ 105
IMAGING
CONTROL UNIT
SIGNAL I, 104
PROCESSING _-> BUFFER |
UNIT
102
IMAGE PICKUP
101 ELEMENT
IMAGE PICKUP
OPTICAL SYSTEM ,, III
REGION
sEGMENTATION
UNIT
v V 112
DEPTH CALCULATION ‘
UNIT
1 /—113
MEMORY
DEPTH MEASUREMENT
APPARATUS
Patent Application Publication Nov. 21, 2013 Sheet 1 0f 9 US 2013/0307966 A1
/-—-10O IMAGE PICKUP APPARATUS
/- 105
IMAGING
CONTROL UNIT
/—103
SIGNAL v r104
> PROCESSING = BUFFER
UNIT
\- I02
IMAGE PICKUP
101 ELEMENT
IMAGE PICKUP
OPTICAL SYSTEM I, /—1 II
REGION
SEGMENTATION
UNIT
v i r112
DEPTH CALCULATION ‘
UNIT
v /—1 13
MEMORY
\- I10
DEPTH MEASUREMENT
APPARATUS
FIG. 1
Patent Application Publication Nov. 21, 2013 Sheet 2 0f 9 US 2013/0307966 A1
( START D
(A PLURALITY OF IMAGES)
l /—S11
DEPTH MEASUREMENT APPARATUS /
ACQUIRES IMAGES
l
REGION SEGMENTATION /_ °"
PROCESSING
' s13
DEPTH CALCULATION /_
PRoGEssING
( END )
FIG. 2
Patent Application Publication Nov. 21, 2013 Sheet 3 0f 9 US 2013/0307966 A1
REGION SEGMENTATION S12
PROCESSING
FOCUS
IMAGE
I /— S121
INITIAL REGION
SEGMENTATION STEP
, S122
REGION SELECTION /—
STEP
/— S123
I'EAIUHI: AMUUNI I
CALCULATION STEP
I /- S124
EVALUATION VALUE
DETERMINATION STEP
No i 3125
__<EXECUTED ON /—
ALL REGION?
S126
REPEAT REPETITION %
<DETERMINATION
END
FIG. 3
Patent Application Publication Nov. 21, 2013 Sheet 4 0f 9 US 2013/0307966 A1
d1
FL
F4IG.
Patent Application Publication Nov. 21, 2013 Sheet 5 0f 9 US 2013/0307966 A1
813
@EPTH CALCULATION PROCESSIN
// // /
LABEL TWO INPUT SHOOTING
IMAGE IMAGE PARAMETER
V
S131
REGION SELECTION /_
V
STEP
I
DEPTH DEPENDENCE VALUE /— 3132
CALCULATION STEP
I
V
DEPTH CORRECTION STEP
S134
FIG. 5
Patent Application Publication Nov. 21, 2013 Sheet 6 0f 9 US 2013/0307966 A1
/-200 IMAGE PICKUP APPARATUS
/- 205
IMAGING
CONTROL UNIT
F203
SIGNAL v r204
= PROOESSING = BUFFER
UNIT
\- 202
IMAGE PICKUP
201 ELEMENT
IMAGE PICKUP
OPTICAL SYSTEM I, /--211
REGION
SEGMENTATION
UNIT
v I, /-212
DEPTH CALCULATION ‘
UNIT
I, /-213
MEMORY
\- 210
DEPTH MEASUREMENT
APPARATUS
FIG. 6
Patent Application Publication Nov. 21, 2013 Sheet 7 0f 9 US 2013/0307966 A1
REGION SEGMENTATION S12
PROCESSING
II
A PLURALITY OF
INPUT IMAGES
8221
v
INITIAL REGION /—
SEGMENTATION STEP
II /_
REGION SELECTION
STEP
FEATUREVAMOUNT /_ S223
CALCULATION STEP
V
/_
EVALUATION VALUE
DETERMINATION STEP
I
EVALUATION VALUE
/_
DETERMINATION STEP
_NO<EXEOIITED ON&
ALL REGION?
YES
227
REPEAT< REPETITION >—s
DETERMINATION
FIG. 7
I END I
Patent Application Publication Nov. 21, 2013 Sheet 8 0f 9 US 2013/0307966 A1
/-s00 IMAGE PICKUP APPARATUS
F306
UI
‘, 305
IMAGING
i CONTROL UNIT
r303
SIGNAL v r304
= PROCESSING > BUFFER
UNIT
\- 302
IMAGE PICKUP
301 ELEMENT
IMAGE PICKUP
OPTICAL SYSTEM " 1,-311
REGIoN
SEGMENTATIoN
UNIT
DEPT\Hr CALCULA‘TV ION ‘
UNIT
MEMORY
\. 310
DEPTH MEASUREMENT
APPARATUS
FIG. 8
Patent Application Publication Nov. 21, 2013 Sheet 9 0f 9 US 2013/0307966 A1
12
REGION SEGMENTATION S
PROCESSING
V V
A PLURALITY OF SHOOTING
INPUT IMAGES PARAMETER
v 8321
SEGMENTATION NUMBER /_
DETERMINATION STEP
V
INITIAL REGION
r
SEGMENTATION STEP
L
7
V S323
REGION SELECTION /—
STEP
I
v /—
FEATURE AMOUNT
CALCULATION STEP
YES
REPEAT /I\ /— S325
REPETITION >
< DETERMINATION