Ubercalibration of the Deep Lens Survey

3.2 MB·English

by  D. Wittman

Draft version January 17, 2012
PreprinttypesetusingLATEXstyleemulateapjv.5/2/11
UBERCALIBRATION OF THE DEEP LENS SURVEY
D. Wittman, R. Ryan and P. Thorman
Physics Department, University of California, Davis, CA 95616
[email protected]
Draft version January 17, 2012
ABSTRACT
We describe the internal photometric calibration of the Deep Lens Survey, which consists of five
widely separated fields observed by two different observatories. Adopting the global linear least-
squares (“ubercal”) approach developed for the Sloan Digital Sky Survey (SDSS), we derive flatfield
2
corrections for all observing runs, which indicate that the original sky flats were nonuniform by up to
1
0 0.13 mag peak to valley in z band, and by up to half that amount in BVR. We show that application
2 of these corrections reduces spatial nonuniformities in corrected exposures to the 0.01-0.02 mag level.
Weconcludewithsomelessonslearnedinapplyingubercaltoasurveystructuredverydifferentlyfrom
n
SDSS, with isolated fields, multiple observatories, and shift-and-stare rather than drift-scan imaging.
a
Although the size of the error caused by using sky or dome flats is instrument- and wavelength-
J
dependent, users of wide-field cameras should not assume that it is small. Pipeline developers should
4
facilitate routine application of this procedure, and surveys should include it in their plans from the
1
outset.

Subject headings: surveys—methods: statistical—techniques: photometric
]
M
I 1. INTRODUCTION corresponding structure of the ubercal model; in §3 we
.
h describethefittingprocess;in§4weshowtheresultsand
As CCD camera fields of view have become dramat-
p examine their robustness; in §5 we report the impact on
ically larger in the past decade, ensuring uniform pho-
- surveyphotometryaccuracy; andwepresentconclusions
o tometry has become more challenging. Traditionally, it
in §6.
r isassumedthatthecameracanbeuniformlyilluminated,
t byskyemissionorbyanilluminatedscreen,anddividing
s 2. UBERCALFORMALISMFORTHEDLS
a by the response of each pixel to this illumination is all
[ that is necessary to make images internally consistent. The basic idea of ubercal is to write down a model
describing the observing process (true magnitudes are
But the sensor may not be uniformly illuminated due to
2 offset by a zeropoint, an airmass term, and a “flatfield”
vignetting or to nonuniform sources of illumination such
v term describing the spatial variation of the camera re-
as scattered light, moonlight, or ghosting.
8 sponse), and constrain the model parameters using the
Theseproblemsareexacerbatedwhenthefieldofview
5
measured instrumental magnitudes. Before writing a
islarge. Forsmallfieldsofview,areasonableworkaround
0
specificmodelfortheDLS,wefirstdescribethestructure
2 for nonuniform illumination is to observe a given star
of the DLS.
. with a grid of pointings, so that spatially-varying pho-
1
tometrycanbedirectlyidentifiedandmodeledout. This
1 2.1. Structure of the DLS
is impractical for very large cameras, where the observa-
1 tion and readout times would be prohibitive. However, The DLS consists of five well-separated 2◦ ×2◦ fields
1
Padhmanabhan et al. (2008, hereafter P08) presented a (Table 1). The northern fields (F1 and F2) were ob-
:
v nearly-equivalent solution involving multiple stars. In served using the Kitt Peak Mayall 4-m telescope and
i anysetofoverlappingexposuresordriftscans,somestars Mosaic prime-focus imager (Muller et al. 1998). The
X
are multiply observed, in different parts of the camera, southern fields (F3 through F5) were observed with a
r and these observations can be used to constrain a model similar setup at the Cerro Tololo Blanco 4-m telescope.
a
ofthespatialvariationofthecameraresponse. Thistech- EachMosaicimagerconsistsofa4×2arrayofthree-edge-
nique, widely referred to as ubercal, analyzes the survey buttable2k×4kCCDs,providinga35(cid:48) fieldofviewwith
data per se, and does not require additional calibration 0.26(cid:48)(cid:48) pixelsandminimalgapsbetweenthedevices. Each
data. Thisneatlyseparatestheproblemsofinternaland DLSfieldisdividedintoa3×3gridof40(cid:48)×40(cid:48) subfields.
external calibration. These subfields are slightly larger than the Mosaic field
Asmoreandbiggerimagingsurveyscomeonline,uber- of view, but are synthesised with dithers of up to 800
cal is sure to be an important tool. P08 applied it to a pixels (208(cid:48)(cid:48)). In some cases, dithers were much larger.
drift-scan survey, the Sloan Digital Sky Survey (SDSS). For example, CCD 3 was dead during two Cerro Tololo
We apply it here to the Deep Lens Survey (DLS) which, runs,andweditheredbyanentirechipwidth(8.5(cid:48)). Ina
like the vast majority of imaging surveys, is a shift-and- small number of exposures, telescope pointing errors re-
staresurvey. TheDLShassomeotherpeculiarities,such sulted in very large dithers and/or odd pointing centers,
as widely separated fields and multiple observatories, further tying the subfields together.
that make it an informative example for the application The DLS was observed in four filters, BVRz. R was
of this technique. In §2 we describe the DLS and the observedwhentheseeingFWHMwas<0.9(cid:48)(cid:48); ifthesee-
ingwasworse, thechoiceoffilterswasdictatedbymoon
2
as receiving its own zeropoint, but happens to be sim-
TABLE 1
DLS field centers (J2000) and mean extinctions from pler to implement in our code. A few images were taken
Schlegel, Finkbeiner & Davis (1998). with nonstandard exposure times. For fitting purposes,
weconvertedtheinstrumentalmagnitudesfromtheseex-
Field RA DEC l,b E(B-V)
posurestotheinstrumentalmagnitudesthatwouldhave
F1 00:53:25.3 +12:33:55 125,-50 0.06
beenobservedhadthestandardexposuretimebeenused.
F2 09:18:00 +30:00:00 197,44 0.02
We choose not to model any time variation of the air-
F3 05:20:00 -49:00:00 255,-35 0.02
mass term. P08 suggest a time term of 1 mmag per unit
F4 10:52:00 -05:00:00 257,47 0.025
F5 13:55:00 -10:00:00 328,49 0.05 airmass per hour, based on data from the SDSS photo-
metric calibration telescope. The sign is such that the
phase, airmass, and past progress in other filters. Thus, atmosphere grows more clear as the night goes on. We
most nights contain images in multiple filters, but few have no information on similar trends at Cerro Tololo
nights contain images in all four filters. There were 25 and Kitt Peak. DLS differs from SDSS in being able to
observing runs of 2-6 nights each. On a few runs, south- use only one filter at a time, so that time baselines in a
ern fields (F4 at declination -5◦ and F5 at -11◦) were given filter tend to be much shorter. Over a three-hour
observed from Kitt Peak. A northern field (F1 at dec- time baseline, a change of 1 mmag per unit airmass per
lination +12◦) was observed on only one Cerro Tololo hourisquitesmall. Furthermore, becauseDLShasfixed
run, on a nonphotometric night. At both sites, equa- fields rather than a drift-scan strategy, airmass is often
torial standard star fields (Landolt 1992, Smith et al. a monotonic function of time in a given filter on a given
2002) were observed on photometric nights. More de- night,inwhichcasemuchofthetimevariation(ifiteven
tails about the DLS survey design and image processing exists)canbeabsorbedintotheairmassterm. Therefore,
can be found in Wittman et al. (2002) and Wittman et we do not include time variation as part of the model.
al.(2006)respectively. Apaperdescribingthefinaldata This is further justified by the general absence of time
release details (Wittman et al., in preparation) is forth- trends in the residuals.
coming.
2.3. Spatiotemporal degeneracy
2.2. Photometric model
Ourprimarymotivationforthisworkwastoderivethe
We adopt the following model for the instrumental
flatfield corrections, because the DLS photometry had
magnitudes measured through a given filter:
been showing spatial dependence. Recall that each sub-
m =m(s) +z(r)+k(n)A+f(r,c)(x,y) (1) fieldwasimaged20timesineachfilter,with∼5(cid:48)dithers
instr true providingsomeoverlapbetweenadjacentsubfields. Intu-
wherez isarun-dependentzeropoint,Aistheairmassof itively,onemightthinkthatthislargenumberofdithers
theobservation,k istheextinctionperunitairmass,f is would be sufficient to constrain the flatfield corrections
the flatfield function, and superscripts indicate indexing by itself. However, there is an astonishing degeneracy
bystar(s),run(r),night(n),andCCD(c)identification between spatial and temporal variations, as follows.
numbers. Filter superscripts are not explicitly written Consideranimagingdevicewithasensitivityvariation
here because the fitting procedure is carried out totally f(x,y) = ax+by (in magnitudes here for convenience)
independently in each filter. and a time-varying zeropoint z(t). For now we will not
We choose f to be a second-order polynomial in the think of z(t) as a function, but just as a collection of
CCDcoordinatesxandy,withanxy cross-termmaking zeropoints at the times t when various exposures were
i
atotalof6parametersperCCD.Thereisnorequirement taken. So a star at position x,y will have instrumental
that the flatfield be continuous across CCD boundaries. magnitude
The primary motivation for allowing discontinuities is
that sky flats have already been applied to the data, so m =m +ax+by+z(t ) (2)
i true i
we are really modeling systematic errors in the sky flat
process, which may include discontinuous jumps at the Nowconsiderasetofexposureswithditheroffsetsx ,y .
i i
CCD boundaries. The flatfield magnitude offset at the The difference between instrumental magnitudes of a
center of the first CCD is fixed at zero to remove the given star on the ith and jth exposures is
degeneracy with overall throughput parameters such as
thezeropoint. Thisleaves47flatfieldparameterstocon- m −m =a(x −x )+b(y −y )+z(t )−z(t ) (3)
i j i j i j i j
strain for each run (eight CCDs times six polynomial
parameters, minus one fixed offset). Each run has its This quantity is identical for all stars and galaxies, be-
own flatfield model, because sky flats were derived and cause they all experience the same spatial shift.
applied on a per-run basis. Now consider an alternative model where the flatfield
Eachrunalsohasitsownzeropoint, sothatthemodel slopes are quite different:
cantrackchangesinnon-atmosphere-relatedobservatory
throughput such as might occur when the telescope op- m =m +(a+λ )x+(b+λ )y+z(t )−λ x −λ y (4)
i true 1 2 i 1 i 2 i
tics are recoated. Changes in CCD gain would also be
absorbed into this factor. with λ and λ being arbitrary numbers making it clear
1 2
Airmass terms are, in principle, fit for each night. In thatthismodelisdifferentfromthepreviousmodel. Re-
practice, many nights were not photometric, and thus call that x and y are the star’s position on the detector,
cannot be fit this way. On these nights, each image re- while x and y are the dither shifts, so the last two
j j
ceives its own airmass term. This conceptually the same terms affect the entire exposure, effectively changing its
3
zeropoint. Now the difference between exposures is by a uniform decrease in run zeropoints. Fixing this de-
generacyispreciselytheproblemofexternalcalibration,
m −m =(a+λ )(x −x )+(b+λ )(y −y )
i j 1 i j 2 i j which we do not address here.
+z(t )−z(t )−λ (x −x )−λ (y −y ) Relative calibration degeneracy for isolated fields. The
i j 1 i j 2 i j
truemagnitudesinonefieldarefixedrelativetothetrue
=a(x −x )+b(y −y )+z(t )−z(t ) (5)
i j i j i j
magnitudes in another field only if both fields were ob-
Equations 3 and 5 are the same, demonstrating that served from the same observatory on the same photo-
the observed magnitude difference is identical in these metricnight. Non-photometriccross-observationsdonot
twocases,despitetheirverydifferentimplicationsforthe helpbecauseper-imagezeropointsarefreetoabsorbany
flatness of the detector sensitivity, and therefore for all offset in the true magnitudes of one field relative to the
thephotometryderivedtherefrom. Thisisadegeneracy. other field. Almost every photometric night of DLS ob-
Note that the degeneracy parameters λ1 and λ2 do servations touched multiple fields, so most fields are well
not depend on the position of the star on the detec- tied together. However, there is a natural north-south
tor, only on the dither offset xi−xj,yi−yj. Therefore, split with few photometric cross-observations. The B, V
the density and spatial distribution of stars is irrelevant. andzobservationscontainatleastonephotometricnight
Furthermore, the λi depend only on the ith exposure’s withcross-hemisphereobservations. However,inRthere
dither offset, not on any properties of the other expo- are none. This could be fixed with external calibrations
sures. Therefore, the number and spatial distribution of from the standard star fields. However, in the spirit of
dithers is irrelevant. This is a true degeneracy as long ubercal, we make use of the fact that both observatories
as the zeropoints have freedom to soak up any effect of often observed the same equatorial standard star fields,
the flatfield slope. But with observations in photometric in particular SA 98 and SA 101 (Landolt 1992). There-
conditions, we can put a strong prior on z(ti)−z(tj), fore, we included SA 98 and SA 101 data in the ubercal
thus constraining the linear flatfield terms a and b to fit,nottoprovideabsolutecalibration,buttostrengthen
similarprecisioninunitsofthedithersize. Surveyswith thetiesbetweennorthandsouth. (Onecouldmakeuber-
substantial fractions of nonphotometric time should be cal perform both of these functions by fixing m for
true
aware of this issue. For example, they should ensure the Landolt (1992) stars rather than considering them
that the nonphotometric time is not overly concentrated nuisance parameters. However, this would require all
in any one filter, lest it become impossible to solve for ubercal photometry to be measured within a large (14(cid:48)(cid:48)
the flatfield corrections in that filter. diameter)aperturetobeconsistentwithLandolt(1992).
This degeneracy is grasped much more intuitively and Therefore,theabsolutecalibrationisbetterdoneoutside
directly by considering a technique used for small cam- the ubercal framework.)
eras, dithering a given star around the field of view to Airmass/timedegeneracy. OnmanyDLSnights,there
directlymeasuretheflatnessofthephotometry. Clearly, isamonotonicrelationshipbetweenairmassandtimefor
that procedure would work only if done quickly enough some of the filters, because a given filter was used for
toeliminatethepossibilityofchangesinthroughputdue only a few hours before or after a field transited. Thus,
to anything other than star position. Similarly, ubercal atime-dependenttermwouldbedegeneratewithanair-
requires a strong temporal or airmass-based constraint mass term. This degeneracy does not prevent one from
on each exposure’s throughput. producing internally consistent photometry, but it does
One might think that another way to fix the degen- severelyhamperphysicalinterpretationofthefitparam-
eracy would be to rotate the CCD. Although this does eters. Consider a time-dependent airmass coefficient of
constrain the gradient, there are still rotational modes the form k+k t. With k fixed at zero, the size of k is
1 1
which are degenerate with the time behavior. We have an indicator of the clarity of the night, and very large
confirmed this with simulations. or very small k indicates some problem with the data
If f(x,y) contains higher-order polynomial terms, (most likely clouds if k is very large, for example). With
these are not degenerate. For example, the signature k free, we find that the data will be slightly better fit,
1
ofa2nd-ordertermmightbestarsontheleftsideofthe but the time/airmass degeneracy will probably drive k
CCD getting brighter and stars on the right side getting and k to nonphysical values. We therefore choose not
1
fainter after a small shift to the left, and vice versa for to fit for k .
1
a small shift to the right. This cannot be mimicked by
a zeropoint change. However, the linear terms are inde- 3. FITTING
pendent of the higher-order terms and thus will remain Theproceduresdescribedbelowwerecarriedoutinde-
unconstrained even in the presence of the higher-order pendently in each filter, unless otherwise noted.
terms.
3.1. Input data
2.4. Other degeneracies
We ran SExtractor (Bertin & Arnouts 1996) to pro-
Thereareotherdegeneraciesmodelersshouldbeaware duce a catalogue for each CCD of each exposure, and
of. Most of these were recognised by P08, but are worth then matched the catalogues in RA and DEC to pro-
mentioninghereinthecontextofaverydifferentlystruc- duce a master catalogue for each widely separated field.
tured survey. (This is an exception to the per-filter rule; all images in
Absolute calibration degeneracy. The bulk of the fit a given field were matched to ensure consistent astrome-
parameters are the “true” magnitudes of the stars. Of try across filters.) Because the eventual goal is to stack
course, these are not true magnitudes in an absolutely theimagesontoagridwithuniformpixelsize, weadded
calibrated sense. There is a degeneracy in which uni- masterx,y columnscorrespondingtothedesiredoutput
form increase in true magnitudes can be compensated grid. Wealsoassignedauniqueidentificationnumberto
4
each object in the master catalogues. We then matched reject those observations.
each CCD catalogue to the master catalogue and fit for
polynomial coordinate transforms between CCD coordi- 3.2. Photometricity of nights
nates and uniform-grid coordinates. We corrected the Before using ubercal, our previous method of deter-
photometry of each CCD catalogue by the Jacobian of mining photometric offsets between exposures was by
thecoordinatetransform,tocorrectforthevaryingpixel matching exposure catalogues against a master cata-
sizewithoutresamplingthepixels. SeetheDLSdatare- logue. Theseoffsets, alongwithobservinglogsandstan-
lease paper (Wittman et al., in preparation) for a more dardstarobservationstakenonnightsthoughttobepho-
completeexplanationofthiscorrection. Asnotedabove, tometric by the observers, gave us a good starting point
in a few cases we also adjusted the magnitudes to the for marking nights as photometric (and thus requiring
standard exposure time.
a single k(n) parameter for the night) or nonphotomet-
We then cleaned the catalogues. We rejected obser-
ric (and thus requiring an exposure-specific magnitude
vations with SExtractor FLAGS>0, and cut on object
offset). However, we found (as did P08) that ubercal it-
peakintensity(SExtractor’sFLUX MAXoutputparam-
self was the best tool for picking up on subtle levels of
eter). On this attribute, we imposed both a maximum
cloud. Nonphotometricnightsstoodoutashavingmany
(20% below saturation after taking into account the sky
exposures with large mean residuals from the fit. When
level which is not included in FLUX MAX) and a filter-
a night was identified as nonphotometric, the k(n) pa-
dependent minimum designed to prevent the catalogues
rameter for that night was removed from the model, one
from becoming extremely large. The minimum was set
extinction parameter per exposure was inserted, and the
around 35σ (for the peak pixel, not the entire detection)
fitting was restarted from scratch. The process was iter-
in z band, and higher in the other bands. Roughly 100
ative, withever-betteroverallfitshighlightingevermore
objectsperCCDpassthesecuts,yieldingroughly800ob-
subtle photometricity variations. As described below,
jectsperexposureandroughly800,000objectsperfilter.
we found that even the best-fit nights have unmodeled
The time required to fit a catalogue of this size turned
variations, so that the cutoff between photometric and
out to be surprisingly short (see §3.3), so that the cat-
nonphotometric is somewhat arbitrary. We adopted a
alogs could have been made more inclusive. However,
rule calling a night photometric if the vast majority of
some lower signal-to-noise ratio (S/N) cutoff will always
its exposures were modeled to within 0.02 mag.
be necessary, as low-S/N objects add limited constrain-
Figure 1 illustrates the process of identifying nonpho-
ingpowerwhileincreasingthecomputetimelinearly. We
tometric nights. Each color in the figure represents a
prefer a conservative S/N cut because SExtractor over-
different night on one run (all in B band). We rearrange
statestheS/Nbynotaccountingforskymodelingerrors.
Equation 1 to isolate the airmass part of the model:
We used both stars and galaxies. P08 used only stars
“to avoid subtleties of extended object photometry.” m −(m(s) +z(r)+f(r,c)(x,y))=k(n)A. (5)
However, these subtleties are mostly due to the range instr true
of galaxy profiles and morphology. Because we are only We then average the left-hand side over the hundreds of
concerned with relative photometry of the same galaxy objectsphotometeredinthatexposuretoobtainasingle
appearing in different places on the sensor, galaxy pho- point for that exposure. These are the points plotted
tometry should be usable for this purpose. In tests, we in the top panel of Figure 1. For one night (shown in
found SExtractor’s MAG AUTO (the magnitude within black), the points cluster around a straight line. This is
an aperture of size and shape determined by the object the airmass model on the right-hand side of Equation 5,
itself)toberelativelyimmunetoseeingvariations,sowe which is also drawn on the figure. For two other nights
used MAG AUTO for the fit. (showninblueandgreen),thereisahugevarianceinthe
However, the stated uncertainties on MAG AUTO are upper panel. It appears that the telescope was pointed
unrealisticallysmall(∼1mmag)becausetheyarebased atrelativelyclearskyforpartofeachnight(pointslined
on photon noise. For bright objects, where photon noise up near the black line), but obviously cloudy sky for a
is relatively small, measurement uncertainty is likely substantial number of exposures (points well above 0.3
dominatedbybackgroundmodelingerrors. Atthebright mag on the vertical axis). The conclusion is that those
end, we found a noise floor of ∼0.02 mag as determined nights were not photometric, and therefore no airmass
bythermsdifferencesbetweenmagnitudesmeasuredfor fit is drawn. Instead, each nonphotometric exposure is
the same object on pairs of images closely matching in assignedauniquevalueofkAinEquation5,equaltothe
time,airmass,anddetectorposition. Thisvalueisonlya meanoftheleft-handsideoftheequation. Forthenight
rough approximation, as it is certainly filter-dependent, whichwasfitwithasingleparameter,theresiduals(asa
and may decrease after taking flatfield corrections into function of airmass and of time) are shown in the lower
account, but it is much more realistic than the nominal two panels.
uncertainties, so we assign an uncertainty of 0.02 mag The night shown in red has only three exposures in
to each measurement. When interpreting the χ2 values this filter, so the evidence is not as strong as for the
from the fit, we must keep in mind the roughness of the other nights, but variable cirrus may have been a factor.
approximation here. In cases like these, the presumption should be nonpho-
The cleaned catalogue in any given filter contains 5- tometric. The time and airmass baselines are short, so
7×105observationsand6-7×104uniqueobjects,depend- even if the data were photometric they would not pro-
ingonfilter. Thecleanedcataloguedoesnotcontainany videstrongadditionalconstraintsbeyondtheirbasenon-
objects observed only once. Thus it should not contain photometric value. We stress that even nonphotometric
cosmic rays or asteroids, except where they happened to data help constrain the flatfields. The extinction due to
land on top of real objects. In §3.3 we describe how to cloudsisunlikelytohavespatialstructureonthescaleof
5
Fig. 1.— Airmass fits and residuals for a run with one photo-
metric and three nonphotometric nights in B band. Each point
represents one exposure and each night is a different color. The Fig. 2.—Histogramsofthebest-fitnightlyairmassterms,which
bottom panel shows the residuals from the fit (after averaging all
clusteraroundthephysicallyexpectedvaluesineachfilter. Nights
objects in each exposure) as a function of time, and the middle
which deviated greatly from these values were recognized as non-
panelshowstheresidualsasafunctionofairmass. Thetoppanel
photometric. Typically,suchnightswerepoorfitsanyway.
showsthemodelanddatarelativetotherunzeropoint,suchthat
thelinesshouldinterceptzeromagnitudesatzeroairmass.
We could obtain fits with slightly smaller residuals by
the camera (35(cid:48)), especially integrated over an exposure freeing this constraint, but there is little physical moti-
time of 10-15 minutes. Therefore, as long as other data vation for doing so.
inthesurvey(onotherruns,perhaps)constrainthetrue TwoofthenightsinFigure1areveryobviouslycloudy,
magnitudesoftheobjectsintherelevantareasofsky,the but many cases were more borderline, with ∼ 0.1 mag
spatial variation of the photometry in the nonphotomet- departures from the best airmass fit. Before defini-
ricexposuredoesprovideinformationabouttheflatfield. tively marking a night as nonphotometric, we investi-
So in the context of a larger survey with sufficient pho- gated other possible explanations for a poor fit. In par-
tometric data, ubercal can solve for the flatfield even for ticular, because good flatfield solutions were our main
a completely nonphotometric run. goal, we investigated the possibility that a poor nightly
Itisinstructivetoexaminetheresidualsonphotomet- fitcouldbecausedbypoorlyfittingflatfieldsratherthan
ricnights(lowerpanelsofFigure1). Thefitisgoodover nonphotometricity. We did this by checking for correla-
a wide range of airmass and time. The worst residu- tions between dither directions and residuals. For exam-
als are 6-7 mmag, whereas the measurement uncertainty ple,withrespecttothefirstexposureofthefive-exposure
on each residual is about 1 mmag. The residuals are dither pattern, subsequent exposures were shifted 3.5(cid:48)
also structured in time, with a quasiperiodic variation shift to the northeast, 3.5(cid:48) shift to the southwest, 1.75(cid:48)
of ∼ 2 hours and a peak-to-valley amplitude of ∼ 10 shifttothenorthwest,andfinally1.75(cid:48) shifttothesouth-
mmag. These data do not tell us whether this variation east. Thus,ifaseriesofimagesfitpoorlyduetopoorflat-
inextinctionoccurredinalldirections,oronlyinthepar- field solutions, one would expect the residuals to reverse
ticular field being followed. It seems likely that a local signbetweenthesecondandthird,and/orthefourthand
variation such as a band of very thin cirrus would move fifth, exposures of a dither pattern, and for the residuals
across the field on faster timescales, and that on photo- to be largest on the second and third exposures. In fact,
metric nights the long DLS exposures are more sensitive we never found this pattern, instead consistently finding
toglobalvariations. However,surveyswithmuchshorter the ∼2 hour quasiperiodic variation described above.
exposuretimeswillnotnecessarilyfindspatialvariations Incasesofborderlinephotometricitywealsolookedat
to be trivial, and may find a wider-field boresighted cir- behavior in all filters on the same night. If the collective
rus monitor to be a good investment. behavior seemed nonphotometric, we marked the night
To be declared photometric, a night had to have not as nonphotometric. Once marked as nonphotometric for
onlyagoodairmassfit,butaphysicallyplausibleairmass any reason, a night is considered nonphotometric in all
coefficient(∼0.05, 0.08, 0.14, and0.22inz, R, v, andB filters.
respectively). Figure 2 shows histograms of the best-fit A few images were flagged as outliers despite being on
nightly airmass terms in the four filters, for those nights apparently photometric nights. About half of these had
which were considered photometric. To prevent overfit- clearlyidentifiablecauses,suchasincorrectheaderinfor-
ting,themodelenforcesacommonzeropointthroughout mation, or the last exposure of a night pushing too far
arun. Inotherwords,thefitsfortwonightsonthesame into dawn and corrupting the input photometry. In the
run must intersect at zero airmass. Although there are case of bad header information on a photometric night,
physicalmechanismswhichcouldchangethezeropointat theimagewasreturnedtothephotometriclistifthecor-
zeroairmass(recoatingoftheoptics,forexample),fewof rectmetadata(primarilyairmass)couldbereconstructed
them are likely to operate on a night-to-night timescale. from observing logs. Otherwise, the image was consid-
6
ered as a singleton, receiving its own exactly fitted value of the exposure). The rms residual is about 0.02 magni-
of kA just as if it were nonphotometric. The most inter- tudes (ranging from 0.015 in R to 0.025 in z), and the
estingcasewasanexposurewhereapparentlytheguider mean residual of the worst-fitting exposure ranges from
jumped stars at some point, resulting in double images 0.014 in V to 0.025 in B, thus setting a clip threshold of
of everything (but one image several times fainter than 0.10 to 0.17 magnitudes depending on the filter. Clip-
the other). The fact that this was not flagged manually ping removes the one outlying observation of an object,
at the time of observation demonstrates the porosity of rather than all observations of that object. Manual in-
manual checks and the benefits of fitting all the survey spection of some of the clipped observations confirmed
data simultaneously. that the causes were generally identifiable as cosmic-ray
hits, proximity to an unmasked bad pixel or bleed, fore-
3.3. Writing and running the code
ground asteroid, etc.
The core fitting code was kindly provided by N. Afterclipping,thereducedχ2isaboutunity. However,
Padhmanabhan, written in C++ with a conjugate gra- the value of χ2 is not a rigorous indication of goodness-
dient optimiser based loosely on the one in Press et al. of-fit because we manually set the instrumental magni-
(1992). This is not the same code used in P08; it was tude uncertainties to 0.02 mag in the absence of realistic
written from scratch to assess the feasibility of ubercal uncertainty estimates from SExtractor. The reduced χ2
fortheLargeSynopticSurveyTelescope(LSST,Abellet rangesfrom0.56inRto1.65inz becausetheRphotom-
al. 2009), and only solved for per-image zeropoints. We etry is more precise than 0.02 mag (R being the deepest
modified the code for DLS, adding indexing by run, by filter and the one with the best seeing) and the z pho-
night, and by CCD, and solving for polynomial flatfield tometryislessprecise(duetothelargeskynoise). Thus
solutions, airmass terms, and per-run zeropoint offsets. the χ2 values indicate that the fits are plausibly good,
To avoid potential numerical instabilities, the polyno- but we base our judgment of the fits mostly on other
mial coordinates are centered on each CCD and tracked considerations such as inspection of all the airmass plots
in units of CCD widths rather than pixels. For conve- and(asdescribedbelow)physicalplausibilityofthesolu-
nience, therunzeropointsarereferencedtothecenterof tions,closed-loopcorrectiontests,androbustnessagainst
CCD 1 rather than to an average over the entire array. varying preparation of the input photometry.
The code takes only 6 minutes to converge on a stan-
dard desktop computer with 7 × 105 data points and 4. RESULTS
6.5×104 parameters (mostly nuisance parameters, the There are three main types of output: the nightly air-
truerelativemagnitudesoftheobjects). Wedeclarecon- mass terms, the run zero points, and the spatial sensi-
vergencewhenaniterationyieldsarelativeχ2 reduction tivity functions (“flatfields” for short). We show some
of 10−9 or less. Because the total χ2 is about ∼6×105 examples of each and offer physical interpretations.
after convergence, this criterion corresponds to a χ2 re-
duction of much less than one per iteration, but due to 4.1. Airmass Terms
the poorly defined photometric uncertainties the value We have discussed the airmass fits already, because
of χ2 is less important than the fact that further itera- theyarecrucialtotheiterativeprocessofmarkingnights
tions do not decrease it. This level of convergence takes as nonphotometric and converging on a final fit. We
about 3000 iterations. As an external check of conver- thereforefocushereonsomeexampleswhichareatypical
gence, we restart the fitter with the best parameters but but which provide insight into the process.
nomemoryofthegradient,whichforcesthefittertotest Figure 3 shows the airmass fits for the January 2001
anewdirectioninparameterspace;theresultwasalways CTIO run in R band. Three nights (shown in green,
consistent with the original solution. red, and blue) were well fit by a common run zeropoint
We checked for images with large ((cid:38) 0.05 mag) rms plus nightly airmass terms. In other words, the fitted
scatter in the residuals, even if the mean residuals were lines all converge to a point (the run zeropoint) at zero
acceptable. Afewimageswithhighskylevelsduetoap- airmass. The first night (shown in black) has similar
proaching dawn were identified this way. These images airmass slope but is offset by about 0.15 mag, i.e. it
appeared good to a casual inspection, e.g, the sky was could never converge at the common run zeropoint. Put
not very close to saturation. Although they contributed simply, exposurestakenonthisnightcontainabout15%
relatively little information to the survey due to their greater flux at any given airmass. Possible causes fall
high sky noise, the pre-ubercal judgment was to include into several categories:
thembecausetheyaddedsomething. However,theuber-
1. the camera collected 15% more photons from each
cal residuals were often highly non-Gaussian, hinting at
source on that night. There is no plausible phys-
sky-modeling problems or other subtle problems related
ical scenario enabling this. If the other nights
to the bright sky. Therefore, the judgment changed: the
on the run were photometric with airmass terms
small information contribution could be outweighed by
∼0.07, then even complete removal of the atmo-
other problems, and we decided to exclude these images
sphere would not suffice to increase throughput by
from the survey.
this much.
As mentioned above, asteroids and cosmic rays can
landontopofrealobjectsandcorrupttheirphotometry.
2. the CCD amplifier gain differed by 15% on that
Fortheseandotherreasons,welightlyclipthecatalogue
night. This is unexpected, but not as implausible
before fitting for the final solution. We clip at 6σ plus
as the first scenario.
the mean residual of the worst-fitting exposure (so that
theworst-fittingexposuredoesnothavemanyindividual 3. the data were changed after acquisition, for exam-
objects clipped simply due to some overall characteristic ple, by multiplying all images from this night by
7
Fig. 3.—AirmassfitsfortheJanuary2001CTIOruninRband. Fig. 4.—Thebest-fitairmasstermsfortheNovember2000Kitt
Threenights(showningreen,red,andblue)werewellfitbyacom- Peakrunarepuzzling. Seetextfordetails.
mon run zeropoint plus nightly airmass terms. One night (shown
inblack)hasamagnitudeoffsetwhichistoolargeandtooairmass- (which does not suggest overall extinction greater than
independent to be an atmospheric variation. This kind of change other runs), nor in the other filters observed on these
couldbecausedbyachangeinoverallscalingfactorsuchasCCD nights. Asecondhypothesismightbethatthefitisforc-
gain.
ingthenightshowningreenintounnaturalbehaviorbe-
1.15,orapplyingadifferentflatfieldthantheother causeallthebest-fitlinesmustconvergeatzeroairmass.
nights. There is no evidence in the image headers But freeing that night to float to its own zeropoint does
for this. not change the behavior shown, nor does freeing each
exposure change the pattern. This effect is seen only on
4. the photometry procedure is not robust, for exam-
the z band data from this run, so it could possibly be
plebeingseeing-dependent. However,theseeingon
related to the temperature dependence of the detector’s
thisnightisnotverydifferentfromtheothernights
z bandsensitivity. Thesedatawouldbefitbyascenario
onthisrun,andotherrunswithmuchlargerseeing
in which the detector warms slightly with airmass (not
variations do not show this behavior.
with time) due to the arrangement of (a presumably low
5. exposure times were uniformly 15% longer on this level of) liquid nitrogen in the dewar relative to the cold
night. This is implausible given the headers and finger when pointing at our field.
observing logs. We also explored some alternative hypotheses not in-
volving umodeled physical effects. We tested the local-
Only R data were taken on this night, so behavior in minimum hypothesis by perturbing the solution and
theotherwavelengths,ourcross-checkoffirstresort,can- starting from a variety of places in parameter space; the
notbechecked. Twoothernightsshowsimilarbehavior, fitter always returned to this solution with a few nights
but at a lower (4-5%) level. One of those nights con- of negative airmass coefficients. We also tested a hy-
tains observations in only one band, but the other con- pothesis in which too much of the z data were nonpho-
tainsobservationsinthreebands,andallthreebandsare tometric, thus providing too few constraints to arrive at
consistent. Because different bands are reduced totally a physical solution. We tested this by fixing the magni-
independently, this makes it very unlikely that a data tudesofobjectsappearingintheSDSScatalogueattheir
processing misstep is responsible, and more likely that SDSS magnitudes. This provided many hundreds of ab-
a gain change is responsible. Whatever the cause, the solute reference points to better constrain the solution,
practical impact is that these nights cannot be fit with butthenegativeairmasscoefficientsdidnotgoaway. We
the usual model of nightly airmass term and overall run conclude that an unmodeled physical effect is indeed at
zeropoint. In principle, they could be fit with a nightly work, even if it is not the specific dewar-related effect
airmass term plus nightly zeropoint, but for convenience described in the previous paragraph. Fortunately, the
we simply fit them with per-exposure zeropoints as if best-fit parameters of interest (flatfield corrections and
they were nonphotometric. relative throughputs of each exposure) do not change
In Figure 4, we show a second example of unexpected substantially depending on how we model this run, so
behavior of the airmass fits. The night shown in green is that we can adopt these parameters regardless of the
best fit by a negative airmass term. This is completely physical interpretation.
unphysical, so the natural conclusion is that the night Westressthattheseexamplesareinterestingprecisely
had varying amounts of clouds which happened to anti- because they are atypical. Most runs show no such un-
correlate with the airmass. However, if that night had expectedbehavior,yieldingfitssimilartotheuppermost
clouds, then the other nights on the run had even more threenightsinFigure3andresidualssimilartothelower
clouds,whichalsoimplausiblyconspiredtoyieldairmass panels of that figure. The atmospheric variations are of-
fits with small residuals. There is no indication of this tenmodeledtowithinafewmillimagnitudesoverawide
in the observing logs, nor in the best-fit run zeropoint range of airmass. Figure 5 shows histograms of the ex-
8
Fig. 5.—Histogramsoftheexposureresidualsineachfilter,for Fig. 6.—Runzeropoints,asdefinedinEquation1,foreachrun
exposures in photometric conditions. The R histogram is super- with at least one photometric night in a given filter (B, blue; V,
posedontheBVz histogramsforcomparison. green;R,red;z,black)atKittPeak(left)andCerroTololo(right).
posure residuals in each filter, for exposures in photo-
4.3. Flatfield Corrections
metric conditions (recall that the residual is identically
zeroforexposuresinnonphotometricconditions). These Figure 7 shows the flatfield corrections in magnitudes,
histograms indicate that a simple airmass term is suffi- averaged over all runs at Kitt Peak (left column) and
cient to model many hundreds of photometric exposures nearly all runs at Cerro Tololo (right column), in B, V,
to within ∼5 mmag. R, and z (top row to bottom row). This is a visualisa-
tion of the last term in Equation 1, so a larger positive
4.2. Run Zeropoints numbermeansthatastarinthesky-flattenedimagesap-
pearsfainterthanitwouldhaveiftheskyflatshadbeen
Figure6showstherunzeropointsforeachrunwithat
perfect. The variation from run to run is much smaller
leastonephotometricnightinagivenfilter. Onlyrelative
than the systematic differences between telescopes and
zeropointsaremeaningful,andonlywithinagivenfilter.
among filters, so we begin by discussing those patterns:
The most striking feature is that the four filters vary so
nearly in unison, especially at CTIO, despite the com-
1. TheKittPeakandCerroTololocorrectionsclearly
pletely independent fit in each filter. This, along with
form two different families. We stress that there is
the month-to-month stability, suggests that the varia-
nothinginthemodeltoenforceorevensuggestthat
tions reflect real long-term changes rather than noise.
thisshouldbethecase. Eachrunhasitsownsetof
(The referencing of the zeropoint to the center of CCD
flatfieldparameters,unconnectedtotheotherruns.
1 rather than to an average over the entire array causes
ThefactthattheflatfieldparametersfortheCerro
only(cid:46)0.02magshiftsinthisplot,whichweneglecthere-
Tololorunsaresimilartoeachotheranddissimilar
after.)
to those for the Kitt Peak runs is therefore due
Because the zeropoint appears on the right side of
entirely to the data rather than the model. This
Equation 1 rather than on the left, an increase in ze-
givesusconfidencethatthesolutionisrepresenting
ropoint implies lower pixel values for a given star (this
real systematic errors in the flatfielding.
is contrary to how observational astronomers usually
think about a zeropoint). Upward excursions in zero-
2. In BVR, the corrections are more or less continu-
point could be caused by decreases in the atmosphere-
ous across CCDs. Again, we stress that this is not
independent part of the throughput, higher CCD ampli-
required or encouraged by the model, and the fact
fier gain (i.e., lower pixel value per photoelectron in the
that data prefer it suggests that the data are re-
device), or similar factors. The variations do not appear
sponding to physical effects which operate mostly
tobecorrelatedwithrecoatingsoftheopticsatCTIO(A.
on Mosaic-wide scales. However, z band is not
Walker,privatecommunication). Wedohaveevidenceof
verycontiguousbecauseitisverysensitivetoCCD
undocumentedgainchangesonshortertimescales(§4.1),
thickness, whichisobviouslynotcontinuousatthe
so longer-term drifts in gain could be responsible for the
boundaries. Because there can be real discontinu-
coherent part of the time variation. However, because
ities, we retain the chip-by-chip model despite the
the variation at shorter wavelengths is larger than at
dominance of the larger-scale patterns.
longer wavelengths, this cannot be the entire explana-
tion. Wavelength-dependence suggests a physical cause, 3. At each observatory, B, V, R are similar to each
and the blueness of the effect rules out some physical other and very different from z band, which has
causes such as CCD temperature changes. We conclude by far the biggest corrections. Again, this was not
that there are long-term (∼2-year) variations in hard- suggested by the model, which is completely inde-
ware throughput due to as-yet unexplained causes. pendent in each filter.
9
the optical system through paths which are forbidden to
object photons), then dividing the raw data by the sky
flats will make objects near the center appear to be too
faint, which is the sign of the pattern shown here. This
is about a 5% or 0.05 mag effect from center to edge in
BVR, and a bit more than twice as large in z. In z band
at Kitt Peak an additional complication is the presence
in raw images of a ghost image of the pupil, covering
much of the array and with an amplitude ∼3% of the
sky. Mosaicz imagesarethereforereducedwithanextra
step, the IRAF task rmpupil which attempts to model
andremovethisadditiveeffectwhichisroughlycentered
on the array. Undersubtracting this feature will lead to
the same sign error shown here, centered on the array.
However, the ghost pupil hypothesis does not explain
why the corrections derived here are sizable even in the
bandswheretheghostpupilisnotevident. Furthermore,
theamplitudeofanyghostpupilimageundersubtraction
is likely to be negligible on the scale shown here, as only
a ∼3% error would result in the case of no subtraction
at all.
ThecentralpatternseeninBVRatKittPeakisabout
the same amplitude as the Jacobian correction, which is
also centrally symmetric. However, we rule out an in-
correct Jacobian correction as the cause based on two
lines of reasoning. First, the correction is applied the
samewayfortheCerroTololo,whichshowsnosuchcen-
tral pattern. Second, if ubercal were compensating for
a mistake in the input photometry, then the corrected
final images would contain the negative of this mistake.
Analysis of such images (§4.4 and §5) demonstrates that
this is not the case.
Theusualapplicationofskyflatsisknowntobewrong
for another reason (Stubbs et al. 2007). Sky flats rep-
resent one integral over wavelengths, and object fluxes
represent a different integral over wavelengths, but the
proper correction would involve dividing the integrands,
Fig. 7.— Flatfield corrections in magnitudes for Mosaic at Kitt nottheintegrals. Ifthisisthecauseofflatfieldingerrors,
Peak (left column) and Mosaic 2 at Cerro Tololo (right column), theubercalisperformingacorrectionforthemeancolors
inB,V,R,andz(toprowtobottomrow). Eachimageisamean of the objects in the input photometry. However, given
over many observing runs; the variation from run to run is much the Mosaic-wide scale and central symmetry of most of
smaller. Seetextforphysicalinterpretation.
the corrections derived here, we favor an optics-related
We attribute the differences between z and BVR to explanation for the bulk of these corrections.
physical factors. First, across the z band the CCD sen- For a given filter and observatory, the flatfield correc-
sitivity is a strong function of CCD thickness and wave- tionsarefairlystableovertime. Atagivencameraloca-
length. The discontinuous nature of the z band correc- tion, the standard deviation across runs is typically 4-5
tions is likely due to varying device thickness. Second, mmag, an order of magnitude smaller than the correc-
the larger amplitude of the radial pattern in z may be tions shown here. This consistency might suggest con-
due to the effective wavelength of the z filter changing straining the flatfield corrections even more strongly by
across the field, because it is an interference filter. solving for a single pattern per observatory and wave-
Although the hardware setups at Kitt Peak and Cerro length. However, runs closer in time seem to be some-
Tololoarequitesimilar, theyarenotidentical. Thetele- what more consistent with each other, suggesting that
scopes have different prime focus correctors, and some some of the time variation is real. Therefore, we con-
differences in the raw images produced by Mosaic and tinue to model each run separately.
Mosaic 2 are evident upon casual inspection, depending One striking example of variation with time is when
onwavelength. Thusitisnotsurprisingtoseetelescope- CCD3atCerroTololofailedandwasreplaced. Figure8
and wavelength-dependent systematics reflecting our in- shows the z correction over the whole field before and
ability to completely remove these effects in the image afterreplacement. CCD3hasclearlychanged,whilethe
processing. The fact that the data, unguided by the restofthefieldhasnotchangedsubstantially. Beforere-
model, produce the physically plausible patterns listed placement, the corrections for CCD 3 were just as stable
above lends confidence to the results. fromruntorunastheotherCCDsshownhere. TheBVR
The sign of the central pattern seen in the Kitt Peak corrections for this CCD also changed at the same time,
correctionscanbeinterpretedasfollows. Iftherearetoo butareofsmalleramplitude. Again,westressthatthese
many sky photons near the center (perhaps traversing patterns come from the data because the model knows
10
Fig. 8.— Flatfield corrections for Cerro Tolo in z before and
after the replacement of CCD 3 (third from left in bottom row).
AsshownbytheotherCCDs,theflatfieldcorrectionsaregenerally
stableovertimealthoughthemodelallowsindependentparameters
for each run. This one-time change of hardware clearly shows up
intheubercal-derivedcorrections.
nothing about the CCD replacement.
We can test the appropriateness of the polynomial
model by mapping the residuals back onto camera coor-
dinates. To build up a statistically meaningful map, we
averageoverallrunsinagivenobservatory/filtercombi-
nation. This is shown in Figure 9 for Mosaic (left) and
Mosaic2(right)forthefiltersintheusualorder. AtKitt
Peak,weseefaintechoesoftheghostpupil,mostclearly
in B, where it is barely visible in the raw images at the
< 1% level but not modeled out in the image reduction
stage. At Cerro Tololo, three ghost pupils are evident in
most filters—a large one at the center and two smaller
ones at lower right—and some additional process must
be at work in V. Despite the clarity of these patterns,
their amplitude is small, about an order of magnitude
smaller than the patterns captured by the polynomial Fig. 9.— Map of the residuals (in magnitudes) back onto the
camera coordinates at Kitt Peak (left) and Cerro Tololo (right)
model. Their spatial frequencies are also high, so that
for, from top to bottom, B, V, R, and z. For example, at Kitt
they will be washed out by dithering. A way to capture PeakinB,objectsinthecenterofthecamerafieldareobservedto
this high spatial frequency information without adding besystematicallybrighterthanmodeled,revealinganinadequacy
multiple parameters would be to add a smoothed ver- (at the 15 mmag level) of the second-order polynomial flatfield
correctionmodel.
sionoftheappropriatemapbackintotheflatfieldmodel
as an extra term, perhaps with a free parameter for its
To implement the corrections, we modified our stack-
amplitude on each image or on each run.
ing software (dlscombine, Wittman et al., in prepara-
A few other features in these maps are worth noting.
tion) to perform the flatfield corrections on the fly as it
Badcolumnsarevisibleinplaces, indicatingthatthein-
stacks. Wealsoconfiguredittousetheextinctioncorrec-
put photometry could have been cleaned better. More
tionsderivedfromubercal,ratherthanfromourprevious
interesting is the edge behavior. In BVR, objects near
method of estimating extinction corrections.
inside edges are consistently fainter than expected (the
We tested the corrections by having dlscombine write
white bands are not depictions of the CCD gaps), but
outcorrectedpostage-stampimagesforeachobservation
whatevercausesthisdoesnotaffectphotometrynearthe
of each ubercal object. This is an end-to-end test in-
outside edges. This inside/outside distinction points to
volving many interrelated pieces, such as applying the
a physical cause rather than a SExtractor issue. In z
correct Jacobian determinant to the ubercal input cata-
the sign of the effect reverses for the long inside edges,
loguessothatthecombinationofflatfieldcorrectionsand
and objects near the outside edges are now affected as
repixelisation produces uniform photometry. We pho-
well. Thesearesmalleffects,butconsistentacrossobser-
tometered the postage stamps using the same procedure
vatories, so it is presumably related to the CCDs, read-
as for the ubercal input catalogues, and subtracted the
outelectronics,and/orimageprocessingsteps,whichare
model true magnitude for each object. The resulting
largely the same at both observatories. Future surveys
residuals should have zero mean, with rms similar to the
wishing to obtain very high precision should examine
rmsresidualsfoundintheubercalfit. Moreimportantly,
spatial patterns in the residuals and use them to refine
anysubsetofresidualsshouldhavezeromean,sowecan
their models.
subdividethedatabyfield,subfield,CCD,etc,andcheck
forsystematics. Noneoftheseattributesshowanysignif-
4.4. Closing the loop
icant trends. For example, Figure 10 shows histograms