Real-time system for studies of the effects of acoustic feedback on animal vocalizations.

1.1 MB·English

by  Skocik, Mike

METHODSARTICLE
NEURAL CIRCUITS published:07January2013
doi:10.3389/fncir.2012.00111
Real-time system for studies of the effects of acoustic
feedback on animal vocalizations
MikeSkocik1 andAlexayKozhevnikov1,2*
1DepartmentofPhysics,PennsylvaniaStateUniversity,UniversityPark,PA,USA
2DepartmentofPsychology,PennsylvaniaStateUniversity,UniversityPark,PA,USA
Editedby: Studiesofbehavioralandneuralresponsestodistortedauditoryfeedback(DAF)canhelp
AhmedElHady,MaxPlanck shed light on the neural mechanisms of animal vocalizations. We describe an apparatus
InstituteforDynamicsandSelf for generating real-time acoustic feedback. The system can very rapidly detect acoustic
Organization,Germany
featuresinasongandoutputacousticsignalsifthedetected featuresmatchthe desired
Reviewedby:
acoustictemplate. The systemuses spectrogram-baseddetectionof acousticelements.
AhmedElHady,MaxPlanck
InstituteforDynamicsandSelf It is low-cost and can be programmed for a variety of behavioral experiments requiring
Organization,Germany acousticfeedbackorneuralstimulation.Weusethesystemtostudytheeffectsofacoustic
NoahCowan,JohnsHopkins feedbackonbirds’vocalizationsanddemonstratethatsuchanacousticfeedbackcancause
University,USA
bothimmediateandlong-termchangestobirds’songs.
*Correspondence:
AlexayKozhevnikov,Departmentof
Keywords: acoustic feedback, animal vocalizations, behavioral neuroscience, sensory feedback, real-time data
Physics,PennsylvaniaState
processing
University,UniversityPark,PA,USA.
DepartmentofPsychology,
PennsylvaniaStateUniversity,
UniversityPark,PA,USA.
e-mail:[email protected]
INTRODUCTION of the song and the mechanisms of song learning (Brainard
Distortedauditoryfeedback(DAF)isusedforassessingtheeffects and Doupe, 2000). To study the questions about the effects
of auditory input on vocal production. Presenting DAF and of time-localized DAF on birdsong, it is important to be able
assessing its effects on the song and on the neural activity have to deliver DAF with high temporal precision in relation to
been used in songbirds to study the mechanisms of song pro- vocalization. To do this, it is necessary to rapidly and reliably
duction and learning (Leonardo and Konishi, 1999; Sakata and detect the specific acoustic elements of the bird’s song and,
Brainard,2006;AndalmanandFee,2009;KellerandHahnloser, after detection of the acoustic element, generate an acoustic
2009;TschidaandMooney,2012).Humanspeechissensitiveto output.
certain typesofDAF(Lee, 1950;HoudeandJordan,1998),and It is a challenging technical task for an acoustic feedback
DAFisusedtostudyspeechmechanisms.Itisoftendesirableto system to be real-time. Real-time performance is most easily
havereal-timeDAF,i.e.,torapidly(inafewmillisecondsorfaster) achievedwithanalogsystems(CynxandVonRad,2001),butdig-
detecttheoccurrenceofspecificacousticelementsinvocalization italsystems offersignificantadvantagesintermsofconvenience
andpresentanauditorystimulusoncethetargetacousticelement and flexibility. However, the advantages of a digital system are
isdetected. accompanied by the difficulties ofmaking a digital system have
Inthispaper,wedescribeanautomated systemforreal-time small and constant processing delays. The system has to per-
DAF and demonstrate its use to study both the immediate and form analog-to-digital conversion, fast analysis of the recently
the long-term effects of DAF on the song of Bengalese finches. acquired data and digital-to-analogconversion, and these oper-
Thesystemusesopen-sourcesoftwareand,therefore,isextremely ations have to take place with reliable timing and concurrently
flexibleandcustomizablebytheuser.Ithasasignificantlylower with savingtheacquired data.Custom-madeDAFsystems have
costthancommercialsystems. been developed and used in behavioral studies (Leonardo and
Songbirds use auditory feedback to learn to sing when they Konishi, 1999; Kao et al., 2005), but their real-time processing
are young and to maintain their songs in adulthood (Konishi, characteristicshavenotbeenreported.Oftentimes,custom-made
1965; Brainard and Doupe, 2000). Long-term exposure to DAF systems have significant and not well-controlled delays, espe-
hasbeenshowntocausesongdegradationinsongbirds(Okanoya ciallyforsystemsbasedonPC’srunningWindows.Commercial
and Yamaguchi, 1997; Woolley and Rubel, 1997; Leonardo and systems for real-time acoustic processing are available but
Konishi, 1999). Some bird species’ songs exhibit immediate areexpensive.
sensitivity to acoustic input. For these birds, DAF can have We developed a real-time DAF system based on a PC run-
an immediate effect on the timing and acoustic structure of ning Linux and the Real-Time eXperiment Interface (RTXI)
the song (Sakata and Brainard, 2006). Analyzing the effects of software (Lin et al., 2010) and a National Instruments A/D
DAF can yield new understanding of the neural organization card. The system is low-cost (the cost is only the cost of the
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article111|1
SkocikandKozhevnikov Real-timeacousticfeedbacksystem
hardware, the software is free). The system is capable of A/D In triggered mode, the system does real-time processing of
bandwidthof over30kHz with real-time processing of acoustic auditorydata.InFigure1(bottom),weshowtheprocessingdone
signals. for recognizing the song syllable of a Bengalese finch. The FFT
of the past 256 data points (∼8.4ms) is computed every 1ms
METHODS and stored in an FFT circular buffer. Every 1ms, the spectro-
A PC with an Intel i7 six-core processor (2.66GHz) and 4GB gram of the most recent 40ms of the input signal is obtained
or RAM running Ubuntu Linux 2.6.29.4-rtai with RTXI ver- from the FFT circular buffer. A correlation coefficient between
sion 1.1.2 and a National Instruments PCIe-6251 A/D card is theinputsignalspectrogramandthespectrogramofthetemplate
used. The A/D card receives audio input from a microphone iscomputed.Ifthecorrelationcoefficientexceedsthethreshold,
(AudioTechnika PRO-44, used with Behringer Shark DSP110 thesystemdetectstheoccurrenceofthetargetsongsyllable,and
microphone amplifier). The output is sent to a speaker ampli- acousticfeedbackcanbegenerated,orfurtherprocessingcanbe
fier(SLA-1,AppliedResearchandTechnology);theoutputofthe done.Whilegeneratingacousticfeedback,thesystemkeepsgoing
amplifierisconnectedtoaspeaker. through all of the above steps, but is disallowed from register-
TheDataRecordersoftwarewithinRTXIiscustommodified. ing another detection to prevent it from triggering on its own
ThesimplifieddiagramofsignalprocessingisshowninFigure1. output.
Thesystemhastwomodesofoperation—anon-triggered(idle) The presence of the data circular buffer allows very fast
modeandatriggered(active)mode.Atthecoreofthemodified access to chunks of the most recent data for processing with-
softwareisthe circularbufferthattakesdatapoints one-by-one out affecting the timing of the data acquisition process. The
from the data acquisition engine once they become available. FFT circular buffer also allows extremely fast computations of
In the non-triggered mode [Figure1 (top)], the system contin- thespectrogramsofthesound(computingthespectrogramisa
uously (every 1ms) computes the rms of the last 10ms of the computationally-intensive task). This enablesthe system to rec-
inputsignal. If the signal rms exceeds the threshold, the system ognizecomplexvocalelementsbasedontheirspectrogram(e.g.,
isswitchedintotriggeredmode. frequencysweeps)withoutcompromisingthetiming.Whiletrig-
gered, every 1s, the system computes the rms of the previous
200ms of the input signal to check if the acoustic input is still
present. If the rms is below a threshold (no signal), the system
goesintotheidlemode.Whiletriggered, thesystemcontinually
savesallthedataacquiredinaseparatearrayandsavesthedata
to the harddriveonceitisswitched backto idlemode. Amore
detaileddescriptionofthissystem,alongwiththesourcecode,is
availableathttp://www.phys.psu.edu/∼akozhevn/ac_feedback/.
RESULTS
We tested the performance of our DAF system in several tasks
whichareoftenneededinbehavioralexperimentsusingacoustic
feedback.Wealsousedthesystemtoassesstheeffectsofacoustic
feedback on the song of Bengalese finch. All animal procedures
werecarriedoutinaccordancewiththelocallyapprovedIACUC
protocol.
DELAYBETWEENINPUTANDOUTPUTTEST
Asimpletaskisgeneratingacousticfeedbackwhentheinputlevel
exceedsacertainthreshold.Althoughthistaskmaybetoosimple
formostbehavioralexperiments,thedelayinthesystembetween
detectingthecrossingofthethresholdandproducingtheoutput
isausefulfigureforindicatinghowfastthesystemcanbewhen
itissolelyconvertingA/DandD/Aandsavingdatawithoutany
complexdataprocessing.
Thesystemwasprogrammedsothat,oncetheinputexceeded
FIGURE1|Blockdiagramoftheacousticfeedbacksystem.Whennot a fixed threshold, theacquiredinputsignal wassentto theD/A
triggered(top),thesystemcomputesthermsoftheinputsignal.When
outputwithnoextraprocessing.Asquarewavewiththeampli-
thermsexceedsthethreshold,thesystemistriggered.Whentriggered
tudeexceedingthethreshold wasappliedtotheinput;thedelay
(bottom),thesystemcomputesthespectrogramofthemostrecent20ms
ofsignalandcomputesthecorrelationcoefficientofthisspectrogramwith between the input and the output was measured with the digi-
thespectrogramofthetemplatesound(e.g.,songsyllable).Thetemplate taloscilloscope.Themeasureddelaybetweentheoutputandthe
soundisdetectedwhenthecorrelationcoefficientexceedsathreshold inputwas27±9µs(mean±SD,min=9µs,max=43µs).The
value;inthiscase,acousticfeedbackcanbegenerated.Boththeinputand
sample rate was 30.3kHz, so the observed delays corresponded
theacousticoutputaresavedtothecomputerharddrive.
to a delay of 1 data point between the input and the output.
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article111|2
SkocikandKozhevnikov Real-timeacousticfeedbacksystem
Theobservedvariationsofthedelayareduetothedifferencein
timing between the external input signal and the timing of the
A/Devents.Inallcases,however,thedelaybetweentheinputand
theoutputdoesnotexceed1datapoint.Therefore,thesystemhas
real-timecapability.
DETECTIONOFSPECIFICVOCALELEMENTSINTHEBIRD’SSONG
A typical task in experiments using acoustic feedback is detec-
tion of a certain “template” sound. The template can be either
a sound of a certain frequency or a more complex combina-
tion of frequencies, frequency sweeps, etc. Once the template is
detected, the acoustic feedback can be played back to the ani-
mal.Thistaskiscomputationallyintensivebecauseoneneedsto
compute the characteristics of the recently acquired input sig-
nal, then comparethese with the characteristics ofthe template FIGURE2|Top:spectrogramofthesongofaBengalesefinchandthe
timesofoccurrenceofoneofthesongsyllables.Thesystemwas
and, if the input is sufficiently similar to the template, decide
programmedtoonlydetecttheoccurrencesofthetargetsyllableinreal
that the detection has occurred and generate acoustic output.
time,noacousticfeedbackwasgenerated.Thedetectiontimesareshown
The computation has to be done fast enough to enable real- asverticalredlines.Bottom:thesystemisdetectingthetargetsyllables
time performance and not interfere with the data acquisition (verticalredlines)andisgeneratingacousticfeedbackafterdetection.The
process. acousticfeedbackwaveformisshownbelow.Thefeedbacksignalisoneof
thebirdsongsyllables;theacousticfeedbackpickupbythemicrophoneis
Commontechniquesthathavebeenusedfordetectingacous-
visibleonthespectrogram.Thezoomed-inspectrogramofthetemplateis
tic elements are spectrogram-based techniques (Leonardo and
shownontheright.
Fee, 2005) and feature-based techniques (Tchernichovski et al.,
2000). In a spectrogram-based approach, the spectrogram of
the recently acquired signal is computed and compared to the
template spectrogram. A common way to accomplish this is to tothebird.Theverticalredlinesindicatethedetection timesof
compute the correlation coefficient between the two spectro- the target syllable. The zoomed-in spectrogram of the template
grams.Detectionofthetemplatesoundoccursifthecorrelation is shown on the right. The template contains part of the inter-
coefficientexceedsathresholdvalue. syllableintervalandthefirst20msofthetargetsyllable,sotheend
Wetestedtheperformanceofthesystemfordetectionofspe- ofthetemplate(detectiontime)isapproximatelyinthemiddleof
cific syllables in the song of a Bengalese finch. The Bengalese the40-mslongsyllable.
finchsongconsistsofasequenceofsyllablesseparatedbysilences Performance of the system was checked by comparing the
(inter-syllablegaps)(Figure2).Theacousticstructureofthesong results of automatic detections of the system with the manual
syllables is fairly stable; the mainsource ofvariability fromone identification of the target song syllables carried out by off-line
songtoanotheristhesequenceofsyllablesineachsong(Honda examinationofthespectrograms.Outof659targetsyllables,610
andOkanoya,1999). werecorrectlydetectedand49weremissed.Therewerezerofalse
Todetectaspecificsongsyllable,thesystemcontinuouslycom- positives. Thus, the system shows robust performance with the
putes the correlationcoefficient ofthe spectrogram ofthe most real-timesyllablerecognitiontask:over92%ofthetargetsyllables
recent20mssegmentoftheacquiredsignalwiththespectrogram werecorrectlyidentified.
ofa20-mssyllabletemplate(seeMethods,Figure1).Thetarget This demonstrates that the system is capable of real-time
syllableisdetectedbythesystemwhenthecorrelationcoefficient detection of target syllables in the song. Note that the sylla-
exceedsthethresholdvalueof0.8.Thevalueofthethresholdwas blesoccurringafterthetargetsyllablesinFigure2arefrequency
chosenbyexaminingthetargetsyllabledetectionsbytheDAFsys- sweeps that overlap with the template’s frequencies. The system
teminasetofabout20songsandcomparingthedetectedsyllable discriminatesthemfromthetargetsyllablesbecausetheyhavea
occurrences with the actual occurrences of the target syllables different frequencyprofile. Suchdiscrimination is anadvantage
determined by visual examination of the song spectrograms. If ofthespectrogram-baseddetection;thiswouldnotbepossibleif
the threshold is set too high, the probabilityof missing the tar- onlyinstantaneousfrequenciesweredetected.
getsyllableincreases.Settingthethresholdtoolowincreasesthe Additionally, we tested the system on the detection of sylla-
probability of false positive detections. After the syllable detec- blesinthesongofazebrafinch—anotherbirdspecies.Weused
tion, acoustic feedback (either white noise or the song syllable) our dataset of zebra finch songs with known syllable sequences
canbeplayedbacktothebird. obtainedinapreviousstudy(KozhevnikovandFee,2007).Zebra
Typical performance of the system on the real-time syllable finchsongswereplayedbackthroughthespeaker,andtheresults
recognitiontaskisshowninFigure2.Thetopspectrogramshows ofthereal-timedetectionbyourDAFsystemwerecomparedto
“detection only”mode—thesystemdetects thetargetsyllablein theknownoccurrencesofthetargetsyllable.
real time, but no playback is generated. The bottom spectro- Asmallsubsetofsongs(10songs) wasused asatest set: the
gramshowsdetection and playbackgeneration—after detecting threshold value for the syllable detection was adjusted to opti-
the target syllable, the system plays back another song syllable mize the percentage of correctly detected syllables in this small
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article111|3
SkocikandKozhevnikov Real-timeacousticfeedbacksystem
test set. After this, the threshold kept was fixed, and the per-
formance of the system was tested on the whole dataset (about
100 songs). Out of 756 target syllables in the dataset, 728 were
correctly detected, 28 were missed; there were 4 false positives.
The system correctly detected over 96% of the target syllables
in the dataset; the probability of a false positive detection was
lessthan1%.
EFFECTSOFAUDITORYFEEDBACKONTHETIMINGOFTHEBIRDSONG
Auditoryfeedbackhasbeenshowntohaveimmediateeffectson
someanimalvocalizations.ForBengalesefinches,DAFhasbeen
shown to affect the timing of song syllables. DAF played after
the song syllable increases the time interval between that sylla-
bleandthenextsyllableinthesong(SakataandBrainard,2006).
We tested whether our feedback system is effective in causing
real-time changes to the Bengalese finch song. The system was
programmed to detect one of the song syllables and, once the
syllable was detected, to play back the same song syllable with
FIGURE3|DAFincreasesthedurationofthetimeintervalbetween
a probability of 0.05. This ensured that the feedback was suffi-
Bengalesefinchsongsyllables.Shownabovearethehistogramsofthe
cientlysparsesoalmostinallcasestherewasonlyoneplayback timeintervalsbetweentwosubsequentsyllablesinthesonginthe
duringeachsong.Thefeedbackandcontroltrialswererandomly presenceofDAF(blue)andwithoutDAF(red).Themeansare:
interleaved. This simplified the analysis of the syllable timing (cid:2)tmean=74.8ms(control,N=637syllables)and(cid:2)tmean=75.7ms
(feedback,N=97syllables),thedifferenceisstatisticallysignificant
andeliminatedanyconfoundingeffectsfromplaybacksbeingtoo
(p=0.001,two-wayKolmogorov–Smirnovtest).
closetooneanother.Thedelaybetweenthesyllablesungbythe
birdandthesyllableplaybackwas40ms.
Theplaybackcausessomepickupontheinputchannel,which
can cause difficulties in precise determination of the timing of can cause gradual change of the song (Leonardo and Konishi,
thesyllablethatisoccurringduringtheplayback.Therefore,the 1999; Warren et al., 2011). We tested out system on the task of
timeintervalbetweenthetargetsyllableandthefollowingsylla- causing long-term changes of the frequency of one of the song
ble(whichispartiallyoverlappedwithplayback)wascomputedas syllables.
onehalfofthedifferencebetweenthedetectiontimeofthetarget The system is programmed to detect the fundamental fre-
syllableandthedetectiontimeofthesecondsyllableafterthetar- quencyofoneofthesongsyllables.Afterthetargetsongsyllable
getsyllable.Thesameprocedurewasperformedincontroltrials isdetected,thetemporalprofileofthepitch(definedasthelargest
to ensureconsistency indataanalysis.Sincethedistributionsof peakintheFFTofthelatest256points)wascomputed.Thelow-
timeintervalsmaynotbeGaussian,weuseanon-parametricsta- estvalueinthepitchprofileinthetimewindowbetween 3and
tistical test—two-way Kolmogorov–Smirnov test—to assess the 12msafterthedetectiontimewastakentobethepitchofthesyl-
statisticalsignificanceofDAFeffectsonthesongtiming. lable. The feedback(white noise) isconditional onthe detected
Figure3showsthedistributionsofthetimeintervalsbetween pitchofthesongsyllable.Forexample,thefeedbackcanbegener-
the target syllable and the following syllable when the feedback atedifthedetectedsyllablepitchissmallerthanathresholdvalue.
is present (bluehistogram) and when there is no feedback (red Continuousexposuretosuchfeedbackhasbeenshowntocause
histogram). The widths of the distributions are due to the nat- the birdto graduallyshift themeanpitch ofthesyllableso that
ural variability of the song timing. In the presence of feedback, the feedbackisgenerated lessoften—the birdadaptsitssongto
thetimeintervalsbetween thesyllablesbecomelonger.Without avoidhearingDAF(Warrenetal.,2011).
DAF,themeanintervalis74.8ms(N =637syllables);inthepres- WetestedwhethersuchconditionalDAFcouldshiftthemean
ence of DAF, the mean interval is 75.7ms (N =97 syllables). syllable pitch in both directions. Figure4 shows the long-term
Although the change of the mean duration is small compared effectsofDAFwhichisconditionalonthesyllablepitch.During
to the widths of the distributions, the effect is highly statisti- days1–6,thefeedbackwasplayedbackifthepitchwaslessthan
callysignificant(p=0.001,two-wayKolmogorov–Smirnovtest). 3530Hz; during days7–12, the feedback wasplayed back if the
Theobservedlengtheningofthetimeintervalbetweenthesong pitchwasgreaterthan3530Hz;duringdays13–18,thefeedback
syllables is consistent with previous observations (Sakata and wasplayedbackifthepitchwaslessthan3510Hz.
Brainard, 2006). Thus, our acoustic feedback has an immedi- Ourfeedbackiseffectiveincausinggradualchangesinthesyl-
ate effect on the song: DAF immediately and reversibly affects lablepitch. PlayingbackDAFwhenthe frequencyislowerthan
songtiming. thethresholdvaluecausesanupwarddriftofthemeanpitch(days
1–6 and 13–18). Presenting feedback when the pitch is higher
LONG-TERMEFFECTSOFDAFONACOUSTICSTRUCTUREOFTHESONG thanthethresholdcausesadownwarddriftinpitch(days7–12).
DAFhasbeenshowntocauselong-termchangestoanimalvocal- This shows that the system is suitable for studies of long-term
izations.Forsongbirds,prolongedrepeatedpresentationofDAF effectsofDAFonanimalvocalizations.
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article111|4
SkocikandKozhevnikov Real-timeacousticfeedbacksystem
dissimilar to other vocal elements. To achieve optimum perfor-
mance,adjustmentstothethresholdordetectionalgorithmmay
be needed; thus, it is important to have a highly customizable
system.
It is worth mentioning that, when the song syllables are
detected, dataprocessingisnotatime-limiting step, andsignif-
icantlymorecomplexprocessingcanbedonewithoutdecreasing
theA/Drate.Wetestedthesystemwithlongertemplates(60and
100ms); they did not affect performance. We also tested the
simultaneousdetectionoftwotemplates,sothat,every1ms,the
systemcomputedthecorrelationcoefficientofthesoundspectro-
gramwithtwotemplatespectrograms,andthatalsodidnotaffect
theA/Drate.Foratemplate60mslong,thecomputationofthe
correlation coefficient takes 16µs; this time scales linearly with
thelengthofthetemplate.ThecomputationoftheFFT(tofillthe
columninthespectrogrambuffer)takes7–8µs.FFTandcorre-
lationcoefficientcomputationsaretheslowestsignalprocessing
FIGURE4|ProlongedexposureofthebirdtoDAFcausesgradual
changesinthesong.Themeanpitchofoneofthetargetsongsyllablewas steps; allotherstepscombinedtakelessthan1µs.Thus—Ifthe
manipulatedbyfeedbackconditionalonpitch.Eachdatapointisthepitchof spectrogram update rate is kept at 1ms—the system should be
thetargetsyllableaveragedoveralltherenditionsofthetargetsyllablesung capableofsimultaneouslydetectingofover10differentsongsyl-
bythebirdonagivenday.Thenumberofrenditionsvariesfromdaytoday
lables.Therefore,fairlycomplexreal-timeanalysisanddetection
(mean=208,min=126,max=288).Errorbarsares.e.m.Dashedlinesare
of multiple vocal elements can be done without compromising
thevaluesofthethreshold.Arrowsindicatethedirectioninwhichthesyllable
frequencywasexpectedtochangeinresponsetoDAF.Largererrorbarson thespeedofthesystem.
day12aremostlyduetothesmallestnumberoftargetsyllablerenditions Thesystemisusuallyusedwith1outputchanneland2input
(n=126)sungonthatday. channels(oneinputchannelforacousticinputandonechannel
forrecordingtheactualoutputoftheA/Dcard).Itispossibleto
DISCUSSION increasethenumberofinputchannels. Thisway,onecoulduse
The described real-time acoustic feedback system is a versatile thedatarecordingcapabilityofthesystemtocollectphysiological
tool for studies of the effects of auditory feedback on ongoing data(e.g.,EEGorneural)duringacousticfeedbackexperiments.
animal vocalization.Theadvantageofthe system isthe flexibil- However,theprocesslimitingthespeedofthesystemappearsto
ity of the processing than can be realized. The circular buffer be reading the data from the A/D card and sending the data to
allows real-time acquisition of arbitrary-length segments of the the D/A. Thus,increasing thenumber ofchannels will slowthe
most recent data without affecting the timing of data acquisi- system down and decrease the A/D rate. We tested the perfor-
tion.Inaddition,havingaseparateFFTbufferfacilitatesreal-time manceofthesystemwith3inputchannelsand1outputchannel.
spectralprocessingofacquiredsignals.Thisfeatureenablesavery Toachievestableoperation,theA/Dratehadtobedecreasedto
quick creation of the spectrograms of long (tens or even hun- 20kHz.Despitethisdecrease,thisisanacceptablerateformany
dreds of milliseconds) segments of signals at a high rate (the experimentswhereacousticandelectrophysiologicaldatahaveto
spectrogramisupdatedevery1ms). becollected.
Thiscapabilityisveryusefulforthedetectionofcomplexvocal Finally, the real-time processing capabilities of the system
signals. Often, itisnotjustacertain frequencythatneedsto be couldbeusedforneuralfeedbackexperiments.Thespectrogram-
detected, butratheracertainspectrogram pattern,likemultiple basedsignalprocessingcapabilitycanbeusefulforthedetection
frequenciesorthefrequencysweepsfrequentlyseeninbirdsong ofneuraloscillations.Theoutputcanbeusedfortargetedmicros-
syllables. Sincethe samefrequency canoccurinmany syllables, timulation. Thedescribed auditoryfeedbacksystem isaflexible
it is the whole pattern of the spectrogram that allows real-time low-costtoolforbehavioralneuroscienceresearch.
detectionofthesyllable.Oursystemisverywell-suitedforrapid
spectrogram-baseddetectionofacousticelements. ACKNOWLEDGMENTS
The performance of the system will vary depending on the The authors wish to acknowledge Ryan Wendt for his contri-
typeofanimalvocalizationandthenatureoftheacousticelement bution to the initial stages of the acoustic system development
beingdetected fortwomainreasons.First,thereisalwaysanat- andJonathanBettencourtandJohnWhitefortheirexpertadvice
uralvariabilityinthe acousticstructure ofavocalelement, and on RTXI capabilities and installation. This work was supported
thedegree ofthisvariabilitymaybedifferent fordifferentvocal by NSF Grant 0827731 and the NSF Research Experience for
elements;thiswillaffectthereliabilityofdetection.Forexample, UndergraduatesprogramatthePennsylvaniaStateUniversity.
a birdsong syllable canpossess a more orless stereotyped spec-
trogram;thedetectionwillbeeasierforamorestereotypicalsong SUPPLEMENTARYMATERIAL
syllable.Second,agivenvocalelementcanbemoreorlesssimilar TheSupplementaryMaterialforthisarticlecanbefoundonline
in its acoustic structure to other vocal elements; reliable detec- athttp://www.frontiersin.org/Neural_Circuits/10.3389/fncir.2012
tionofatargetvocalelementwillbeeasierifitisspectrallymore 00111/abstract
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article111|5
SkocikandKozhevnikov Real-timeacousticfeedbacksystem
REFERENCES avianbasalganglia–forebraincir- interface for biological control Woolley, S. M. N., and Rubel, E. W.
Andalman, A. S., and Fee, M. S. cuit to real-time modulation of applications,” in 2010 Annual (1997).BengalesefinchesLonchura
(2009). A basal ganglia-forebrain song.Nature433,638–643. InternationalConferenceoftheIEEE striata domestica depend upon
circuit in the songbird biases Keller,G.B.,andHahnloser,R.H.R. EngineeringinMedicineandBiology auditory feedback for the mainte-
motoroutputtoavoidvocalerrors. (2009).Neuralprocessing of audi- Society (EMBC) (Buenos Aires, nanceofadultsong.J.Neurosci.17,
Proc. Natl. Acad. Sci. U.S.A. 106, tory feedback during vocal prac- Argentina),4160–4163. 6380–6390.
12518–12523. tice in a songbird. Nature 457, Okanoya, K., and Yamaguchi, A.
Brainard, M. S., and Doupe, A. J. 187–190. (1997). Adult Bengalese finches Conflict of Interest Statement: The
(2000). Auditory feedback in Konishi, M. (1965). The role of (Lonchura striata var. domestica) authors declare that the research
learningandmaintenance ofvocal auditory feedback in the control requirereal-timeauditoryfeedback was conducted in the absence of any
behaviour. Nat. Rev. Neurosci. 1, of vocalizations in the white- to produce normal song syntax. commercial or financial relationships
31–40. crowned sparrow. Z. Tierpsychol. J.Neurobiol.33,343–356. thatcouldbeconstruedasapotential
Cynx, J., and Von Rad, U. (2001). 22,770–783. Sakata,J.T.,andBrainard,M.S.(2006). conflictofinterest.
Immediateandtransitoryeffectsof Kozhevnikov, A. A., and Fee, M. S. Real-timecontributionsofauditory
delayed auditory feedback on bird (2007). Singing-related activity of feedbacktoavianvocalmotorcon- Received: 31 July 2012; accepted: 08
songproduction.Anim.Behav.62, identified HVC neurons in the trol.J.Neurosci.26,9619–9628. December 2012; published online: 07
305–312. Zebra Finch. J. Neurophysiol. 97, Tchernichovski,O.,Nottebohm,F.,Ho, January2013.
Honda, E., and Okanoya, K. (1999). 4271–4283. C.E.,Pesaran,B.,andMitra,P.P. Citation:SkocikMandKozhevnikovA
Acousticalandsyntactical compar- Lee, B. S. (1950). Effects of delayed (2000). A procedure for an auto- (2013) Real-time system for studies of
isons between songs of the white- speechfeedback.J.Acoust.Soc.Am. mated measurement of song simi- the effects of acoustic feedback on ani-
backed munia (Lonchura striata) 22,824–826. larity.Anim.Behav.59,1167–1176. malvocalizations.Front.NeuralCircuits
and its domesticated strain, the Leonardo, A., and Fee, M. S. (2005). Tschida, K., and Mooney, R. (2012). 6:111.doi:10.3389/fncir.2012.00111
Bengalese finch (Lonchura striata Ensemblecodingofvocalcontrolin The role of auditory feedback in Copyright © 2013 Skocik and
var. domestica). Zoolog. Sci. 16, birdsong.J.Neurosci.25,652–661. vocal learning and maintenance. Kozhevnikov. This is an open-access
319–326. Leonardo,A.,andKonishi,M.(1999). Curr.Opin.Neurobiol.22,320–327. articledistributedunderthetermsofthe
Houde, J. F., and Jordan, M. I. Decrystallizationof adultbirdsong Warren, T. L., Tumer, E. C., CreativeCommonsAttributionLicense,
(1998). Sensorimotor adaptation by perturbation of auditory feed- Charlesworth, J. D., and Brainard, which permits use, distribution and
inspeech production. Science 279, back.Nature399,466–470. M. S. (2011). Mechanisms and reproduction in otherforums,provided
1213–1216. Lin, R. J., Bettencourt, J., White, J. time course of vocal learning and theoriginalauthorsandsourcearecred-
Kao,M.H.,Doupe,A.J.,andBrainard, A., Christini,D.J.,and Butera,R. consolidationintheadultsongbird. itedandsubjecttoanycopyrightnotices
M. S. (2005). Contributions of an J. (2010). “Real-time experiment J.Neurophysiol.106,1806–1821. concerninganythird-partygraphicsetc.
FrontiersinNeuralCircuits www.frontiersin.org January2013|Volume6|Article111|6