Leif Andersen et al.

28 Pages·2017·16.43 MB·English

Super 8 Languages for Making Movies
(A Functional Pearl)
LEIFANDERSEN,STEPHENCHANG,MATTHIASFELLEISEN,PLT@NortheasternUniversity
TheRacketdoctrinetellsdeveloperstocreatelanguages(aslibraries)tonarrowthegapbetweentheterminology
ofaproblemdomainandgeneralprogrammingconstructs.Thispearlillustratesthisdoctrinewiththecreation
ofarelativelysimpledomain-specificlanguageforeditingvideos. Toproducethevideoproceedingsofa
conference,forexample,videoprofessionalstraditionallyuse“non-linear”GUIeditorstomanuallyediteach
talk,despitetherepetitivenatureoftheprocess.Asitturnsout,thetaskofvideoeditingnaturallysplitsinto
adeclarativephaseandanimperativerenderingphaseattheend. Henceitisnaturaltocreateafunctional-
declarative language for the first phase, which reduces a lot of manual labor. The implementation of this
user-facingDSL,dubbedVideo,utilizesasecond,internalDSLtoimplementthesecondphase,whichisan
interfacetoageneral,low-levelClibrary.Finally,weinjecttypecheckingintoourVideolanguageviaanother
DSLthatsupportsprogramminginthelanguageoftypeformalisms. Inshort,thedevelopmentofthevideo
editinglanguagecleanlydemonstrateshowtheRacketdoctrinenaturallyleadstothecreationoflanguage
hierarchies,analogoustothehierarchiesofmodulesfoundinconventionalfunctionallanguages.
CCSConcepts:•Softwareanditsengineering Functionallanguages;Preprocessors;Macrolanguages;
Ñ
Specificationlanguages;Designingsoftware;•Informationsystems Multimediacontentcreation;•Applied
Ñ
computing Mediaarts;•Human-centeredcomputing Graphicaluserinterfaces;
Ñ Ñ
AdditionalKeyWordsandPhrases:Domain-SpecificLanguage,DeclarativeLanguages,VideoEditing,Syntax
Elaboration,LanguageOrientedDesign,Movies,IntegratedDevelopmentEnvironment
ACMReferenceformat:
LeifAndersen,StephenChang,MatthiasFelleisen.2017.Super8LanguagesforMakingMovies.PACMProgr.
Lang.1,1,Article30(September2017),28pages.
DOI:10.1145/3110274
1 BEINGIAVORDIATCHKI
ImaginebeingIavorDiatchki. HeisthefriendlyguywhorecordsallthewonderfulICFPpresen-
tations,editsthemintodigestiblevideoclips,andfinallycreatesaYouTubechannelforthewhole
conference. Whenhecreatesthevideoclips,hecombinesafeedofthepresenterwiththepresenter’s
screen,thesoundfeedforthespeaker,andyetanotheroneforaudiencequestions. Additionally, 30
Diatchkimustaddastartandendsequencetoeachvideoplusvariouswatermarksthroughout.
Onceonevideoisputtogether,thesameprocessmustberepeatedforthenextconferencetalkand
thenextandsoon. Worse,eventhoughsomeeditingstepsinvolvecreativity,theprocessbecomesso
monotonousthatitreducesthecreativespiritforwhenitistrulyneeded.
Theproblemcriesoutforadeclarativelanguage,especiallybecausethestateoftheartforvideo
editingsuggests(seesection3)thatprofessionalsinthisdomainalreadythink“functionally.” Towit,
professionalsspeakof“non-linearvideoediting”(NLVE)tohighlighttheideathattheprocessis
Permissiontomakedigitalorhardcopiesofpartorallofthisworkforpersonalorclassroomuseisgrantedwithoutfee
providedthatcopiesarenotmadeordistributedforprofitorcommercialadvantageandthatcopiesbearthisnoticeandthe
fullcitationonthefirstpage.Copyrightsforthird-partycomponentsofthisworkmustbehonored.Forallotheruses,contact
theowner/author(s).
©2017Copyrightheldbytheowner/author(s). 2475-1421/2017/9-ART30$15.00
DOI:10.1145/3110274
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
30:2 LeifAndersen,StephenChang,Ma￿hiasFelleisen
non-destructive. Technically,theeditingprocessseparatesadescriptivephase—whattheeventual
videoissupposedtolooklike,givenexistingtracks—fromtherenderingphase—whichactually
createsthevideoclipfromthesedescriptions.
ThispearlpresentsVideo(section4),ascriptinglanguageforvideoproduction. Videoturnsvideo
editingupsidedown. InsteadofsittingforhoursonendinfrontofsomeNLVEGUI,aprofessional
cannowspendafewminutesinfrontofanIDEtocreateaVideoscriptand,voilà,avideoclippops
out. AVideoscriptisjustasequenceofexpressions,whichdescribefragmentsofavideoclip,and
definitions,whichintroduceconstantsforandfunctionsonvideoclips. Runningsuchascriptturns
thisdescriptionintosuitable“assemblycode”foravideorenderer.
SpeakingofanIDE,everyoneknowsthatinthisdayandageaprogramminglanguagecomeswith
awholesuiteofgadgets,thatis,theprogrammingenvironmentaroundthelanguage. Wetherefore
throwinanIDE(section7)andadependenttypesystem(section6),notonlybecausethesemightbe
usefulforproducingvideochannels,butbecausearealfunctionalpearldeservesthismuchattention.
ThusanAgda-trainedprogrammermayaddtypestoVideomodules. Andbetterstill,Videocode
mayincludeanNLVEwidget,whichmayofcoursecomewithembeddedVideocode,whichmay
containanotherNLVEwidget,... Didwementionturtlesyet?1
Abstractlyspeaking,VideoonceagaindemonstratesthepoweroftheRacketdoctrine(section2).
Racket hosts Video as an embedded domain-specific language. Implementing (section 5) this
language in Racket takes only a small effort because of Racket’s powerful language-production
language. Indeed,addinganIDEandatypesystemisalsoamatterofafewhoursofthinkingtime
andcoding. Inthisspirit,thepearlpointsoutakeydifferencebetweentheconstructionofembedded
DSLsinconventionalfunctionallanguagesandtheonetrue-bluekidontheblock,Racket.
2 DRSTRANGELOVE:HOWILEARNEDTOSTOPWORRYINGANDLOVERACKET
TheRacketdoctrine(Felleisenetal. 2015)saysthatdevelopersmustfeelempoweredtoeasilycreate
anddeployanewlanguagewhenasoftwaredevelopmentsituationdemandsit. Theemphasison
languagereflectstheRacketteam’sdeepembraceoftheSapir-Whorfhypothesis. Intheworldof
software,thishypothesismeansthatlanguageframesthedeveloper’sabilitytounderstandaproblem
andarticulateasolution—ateverystageinthedevelopmentprocess.
Philosophically,Racketachievesthisidealwitharadicalemphasisonlinguisticreuse(Krishna-
murthi2001). Technically,thisreuseisenabledviaRacket’sdistinctivefeature: amodularsyntax
system(Flatt2002). Inthissystem,itiseasytoimportalinguisticconstructasafunction;indeed
thesystemblursthedistinctionbetweenlanguagesandlibraries,e.g. Tobin-Hochstadtetal.(2011).
Whilealibrarymoduleexportsfunctionswitharelatedpurpose, alanguagemoduleexportsthe
constructsofaprogramminglanguage.
In Racket, every module must first import its language definition, via a one-line specification.
Forexample,#lang racket/base—pronounced“hashlangracketbase”—tellsRacketandafuture
readerthatthemoduleiswrittenintheracket/baselanguage. Thisspecificationpointstoafile
containingthelanguageimplementationthat,approximatelyspeaking,consistsofasuiteoflinguistic
featuresandrun-timefunctions. AdevelopercanthuseditalanguageLinonebufferofanIDEand
anLprograminasecondone. Anychangetothefirstisimmediatelyvisibleinthesecondone,just
byswitchingbetweentabs. Thus,languagedevelopmentinRacketsuffersfromnopointsoffriction.
Developinganewlanguagetypicallystartsfromabaselanguageclosetothedesiredone. From
there,aRacketdevelopercreatesanewlanguagewithsomeorallofthefollowingactions:
addingnewlinguisticconstructs;
‚
hidinglinguisticconstructs;and
‚
1Seeen.wikipedia.org/wiki/Turtles_all_the_way_down,lastvisitedFeb20,2017.
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
Super8LanguagesforMakingMovies 30:3
re-interpretinglinguisticconstructs.
‚
Videoexploitsalloftheabove. Here,linguisticconstructsareanyfunctionsorsyntacticextensions
addedtoalanguagesuchaslistcomprehensionsorpatternmatching. Forthere-interpretationof
linguisticconstructs,Racketdevelopersheavilyrelyonlinguisticinterpositionpoints,thatis,anchors
inthesyntaxelaborationprocesswhereaprogrammayinjectadditionalsyntaxtransformations.
DuetotheeaseofdevelopingandinstallinglanguagesintheRacketecosystem, thelanguage
creation “warhead” is Racket’s distinguishing weapon, akin to Haskell’s type classes or ML’s
functors,initsarsenalofsoftware-engineeringtools.Whendevelopersrealizethatitisbesttoexpress
themselvesinthelanguageofadomain,theydonothesitatetodevelopamatchingprogramming
language. Afterall,domainexpertshavedevelopedthisspecializedterminology(andontology)so
thattheycandiscussproblemsandsolutionsefficiently.2
Thedomainofvideoeditingisaparticularlywell-suiteddomainforillustratingtheabovepoints.
WhiletheevolutionofthelanguagefollowsthestandardpathfromaveneerforaClibrarytoa
full-fledgedlanguage(FowlerandParsons2010), Racketreducesthispathsignificantlyandthis
pearldemonstrateshow. BeforewecandescribeVideoanditsimplementation,however,weneedto
surveytheworldofeditingvideos.
3 PRIMER
Peopleuseso-callednon-linearvideoeditors(NLVEs)tocomposevideoclips(Dancyger2010).
Inthecontextoffilmproduction,non-linearmeansnon-destructive,thatis,thesourcevideosdo
notdegradeinqualityduetoediting.3 ANLVEisagraphicaltoolwithatimelineoftracks. Each
track describes a composition of video clips, audio clips, and effects playing in sequence. The
NLVErendersthesetracksbyplayingthemallsimultaneously,placingonetrackontopoftheother.
Obviouslyascreendisplayingtheresultcanplayonlyasingletrackforvideo;4byconventionthisis
thelastortoptrack. Videoeditorsuseeffectstocompositetracks. Thatis,effectssplicetwoormore
trackstogethersothattheyappearonthescreenatthesametime. Technically,therendereruses
theseeffectstocombinetracksastheyplayandeitheroutputtheresulttoafileorplayitonascreen.
Overtime,professionalshavedevelopedtoolsanddesignpatternstoreducetheamountofrepetitive
manuallaborinvideoediting. Theyfrequentlydevelopso-calledmacros—ascriptedsequenceof
userinterfaceelements—inlanguagessuchasAppleScript(Cook2007). Someprofessionaltools,
such as Adobe Premiere (Jago and Adobe Creative Team 2017), even include an API to create
script-style plug-ins directly. Extending tools in this fashion has limits. These kinds of macros
areextremelybrittleandfrequentlybreak,evenwithinasingleapplication,becausethesemacro
languagesessentiallyspecifydialogboxclickswithoutunderstandingtheunderlyingtools.
Using a tool’s official plug-in interface produces reasonably robust scripts but yields plug-ins
thataretightlycoupledwithitstool. Theycanbeusedonlywhentheentiretoolchainispresent.
Blender(RoosendaalandHess2007),forexample,isonlyscriptablewithaBlender-specificPython
interpreterthatrunswhenBlenderislaunched.
Alternativeapproachesusegeneral-purposemultimediaframeworkssuchasGStreamer(Taymans
etal. 2013)ortheMLTFramework.5 TheseframeworksareAPIsforC-likelanguagesthatprovide
datatypesforbuildingandrenderingvideos. Theseframeworksareprimarilyusedintwosituations.
First,theyaretheback-endstoNLVEs. Forexample,MLTisthebackendforbothShotcut6 and
2NoteveryDSLisaweekormonth-longproject.Turnstile(Changetal.2017)tookalmostayearofdevelopmenttime!
3Digitaleditorsachievethisresultbyoperatingonreferencestovideos,ratherthanoperatingonthevideosthemselves.
4Audiotrackscanactuallybeplayedsimultaneously.
5mltframework.org/
6shotcutapp.com/
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
30:4 LeifAndersen,StephenChang,Ma￿hiasFelleisen
Kdenlive.7 Second,professionalsusetheseframeworkstobatch-processvideos,particularlywhen
interactivedevelopmentisnotdesired.
Theappealoftheseframeworkscomesfromtheirabilitytocreateabstractions,suchasfunctions,
tohandleotherwiserepetitivetasks.Usingtheseframeworksquicklybecomescumbersome,however,
whenthereisaneedtocombineinteractiveandprogrammaticworkflows, asisthecaseforthe
creationofconferencerecordings. Thus,studiostendtostickwithNLVEsandusetheseframeworks
onlysparingly.
Professionalsalsousedomain-specificlanguagesforvideoediting. TheseDSLsprimarilyfallinto
twocategories: XML-basedDSLsandscripting-basedDSLs. XMLDSLssuchasMLTXMLand
thenow-deprecatedSMIL(Bultermanetal. 2008)offerdeclarativelanguagesforprocessingvideos.
Theselanguagesgenerallydonothavefunctionsoranyothertypeofabstraction,however,andthus
professionalstendnottodealwiththeseXMLlanguagesdirectlywhenvideoediting. Rather,NLVEs
usetheselanguagesasafileformattosavevideoprojects.
Scripting-basedDSLssuchasAVISynth8aredeclarativeandsupportfunctionsandotherabstrac-
tions,buthavetheirownlimitations.Theytypicallysupportonlythesimplestoftaskssuchasplaying
videos in sequence with transitions and minor visual effects. They also tend to lack any formal
grammarsanduseasmallscriptforaparser. Finally,theylackmanycodereusefeatures. AVISynth,
forexample,allowsprogrammerstocreatesimplefunctionsbutcomeswithoutanycontrolflow
constructssuchasconditionalbranching.
4 THEPRODUCERS
The literature survey in the preceding section suggests that non-linear video editing distinctly
separatesthedescriptionofavideoclipfromtherenderingactiononit. Specifically,aneditor(asin
thetool)needsadescriptionofwhatthefinalvideoshouldlooklikeintermsofthegivenpieces.
Theactionofrenderingthisvideoisadistinctsecondstep. Goingfromthisassessmenttoalanguage
design requires one more idea: abstraction. For example, a description of a video composition
shouldbeabletouseasequencecomprehensiontoapplyawatermarktoallimages. Similarly,a
professionalmaywishtocreateonemoduleperICFPpresentationinordertomakeupacomplete
ICFPchannelinamodularfashion. Andofcourse,thelanguagemustallowthedefinitionanduseof
functionsbecauseitisthemostcommonformofabstraction.
TheVideolanguagegetstotheheartofthedomain. EachVideoprogramisacompletemodule
thatinterminglesdescriptionsofvideoclipsandauxiliarydefinitions. ItdenotesaRacketmodule
thatexportsaplaylistdescriptionofthecompletevideo. OnewaytouseaVideomoduleistocreate
avideowitharenderer. AdifferentwayistoimportitintoasecondVideomoduleandtoincorporate
itsexportedplaylistintoanothervideo.
Figure1displaysasimpleVideoscript. Itconsistsoffiveexpressions,eachdescribingapiece
ofavideoclip. Rightbelowthethirdpartofthevideodescription,italsocontainstwodefinitions,
whichintroduceonenameeachsothattheprecedingmultitrackdescriptiondoesnotbecometoo
deeplynested. Whentherendererturnsthisscriptintoanactualvideo,itturnsthefivepiecesinto
sequenceofimages,takingintoaccountthetransitionsbetweenthefirstandsecondfragmentand
thefourthandthefifth.
Inessence,thescriptinfigure1assemblesthevisualpartofasimpleconferencevideo. What
ismissingistheaudiopart. Naturally,aVideoprogrammershouldabstractoverthissequenceof
expressions,plustheaudioprocessing,andcreateasuitablelibrary. Figure2showswhatthesame
scriptroughlylookslikeafteraVideoprogrammerhasencapsulatedanabstractionoverthescriptin
7kdenlive.org/
8avisynth.nl
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
Super8LanguagesforMakingMovies 30:5
01 #lang video
02
03 (image �splash.png� #:length 100)
04
05 (fade-transition #:length 50)
06
07 (multitrack (blank #f)
08 (composite-transition 0 0 1/4 1/4)
09 slides
10 (composite-transition 1/4 0 3/4 1)
11 presentation
12 (composite-transition 0 1/4 1/4 3/4)
13 (image �logo.png� #:length (producer-length talk)))
14
15 ; where
16 (define slides
17 (clip �slides05.MTS� #:start 2900 #:end 80000))
18
19 (define presentation
20 (playlist (clip �vid01.mp4�)
21 (clip �vid02.mp4�)
22 #:start 3900 #:end 36850))
23
24 (fade-transition #:length 50)
25
26 (image �splash.png� #:length 100)
time
Figure1: AfirstVideoscript
01 #lang video
02
03 (require �conference-lib.vid�)
04
05 (conference-talk video slides audio 125)
06 ; where
07 (define slides (clip �slides05.MTS� #:start 2900 #:end 80000))
08 (define video (playlist (clip �vid01.mp4�) (clip �vid02.mp4�)
09 #:start 3900 #:end 36850))
10 (define audio (playlist (clip �capture01.wav�) (clip �capture02.wav�)))
time
Figure2: AVideodescriptionofaconferencetalk
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
30:6 LeifAndersen,StephenChang,Ma￿hiasFelleisen
figure1asautilitylibrary. Thisprogramusestheimportedconference-talkfunctiontocombine
arecordingofthespeaker,acaptureoftheslides,andtheaudio. Asmentioned,line1oftheprogram
specifiesthatthismoduleiswrittenintheVideolanguage. Line3importsthelibrarythatdefines
theconference-talkfunction. Line5producesthevideothatthismoduledescribes. Finally,the
remainderisasequenceofdefinitionsthatintroduceauxiliaryconstants.
Figure3showstheessenceoftheutilitylibrary,alsowrittenasaVideomodule. Explainingits
constructionintroducesenoughofVideo’sprimitivesandcombinatorstogetasenseofwhattherest
ofthelanguagelookslike. FirstweexplainVideo’sprimarylinguisticmechanisms,modulesand
functions(section4.1). Then,wedescribebasicproducers(section4.2): images,clips,colorsandso
on,followingupwithadiscussionofthebasicsofhowtocombinetheseproducersintoplaylists
andmultitracks(sections4.3,4.5). Tomakecompellingexamples,weintroducetransitions,filters,
andproperties(sections4.4,4.6,4.7). Finally,wedescribetheinterfacefordisplayingvideosand
renderingVideoprograms(section4.8).
4.1 EssentialVideo
Videomodulesconsistofaseriesofinterleavedexpressions,definitions,andimport/exportforms;
functions have the same shape as modules but without the import/export forms. Video enforces
differentscopingrules,andassignsslightlydifferentmeaningtotheseconstructs,thanRacketdoes.
Inbothcases,definitionsarevalidintheentirescope—thatis,theentiremoduleortheentirefunction
body. Theexpressionsmustdescribevideoplaylists. Modulesandfunctionsdifferinthattheformer
providesavideo,whilethelatterreturnsone. Furthermore,Videomodulesarefirst-orderentities
thatcanbecompiledseparately,whilefunctionsareactuallyfirst-classvalues.
Now,takeasecondlookatfigure3. Lines5through7showthefunctionheader. Therestofthe
codedescribesthefunctionbody(lines7–28). FunctionsinVideoaredeclarative;inparticular,line
8istheproducerreturnedbythisfunction. TheremainingsubsectionsexplaintheVideolanguagein
sufficientdetailtounderstandtherestofthiscode. Specifically,weexplainindividualfeaturesofthe
languageandhowtheyimprovethevideoeditingprocess.
4.2 Producers
TheproduceristhemostbasicbuildingblockforaVideoprogram. Aproducerevaluatestoadata
structurethatdenotessomesortofmultimediaobject: audioclips,videoclips,pictures,andsoon.
Combinationsofproducersarethemselvesproducers,andtheycanbefurthercombinedintoyet
morecomplexproducers.
The simplest type of producer, clip, incorporates traditional video files. The clip producer
convertsthefileintoasequenceofframes. Developersusecliptoimportrecordingsfromfiles,
suchasaconferencetalk,intoscripts:
#lang video
(clip �talk00.mp4�) time
Unlike clip, image creates an infinite stream of frames. Video’s combination forms truncate
thesestreamstofitthelengthofotherproducers. Additionally,developerscanusethe#:length
keywordwhentheywantaspecificnumberofframes:
#lang video
(image �splash.png� #:length 3) time
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
Super8LanguagesforMakingMovies 30:7
01 #lang video
02
03 (provide conference-talk)
04
05 ; Describes an edited conference video with appropriate feeds
06 ; Producer Producer Producer Positive-Integer -> Producer
07 (define (conference-talk video slides audio offset)
08 (multitrack clean-video clean-audio)
09 ; where
10 (define clean-audio
11 (playlist (blank offset)
12 (attach-filter audio
13 (envelope-filter 50 #:direction �in)
14 (envelope-filter 50 #:direction �out))))
15 (define spliced-video
16 (multitrack (blank #f)
17 (composite-transition 0 0 1/4 1/4)
18 slides
19 (composite-transition 1/4 0 3/4 1)
20 video
21 (composite-transition 0 1/4 1/4 3/4)
22 (image �logo.png� #:length (producer-length talk))))
23 (define clean-video
24 (playlist (image �splash.png� #:length 100)
25 (fade-transition #:length 50)
26 spliced-video
27 (fade-transition #:length 50)
28 (image �splash.png� #:length 100))))
Figure3: AVideofunctiondefinition
4.3 Playlists
Avideoisusuallyacompositionofmanyproducers. Videoprovidestwomainwaysforcombining
them: playlistsandmultitracks. Roughlyspeaking,playlistsplayclipsinsequence,whilemultitracks
playclipsinparallel. Anyproducercanbeputinaplaylist,includinganotherplaylist. Each
clipintheplaylistplaysinsuccession. Frequently,videocamerassplitrecordingsintomultiple
files. Withplaylist,developerscaneasilystitchthesefilestogethertoformasingleproducer:
#lang video
(playlist (clip �talk00.MTS�)
time
(clip �talk01.MTS�))
Developers cut playlists and other producers to desired lengths with the #:start and #:end
keywords. Thiscapabilityisincludedbecausevideorecordingsfrequentlystartbeforeatalkbegins
andendafterthetalkfinishes;seefigure4a. Recallthatwhiledefineislocatedbelowthevideo
description,itsscopeincludestheprecedingexpressions.
4.4 Transitions
Jumpingfromoneproducerinaplaylisttoanothercanberatherjarring. Scriptscanreducethis
effectwithtransitions: fading, swiping, etc. Thesetransitionsmergethetwoadjacentclipsina
playlistandareplaceddirectlyinsideof(possiblyimplicit)playlists. Expandingontheexample
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
30:8 LeifAndersen,StephenChang,Ma￿hiasFelleisen
#lang video
(image �logo.jpg� #:length 100)
talk
(image �logo.jpg� #:length 100)
; where
time
(define talk
(playlist (clip �talk00.MTS�)
(clip �talk01.MTS�)
#:start 100
#:end 8000))
Figure4: (a). Exampleofcuttingaplaylist
#lang video
(image �splash.png� #:length 100)
(fade-transition #:length 50)
talk
(fade-transition #:length 50)
time
(image �splash.jpg� #:length 100)
; where
(define talk (cid:1)(cid:2)elided(cid:2)(cid:3))
Figure4(continued): (b). Examplewithfadingtransition
#lang video
(multitrack
(clip �slides.mp4�)
(composite-transition 0 0 1/4 1/4)
talk) time
; where
(define talk (cid:1)(cid:2)elided(cid:2)(cid:3))
Figure4(continued): (c). Exampleusingmultitracks
fromfigure4a,fadetransitionsareusedtosmooththetransitionfromlogotothetalk. Figure4b
illustratesthispoint.
Every transition in a playlist actually shortens the length of its frame sequence, because
transitionsproduceoneclipforeverytwoclipstheyconsume. Additionally,aplaylistmaycontain
multipletransitions. Suchaplayliststillspecifiesauniquebehaviorbecauseplaylisttransitions
areassociativeoperations.Thus,multipletransitionsplacedinasingleplaylistdescribethedesired
clipwithoutanysurprises.
4.5 Multitracks
Multitracksplayproducersinparallel. Likeplaylists,theyemploytransitionstocompositetheir
producers. Syntactically, multitrackissimilartoplaylist. Themultitrackformconsistsof
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
Super8LanguagesforMakingMovies 30:9
#lang video
(multitrack
(blank #f)
(composite-transition 0 0 1/4 1/4)
(clip �slides.mp4�)
(composite-transition 1/4 0 3/4 1) time
talk)
; where
(define talk (cid:1)(cid:2)elided(cid:2)(cid:3))
Figure4(continued): (d). Exampleusinginlinedtransitionsinamultitracks
asequenceofproducersandcreatesanewmultitrackproducer. Again,transitionsareincluded
withinthesequencetocombinetracks;seefigure4c. Thisexampleusescomposite-transition,
whichplacesoneproducerontopoftheother. Thefourconstantsspecifythecoordinatesofthe
top-leftcorneroftheproducerandthescreenspacethatthetopproducertakes. Here,theproducer
followingthetransitionappearsinthetop-lefthandofthescreenandtakesuponequarterofthe
widthandheightofthescreen.
Transitionswithinamultitrackarenotassociative;instead,multitrackinterpretstransitionsin
lefttorightorder. Videosthatrequireadifferentordercanembedamultitrackinsideofanother
one,becauseamultitrackisitselfaproducer. Usingmultipletransitionsallowproducerstoappear
sidebysideratherthanjustontopofeachother. Expandingontherunningexampleinfigure4c,
figure4ddescribesaconferencevideowheretherecordingofthepresentergoesinthetopleftwhile
theslidesgoontheright. Thisexampleaddsacomposite-transitiontothepreviousexample
andplacestheslidesandtherecordingoverasingleblankproducer. Blankproducersareempty
slidesthatactaseitherabackgroundforamultitrackorafillerforaplaylist. Inthiscase,the
blankproducerisprovidingabackgroundthattheslidesandcamerafeedsappearon.
4.6 Filters
Filtersaresimilartotransitions,buttheymodifythebehaviorofonlyoneproducer. Inotherwords,
filtersarefunctionsfromproducerstoproducers. Amongothereffects,filterscanremovethecolor
fromacliporchangeaproducer’saspectratio. Conferencerecordingsfrequentlycaptureaudioand
videoonseparatetracks. Beforesplicingthetrackstogether,adevelopermayaddanenvelopefilter
toprovideafadeeffectforaudio.
Ascriptmayusefunctionapplicationnotationtoapplyfiltersorattachthemtoproducerswiththe
#:filterskeyword. Figure4eshowsanexampleofafilterbeingattachedtoanaudiotrackthatis
itselfcompositedwiththevideoofthecompositedtalkinfigure4d.
4.7 PropertiesandDependentClips
Producersusetwotypesofpropertiestostoreinformation: implicitandexplicitproperties. Implicit
propertiesareinnatetoclips,forexample,lengthanddimensions. Explicitpropertiesmustbeadded
bytheprogramitself.
ThepropertiesAPIcomeswithtwofunctions:
(set-property producer key value)createsanexplicitproperty.Itreturnsanewproducer
‚
withkeyassociatedwithvalue.
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.
30:10 LeifAndersen,StephenChang,Ma￿hiasFelleisen
#lang video
(multitrack
composited-talk
(clip �0000.wav�
#:filters
(list
(envelope-filter 50 #:direction �in)
(envelope-filter 50 #:direction �out))))
time
; where
(define composited-talk (cid:1)(cid:2)elided(cid:2)(cid:3))
Figure4(continued): (e). Exampleofaddingaudiotracks
#lang video
(multitrack
(blank #f)
(composite-transition 0 0 1/4 1/4)
(clip �slides.mp4�)
(composite-transition 1/4 0 3/4 1)
talk
time
(composite-transition 0 1/4 1/4 3/4)
(image �logo.png�
#:length (get-property talk �length)))
; where
(define talk (cid:1)(cid:2)elided(cid:2)(cid:3))
Figure4(continued): (f). Exampleofaddingawatermark
(get-property producer key)returnsthevalueassociatedwithkey. Ifthepropertyisset
‚
bothimplicitlyandexplicitly,theexplicitpropertyhaspriority.
Explicit properties provide a protocol for communicating information from one clip to another.
Implicitpropertiesexistforthesamepurposeexceptthattheystoreinformationthatisimplicitly
associatedwithaproducer,suchasitslength. Forexample,aconferencevideomayhavetocome
withawatermarkthatisthesamelengthasthecapturedconferencetalk. Ascriptthatperformsthis
operationcanbefoundinfigure4f.
4.8 FromProgramstoVideos
AVideomodulemaybeincorporatedintoaprogramorunderstoodasastand-aloneprogram. In
thefirstcase,anotherVideoorRacketmodulemayrequiretheVideomoduleandincorporateits
exportintoitsowncode. Inthesecondcase,ausercanhandtheVideoscripttoarendererthateither
playsthevideoonascreenorsavesittoafile.
Bydefault,aVideomoduleimplicitlyprovidesaproducer. Anymodulethatwantstousethis
implicitlycreatedvideoimportsitwiththeexternal-videoform. Foranexample,considerthis
two-linemoduleandwhatitdenotes:
PACMProgr.Lang.,Vol.1,No.1,Article30.Publicationdate:September2017.