Table Of ContentComputational Synthesis and Creative Systems
Filipe Calegario
Designing
Digital Musical
Instruments
Using Probatio
A Physical Prototyping Toolkit
Computational Synthesis and Creative Systems
SeriesEditors
FrançoisPachet,Paris,France
PabloGervás,Madrid,Spain
AndreaPasserini,Trento,Italy
MirkoDegliEsposti,Bologna,Italy
Creativity has become the motto of the modern world: everyone, every institution,
andeverycompanyisexhortedtocreate,toinnovate,tothinkoutofthebox.This
callsforthedesignofanewclassoftechnology,aimedatassistinghumansintasks
thataredeemedcreative.
Developingamachinecapableofsynthesizingcompletelynovelinstancesfroma
certain domain of interest is a formidable challenge for computer science, with
potentially ground-breaking applications in fields such as biotechnology, design,
andart.Creativityandoriginalityaremajorrequirements,asistheabilitytointeract
with humans in a virtuous loop of recommendation and feedback. The problem
calls for an interdisciplinary perspective, combining fields such as machine
learning, artificial intelligence, engineering, design, and experimental psychology.
Related questions and challenges include the design of systems that effectively
explore large instance spaces; evaluating automatic generation systems, notably in
creative domains; designing systems that foster creativity in humans; formalizing
(aspects of) the notions of creativity and originality; designing productive
collaboration scenarios between humans and machines for creative tasks; and
understandingthedynamicsofcreativecollectivesystems.
Thisbookseriesintendstopublishmonographs,textbooksandeditedbookswith
a strong technical content, and focuses on approaches to computational synthesis
thatcontributenotonlytospecificproblemareas,butmoregenerallyintroducenew
problems,newdata,ornewwell-definedchallengestocomputerscience.
Moreinformationaboutthisseriesathttp://www.springer.com/series/15219
Filipe Calegario
Designing Digital Musical
Instruments Using Probatio
A Physical Prototyping Toolkit
FilipeCalegario
CentrodeInformática(CIn-UFPE)
FederalUniversityofPernambuco
Recife,Pernambuco,Brazil
ISSN2509-6575 ISSN2509-6583 (electronic)
ComputationalSynthesisandCreativeSystems
ISBN978-3-030-02891-6 ISBN978-3-030-02892-3 (eBook)
https://doi.org/10.1007/978-3-030-02892-3
LibraryofCongressControlNumber:2018959876
©SpringerNatureSwitzerlandAG2019
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartofthe
materialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
bookarebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
To Alissa
¼)
Preface
The last decade has witnessed a considerable rise in physical, programmable,
interactive artifacts. Sensors, devices, platforms, and frameworks have become
more accessible, andmore people areprogramming thephysical worldbeyondthe
screen. Interactive devices for artistic expression present challenges that are worth
investigatingbecausetheinteractionoftenneedsahighlevelofskillthatishardto
achieve. Therefore, interactive artistic approaches can teach us valuable lessons
applicabletootherlevelsofinteractiondesignandhuman-computerinteraction.
Oneclassofartistic,physicalinteractiveobjectsisthedigitalmusicalinstrument
(DMI). DMIs are artifacts in which gestural control and sound production are
physically decoupled but digitally mapped. This provides freedom for a DMI
designer since several combinations are possible, but this also increases the com-
plexity ofthedesignspace. Besides,structured methods andguidelines that would
helpinthedesignhavenotyetbeenestablished.Toaddressthisissue,prototyping
seemsto be a promising approach, as a prototype isa tool not only for testing and
communicatingideasbutalsoforgeneratingthem.AsaDMIisameanstoproduce
music, its prototype should provide real-time sound feedback for control gestures.
For that reason, in the DMI context, nonfunctional prototypes are not entirely
suitable. On the other hand, the development of functional prototypes demands
moretimeandeffort,andconsequentlytheycanbeabottleneckiniterativedesign.
HowcanweprovidestructuredandexploratorypathstogenerateDMIideas?How
canwedecreasethetimeandeffortinvolvedinbuildingfunctionalDMIprototypes?
Todealwiththesequestions,weproposetheconceptofinstrumentalinheritance,
that is, the application of gestural and/or structural components of existing instru-
mentstogenerateideasfornewinstruments.Tosupportanalysisandcombination,
weleverageatraditionaldesignmethod,themorphologicalchart,inwhichexisting
artifacts are split into parts, presented in a visual form, and then recombined to
produce new ideas. Finally, integrating the concept and the method in a concrete
object, we developed a physical prototyping toolkit for building functional DMI
prototypes: Probatio,a modular system of blocks and supports to prototype instru-
mentsbasedoncertainwaysofholdingandgesturalcontrolsformusicalinteraction.
vii
viii Preface
The evaluation of the toolkit showed that it contributed to reducing the time to
achieve a functional prototype and also influenced the increase in the number of
cycles of idea exploration. Also, users reported more musical engagement with
Probatio in comparison to a generic sensor toolkit. We believe that the current
version of the system provides a set of features that can work as a platform for
userinteractionanalysisandsubsequentrecommendationofmodulecombinations.
Recife,Brazil FilipeCalegario
December2017
Contents
1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objectives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 DocumentOutline. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 ChallengesinDesigningDMIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1 DigitalMusicalInstruments. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 DMIClassification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 TheChallengeofMultipleCombinations. . . . . . . . . . . . . . . . . . . 7
2.2.1 GesturalController. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.2 SoundOutput. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2.3 Mapping. . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . .. 10
2.2.4 Feedback. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 TheChallengeofExpressivityandVirtuosity. . . . . . . . . . . . . . . 13
2.4 TheChallengeofEvaluationandEvolution. . . . . . . . . . . . . . . . . 13
2.5 TheChallengeofNoPreviousKnowledge. . . . . .. . . . . . . . . . . . 14
2.6 TheChallengeofMultipleStakeholdersandContextsofUse. . . . 15
2.7 FinalConsiderations. . . . . . .. . . . . . .. . . . . .. . . . . . .. . . . . . . 16
3 DesignProcess. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1 IdeaExploration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 Prototyping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 FinalConsiderations. . . . . . .. . . . . . .. . . . . .. . . . . . .. . . . . . . 30
4 StateoftheArt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.1 FrameworksandApproachesforDMIDesign. . . . . . . . . . . . . . . 31
4.2 FunctionalPrototypeinDMIDesign. . . . . . . . . . . . . . . . . . . . . . 36
4.2.1 ToolsforPhysicalandFunctionalPrototyping. . . . . . . . . 36
ix
x Contents
4.2.2 TheTrade-OffArea. . . . . . .. . . . . . .. . . . . . .. . . . . . .. 38
4.3 FinalConsiderations. . . . . . .. . . . . . .. . . . . .. . . . . . .. . . . . . . 42
5 EarlyExploration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 45
5.1 MethodologicalApproach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.2 ProjectBatebit. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.2.1 Interviews. . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . .. 47
5.2.2 Workshops. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.2.3 Pandivá. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
5.2.4 SandboxWow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.3 Summary. . . .. . . . . . . .. . . . . . .. . . . . . . .. . . . . . .. . . . . . .. 53
6 Proposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.1 ScopeandBasis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.2 InstrumentalInheritance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.2.1 RelatedConcepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.2.2 PossibleEvidence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.2.3 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.3 MorphologicalChartforDMIIdeaGeneration. . . . . . . . . . . . . . . 60
6.3.1 WhatIsMorphologicalAnalysis?. . . . . . . . . . . . . . . . . . . 60
6.3.2 MorphologicalChartBasedonInstrumentalInheritance. . . 62
6.4 DevelopmentoftheFunctionalPrototypingToolkitforDMI. . . . 64
6.4.1 Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.4.2 ImplementationDecisions. . . . . . . . . . . . . . . . . . . . . . . . 66
6.4.3 PhysicalStructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.4.4 ConnectionSlots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.4.5 Blocks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.5 FinalConsiderations. . . . . . .. . . . . . .. . . . . .. . . . . . .. . . . . . . 75
7 EvaluationofProbatio0.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
7.1 Description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
7.2 Evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
8 EvaluationofProbatio0.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
8.1 EvolutionfromProbatio0.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
8.1.1 NumberofBlocksandMultipleSensors. . . . . . . . . . . . . . 82
8.1.2 ChangingMappingStrategy.. . . .. . . . .. . . .. . . .. . . . . 83
8.1.3 SoundOutputModule. . . . . . . . . . . . . . . . . . . . . . . . . . . 83
8.1.4 CurvedShapes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
8.1.5 ConnectionArmSupport. . . . . . . . . . . . . . . . . . . . . . . . . 84
8.1.6 ProtectionandConnectionstotheHub. . . . . . . . . . . . . . . 84
8.1.7 FrictionofBlocksandSlots. . . . . . . . . . . . . . . . . . . . . . . 84
8.2 Experiment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
8.2.1 Objectives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
8.2.2 Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
8.2.3 MethodsforDataCollection. . . . . . . . . . . . . . . . . . . . . . 86
