Sockets and Pipes

288 Pages·2022·1.298 MB·English

by  Chris Martin, Julie Moronuki

SocketsandPipes
byChrisMartinandJulieMoronuki
©2022ChrisMartinandJulieMoronuki.Allrightsreserved.
2020-02-25:Firstdraft
2020-05-14:Seconddraft
2020-09-21:Thirddraft
2021-02-19:Fourthdraft
2021-05-01:Fifthdraft
2022-09-01:Sixthdraft
Contents
Preface 7
Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
What’sinside . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Proximitytoreality . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1 Handles 15
1.1 Thenecessityofindirection . . . . . . . . . . . . . . . . . . . 15
1.2 Writingtoafile . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.3 Diligentcleanup . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.4 MonadIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2 Chunks 31
2.1 Packedcharacters . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2 Readingfromafile,onechunkatatime . . . . . . . . . . . . 34
2.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3 Bytes 41
3.1 Packedoctets. . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Copyingafile. . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.3 Characterencodings . . . . . . . . . . . . . . . . . . . . . . . 45
3
3.4 TheShowandIsStringclasses . . . . . . . . . . . . . . . . . 50
3.5 Avoidingsystemdefaults . . . . . . . . . . . . . . . . . . . . 56
3.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4 Sockets 62
4.1 Openupandconnect . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Extradetails . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3 Namesandaddresses . . . . . . . . . . . . . . . . . . . . . . 68
4.4 Addressinformation . . . . . . . . . . . . . . . . . . . . . . . 71
4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5 HTTP 78
5.1 Thespecification. . . . . . . . . . . . . . . . . . . . . . . . . 79
5.2 HTTPrequests . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.3 ASCIIstrings . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.4 HTTPresponses . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.5 Servingothers . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6 HTTPtypes 93
6.1 Requestandresponse . . . . . . . . . . . . . . . . . . . . . . 94
6.2 Requestline . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.3 Statusline . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.4 Headerfields . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
6.5 Messagebody . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.6 HTTPversion . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
7 Encoding 105
7.1 Stringbuilders . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.2 Measuringtime . . . . . . . . . . . . . . . . . . . . . . . . . 108
7.3 Requestandresponse . . . . . . . . . . . . . . . . . . . . . . 112
7.4 Higher-orderencodings. . . . . . . . . . . . . . . . . . . . . 116
4
7.5 Thestartline . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
8 Responding 124
8.1 Ameasureofsuccess . . . . . . . . . . . . . . . . . . . . . . 124
8.2 Response-buildingutilities . . . . . . . . . . . . . . . . . . . 127
8.3 Integers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
8.4 Responsetransmission . . . . . . . . . . . . . . . . . . . . . 132
8.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
9 Contenttypes 137
9.1 Somecommontypes. . . . . . . . . . . . . . . . . . . . . . . 138
9.2 UTF-8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
9.3 HTML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
9.4 JSON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
9.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
10 Change 152
10.1 STM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
10.2 Increment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
10.3 Atomically . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
10.4Thecountingserver . . . . . . . . . . . . . . . . . . . . . . . 159
10.5 OtherSTMtopics . . . . . . . . . . . . . . . . . . . . . . . . 160
10.6Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
11 Streaming 164
11.1 Chunkedhello . . . . . . . . . . . . . . . . . . . . . . . . . . 166
11.2 Chunktypes . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
11.3 Encodingachunk . . . . . . . . . . . . . . . . . . . . . . . . 171
11.4 Transfer-Encoding . . . . . . . . . . . . . . . . . . . . . . . 173
11.5 Servingthefile . . . . . . . . . . . . . . . . . . . . . . . . . . 174
11.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
12 ListTIO 179
5
12.1 Thenewresponsetype . . . . . . . . . . . . . . . . . . . . . 179
12.2 WhatisListT . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
12.3 Constructingaresponse . . . . . . . . . . . . . . . . . . . . . 187
12.4 Encodingaresponse . . . . . . . . . . . . . . . . . . . . . . . 189
12.5 Sendingaresponse . . . . . . . . . . . . . . . . . . . . . . . 193
12.6ListTinotherlibraries . . . . . . . . . . . . . . . . . . . . . . 194
12.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
13 Parsing 198
13.1 Encodingvsdecoding . . . . . . . . . . . . . . . . . . . . . . 200
13.2 Attoparsec . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
13.3 Requestline . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
13.4 Explainingwhat’swrong . . . . . . . . . . . . . . . . . . . . 216
13.5 Incrementalparsing . . . . . . . . . . . . . . . . . . . . . . . 218
13.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
14 Errors 226
14.1 Statuscodes . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
14.2 Constructingresponses . . . . . . . . . . . . . . . . . . . . . 229
14.3 Visibilityintwoplaces . . . . . . . . . . . . . . . . . . . . . . 230
14.4Thread-safelogging . . . . . . . . . . . . . . . . . . . . . . . 236
14.5 Either . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
14.6ExceptT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
14.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
Comingup 249
15 Readingthehead 250
16Readingthebody 251
17 Connectionreuse 252
A Solutionstoexercises 253
6
Preface
Thecontentthateventuallygrewintothisbookbeganwiththequestion:
Whatexactlyisawebserver? Asatisfactoryanswerthatdoesnotassume
substantialbackgroundknowledgerequiresspanningquiteafewareasof
computing. Fortunately,theyallserveasfruitfulmotivationsforsimulta-
neouslylearningabouthowtouseHaskell,whichisthelargerobjectiveof
theJoyofHaskellcollection.
ThelanguageawebserverspeaksistheHypertextTransferProtocol
(HTTP),whichthisbookexploresingreatdetailwhilewalkingthroughthe
creationofaserverfrom“scratch”.Weencouragereaderstofollowalong
inreadingtheofficialdefinitionofHTTP(RFC7230publishedbytheInter-
netEngineeringTaskForce)asweimplementthespecificationinHaskell.
Whilehigh-levellibrariesmakeitpossibletocreatewebapplicationswith-
outdetailedknowledgeofHTTP,webelievethatafullunderstandingof
theunderlyinglayerswebuilduponhelpsususeaplatformmoreeffec-
tively. BystudyingHTTPwealsogainanappreciationforwhatitisand
isnotgoodfor,andforwhatapplicationswemightstandtobenefitfrom
choosingadifferentnetworkprotocolinstead.
7
Prerequisites
ThisbookisforHaskelllearnerswhohavesomebasicfacultywiththelan-
guageandarenowreadytoworkuptoasubstantialproject.Weexpectthat
youunderstandthebasicsyntaxandcandothingslike:
(cid:228) writeacaseexpressiontopatternmatchoverasumtype
(cid:228) sequenceIOactionsinadoblock
(cid:228) usequalifiedimports
(cid:228) definedatatypes
(cid:228) useGHCi
(cid:228) installHaskelllibraries
Fromthebasepackage,weassumesomefamiliaritywith:
(cid:228) typesMaybe,Either,and[]
(cid:228) classesEq,Show,Monoid,Foldable,Functor,andMonad
WedonotassumepriorknowledgeofanyadditionallibrariesorGHC
languageextensions.
What’sinside
Bytesandcharacters Thefirstseveralchaptersintroducethebytestring
andtextlibrariesandarelargelydedicatedtotearingapartatraditional
helloworldprogram,lookingunderneaththeabstractnotionof“printing
text”tostartgreetingtheworldintermsofwritingbytestoafilehandle.
Afterdiscussingbytes,weneedonlyashorthoptosockets,ourmeansof
writingbytesacrossgreatdistancesusingthenetworklibrary.
Encodingandparsers FirstweencodingHTTPmessagesasbytestrings.
That’stheeasypart;next,wegointheoppositedirectionandlearnhowto
interpretbytestringsusingtheattoparseclibrary. Thiswillacquaintus
evenmorecloselywiththeHTTPmessageformat.
8
Monadtransformers WeintroducethreeMonadtransformersthatarees-
peciallyapplicabletooursubjectmatter:ResourceT,ListT,andExceptT.
Nopriorexperiencewithtransformersisrequired.Wedonotlingeronthe
generalconcept,preferringinsteadtofocusoneachofthethreeexamples
andtocreatefamiliaritywithtransformersandliftingchieflybydemon-
stration.
Resource safety Use of ResourceT begins in chapter 1, and we use it
throughoutthebook. Thismakesitabreezetodealwithfilesandsockets
withoutresourceleaks.
Streaming To move past toy examples that fit easily into memory, we
havetostartwritingstreamingprocessesthatcandealwithlargeamounts
ofdatabyhandlingitinsmallerpieces. Allofthecodewithinthisbookis
writtenwithmemoryusageinmind.ListT,thesubjectofchapter12,pro-
videsanespeciallyconvenientfacilityforworkingwithstreams.
Errorhandling Astheamountoffunctionalityofourserverbuildsup,the
numberofpossibleerrorconditionsstartstorise. Chapter14introduces
ExceptTtoworkwitherrorsinacleanandwell-typedmanner.
ThisbookhasacompanionHaskelllibrarycalledsockets-and-pipes,
availablefromthestandardpackagerepository.
https://hackage.haskell.org/package/sockets-and-pipes
Thelibraryre-exportsallofthemodulesfromotherlibrariesthatwe
useinthebook; prospectivereadersareencouragedtobrowsethedocu-
mentationatthewebaddressgivenabove,asitprovidesanoverviewofthe
librariesthatyouwilllearntousefromreadingthisbook.
9
Setup
Westronglyencourageyoutofollowalongwiththebookandtypethecode
asyouread. Theexercisesattheendofeachchaptermakeuseofthecode
giveninthechapter. Subsequentchapterswillalsoreferbacktodefini-
tionsfromearlierinthebook,soitisimportanttokeepeverythingasyou
progress.
Youcanorganizethecodehoweveryoulike,butherewegivearecom-
mendedsetupfortheconvenienceoflessopinionatedreaders.
book.cabal
cabal-version: 3.0
name: book
version: 0
data-files: **/*.txt
library
default-language: Haskell2010
default-extensions: BlockArguments QuasiQuotes
TypeApplications ScopedTypeVariables
ghc-options: -Wall -fdefer-typed-holes
build-depends: sockets-and-pipes ^>= 0.3
exposed-modules: Book
cabal-version,name,andversionarenecessitiesinanyCabalpack-
agefile.
Thedefault-extensionsfieldenablesafewlanguageextensions:
(cid:228) BlockargumentsisasmalladjustmenttotheHaskellsyntaxwhich
allowsadoblocktobeusedasafunctionparameter,ataskwhich
hastraditionallybeenaccomplishedusingthe($)operator.
10