BitAV: Fast Anti-Malware by Distributed Blockchain Consensus and Feedforward Scanning

5.2 MB·

by  Charles Noyes

1
BitAV: Fast Anti-Malware by Distributed Blockchain
Consensus and Feedforward Scanning
Charles Noyes, [email protected]
Synopsis—In the age of information, of the Internet, each new malware sample, and variant sub-sample,
the protection of our most vital infrastructure becomes requires its own signature [2]. Thus the size of
increasingly important. Moores law continues to prove a anti-virus’s signature set S is related to γ, the
accurate, with the number of transistors on standard
number of all known malware samples, S γ.
6 integrated circuits doubling about every two years, but ∝
Please do note that ’signature’ is used interchange-
1 virusscanningapplicationshavenotinnovatedonthesame
0 ably with’identifier,’ both ofwhich mean theunique
level and development has stagnated. Thus, the attack
2
surfaces become larger and the targets more lucrative, value resulting from the inputting of the candidate

n while the defensive mechanisms are failing to improve at malware sample into a cryptographic hash function
a
J a comparable rate. (a function which maps an arbitrary input to a set-
Ipresentthedesignandimplementationofanovelanti-
7 length output with uniform distribution).
malwareenvironmentcalledBitAV.BitAVallowsforthede-
Because virtually all anti-malware programs de-
] centralization of the update and maintenance mechanisms
R
of the software, traditionally performed by a central host, vote most of their resources to the matching of
C
and uses a staggered scanning mechanism in order to im- these signatures S to some arbitrary input stream,
s. proveperformance.Thepeer-to-peernetworkmaintenance usually with an exact matching algorithm, the two
c mechanism lowered the average update propagation speed
[ main factors that determine the effectiveness of
by 500% and is far less susceptible to targeted denial-
a solution are the ratio of detected to undetected
1 of-service attacks. The feedforward scanning mechanism
v significantly improved end-to-end performance of the mal- inputs (possibly taking into account the rate of
5 ware matching system, to a degree of an average 14 false-positives, although this is only a problem in
0 ×
increase, by decomposing the file matching process into solutions that utilize regular-expression-based multi-
4
1 efficient queries that operate in verifiably constant (O(1)) pattern algorithms) and the scalability of the signa-
0 time.
ture set. The second factor is due to the possibility
.
1 of seemingly highly efficient implementations that
0
are really only efficient in-memory, as they rely
6 I. INTRODUCTION
1 heavily on constant (very expensive) disk accesses,
: As the use of the Internet, and other massively
v and therefore degrade user experience elsewhere in
networked systems like it, becomes increasingly
Xi the system.
widespread, the ease with which viruses proliferate
r Obviously all of these possible pattern matching
a grows with it. The result is the need for tech-
schemes rely on having a known pattern set that acts
nologies designed to block these viruses, generi-
as the corpus that inputs are matched against. In this
cally called malware (malicious software), at all
case, it is the known malware identifiers. Thus, any
major network stops, but especially at the terminal
anti-virus solution that aims to protect its users from
end-user point. The throughput of most end-user’s
future malware types and variants must have update
network connections, and thus the amount of po-
mechanisms in place which are able to update the
tential data consumption, is greatly increasing as
known pattern set. The apparent solution is just to
well. While network-based detection systems have
have a centralized update server, but this is sub-
reached speeds of over 1Gb/s, the speed of actual
optimal, especially for open-source efforts, because
virus scanning and malware preventive systems has
of the cost associated with it and the fact that it acts
not kept pace.
as an obvious and openly-facing target for malicious
The amount of new malware released onto the
attackers.
public Internet is exploding [1]. As most anti-
virus software currently filters suspect files through An ideal anti-malware system would be wholly
string matching against pseudo-unique identifiers, efficient and extremely fast, but the two are gen-
2
erally at odds with one another, and thus trade- matching, the others should not be discounted. In
offs must be made in the search for an acceptable the future, when computational resources are so vast
middle-ground. Really the aim of this project is to as to render their cost moot, they will likely be the
find that middle-ground. best choice. They are not currently because of the
Objective evaluation shows that our solution, associated reduction in end-to-end speed. Thus, this
BitAV, is an effective architecture that is more paper really focuses on the creation of a bridge be-
optimal than any currently available commercial tween the currently used, and increasingly outdated,
or researched/published solution. Specifically, we method and the likely future of anti-malware. The
show: fostered debate around this transitional period [3],
Fast scanning speed with less memory [4], is ongoing, but the consensus view seems to be
•
usage: By layering a cache-efficient bloom that, for the time being, signature matching is the
filter on top of the more costly bloomier best option.
filter, BitAV manages to increase end-to-end There are two main approaches to signature
throughput of the average-case input by 14 , matching: exact and rolling. Exact signatures are of
×
and requires less memory to do so than tradi- whole files, whereas rolling signatures ’roll’ over
tional algorithms. some sections n of the file F, such that n = F/k
Scalability: BitAV can handle large numbers where k is the size of each section. Rolling signa-
•
of signatures with ease, and further space- tures are useful because if even one bit is flipped
efficiency improvements in order-and-match in a malware sample that codes for an identifier,
construction within our data structures will the corresponding exact signature will change dra-
further improve scalability. matically (as a result of the waterfall effect of hash
Decentralized updates and maintenance: function computation), while the rolling signatures
•
The community of users using and maintain- will remain mostly the same (perhaps only one
ing BitAV are provided a trustable, timeless section will be divergent) [5]. Thus, exact signatures
conduit by which to work together that is are more precise, whereas the rolling signatures are
not dependent on any centralized authority or more likely to detect slightly modified variants.
schema other than cryptographic verification.
This is accomplished through the use of a
II. METHODS/DESIGN
novel blockchain variant.
Easily implementable on all types of de- This paper centers around the design and im-
•
vices: BitAV should work on any architecture, plementation of ’BitAV,’ an anti-malware system
provided that it has enough RAM and disk that uses novel techniques to propagate malware
space to store the identifiers and load them identifiers along a network of users in conjunction
into the memory. The low memory and disk with an extremely efficient pattern-matching scheme
space usage contributes to this. ofmyowndesigntocreatetheoptimalanti-malware
solution. The scanning mechanism utilizes a bloom
A. Virus Scanning Techniques Overview
filter [6] and one of its derivative data structures,
Signature Matching: checking if a file is a known the bloomier filter [7], to create a structure that
virus, or contains bytecode known to be mali- allows for constant time key-value queries, without
cious, by searching for the hash of the identi- the high probability of false-positives that comes
fier. with probabilistic data stores.
Heuristic Analysis: testing for polymorphism by This is not the first use of bloom filters to speed
executing the virus and searching for known up pattern matching ( [8], [9]), nor the first to use
malicious identifiers in-memory. tiered look-up systems based on stratified bloom
Behavioral Analysis: checking if a file contains a filter layering ( [10], [11]). This is, however, the
completely unknown virus by running the file first implementation that takes advantage of cheap
in an emulated environment; the downside is hash functions, feedforward logical flow, and cache-
the large overhead of the emulation. resident (or in the case of systems that run a
While this paper explores novel methods to apply dedicated processor, texture memory-resident and
the first type of scanning mechanism, signature massively parallelized) architecture. This module is
3
Transaction Transaction Transaction
Owner 1's Owner 2's Owner 3's
Public key Public key Public key
Hash Hash Hash
Verify Verify
Owner 0's Owner 1's Owner 2's
Signature Signature Signature Fig. 2: Orphan Chain Competition
Sign Sign
Owner 1's Owner 2's Owner 3's
Private Key Private Key Private Key
value associated with these contracts, there is an in-
centive to make sure that they are ’correct,’ and this
Fig. 1: Currency Blockchain [15]
(coupled with the proof-of-work [15] system present
in these applications) allows for the coordination of
networks that control valuable information over an
anonymous network (in this case the information is
a more final refinement of previous works, namely
the transactions that determine currency ownership).
[5] and [10].
Prior the invention of these mechanisms, it was
The networking module acts as a way for users
simply not possible to coordinate large numbers of
across a decentralized [12] network to both receive
individual activities into a cohesive network with-
and transmit information trustlessly, and works to
out a centralized governing body to watch over
improve the reliability and efficiency of the update
and verify the proceedings [17]. This problem of
network. It does this using blockchain architecture
coordination is a well-known problem in the field
that allows for distributed anonymous consensus
of distributed networking, originally outlined in the
among peers, with the ’vote’ (an abstracted rep-
80’s [18] and more recently encapsulated by the um-
resentation of influence) weighting being a result
brella ’Byzantine Generals Problem’ [19], alluded to
of computational power expended. Thus, in order
earlier.
to overpower the rest of the network, a user (or
The Generals problem questions how individual
a group of malicious actors) would need > 50%
computer systems can come to a consensus without
network power to gain control [13]. This solves the
a method of omniscient verification (which a central
Byzantine Generals’ problem nicely, as the expen-
body would provide), in such a way that the network
diture of computational power scales directly with
is resilient to attacks by bad actors1. It posits that
the amount of capital required to obtain it (either
three divisions of the Byzantine army are camped
through hardware or, more often, electricity costs)
outside an enemy city in hopes of conquering it; an
[14]. This network model is very similar, and is
independent commander directs each division and,
actually derived from, the core Bitcoin protocol, laid
in order to be successful, all three must attack at the
out by the pseudonymous Satoshi Nakamoto in [15].
same time [19]. The generals can only communicate
through an unreliable messenger, which may be
A. Blockchain Architecture Overview influenced by a traitor in the group who is actively
trying to derail the generals’ efforts.
Originally created by the visionary Satoshi
A blockchain solves this problem by forcing
Nakamoto, Bitcoin has revolutionized the business
transparency among the groups using it, and uses
of digital currencies [16]. Bitcoin is, however, only
cryptographic measures to allow for independent
one of the innumerable number of potential applica-
verification of transmitted information by all groups
tions of the blockchain (illustrated in Fig. 1). It has
the potential to completely decentralize data storage,
1”The Byzantine Generals Problem seems deceptively simple. Its
reputation systems, even democratic voting. These difficulty is indicated by the surprising fact that if the generals can
are all done through the creation of self-executing sendonlyoralmessages,thennosolutionwillworkunlessmorethan
two-thirds of the generals are loyal. In particular, with only three
digital contracts that are backed by intelligent assets
generals, no solution can work in the presence of a single traitor.”
(cryptocurrency ’coins’). Because there is monetary [19]
4
identifier field allows for the addition of novel
Block 1
Header malware identifiers; this does not mean ’new’ sig-
natures, just those not currently present on the chain
Genesis Bl ock Prev. Hash in some other block. This is illustrated in Fig. 3.
Time
Each field in the block header (essentially the
Version
meta-data section) is needed to ensure verifiability
Merkle Root
and consensus. First, the version number is needed
to prevent errors resulting from hard forks (updates
Block 1 Transactions that would break compatibility with previous ver-
sions). The time is the approximate creation time
New Invalidated
Signatures Signatures of the block and must be within the calculated
acceptable range for each new block; BitAV uses
v894kjf… fk58gurj…
fkr8g85… ap5qb3… the timestamp both to recalculate difficulty (for
b62jf16… c82kf06… use in the proof-of-work verification [’mining’ in
w09we0… 0a3kdf9…
Bitcoin’s terminology]) and in some checks that
20er0fj3… lg95ks0…
wq0er80… t4dv86j… work to ensure chronological sanity and canonicity.
209w9g… 05kd9g5…
The previous hash field is what really makes the
w0ie0er… h58fkjd3…
blockchain a ’chain,’ as it necessitates the inclusion
Fig. 3: BitAV Blockchain of the most recent block’s hash in the next block
to be created; because of the difficulty of reverse-
engineering the has function used (SHA256, in this
case), the longer the chain the more difficult it is
in the network. In order to send a new message to recreate a verifiable blockchain. For reference,
(or ’mine’ a new block), significant computational the difficulty in recreating the current Bitcoin chain
power must be expended; this makes it both expen- is approximately 1 , where n is the current
(2256)n
sive and cumbersome for a bad actor to coordinate blockchain length. Note that this is the worst-case
an attack against the network. Blockchain protocols time when attempting to recreate the exact hashes
thusensurethattransactions(includedineachblock, of all current blocks using falsified transactions;
and the truly ’valuable’ part of the system) are individual blocks would reduce it to 1 , where
(2256)n k
valid and secure, as long as > 50% of the network k is the depth of the recreation attempt−from the
is non-malicious [14], [16], [20]. The possibility ’top’ of the chain. Additionally, this is assuming
of multiple competing chains being extended at SHA-256 is used as a hash function; should another
once, illustrated in Fig. 2, is addressed by having hash function of digest length d be used, the ap-
each individual select the longest chain they know proximate odds of finding a collision are 1 . Finally,
2d
of; as long as the proof-of-work computations are not pictured is the ’nonce’ that is included in the
accurate, the longest chain is verifiably the most header so that the hash of said header can be quickly
difficult to compute, and therefore the least likely modified by changing the nonce. Users extend the
to have been compromised or be a false chain. chain by finding a nonce that results in the hash
of the blocks header being a lower value than the
calculated difficulty level [15].
B. BitAV Blockchain
This architecture can actually be used for any
The BitAV blockchain is similar to the core networks whose aim is to share inherently valuable
Bitcoin implementation’s, in that it uses crypto- information across a network of users. The only
graphic proofs to extend and verify the chain, but stipulation is that the information have some unique
it radically differs in the way that the transaction characteristics. In this case, BitAV peers working to
field works. Traditionally, the transaction (’tx’) field extend the chain can leverage open-source databases
houses a forward-flowing narrative of all known to check whether a submitted identifier is known to
state-changes for the currency it is recording [21]. be good or bad; in either case, it allows for some
BitAV replaces the tx field with two information level of pre-screening. There is still the possibility
stores: the identifier and invalidation fields. The of attack by the submission of identifiers that are
5
Signing on a case-by-case level, whereas the possibility of
0.5% of all malware identifier invalidations being
malicious would only marginally effect the network
Hash 101100110101
function as a whole.
Hash
We understand that given the newness of the
Data
Encrypt hash blockchain architecture and the nature of these prob-
using signer's
private key
lems being less scientific and more game-theory
oriented, the authors encourage all interested parties
toreadupmorefullyontheseconceptsin[13]–[15],
111101101110
and especially [17], which is far more approachable
Certificate Signature
than many of the more highly specialized studies.
Each new identifier submission holds the general
format of:
Attach
to data
[Identifier] [Pubkey] [Signature]
Thesignatureisthecryptographicallysignedhash
of the identifier, and to verify that the submission is
not a forgery. Verification can take place by using
the included public key to decrypt the signature,
Digitally signed data hashing the decrypted value, and then checking
Fig. 4: Cryptographic Signing of Data [22] the hash of the identifier against the digest of the
decrypted value’s hash [23], [24]. It is a very simple
and well established procedure, and the signature
step is illustrated in Fig. 4.
currently unknown to be good or bad, or more Because we are able to ensure that all of the
generally the submission of universally unknown submissions under a specific key-pair are, in fact,
identifiers for some malicious purpose. generated by using that key-pair to sign the sub-
This vector of attack is mitigated, almost in its mission, we can ’track’ a user through the network
entirety, by a novel voting scheme that we call by search from all the occurrences of their public
’minority transaction consensus.’ This is perhaps the key in the ’new identifiers’ transaction field of each
most innovative feature of our network’s design, block on the chain. Knowing this, we posited that a
and it is what drives the ’invalidation’ field of our user’s relative ’trust’ could be evaluated by counting
blockchain. To fully appreciate the significance of these occurrences and using the number of times
this scheme one must understand why it is that they have altruistically added value to the network
this would never work on a currency network, such (in the form of new identifier submissions) as the
as Bitcoin (we theorize this is the reason that no weighting factor in deciding their trust.
other organization has though to implement such a When users trying to extend the chain are actively
protocol). When dealing with immutable currencies, broadcasting their status as current miners, they re-
the ability to just ’rewind’ transactions and return ceivebothrequeststoaddnewidentifiersand’votes’
currency to an entity that had appeared to have from users attempting identifiers they believe are
spent it is not allowable, as it opens the door for invalid.Invalidationsareonlypossibleforsignatures
far too much fraud and actually would turn Bitcoin addedwithinthelast10(notethatthisisanarbitrary
into an even more abusable version of credit card limit and can be easily modulated once large-scale
chargebacks (the preclusion of which Bitcoin touts testing is done) blocks, so that the backbone of the
as one of its greatest strengths). In our network, blockchain is unmanageably canonical. A nice side
however, each signature is not actually valuable to effect of using new blocks to change the state of
an individual, but to the network as a whole in a identifiers in older blocks is that the data within
more probabilistic sense. Allowing individuals to those older blocks is never changed (and thus the
reclaim currency which they have appeared to have hash remains constant), so the prev. hash field is
spent, and is accepted into the network as canon and not compromised.
shown to have been validated to all peers, is harmful The actual calculation done by a miner in deter-
6
{x, y, z}
Block 1 Transactions Block 2 Transactions
New Invalidated New Invalidated
Information Information Information Information
0 1 0 1 1 1 0 0 0 0 0 1 0 1 0 0 1 0
v894kjf… fk58gurj… 0w09wb… v894kjf…
fkr8g85… ap5qb3… kdp439… 320dlsd…
dffsg8r4… c82kf06… w435yb… dffsg8r4…
w
w09we0… 0a3kdf9… 4mfgsas… w09we0…
20er0fj3… lg95ks0… 45ufgjks… 2asdlfk3…
Fig. 6: Bloom Filter [9]
wq0er80… t4dv86j… muwe3f… 389dfjsd…
209w9g… 05kd9g5… 29qiucm… 209w9g…
w0ie0er… h58fkjd3… 1i8rffds… w0ie0er…
TXID: w0ie0er…Pubkey: pNy3GikSig: ... ID: w0ie0er…Pubkeys: nV2d3km,.. Sigs: ...
digests h (α),h (h (α)+α),...h ((h (α))+α).
1 2 1 k k 1
Increase Trust of pNy3Gik Proportionally Decrease Trust of nV2d3km,.. −
Note the use of the result of the first hash digest
Decrease Trust of pNy3Gik
in the second, and the second in the third, etc., as
Fig. 5: BitAV Trust Determination Scheme this allows us to use just one hash function while
still being able to efficiently modulate k. To ensure
the digests h (α),...h ((h (α))+α) are within
1 k k 1
−
the bounds of 1,...m, the result of each digest is
mining if an invalidation is allowable is simply:
modulated by m.
Trust of Invalidators Trust of Submitter
−
X X (1) C. BitAV Scanning Mechanism
If the result of this is > 0, the invalidation
Because of the possibility of hash collisions, in
goes through. There is no punishment for a failed
which two inputs result in the same output digest,
invalidation vote because it would bloat the chain. If
we must consider the possibility of false positives.
the outcome is of an invalidation, the submitter loses
Given that after inserting n keys into our table of
trust equal to that of the result of the subtraction,
size m, the probability of a specific bit being 0 is
and the invalidators lose trust proportionally to their
exactly
own trust level. For example, if an invalidation with
1
10 voters 10 trust total, and they have equal initial (1 )kn, (2)
− m
trust, they will all lose one ’trust.’ However, if one
the probability of a false positive p is exactly
has one more trust initially, they will lose more than
the rest. While it may not seem intuitively ’fair’ to 1
p = (1 (1 )kn)k (1 e−mkn)k. (3)
give penalties for invalidations to the voters, it is − − m ≈ −
necessary to ensure that a bad actor who has accrued
Finally, we can derive that given a target false
lots of trust does not take over the network by inval-
probability p, the minimum value of k that will
idating the submissions of any opposers. Note that
produce this probability is:
the trust points here are simplified for the example
m
and are abstracted from the real implementation; k = ln2. (4)
n
never the less, the core ideas hold true.
Bit-AV exploits the knowledge that most input After construction of this bloom filter, we are
files are not malicious by using probabilistic data left with a basic probabilistic data structure that can
structures with exactly precise no-match accuracy to perform lookups in constant time while residing in
greatly speed up the process of file scanning. It first the level 2 (L2) CPU cache. This filter will never
constructsacountingbloomfilter[6](aprobabilistic produce a false-negative result, assuming proper
data structure that uses bit vectors to efficiently implementation, but each lookup has the probability
map data [25]), illustrated in Fig. 6 from the set of being falsely positive p.
of known identifiers S. We start with an integer In a bloom filter’s worst case scenario, the entire
vector (as we want to allow for deletions [26]) of known set of data must be looked up in the filter
size m buckets, all of which are set to 0 during to confirm an uncertain match from the hashing
creation. For each identifier, k hash functions are operation; this scenario can occur when a filter is
applied to its each signature α S, resulting in hash too small for the number of elements it contains
∈
7
(and thus the false-positive rate would be extremely
Suspect
high). Hash collisions that result even before the
Files ∑F
table size modulation operation can occur, but they
are far less likely.
Bloomier filters [7] solve this problem well in
this case, as they allow for key-value lookups using
Target File F Target File F
a vector structure similar to those in bloom filters. (Fi lter H it)1 (Fil ter M iss)2
The query runs in constant time and the space
requirementisonlyO(nτ),whereτ isthesizeofthe
bucket used in the order-and-match finder operation
[11], [27]. 0 3 1 0 0 2 0 3 0 1 0 0 0 2 0 0
XOR Operation XOR Operation
k
0 1 0 0 1 0 0 1 0 0 0 0 0 1 0 0
Info = knownPatterns(Hash (Suspect))
i .1.. .0.. .0.. .1.. .1.. .1.. .0.. .0.. .0.. .0.. .0.. .0.. .0.. .1.. .0.. .0..
i=1
M (5)
Equation 5 denotes the method by whichinforma-
tion is returned from the bloomier structure. An n- Signatures for Exact
Matching
ary XOR operation (symbolized by the operator)
is performed on the hash digests, 1 to k, of the Fig. 7: Feedforward Bloom-Bloomier Filter
L
suspect string(s). The digests h (α),...h (α) are
1 k
used as indices in the bloomier index table (for
example, the first position would be at h (α)). The
1
data in the index table at all of these positions, set of the larger signature set (that was still fairly
collectively, is XORed with the total index table, substantial). Our mechanism doesn’t require this
returning the information. step, as it operates in constant time throughout.
Because of the ease and efficiency with which BitAV-Hash( ) δ takes the set of signatures
→
Eq. 5 performs simple key-value lookups, and the andoutputsanintegervectorδ thatcontainsthe
P P
usage of index table XORing, many parallels can hash digests (signatures) of all known malware.
be draw between our bloomier filters and structures BitAV-Screen(δ, F) (λ, F ) constructs
suspect
→
like IBLT’s [28], KBF’s [29], and other bloomier- a feedforward filtering mechanism from δ
like data stores. While it is possible that one of these and a bloomier filter. Each file f F is
∈
similar data structures outperforms a bloomier filter, scanned using φ. The tuple (λ, F ), where
suspect
our model can be easily adapted should that prove F F, is the list of files matched
suspect
⊆
to be the case. As of now no clear consensus exists by δ, and λ is string that the indices of the
on the matter. signatures actually matched in F .
suspect
The integration of these two filtering methods BitAV-HitScan(λ, ) 0 takes λ and outputs
→
naturally leads to a system in which information thesetofsignaPtures P0 ⊆ thatwerematched
flowsdownward,stoppingwhenithitsanimpassible during BitAV-Screen by querying our bloomier
P P
junction, but sometimes slipping through the cracks filter.
intheformoffalse-positives.Icallthismechanisma BitAV-HitMatch( 0,Fsuspect) Fmalware takesin
→
feedforward bloom-bloomier filter (FBBF) because the set of signPatures 0, the set of files Fsuspect,
of the preclusion of a feedback loop developing and outputs the set oPf files Fmalware ⊆ Fsuspect
and the necessitation of downward flow in the data matching 0 Fsuspect.
stream. The only modifications come from the input Note that the entire operation, including the final
P L
disturbances caused by the networking module. string matching, runs in constant time. The final
The full FBBF mechanism is illustrated in Fig. 7. ’exact matching’ stage does not actually utilize an
The general format of the scanning algorithm is exact matching algorithm, as they run in linear time
derived from [10], which uses a two-tiered bloom and would reduce the worst-case run to O(n), but
filtering mechanism. Unlike in our filter, their last actually just XOR’s the hash of the suspect file with
step was a full pattern matching against a sub- the bloomier output. If the result is not 0 (as the
8
End-to-End Speed vs. Industry Solutions
400
)
s
/
B 300
M
(
t
u
p
h
g 200
u
o
r
h
T
Fig. 9: Social Network Analysis of Real-World
100
Simulation
0
Bi(tCAUVDA)Bit(ACVPU)Microsoft BDietfenderSophos Avast McAfee Panda Avira MhaDsh5inigmlpalregmeebnutfafteiorsntthhaant wCPasUs-idgrnivifiencamntelychfaansitsemr ast.
It was not, however, faster when the input stream
Best Average Worst wasinconsistent(whichwouldtranslatetoon-access
in the real world). Disk I/O speeds are increasing
Fig. 8: Throughput Graph
fast enough that within the next decade they will
most likely not be the limiting factor for these kinds
of pattern matching schemes.
XOR of two exactly similar objects is always 0), Measurements of propagation time were much
then the result was a false positive. Otherwise, the more difficult to obtain than scanning speed. To
file is recognized as malicious and deleted. measure it, a network of servers (mostly AWS
micro instances or virtual machines in accessible
datacenters) was constructed to run my software.
III. RESULTS
I set up a honeypot server on my own remote
My anti-virus implementation was, on average, machine, and piped all of the garnered binaries
1,400% faster at the process of scanning files than into a test blockchain (which was shared with all
the mean of the industry solutions tested against my of the other servers). I measured the difference
implementation. This is including test data sets that between VirusTotal’s first sample seen history and
werelargelycache-missesornotalreadyinthesetof the time it took me to identify the sample (all
known malware for BitAV. I considered my average of which I had to manually check to make sure
speed to be the mean of the ’average case’ end-to- they were, in fact, malicious); this yielded BitAVs
end speed tests ( 70% clean files, 30% malware data, and the data of the industry solutions was
binaries), with all of the end-to-end throughput done by periodically checking VirusTotal’s report
test results represented in Fig. 8. Eliminating those API to monitor the detection status. The reason
cases brings the average speed increase to 8,500%, that most of my results are clustered into tiers is
twice the average performance gain of [5] without because VirusTotal’s database of provider results for
optimizations for cache residency. each sample is only updated periodically, leading
One of the main improvements on my original to highly clustered results. Once again BitAV ran
designwastheparallelizationofthesignaturegener- significantly faster than the tested industry solutions,
ationforcasesinwhichbulkfilebufferswerepassed 500% on average, and our results were even more
to BitAV. This was done through a simple CUDA statistically significant.
9
The last measurement I took was of the con- a reality (most likely before the year is out), my
nections made between test network servers. The networking architecture will have the potential to
relationships are represented in a social network disrupt every industry based around valuable, time-
analysis graph in figure 4 (note that the Open- critical data.
Ord ranking algorithm was used to arrange the
nodes). The graph clearly shows that the nodes natu-
V. ACKNOWLEDGEMENTS
rally clustered into a P2P-structured network. There
I would like to thank prof. Chi So, information
are quite a few advantages to a networks hosting
security department at the Viterbi School of Engi-
and maintenance being decentralized, namely the
neering at the University of Southern California, for
preclusionofdenial-of-serviceattacks(assumingthe
introducing me to the field. In addition I would
protocol is not inherently vulnerable),
like to thank Pei Cao, Ozgun Erdogan, Sungmin
Cho, prof. David Brumley, and Sang Kil Cha for
IV. CONCLUSION providing useful discussion and source code for
their implementations of Hash-AV and SplitScreen,
My findings showed that the implementation of
respectively. Finally, VirusTotal provided access to
the proposed design exceeded expectations in all
their private research A.P.I. to gather data for the
areas of performance. When fully optimized for
propagation speed test and malware samples for the
cache residency and with manually tuned bytecode
scanning speed tests.
our anti-virus was able to scan at a speed of over
350MB/s, making it a viable solution for network
based scanning. Yes, it is very possible that an REFERENCES
entirenetworkcouldbesecuredthroughdeeppacket
[1] M. Garetto, W. Gong, and D. Towsley, “Modeling malware
inspectionbywayofBitAV’srevolutionaryscanning spreading dynamics,” IEEE INFOCOM 2003. Twenty-second
mechanisms. The propagation speed tests similarly Annual Joint Conference of the IEEE Computer and Commu-
nications Societies (IEEE Cat. No.03CH37428), vol. 3, 2003.
showed BitAV’s clear preeminence over ’industry
[2] T.ChenandJ.Robert,“Theevolutionofvirusesandworms,”
standard’ solutions.
Statistical Methods in Computer, pp. 1–16, 2004.
Asmoredevicesarebroughtonlineitbecomesin-
[3] W. Grossman, “Does antivirus have a future?” The Guardian,
creasingly important to make sure that all networks vol. 20, 2007.
areabletobesecuredwithrelativeeaseandnoother [4] M. Fitzgerald, “The future of antivirus.”
solution provides as much modularity as one that is [5] O. Erdogan and P. Cao, “Hash-AV: Fast virus signature scan-
ning by cache-resident filters,” GLOBECOM - IEEE Global
hosted by a community of its own users. Adaptation
TelecommunicationsConference,vol.3,pp.1767–1772,2005.
is limited only by the enthusiasm of the network’s
[6] B. H. Bloom, “Space/time trade-offs in hash coding with
users to contribute to development, and previous allowable errors,” pp. 422–426, 1970.
open-source software development efforts in similar [7] B. Chazelle, J. Kilian, R. Rubinfeld, and A. Tal, “The
Bloomier filter: an efficient data structure for static support
areas have shown this to be the least likely limiting
lookup tables,” Proceedings of the fifteenth ..., 2004.
factor. Furthermore, with the advent of malware
[8] K. Huang, D. Zhang, and Z. Qin, “Accelerating the bit-split
that is developed and spread by a bad actor with string matching algorithm using Bloom filters,” Computer
influence over the institutions built around cyber Communications, vol. 33, no. 15, pp. 1785–1794, 2010.
defense (e.g. NSA’s REGIN and SIGINT malware [9] D. Eppstein and M. T. Goodrich, “Straggler Identification
in Round-Trip Data Streams via Newton’s Identities and
campaigns, whose identifiers have still not been
Invertible Bloom Filters,” Knowledge and Data Engineering,
added to a majority of the commercially available ..., pp. 1–18, Apr. 2007.
solutions), it is advantageous to users to know that [10] S. K. Cha, I. Moraru, J. Jang, J. Truelove, D. Brumley, and
there is no bias associated with the addition of new D. G. Andersen, “SplitScreen: Enabling efficient, distributed
malwaredetection,”JournalofCommunicationsandNetworks,
malware identifiers.
vol. 13, pp. 187–200, 2011.
To be clear, my blockchain-based consensus
[11] N. Tuan, B. Hieu, and T. Thinh, “High performance pattern
scheme is the only one in existence that works for matching using bloom-bloomier filter,” ...and Information
Technology ( ..., pp. 0–4, 2010.
unvalidatable data across an anonymous network.
[12] J. F. Buford, H. Yu, and E. K. Lua, P2P Networking and
The potential applications of this architecture are
Applications, 2009.
innumerable. I believe that once atomic binding of
[13] F. Reid and M. Harrigan, “An Analysis of Anonymity in the
currencynetworkstoinformatorynetworksbecomes Bitcoin System,” Jul. 2011.
10
[14] J. Kroll, I. Davey, and E. Felten, “The economics of Bitcoin
mining,orBitcoininthepresenceofadversaries,”Proceedings
of WEIS, no. Weis, pp. 1–21, 2013.
[15] S. Nakamoto, “Bitcoin: A Peer-to-Peer Electronic Cash
System,” Consulted, pp. 1–9, 2008.
[16] K. Okupski, “Bitcoin Developer Reference,” 2014.
[17] A. Wright and P. De Filippi, “Decentralized blockchain tech-
nology and the rise of lex cryptographia,” Available at SSRN
2580664, 2015.
[18] B. Preneel and M. Lowry, “Analysis and Design of
CryptographicHashFunctions,”DoctDissertationKULeuven,
2003.
[19] L.Lamport,R.Shostak,andM.Pease,“Thebyzantinegenerals
problem,”ACMTransactionsonProgrammingLanguagesand
Systems (TOPLAS), vol. 4, no. 3, pp. 382–401, 1982.
[20] G. Fox, “Deanonymistion of clients in Bitcoin P2P Network,”
Computing in Science & Engineering, pp. 15–29, 2010.
[21] V.Buterin,“Anext-generationsmartcontractanddecentralized
application platform,” Etherum, pp. 1–36, 2014.
[22] Acdx, “Digital Signature diagram,” 2008.
[23] D.Johnson,A.Menezes,andS.Vanstone,“TheEllipticCurve
Digital Signature Algorithm (ECDSA),” International Journal
of Information Security, vol. 1, pp. 36–63, 2001.
[24] A. Juels, M. Luby, and R. Ostrovsky, “Security of blind
digital signatures,” Advances in CryptologyCRYPTO’97, pp.
150–164, 1997.
[25] E. Karpilovsky, “Bloom Filters,” in COS598E, 2005, p. 106.
[26] F. Bonomi, M. Mitzenmacher, R. Panigrahy, S. Singh, and
G. Varghese, “An improved construction for counting bloom
filters,” in Algorithms–ESA 2006. Springer, 2006, pp. 684–
695.
[27] D. Charles and K. Chellapilla, “Bloomier filters: A second
look,”inAlgorithms-ESA2008. Springer,2008,pp.259–270.
[28] M. T. Goodrich and M. Mitzenmacher, “Invertible bloom
lookup tables,” in 2011 49th Annual Allerton Conference on
Communication, Control, and Computing (Allerton). IEEE,
Sep. 2011, pp. 792–799.
[29] S. Xiong, Y. Yao, Q. Cao, and T. He, “kBF: A Bloom Filter
forkey-valuestoragewithanapplicationonapproximatestate
machines,”INFOCOM,2014Proceedings...,pp.1150–1158,
Apr. 2014.