@@ -6,9 +6,26 @@
is specified as part of the command. This document describes what happens
behind the scenes in order to synchronize the information on the two
repositories.</p>
-<h2>1.0 Transport</h2>
+<h2>1.0 Overview</h2>
+
+<p>The global state of a fossil repository consists of an unordered
+collection of artifacts. Each artifact is identified by its SHA1 hash.
+Synchronization is simply the process of sharing artifacts between
+servers so that all servers have copies of all artifacts. Because
+artifacts are unordered, the order in which artifacts are received
+at a server is inconsequential. It is assumed that the SHA1 hashes
+of artifacts are unique - that every artifact has a different SHA1 hash.
+To first approximation, synchronization proceeds by sharing lists
+SHA1 hashes of available artifacts, then sharing those artifacts that
+are not found on one side or the other of the connection. In practice,
+a repository might contain millions of artifacts. The list of
+SHA1 hashes for this many artifacts can be large. So optimizations are
+employed that usually reduce the number of SHA1 hashes that need to be
+shared to a few hundred.</p>
+
+<h2>2.0 Transport</h2>
<p>All communication between client and server is via HTTP requests.
The server is listening for incoming HTTP requests. The client
issues one or more HTTP requests and receives replies for each
@@ -26,9 +43,9 @@
The client maintains state between all requests. But on the server
side, each request is independent. The server does not preserve
any information about the client from one request to the next.</p>
-<h3>1.1 Server Identification</h3>
+<h3>2.1 Server Identification</h3>
<p>The server is identified by a URL argument that accompanies the
push, pull, or sync command on the client. (As a convenience to
users, the URL can be omitted on the client command and the same URL
@@ -50,9 +67,9 @@
<blockquote>
http://fossil-scm.hwaci.com/fossil/xfer
</blockquote>
-<h3>1.2 HTTP Request Format</h3>
+<h3>2.2 HTTP Request Format</h3>
<p>The client always sends a POST request to the server. The
general format of the POST request is as follows:</p>
@@ -88,15 +105,15 @@
<p>The content type of the reply is always the same as the content type
of the request.</p>
-<h2>2.0 Fossil Synchronization Content</h2>
+<h2>3.0 Fossil Synchronization Content</h2>
<p>A synchronization request between a client and server consists of
one or more HTTP requests as described in the previous section. This
section details the "x-fossil" content type.</p>
-<h3>2.1 Line-oriented Format</h3>
+<h3>3.1 Line-oriented Format</h3>
<p>The x-fossil content type consists of zero or more "cards". Cards
are separate by the newline character ("\n"). Leading and trailing
whitespace on a card is ignored. Blank cards are ignored.</p>
@@ -106,9 +123,9 @@
are arguments. The set of operators understood by servers is slightly
different from the operators understood by clients, though the two
are very similar.</p>
-<h3>2.2 Login Cards</h3>
+<h3>3.2 Login Cards</h3>
<p>Every message from client to server begins with one or more login
cards. Each login card has the following format:</p>
@@ -132,9 +149,9 @@
<p>Privileges are cumulative. There can be multiple successful
login cards. The session privileges are the bit-wise OR of the
privileges of each individual login.</p>
-<h3>2.3 File Cards</h3>
+<h3>3.3 File Cards</h3>
<p>Repository content records or files are transferred using
a "file" card. File cards come in two different formats depending
on whether the file is sent directly or as a delta from some
@@ -166,9 +183,9 @@
server to client. A delta might be sent before the source of
the delta, so both client and server should remember deltas
and be able to apply them when their source arrives.</p>
-<h3>2.4 Push and Pull Cards</h3>
+<h3>3.4 Push and Pull Cards</h3>
<p>Among of the first cards in a client-to-server message are
the push and pull cards. The push card tell the server that
the client is pushing content. The pull card tell the server
@@ -191,9 +208,9 @@
<p>The server will also send a push card back to the client
during a clone. This is how the client determines what project
code to put in the new repository it is constructing.</p>
-<h3>2.5 Clone Cards</h3>
+<h3>3.5 Clone Cards</h3>
<p>A clone card works like a pull card in that it is sent from
client to server in order to tell the server that the client
wants to pull content. But unlike the pull card, the clone
@@ -206,9 +223,9 @@
<p>In response to a clone message, the server also sends the client
a push message so that the client can discover the projectcode for
this project.</p>
-<h3>2.6 Igot Cards</h3>
+<h3>3.6 Igot Cards</h3>
<p>An igot card can be sent from either client to server or from
server to client in order to indicate that the sender holds a copy
of a particular file. The format is:</p>
@@ -222,9 +239,9 @@
The receiver of an igot card will typically check to see if
it also holds the same file and if not it will request the file
using a gimme card in either the reply or in the next message.</p>
-<h3>2.7 Gimme Cards</h3>
+<h3>3.7 Gimme Cards</h3>
<p>A gimme card is sent from either client to server or from server
to client. The gimme card asks the receiver to send a particular
file back to the sender. The format of a gimme card is this:</p>
@@ -237,9 +254,9 @@
the sender wants. The receiver will typically respond to a
gimme card by sending a file card in its reply or in the next
message.</p>
-<h3>2.8 Cookie Cards</h3>
+<h3>3.8 Cookie Cards</h3>
<p>A cookie card can be used by a server to record a small amount
of state information on a client. The server sends a cookie to the
client. The client sends the same cookie back to the server on
@@ -257,9 +274,9 @@
a way that it can tell if the cookie it sees is its own cookie or
a cookie from another server. (Typically the server will embed
its servercode as part of the cookie.)</p>
-<h3>2.9 Error Cards</h3>
+<h3>3.9 Error Cards</h3>
<p>If the server discovers anything wrong with a request, it generates
an error card in its reply. When the client sees the error card,
it displays an error message to the user and aborts the sync
@@ -277,14 +294,14 @@
space and newline, no other whitespace characters nor any
unprintable characters are allowed in
the error message.</p>
-<h3>2.10 Unknown Cards</h3>
+<h3>3.10 Unknown Cards</h3>
<p>If either the client or the server sees a card that is not
described above, then it generates an error and aborts.</p>
-<h2>3.0 Phantoms And Clusters</h2>
+<h2>4.0 Phantoms And Clusters</h2>
<p>When a repository knows that a file exists and knows the UUID of
that file, but it does not know the file content, then it stores that
file as a "phantom". A repository will typically create a phantom when
@@ -317,9 +334,9 @@
the file. There must be one or more M cards. There must be a
single Z card with a correct MD5 checksum. And all cards must
be in strict lexicographical order.</p>
-<h3>3.1 The Unclustered Table</h3>
+<h3>4.1 The Unclustered Table</h3>
<p>Every repository maintains a table named "<b>unclustered</b>"
which records the identity of every file and phantom it holds that is not
mentioned in a cluster. The entries in the unclustered table can
@@ -328,11 +345,11 @@
which might contain still more clusters, and so forth. Beginning
with the files in the unclustered table, one can follow the chain
of clusters to find every file in the repository.</p>
-<h2>4.0 Synchronization Strategies</h2>
-
-<h3>4.1 Pull</h3>
+<h2>5.0 Synchronization Strategies</h2>
+
+<h3>5.1 Pull</h3>
<p>A typical pull operation proceeds as shown below. Details
of the actual implementation may very slightly but the gist of
a pull is captured in the following steps:</p>
@@ -382,9 +399,9 @@
or if servers and clients sometimes change roles. The only negative
effects of these unusual arrangements is that more than the minimum
number of clusters might be generated.</p>
-<h3>4.2 Push</h3>
+<h3>5.2 Push</h3>
<p>A typical push operation proceeds roughly as shown below. As
with a pull, the actual implementation may vary slightly.</p>
@@ -416,17 +433,17 @@
pull, the client attempts to keep the size of the request from
growing too large by suppressing file cards once the
size of the request reaches 1MB.</p>
-<h3>4.3 Sync</h3>
+<h3>5.3 Sync</h3>
<p>A sync is just a pull and a push that happen at the same time.
The first three steps of a pull are combined with the first five steps
of a push. Steps (4) through (7) of a pull are combined with steps
(5) through (8) of a push. And steps (8) through (10) of a pull
are combined with step (9) of a push.</p>
-<h2>5.0 Summary</h2>
+<h2>6.0 Summary</h2>
<p>Here are the key points of the synchronization protocol:</p>
<ol>