222 lines
8.6 KiB
Tcl
222 lines
8.6 KiB
Tcl
#
|
|
# Run this Tcl script to generate the sqlite.html file.
|
|
#
|
|
set rcsid {$Id: arch.tcl,v 1.16 2004/10/10 17:24:54 drh Exp $}
|
|
source common.tcl
|
|
header {Architecture of SQLite}
|
|
puts {
|
|
<h2>The Architecture Of SQLite</h2>
|
|
|
|
<h3>Introduction</h3>
|
|
|
|
<table align="right" border="1" cellpadding="15" cellspacing="1">
|
|
<tr><th>Block Diagram Of SQLite</th></tr>
|
|
<tr><td><img src="arch2.gif"></td></tr>
|
|
</table>
|
|
<p>This document describes the architecture of the SQLite library.
|
|
The information here is useful to those who want to understand or
|
|
modify the inner workings of SQLite.
|
|
</p>
|
|
|
|
<p>
|
|
A block diagram showing the main components of SQLite
|
|
and how they interrelate is shown at the right. The text that
|
|
follows will provide a quick overview of each of these components.
|
|
</p>
|
|
|
|
|
|
<p>
|
|
This document describes SQLite version 3.0. Version 2.8 and
|
|
earlier are similar but the details differ.
|
|
</p>
|
|
|
|
<h3>Interface</h3>
|
|
|
|
<p>Much of the public interface to the SQLite library is implemented by
|
|
functions found in the <b>main.c</b>, <b>legacy.c</b>, and
|
|
<b>vdbeapi.c</b> source files
|
|
though some routines are
|
|
scattered about in other files where they can have access to data
|
|
structures with file scope. The
|
|
<b>sqlite3_get_table()</b> routine is implemented in <b>table.c</b>.
|
|
<b>sqlite3_mprintf()</b> is found in <b>printf.c</b>.
|
|
<b>sqlite3_complete()</b> is in <b>tokenize.c</b>.
|
|
The Tcl interface is implemented by <b>tclsqlite.c</b>. More
|
|
information on the C interface to SQLite is
|
|
<a href="capi3ref.html">available separately</a>.<p>
|
|
|
|
<p>To avoid name collisions with other software, all external
|
|
symbols in the SQLite library begin with the prefix <b>sqlite3</b>.
|
|
Those symbols that are intended for external use (in other words,
|
|
those symbols which form the API for SQLite) begin
|
|
with <b>sqlite3_</b>.</p>
|
|
|
|
<h3>Tokenizer</h3>
|
|
|
|
<p>When a string containing SQL statements is to be executed, the
|
|
interface passes that string to the tokenizer. The job of the tokenizer
|
|
is to break the original string up into tokens and pass those tokens
|
|
one by one to the parser. The tokenizer is hand-coded in C in
|
|
the file <b>tokenize.c</b>.
|
|
|
|
<p>Note that in this design, the tokenizer calls the parser. People
|
|
who are familiar with YACC and BISON may be used to doing things the
|
|
other way around -- having the parser call the tokenizer. The author
|
|
of SQLite
|
|
has done it both ways and finds things generally work out nicer for
|
|
the tokenizer to call the parser. YACC has it backwards.</p>
|
|
|
|
<h3>Parser</h3>
|
|
|
|
<p>The parser is the piece that assigns meaning to tokens based on
|
|
their context. The parser for SQLite is generated using the
|
|
<a href="http://www.hwaci.com/sw/lemon/">Lemon</a> LALR(1) parser
|
|
generator. Lemon does the same job as YACC/BISON, but it uses
|
|
a different input syntax which is less error-prone.
|
|
Lemon also generates a parser which is reentrant and thread-safe.
|
|
And lemon defines the concept of a non-terminal destructor so
|
|
that it does not leak memory when syntax errors are encountered.
|
|
The source file that drives Lemon is found in <b>parse.y</b>.</p>
|
|
|
|
<p>Because
|
|
lemon is a program not normally found on development machines, the
|
|
complete source code to lemon (just one C file) is included in the
|
|
SQLite distribution in the "tool" subdirectory. Documentation on
|
|
lemon is found in the "doc" subdirectory of the distribution.
|
|
</p>
|
|
|
|
<h3>Code Generator</h3>
|
|
|
|
<p>After the parser assembles tokens into complete SQL statements,
|
|
it calls the code generator to produce virtual machine code that
|
|
will do the work that the SQL statements request. There are many
|
|
files in the code generator:
|
|
<b>attach.c</b>,
|
|
<b>auth.c</b>,
|
|
<b>build.c</b>,
|
|
<b>delete.c</b>,
|
|
<b>expr.c</b>,
|
|
<b>insert.c</b>,
|
|
<b>pragma.c</b>,
|
|
<b>select.c</b>,
|
|
<b>trigger.c</b>,
|
|
<b>update.c</b>,
|
|
<b>vacuum.c</b>
|
|
and <b>where.c</b>.
|
|
In these files is where most of the serious magic happens.
|
|
<b>expr.c</b> handles code generation for expressions.
|
|
<b>where.c</b> handles code generation for WHERE clauses on
|
|
SELECT, UPDATE and DELETE statements. The files <b>attach.c</b>,
|
|
<b>delete.c</b>, <b>insert.c</b>, <b>select.c</b>, <b>trigger.c</b>
|
|
<b>update.c</b>, and <b>vacuum.c</b> handle the code generation
|
|
for SQL statements with the same names. (Each of these files calls routines
|
|
in <b>expr.c</b> and <b>where.c</b> as necessary.) All other
|
|
SQL statements are coded out of <b>build.c</b>.
|
|
The <b>auth.c</b> file implements the functionality of
|
|
<b>sqlite3_set_authorizer()</b>.</p>
|
|
|
|
<h3>Virtual Machine</h3>
|
|
|
|
<p>The program generated by the code generator is executed by
|
|
the virtual machine. Additional information about the virtual
|
|
machine is <a href="opcode.html">available separately</a>.
|
|
To summarize, the virtual machine implements an abstract computing
|
|
engine specifically designed to manipulate database files. The
|
|
machine has a stack which is used for intermediate storage.
|
|
Each instruction contains an opcode and
|
|
up to three additional operands.</p>
|
|
|
|
<p>The virtual machine itself is entirely contained in a single
|
|
source file <b>vdbe.c</b>. The virtual machine also has
|
|
its own header files: <b>vdbe.h</b> that defines an interface
|
|
between the virtual machine and the rest of the SQLite library and
|
|
<b>vdbeInt.h</b> which defines structure private the virtual machine.
|
|
The <b>vdbeaux.c</b> file contains utilities used by the virtual
|
|
machine and interface modules used by the rest of the library to
|
|
construct VM programs. The <b>vdbeapi.c</b> file contains external
|
|
interfaces to the virtual machine such as the
|
|
<b>sqlite3_bind_...</b> family of functions. Individual values
|
|
(strings, integer, floating point numbers, and BLOBs) are stored
|
|
in an internal object named "Mem" which is implemented by
|
|
<b>vdbemem.c</b>.</p>
|
|
|
|
<p>
|
|
SQLite implements SQL functions using callbacks to C-language routines.
|
|
Even the built-in SQL functions are implemented this way. Most of
|
|
the built-in SQL functions (ex: <b>coalesce()</b>, <b>count()</b>,
|
|
<b>substr()</b>, and so forth) can be found in <b>func.c</b>.
|
|
Date and time conversion functions are found in <b>date.c</b>.
|
|
</p>
|
|
|
|
<h3>B-Tree</h3>
|
|
|
|
<p>An SQLite database is maintained on disk using a B-tree implementation
|
|
found in the <b>btree.c</b> source file. A separate B-tree is used for
|
|
each table and index in the database. All B-trees are stored in the
|
|
same disk file. Details of the file format are recorded in a large
|
|
comment at the beginning of <b>btree.c</b>.</p>
|
|
|
|
<p>The interface to the B-tree subsystem is defined by the header file
|
|
<b>btree.h</b>.
|
|
</p>
|
|
|
|
<h3>Page Cache</h3>
|
|
|
|
<p>The B-tree module requests information from the disk in fixed-size
|
|
chunks. The default chunk size is 1024 bytes but can vary between 512
|
|
and 65536 bytes.
|
|
The page cache is responsible for reading, writing, and
|
|
caching these chunks.
|
|
The page cache also provides the rollback and atomic commit abstraction
|
|
and takes care of locking of the database file. The
|
|
B-tree driver requests particular pages from the page cache and notifies
|
|
the page cache when it wants to modify pages or commit or rollback
|
|
changes and the page cache handles all the messy details of making sure
|
|
the requests are handled quickly, safely, and efficiently.</p>
|
|
|
|
<p>The code to implement the page cache is contained in the single C
|
|
source file <b>pager.c</b>. The interface to the page cache subsystem
|
|
is defined by the header file <b>pager.h</b>.
|
|
</p>
|
|
|
|
<h3>OS Interface</h3>
|
|
|
|
<p>
|
|
In order to provide portability between POSIX and Win32 operating systems,
|
|
SQLite uses an abstraction layer to interface with the operating system.
|
|
The interface to the OS abstraction layer is defined in
|
|
<b>os.h</b>. Each supported operating system has its own implementation:
|
|
<b>os_unix.c</b> for Unix, <b>os_win.c</b> for windows, and so forth.
|
|
Each of these operating-specific implements typically has its own
|
|
header file: <b>os_unix.h</b>, <b>os_win.h</b>, etc.
|
|
</p>
|
|
|
|
<h3>Utilities</h3>
|
|
|
|
<p>
|
|
Memory allocation and caseless string comparison routines are located
|
|
in <b>util.c</b>.
|
|
Symbol tables used by the parser are maintained by hash tables found
|
|
in <b>hash.c</b>. The <b>utf.c</b> source file contains Unicode
|
|
conversion subroutines.
|
|
SQLite has its own private implementation of <b>printf()</b> (with
|
|
some extensions) in <b>printf.c</b> and its own random number generator
|
|
in <b>random.c</b>.
|
|
</p>
|
|
|
|
<h3>Test Code</h3>
|
|
|
|
<p>
|
|
If you count regression test scripts,
|
|
more than half the total code base of SQLite is devoted to testing.
|
|
There are many <b>assert()</b> statements in the main code files.
|
|
In additional, the source files <b>test1.c</b> through <b>test5.c</b>
|
|
together with <b>md5.c</b> implement extensions used for testing
|
|
purposes only. The <b>os_test.c</b> backend interface is used to
|
|
simulate power failures to verify the crash-recovery mechanism in
|
|
the pager.
|
|
</p>
|
|
|
|
}
|
|
footer $rcsid
|