Cluster Emulator (DCLUE)
Kant, A. Sahoo and N. Jani
DCLUE is a
powerful simulator for studying the performance of clustered database systems.
It includes detailed simulation of an Oracle-9i like clustered
database running a TPC-C like workload.
The model implements the following features:
- Complete creation of the
database tables and their indices and their proper initialization. (Only
the required table columns [fields] are represented, but row sizes are
- Emulation of disk IO and buffer cache
at each node. Each table & its index are cached separately. Disk IO
emulates elevator scheduling, caching and other usual IO features.
- All TPC-C queries are
implemented in detail according to a query plan. Barring a few short-cuts,
each query should actually produce results as in a real database.
concurrency control, where each update effectively creates a new
data version. This scheme obviates the need for read locks.
- Reasonably sophisticated
(write) locking mechanism. In particular, locks are acquired in two
phases: initial intention locking
followed by mass conversion of latches to locks. Locking granularity can
extend from page down to row level.
- Scheduling of message
sends, message receives and application processing are
implemented in a way to mimic processing by current operating systems
- Thread switching is
implemented explicitly and takes into account the working set and whether
the working set fits in the processor cache. Processor caches are not
implemented; instead, their impact is accounted for in thread switching
- Queuing and latency impact of
processor-memory data transfers are
implemented. In particular, latencies of address bus, data bus and memory
channels are accounted for in iteratively estimating the CPI (avg cycles per instruction) for the processor.
- Oracle 9i like cache fusion
architecture which allows pooling of buffer caches from all nodes. As a result, a lot of local buffer cache
misses can be satisfied from some remote buffer cache.
communication (IPC) messages between nodes are implemented
explicitly and are used for a variety of purposes including locking,
directory query/update, data transfer, etc.
- Remote disk accesses use iSCSI protocol which itself requires a couple
of IPC messages. These are called “storage messages” and are carried on
separate TCP connections from regular IPC messages. They may also be
carried on a separate fabric.
- Logging of
all modified data to log disks before a transaction is allowed to commit.
Logging could be directed to one or more nodes.
Owing to its very detailed
nature, the model requires a rather fine-grain calibration, which could
be difficult, but can be done reasonably for a well studied workloads like
TPC-C, TCP/IP processing, iSCSI processing, etc. On the positive side, however, the model
isn't dependent on high level input parameters such as buffer cache hit ratio
or number of IPC messages per transaction – things that will be easily
invalidated by a change in configuration.
The model is developed using the OPNET simulation package and uses the
complete TCP/IP/Ethernet MAC provided by OPNET
for networking. DCLUE also supports a
rudimentary implementation of SCTP as the
transport protocol. For more information on DCLUE, please consult the
- K. Kant and A. Sahoo, “Clustered
DBMS Scalability under Unified Ethernet Fabric”, Proc. of ICPP, May
2005, Oslo, Norway
- K. Kant, A. Sahoo
and N. Jani, “DCLUE:
A Distributed Cluster Emulator”, to appear in IEICE Transactions, Special Issue on Parallel/Distributed Computing
and Networking, 2006. Also appears in Proc. of 2005 Opnetwork, Washington DC, Aug 2005.
use of DCLUE will be in a series of ongoing projects dealing with fabrics for future
data centers. In particular, DCLUE will be used to evaluation real
application impact of a variety of link level, network level and transport
level fabric features. DCLUE can be made available in source form on a request