Transaction Processing Overhead

In transaction logging, every change to the database is recorded in a journal (the transaction log).  In the event of a catastrophic failure, the database can recover, upon restart, by committing or rolling back transactions from log files. Disk-based databases are hard-wired to keep these logs, and to flush log files and cache to disk after transactions are committed.

With IMDSs, this journaling approach is typically optional. For example, in addition to providing transaction logging, McObject's eXtremeDB IMDS offers an in-memory alternative: the database maintains a before-image of the objects that are updated or deleted, and a list of database pages added during a transaction. Upon transaction commit, the memory that held before-images and page references returns to the database memory pool (a very fast and efficient process). If the database must abort a transaction -- for example, if the in-bound data stream is interrupted -- the before-images are returned to the database and the newly inserted pages are returned to the database memory pool.

In the event of catastrophic failure, this database image is lost. In many instances, this is acceptable, which points to a difference between the typical uses of IMDSs, and those of the more business-oriented DBMSs. In embedded applications, an IMDS can often simply be re-provisioned upon restart, as with a program guide application in a set-top box that receives continual updates from content providers, a network switch that discovers network topology on startup, or a wireless access point that is provisioned by a server upstream.

This does not preclude data persistence. At startup or any other point, the application can open a stream (a socket, pipe, or a file pointer) and instruct the IMDS to read or write a database image from, or to, the stream, for example, to create and maintain boot-stage data. Persistence can also be obtained by pairing an IMDS with non-volatile memory such as battery-backed RAM. Other applications guarantee in-memory database survival by maintaining one or more synchronized database copies (replication).

More recently, vendors have introduced hybrid database technology that combines in-memory and on-disk data storage in the same database instance. With McObject's eXtremeDB Fusion, a notation in the database design or "schema" causes certain record types to be written to disk, while others are managed entirely in memory, enabling the developer to make fine-grained tradeoffs between persistence and performance.

Many application types will continue to use DBMSs that inherently store records to disk. But for software requiring a performance edge, or an exceptionally small footprint, in-memory database systems can provide both.