Skip to content
Historical Database

Historical Database Configuration

Three properties control where the Historical Database writes and how many files it holds open, and one further consideration, schema persistence, applies to every deployment.

Overview

The Historical Database needs no configuration to run. A historical table created against a default installation writes to a hdb directory beneath the installation root and works without any property being set.

The properties below matter when the default location is wrong for the deployment, usually because history belongs on separate storage from the installation, or when a table is wide enough that the default file-handle cache throttles it.

All three are read once, when the Center constructs its storage engine. Changing any of them requires a restart.

Warning

None of these properties appears in config/defaults.properties. Their defaults are set in code, so the usual technique of reading the shipped defaults file to discover a property's current value does not work here. The values below are the defaults.

Overrides belong in config/local.properties, alongside every other property override.

Properties

ami.hdb.root.dir

Default:
1
ami.hdb.root.dir=hdb
Description:
  • The directory beneath which every historical table is stored.
  • A relative path resolves against the installation root, so the default places storage in hdb/ inside the installation.
  • Set an absolute path to place history on separate storage from the installation.
  • Each table becomes one directory beneath this root. See Storage layout.

ami.hdb.filehandles.max

Default:
1
ami.hdb.filehandles.max=32
Description:
  • The number of column files the Center keeps open per table.
  • Raised automatically when a table needs more than this, so a wide table is never throttled by the setting. The floor is one handle per index, plus two, plus the files each column occupies.
  • A column occupies one file, except STRING, BINARY, BIGINTEGER, and BIGDECIMAL, which occupy two: the column file and its .3fhdat payload. A table of text columns therefore needs close to twice the handles of an equally wide numeric table.
  • Raise it when many tables are queried concurrently and the operating system permits the resulting handle count.

ami.hdb.blocksize

Default:
1
ami.hdb.blocksize=8192
Description:
  • The block size used when reading and writing column files.
  • Safe to change against existing data. Tables written under a different block size remain readable.

Storage layout

Each historical table is one directory beneath the storage root, holding one subdirectory per partition, holding one file per column:

1
2
3
<root>/Trades.3fhtab/                          the table
<root>/Trades.3fhtab/0001.3fhpar/              one partition
<root>/Trades.3fhtab/0001.3fhpar/qty.3fhcol    one column within it

Partition directories are numbered from 0001 and increment as partitions are created. Indexes are written as .3fhidx files beside the columns, variable-length column types add a .3fhdat payload file, and KEY.3fhkey records the partition's key values.

To find the resolved root on a running system, run DIAGNOSE TABLE against any historical table and read the FILE column, which reports absolute paths.

Schema persistence

Nothing in the storage root records a table's column definitions. The files hold data, not schema.

The Center rewrites data/managed_schema.amisql on every schema change and replays it at startup, and that replay is what recreates the table definitions before the data becomes readable. A historical table whose definition is missing from that file is not recovered by the presence of its data directory.

Warning

Back up data/managed_schema.amisql alongside the storage root. Restoring the data directory without it leaves the data unreadable, because the tables it belongs to no longer exist.

Prerequisites

Nothing beyond a running Center. The Historical Database is part of the Center rather than a separately installed component, so it needs no plugin, adapter, or additional .jar.