vallenae.io¶

IO¶

Database classes¶

Classes to read/write pridb, tradb and trfdb database files.

Warning

Writing is still experimental

`PriDatabase`	IO wrapper for pridb database file.
`TraDatabase`	IO wrapper for tradb database file.
`TrfDatabase`	IO wrapper for trfdb (transient feature) database file.

All database classes implement two different interfaces to access data:

Standard read_*

Read data to pandas.DataFrame, e.g. with PriDatabase.read_hits

>>> pridb = vae.io.PriDatabase("./examples/steel_plate/sample.pridb")
>>> df = pridb.read_hits()  # save all hits to pandas dataframe
>>> df[["time", "channel"]]  # output columns hit and channel
            time  channel
set_id
10      3.992771        3
11      3.992775        2
12      3.992813        4
13      3.992814        1

Streaming iread_*

Iterate through the data row by row. This is a memory-efficient solution ideal for batch processing. The return types are specific typing.NamedTuple, see Data types.

Example with PriDatabase.iread_hits:
>>> pridb = vae.io.PriDatabase("./examples/steel_plate/sample.pridb")
>>> for hit in pridb.iread_hits():
...     print(f"time: {hit.time:0.4f}, channel: {hit.channel}")
...
time: 3.9928,   channel: 3
time: 3.9928,   channel: 2
time: 3.9928,   channel: 4
time: 3.9928,   channel: 1
>>> type(hit)
<class 'vallenae.io.datatypes.HitRecord'>

Data types¶

Records of the database are represented as typing.NamedTuple. Each record implements a class method from_sql to init from a SQLite row dictionary (column name: value).

`HitRecord`	Hit record in pridb (`SetType.HIT`).
`MarkerRecord`	Marker record in pridb (`SetType.LABEL`, `SetType.DATETIME`, `SetType.SECTION`).
`StatusRecord`	Status data record in pridb (`SetType.STATUS`).
`ParametricRecord`	Parametric data record in pridb (`SetType.PARAMETRIC`).
`TraRecord`	Transient data record in tradb.
`FeatureRecord`	Transient feature record in trfdb.
`SetType`	Record types of the pridb.
`HitFlags`	Hit record flags.
`StatusFlags`	Status flags.

Compression¶

Transient signals in the tradb are stored as BLOBs of 16-bit ADC values – either uncompressed or compressed (FLAC). Following functions convert between BLOBs and arrays of voltage values.

`decode_data_blob`	Decodes (compressed) 16-bit ADC values from BLOB to array of voltage values.
`encode_data_blob`	Encodes array of voltage values to BLOB of 16-bit ADC values for memory-efficient storage.