Loading data saved from the LabOne UI in Python

The LabOne User Interface can continuously save instrument data such as demodulator samples to hard disk as binary data in MAT (Matlab®) files or as plain text in CSV (comma separated value) files.

The "Record Data" section of the LabOne User Interface's Config Tab. The tree-like structure in the bottom left allows the user to choose which instrument data should be saved to file.

The “Record Data” section of the LabOne User Interface’s Config Tab. The tree-like structure in the bottom left allows the user to choose which instrument data should be saved to file.

Using MAT files, demodulator data can be recorded at full demodulator rate at up to 100MB/s (disk speed limited) whereas CSV files are always write time limited and will not be faster than ~10MB/s (rounding down, this corresponds to a demodulator rate of approximately 110 kSamples/s). Therefore, CSV files are a bad choice for saving fast data from the LabOne User Interface and for subsequently working with large data sets due to the long loading and saving times associated with text files.

Python is a high-level language which combined with NumPy, SciPy and matplotlib becomes an extensive environment for data analysis and scientific computing. As such, Python is a good choice (it’s also free and open source) for manipulating data saved from the LabOne User Interface in either CSV or MAT formats. The attached python module zifileutils.py contains convenient routines based on NumPy and SciPy functions to quickly load demodulator data saved by either the LabOne User Interface (UHFLI) or ziControl (HF2 Series). More specifically it provides:

  • load_labone_mat(): for MAT files saved from LabOne,
  • load_labone_csv(): for CSV files containing generic data saved from LabOne,
  • load_labone_demod_csv(): for CSV files containing demodulator data saved from LabOne,
  • load_zicontrol_csv(): for CSV files saved from ziControl,
  • load_zicontrol_zibin(): for binary (ziBin) files saved from ziControl.

The following code snippet demonstrates how to load demodulator data from a CSV file saved by the LabOne User Interface into a NumPy structured array using zifileutils.py and how to subsequently work with the data.


import numpy as np
import scipy.signal
import matplotlib.pyplot as plt
import zifileutils

def main():
"""
A function demonstrating the use of
zifileutils.load_labone_demod_csv() and manipulating
the resulting structured array containing
demodulator data.
"""
# Load demodulator samples from file
sample = zifileutils.load_labone_demod_csv('demods_0_sample_00000.csv')
# Calculate the demodulator's magnitude from the X and Y fields
R = np.abs(sample['x'] + 1j*sample['y'])
# Estimate the DC component in the signal by removing
# the linear trend
R_dc = R - scipy.signal.detrend(R)
# Calculate a threshold based on the DC component and
# standard deviation of the signal
threshold = R_dc + np.std(R)
plt.interactive(True)
plt.figure()
plt.grid()
t0 = sample['timestamp'][0]
plt.plot(sample['timestamp'] - t0, R, 'b-')
plt.plot(sample['timestamp'] - t0, threshold, 'k--')
plt.xlabel('Time (timestamps)')
plt.ylabel('R (Vrms)')
plt.legend(('Demod R', 'threshold'))

if __name__ == '__main__':
main()

The above code produces the plot below.

Demodulator R signal with threshold.

The resulting plot showing the automatic threshold resulting from the code snippet above.

 

Other options for working with data from LabOne

If CSV or MAT are not suitable formats, it’s also possible to stream data directly into applications such as Excel or C# via a URL as provided by the LabOne Net Link.

Should Python not be the tool of choice in your lab’s workflow, another obvious choice for loading LabOne’s MAT files is Matlab®. Since CSV is an extremely commonly used file format, LabOne’s CSV files can be used with many other external programs. For example, if the sole requirement is visualisation, kst is a fast, open source plotting tool that is available for Windows, Linux and Mac OSX.

Combined with the LabOne Python API, Python can be used as a complete solution for interacting with your instrument.

Links

Here are downloads for:

More information about our APIs can be found in the LabOne Programming Manual.