Randomized benchmarking in seconds

Randomized benchmarking (RB) is a widely used tool in quantum information science to determine qubit fidelity and to characterize individual qubit gates. It is also an excellent example of a quantum experiment with advanced requirements on the control hardware and software. For an RB experiment, we apply random sequences of unitary gates to the qubit. The sequences are chosen such that the final gate always inverts all the preceding ones. This means that in the ideal case, where all the gates are perfect and there is no decoherence, the initial state of the qubit remains unchanged after we have applied all gates. Gate errors and decoherence will however lead to changes in the final qubit state and will have more influence for longer sequences of gates. We perform the RB experiment by varying the number of random gates M, where we choose the unitary gates from the Clifford group. For each sequence length M, we also choose k different random sequences. Finally, for each random sequence we apply the final recovery gate, which in the ideal case would take the qubit back to its initial state. We experimentally determine the probability of finding the qubit in its initial state and plot the average of the return probability for all k random sequences as a function of the sequence length M. An exponential fit to the return probability as a function of the sequence length M provides a general measure of qubit fidelity, which can be understood as an estimate of the average error rate per qubit operation. Benchmarking qubits in this way is largely independent of state-preparation and measurement errors, making RB the preferred method for benchmarking individual qubits for many research groups worldwide.

Figure 1: Illustration of randomized benchmarking, where each Ck is a random unitary gate from the Clifford group.

Typically, we generate each instance of the random sequence locally on a computer before uploading it to the control hardware and executing it on the qubits. When using a traditional arbitrary waveform generator (AWG), this has to be done on a sample-by-sample basis, and includes not only the waveform envelopes but also the intermediate frequency (IF) components of both I and Q channels of the control signal, which are then upconverted via IQ mixing to the radio-frequency (RF) needed for qubit control.

This process is time-consuming in computer and upload time to the instruments, and often leads to the instruments being effectively idle for a large fraction of the total time of the experiment – time that would be better spent doing measurements. This is especially true for long gate sequences, which will be increasingly required given the increase in coherence times and gate fidelities for superconducting and spin qubits.

In this post, we demonstrate two efficient ways of performing randomized benchmarking with minimal instrument downtime. To this end, we employ two features of the Zurich Instruments HDAWG Arbitrary Waveform Generator: digital oscillators for signal modulation and the command table for pulse-level sequencing. In both these examples we program the AWG sequencers only once, which simplifies and speeds up the experimental procedure. For each individual random sequence, we still need to upload information about the nature of the sequence to the device. Here we will show that this can be done fast, as these methods do not involve a recompilation of the sequencer program or the upload of large amounts of waveform data.

The use of the digital oscillators in our HDAWG allows us to generate a series of fully phase-coherent IQ pulses on the fly while only specifying the waveform envelopes. In practice, the channel output is the multiplication of this envelope with a digital oscillator running on the FPGA hardware directly. Here we can specify different oscillators, i.e., oscillators running at different frequencies for example, for each output channel. Alternatively, as is typical when employing IQ upconversion from an IF to a target frequency in the RF range, two channels may share the same oscillator with a relative phase difference of 90 degrees between them. The major advantages of this approach are that phase coherence of subsequent pulses is guaranteed, as the internal oscillators have a high phase stability, and that the effort in generation and upload of the waveforms is reduced.

Figure 2: Output tab of the LabOne GUI for the HDAWG, with digital modulation of the first two output channels enabled, at an intermediate frequency of 15 MHz. The settings shown here lead to single-sideband-upconversion with an IQ mixer like the HDIQ IQ modulator, for two-channel signals of the type used in the example Python notebook.

The second feature we highlight here is the command table, which we use to program the HDAWG on a pulse level. The command table contains a table of instructions, where each entry may contain the reference to a two-channel waveform as well as amplitudes and phases for each of the channels. Accessing the command table with the executeTableEntry() command on the HDAWG Sequencer then plays the specified waveform with its maximum amplitude defined by the amplitude setting of that entry, and with the phase of the internal oscillators set by its phase setting. One advantage of using the command table for this purpose, as opposed to using separate sequencer instructions to achieve the same goal, is that all these steps are executed simultaneously using a single sequencer instruction.

Generating and uploading command table entries to the HDAWG is as simple as executing the code below, written in Python with our ziPython module.

import zhinst.ziPython as zi
import json
// connect to dataserver and HDAWG
daq = zi.ziDAQServer(dataserver_ip, 8004, 6)
daq.connectDevice(device_ID, connect_type)
// define comand table
ct = {'header': {'version': '0.2'},
'table': [{'index': 0,
'waveform': {'index': wave_ind},
'amplitude0': {'value': 1.0},
'amplitude1': {'value': 1.0},
{'index': 1,
'waveform': {'index': wave_ind},
'amplitude0': {'value': 0.5},
'phase0': {'value': 90.0, 'increment': True},
'amplitude1': {'value': 0.5},
'phase1': {'value': 90.0, 'increment': True}
// upload command table to instrument
node = f"/{device_ID:s}/awgs/{awg_index}/commandtable/data"
daq.setVector(node, json.dumps(ct))

In this standalone example, the command table has two entries, with the same two-channel waveform envelope, identified by a wavetable index wave_ind. The second pulse has the same envelope but only half the amplitude of the first and a relative phase of 90 degrees. In this way, we might define an X_pi and a Y_pi/2 pulse. The command table is then uploaded to the instrument by writing to its node tree, as shown. The example notebook provided with this blog post encapsulates this functionality already in a useful helper class for control of the HDAWG.

Efficient randomized benchmarking on the HDAWG

With these simple ingredients, we are now ready to implement randomized benchmarking in an efficient way. We demonstrate two different methods of using the command table and discuss their relative merits and potential constraints: (1) generating each random sequence locally on a computer and uploading it to the command table and (2) generating the random sequence on the HDAWG itself using the in-built pseudo-random number generator (PRNG) and uploading the set of Clifford gates only once.

A Python notebook with all the code needed to reproduce this demonstration is available at [1]. For these examples, we only concern ourselves with the qubit control pulse aspect of RB, and do not include readout pulses in the sequences. For a detailed description on how to do simultaneous multiplexed readout of up to ten qubits using a single UHFQA, for example, see this recent blog post. Also, the code examples execute each random sequence only once, while for a real experiment one would need to acquire suitable statistics. We do this because we provide timing examples comparing our methods for pulse generation with the traditional approach of uploading the full waveform, here using the same HDAWG. Those examples, as well as the code used to perform the timing, are also included in the notebook.

For both methods that we introduce here, we first need to specify the instructions for each of the Clifford gates. There are 24 elements in the single-qubit Clifford group. Each Clifford gate can be generated from a succession of between one and three single-qubit rotations by integer multiples of 90 degrees around the X or Y axis. Details may be found in [2, 3]. In our code example, we specify a Gaussian envelope with a fixed length for all pulses. The rotation angle is then determined through the pulse amplitude and the axis of rotation is fixed by the phase of the control pulse. Using the definitions of the Clifford gates, we precalculate the waveform envelope for each of the individual members of the Clifford group. The code generating the Clifford gate waveforms can be found in the notebook in the section Clifford gates. There we also define the function calculate_inverse_clifford() which calculates the index of the final recovery gate, ensuring that the full (M+1)-length sequence leads back to the initial state.

(1) Generating a random command table

For our first demonstration of randomized benchmarking, we start by generating a sequencer program and waveform table to upload to the HDAWG. The sequencer program is defined as a string in the Python code. It allocates the memory for each of the wavetable entries, reads the number of gates from a user-defined register, and then steps through the command table entries until it has reached that number. We upload and compile this program together with the waveform information to the instrument. The waveform data here only contains the two-channel waveform envelopes for each of the 24 Clifford gates. Crucially, in our approach, we have to perform this step only once, at the very start of the experiment. To perform RB, we then make use of the command table in a somewhat simple fashion: for each random sequence, we generate a command table of length M+1, where each successive entry references the waveform of the Clifford gate at this position in the random sequence. We upload the new command table and set the user register to the sequence length, and then instruct the AWG to execute the sequencer program. It steps through the command table and thus the random sequence. An example sequence output of the AWG is shown in Figure 3.

Figure 3: Scope recording of the AWG output for an example sequence of 16 random Clifford gates. For illustration purposes, we show simple quadrature pulses, without the digital oscillators added, as well as the full IQ quadrature pulses including digital modulation at an IF of 15 MHz. Our example code enables both possible outputs with the ability to switch between the two options by setting awg.iq_modulation = True/False, without changing the nature of the data uploaded to the AWG.

As for each random instance we only have to upload the command table and set a user register, this method ensures very fast sequence execution. In our simple setup, connected to the HDAWG through ethernet and using an elementary gate length of 240ns, each random sequence including single sequence playback requires on average about 40ms per step. In this case, an example RB experiment with 10 different sequence lengths of between 2 and 1024 gates and 20 different random sequences per length takes a total of 8s of execution time. Compared to the traditional approach, where the full waveform is uploaded for each random sequence, and where in the best case we achieve an average timing per step of 140ms for a total execution time of 28s, we thus achieved a speed-up by a factor of 3.5 in this first example.

(2) Using the native random number generator

Due to the current maximum length of the command table on the HDAWG, the first method we demonstrated is limited to a sequence length of 1024 Clifford gates. Our second example is slightly more complex, but it uses only 24 entries of the command table, and the sequence length can be practically arbitrary – in practice it is limited to 2^16 – 1 as this is the number of internal states of the random number generator employed. Again, we start by generating a sequencer program and wavetable, upload them to the HDAWG and compile the sequencer program. We also generate a single command table containing 24 entries, and upload this to the instrument only once at the start of the experiment. Each command table entry now references one of the 24 gates in the Clifford group.

In this example, we are using the fact that the HDAWG has its own internal PRNG that runs natively on the instrument. This additional feature allows us to generate the random sequences for RB locally and in real time. As the calculation of the recovery gate goes beyond what we want to implement on the HDAWG directly, we make use of the fact that we can model the PRNG in software. For each random sequence, we choose a sequence length M and a random seed for the PRNG. Given the random seed, we can deterministically pre-calculate the PRNG output in Python, which in turn allows us to calculate the required recovery gate for the random sequence. For each random gate sequence, we transmit the gate length, PRNG seed and recovery gate index via the user registers to the HDAWG and execute the sequencer program. The HDAWG now generates the random sequence locally and in real time from the given seed and plays the appropriate command table entry corresponding to the chosen Clifford gate at each step.

This method again leads to low overhead in measurement time and is even faster than the previous method at an average 25ms upload and playback time per sequence and a total execution time of 5.2s. Compared to the traditional approach, we thus demonstrated a speed-up by more than a factor of 5 using the command table and digital oscillators.


In this post, we have shown two ways of generating a randomized benchmarking experiment using the HDAWG Arbitrary Waveform Generator. We demonstrated how to make efficient use of the digital oscillator feature and the command table functionality to reduce sequence generation, compilation and upload time by more than a factor of 5. Combining the efficient pulse sequence generation shown here with fast and high-fidelity readout as well as active qubit reset will help you make the most out of your measurement time and complete qubit benchmarking in seconds. For an example of how to take advantage of the Zurich Instruments solutions for scalable, coherent quantum system control including feedback, also take a look at our recent post on practical active qubit reset.

All example code for this post is written in Python, using the ziPython module, and equivalent functionality is available using our MATLAB or C++ APIs.

Did this post resonate with you? To know more, get in touch with Clemens ([email protected]) or Andrea ([email protected]).


[1] A jupyter notebook with all code needed to reproduce the experiments demonstrated in this post is available here

[2] Epstein et al., Phys. Rev. A 89, 062321 (2014) (arXiv link)

[3] Muhonen et al., J. Phys.: Condens. Matter 27, 154205 (2015) (arXiv link)