performance test for QCodes

[peendebak]

Simple test for performance. Should be extended to log the performance and set tighter boundaries

@giulioungaretti

[AdriaanRol]

Looks great, always nice to have more test. I also think that the overhead bounds are reasonable.
Code is clean and clearly commented.

Let's wait for one of the maintainers to merge this.

[giulioungaretti]

Thanks for the contribution, but I think we should go an extra mile! 🎉

F.ex. we should design for some hard limits, rather than test that things are ok!

In general we should have a clear max and min for:

• size of data created per iteration of loop
• frequency of loop iterations

I am collecting some numbers locally, but the more data the better!

[giulioungaretti]

just playing around with @nataliejpg setup with the alzar card

• 14 kb of data per iteration
• at full speed (i.e. no loop delays), we have 0.7 second

[majacassidy]

We have situations where we run the Alazar in streaming mode. Then the data file size is infinite. Using the RAID arrays from Alazar, you can store the 2GB/s of data in real time.

On Nov 15, 2016, at 2:15 PM, Giulio Ungaretti [email protected] wrote:

just playing around with @nataliejpg https://github.com/nataliejpg setup with the alzar card

14 kb of data per iteration
at full speed (i.e. no loop delays), we have 0.7 second
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub #374 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AClTo2vmAIkrOimi-g0MKfqpcKjDJ9sTks5q-bBygaJpZM4KkYoR.

[giulioungaretti]

@majacassidy interesting thanks for the answer! but It's really hard to correlate the numbers you give with the actual question 🐼 ! Perhaps my question is badly worded.

[majacassidy]

Basically, with this acquisition method our datafile size is unlimited. So, any hard coded limit will cause us problems.

Sometimes we need to stream for days. This means the data file is >> gigabytes.

On Nov 15, 2016, at 2:51 PM, Giulio Ungaretti [email protected] wrote:

@majacassidy https://github.com/majacassidy interesting thanks for the answer! but It's really hard to correlate the numbers you give with the actual question 🐼 ! Perhaps my question is badly worded.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub #374 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AClTo5odPL5aQUOAbTqwv-mtADuBy9cTks5q-bjTgaJpZM4KkYoR.

[giulioungaretti]

@majacassidy cool, that I understood. But the question is not about limits or hardcoded things, but rather what/how/how often you need that data.

I am not asking about files, but "bytes" you need from the card at a specific time namely at every point in the loop in this case.

[majacassidy]

Just to clarify, the 0.7s/14 kb of data you talk about is your ‘integration time’ in an experiment. How long an integration time you select depends on the signal to noise of the experiment. The 0.7s/14 kb here you refer to is probably from a single acquisition rather than the streaming mode and so it corresponds to your Alazar’s onboard memory. Each card has a different onboard memory. In streaming mode, you are no longer limited by onboard memory as it operates as a FIFO stack and pulls the data off onto the hard drive as fast as you can take it. I guess when I say infinite, I mean ‘a very large number' that is set by the user’s system parameters (hard drive size etc).

For example, a ‘loop' experiment I could run is:

At each step in gate voltage, use the Alazar to integrate for 1 hour and store the raw data. Therefore the ‘bytes’ is something on order GB for each point in the loop.

On Nov 15, 2016, at 3:26 PM, Giulio Ungaretti [email protected] wrote:

@majacassidy https://github.com/majacassidy cool, that I understood. But the question is not about limits or hardcoded things, but rather what/how/how often you need that data.

I am not asking about files, but "bytes" you need from the card at a specific time namely at every point in the loop in this case.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub #374 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AClTo78uEOYH9luBJcra1V-JAtwNRI-Xks5q-cELgaJpZM4KkYoR.

[giulioungaretti]

@majacassidy afaik it puts into what the software defines and and streams data as far as the dma does not overflow, but this is too technical anyway. My question was more dumb 🐱 !

But I have your answer now:

• order of GB let's even say tens of GB / hour

[damazter]

@giulioungaretti
@majacassidy
I am not sure that I follow the discussion. As far as I understand we want to take an arbitrary amount of data per loop iteration, which will take an arbitrarily short time. (by which I mean, our current measurements are sometimes of the order of 2 GB/s, but in the future this can increase a lot)

But I am not sure @giulioungaretti what you are going to use these limits for. I think it would be bad to limit the measurements in any way. Could you explain a littlebit more?

[giulioungaretti]

@damazter when I mention hard limits, I mean just that any design that is not fas enough to begin with shall not be used.
Multiprocessing will always be slower because it does have an extra layer of comms (be a tcp socket/ unix socket or windows named pipes). So I want to make sure I know all the experiments data rates before releasing something that is doomed to be too slow.

@damazter arbitrary amount/ arbitrarily short time may not true because you'll be limited by the hardware 🙍‍♀️ !

So a hard limit would probably be the fastest PCIe bus throughput (order of 10GB / second).

[AdriaanRol]

@giulioungaretti for the experiment I presented at the QCoDeS meetup in Delft we've done some detailed benchmarking of our adaptive sequences. All of this was done using QCoDeS + PycQED. If you want you can take a look at the paper https://arxiv.org/abs/1611.04815, especially the timing section that starts on the top of the second page.

Everything was done without use of the qcodes multiprocessing framework but with qcodes instruments. Let me know if this is the kind of information you're looking for.

[giulioungaretti]

Closing as not relevant until we have the new architecture!