qutech-delft / qtt Goto Github PK

Hi all!
I'm new using qtt and while measuring with "scan1D" function I realized that the live plot does not work as expected.

I can see how the Y-axis updates, but the points are not shown. I took a look into the code but I didn't find the problem.

This is some dummy code that I wrote to show the problem:

import sys, os, tempfile
import numpy as np
%matplotlib inline
%gui qt
import matplotlib.pyplot as plt
import qcodes
from qcodes.plots.qcmatplotlib import MatPlot
from qcodes.plots.pyqtgraph import QtPlot
from qcodes.data.data_set import DataSet
from qtt.measurements.scans import scanjob_t  
import qtt
from qtt.instrument_drivers.gates import VirtualDAC
from qcodes.tests.instrument_mocks import DummyInstrument, DummyInstrumentWithMeasurement

# set data directory
path_save = r'K:\Jaime\Testing_qtt'
datadir = os.path.join(path_save, 'Test')
DataSet.default_io = qcodes.data.io.DiskIO(datadir)

dac = DummyInstrument('dac', gates=['ch1', 'ch2'])
dmm = DummyInstrumentWithMeasurement(name='dmm', setter_instr=dac)

station = qcodes.Station(dac, dmm)
mwindows = qtt.gui.live_plotting.setupMeasurementWindows(station, create_parameter_widget=False)
plotQ = mwindows['plotwindow']

# Checking the sensor
scanjob = scanjob_t({'sweepdata': dict({'param': station.dac.ch1,
                                        'start': 0.0, 'end': 30.0, 'step':1.0,
                                        'wait_time': 0.1}),
                                        'minstrument': [station.dmm.v1],
                                        'dataset_label': 'Test1D_liveplot'})

data1d = qtt.measurements.scans.scan1D(station, scanjob, liveplotwindow= None,location=None, verbose=0)

Using "scan2D" the updating works without problem, then it is something really particular for the 1D plot.

Could someone help me to solve this issue?
Thanks!
Jaime

In upstream qcodes the test for MultiParameter (introduced here #140) breaks. In particular the test_scan2D function fails, but probably also some of the PAT measurements.

@CJvanDiepen Do you still use the MultiParameter functionality?
@lucblom @CJvanDiepen Can you look into solving this issue?

pyqtgraph is in the install_requires list in setup.py without any version requirement. This downloads version 0.10 (latest available release). However, version 0.11 is the minimum (as specified in extras). If this version requirement is also added to the install_requires list (as it should be for consistency), that fails because 0.11 does not exist (anywhere). As a workaround, we can download the develop branch from github by adding an extra parameter to the setup() call:

dependency_links=[ 'git+https://github.com/pyqtgraph/pyqtgraph@develop#egg=pyqtgraph' ],

The version number should in that case still be omitted frominstall_requires, because setup() does not properly parse the develop version (0.11.0.dev0+gad7453c) and the >=0.11 check fails.
pyqtgraph should also be removed from extras (no longer needed, and version check fails).

tools.addPPTslide breaks with
module 'qtt.measurements' has no attribute 'videomode'
and after fixing that with
module 'qtpy.QtGui' has no attribute 'QWidget'

also the error treatment in this module is quite mean (just pass), can we at least log or print the error that occur?

@peendebak @lucblom

scans break with
NameError: name 'NotGitRepository' is not defined

@lucblom @peendebak

The qctoolkit package was renamed to qupulse. The code need to be updated to reflect this (mainly the new virtual awg and the setup.py/requirements.txt/dev-requirements.txt).

@lucblom

The test builds that run to validate pull requests use an installation command (pip install -r requirements.txt) which is different from the installation command in the README.md file (pip install -e .), which is what end users would use.
There are 2 problems with this:

1. It violates the DRY principle (setup.py and requirements.txt)
2. The test build does not test the installation procedure that an end user would use.

It there a reason to have 2 different install methods?

For example, issue #616 was not caught in the test build for pull request "OP-61 Improve qtt installation." (to solve issue #613 ). In that particular case, the user install method does not include the PyQt5 dependency (in setup.py), whereas the test build install method does include it (in requirements.txt).

If I (accidentally) close the VideoMode (qtt.measurements.videomode.VideoMode) without pressing the stop button, the behaviour of the rest of the code is unexpected. For example, all AWGs keep running and if I then immediately shut off all communications afterwards (sorry for my setup-specific code):

rfa.off()
rfb.off()
awg.stop()
wet_8dot.close()

I will get the following error:

ERROR:root:could not find instrument digitizer
Traceback (most recent call last):
  File "d:\users\cjvdiepen\qtt\qtt\measurements\videomode.py", line 342, in updatebg
    dd = self.datafunction()
  File "d:\users\cjvdiepen\qtt\qtt\measurements\videomode.py", line 69, in __call__
    minstrumenthandle = qtt.measurements.scans.get_instrument(self.minstrument)
  File "d:\users\cjvdiepen\qtt\qtt\measurements\scans.py", line 249, in get_instrument
    raise Exception('could not find instrument %s' % str(instr))
Exception: could not find instrument digitizer
VideoMode: Exception in updatebg, stopping readout
VideoMode: stop readout

I'm unsure if closing the window without stopping is a feature or a bug, but I would expect the VideoMode to execute the closing routine when I close the window?

Kind regards,
Steven

Before final release remove old code:

• scans.py (moved into measurements/)

• debug.py (move into utilities)

• live and live_plotting.py (move into measurements and/or gui)

• loggingGUI.py (move into gui)

• reports.py (remove)

• tools.py (move into utilities?)

• test.py (need used any more?)

• pgeometry.py (move into utilities?)

• structures.py (?)

• Go through all files and remove dead code

• Document main functions

Optional:

• Convert ttraces to qctoolkit

Scikit-image 0.17 was released with some non-backwards compatible changes to peak_local_max (scikit-image/scikit-image#4501). The qtt tests are broken, and probably the functionality as well.

@QFer @sldesnoo-Delft

ToDo extensions of the videomode:

• Get coordinates via mouse click callback
• Running average / Continue averaging until event occurs
• Naverage in GUI
• Marker at the center to show your current position?

Wishlist:

• "Click and drag"
• Add lines to live plot
• Save current image with higher averaging, hence higher SNR
• Change AWG amplitude while sweeping to change the sweeprange
• Measure and tune sensing dot directly from 2d scan, possibly via push button
• Set all gates to zero with push button

When using the virtualAWG in video-mode, using virtual gates. Using a virtual gate space that uses (ch1,ch3) for x-axis and (ch2,ch4) for the y-axis. All the proper waveforms are generated, but for some reason, channel 4 does not get enabled. When turned on manually, everything works fine, but this is slightly annoying.

qtt should provide a set of example measurement data of common measurements. This data can be used as input during development and demonstration, also for code that is not part of qtt.

There is already some example data, but it is not included in the distribution and does not have descriptive names.

Changes to be made:

• Include example data in distribution
• Change names to descriptive ones
• Add a function to retrieve the names of the available examples

Rename:
"addition_line_scan" Old name: '17-57-24'
"charge_stability_diagram_anti_crossing" Old name: '2017-01-10/09-45-15'
"polarization_line" Old name: '2017-02-21/15-59-56'
"charge_stability_diagram_anti_crossing_virtual_gates" Old name: '2017-09-04/11-04-14_qtt_scan2Dturbovec'
"PAT_scan_background" Old name: '2017-09-04/11-04-45_qtt_scan1Dfastvec'
"PAT_scan" Old name: '2017-09-04/11-05-17_qtt_scan2Dfastvec'
"coulomb_peak" Old name: 'misc/13-13-16_qtt_sensingdot_t.fastTune'
"charge_stability_diagram_double_dot_system" Old name: 'misc/Lever_arm_Charging_energy/10-06-59_qtt_scan2Dfast' (and perhaps resample to make loading faster)
"charge_stability_diagram_double_dot_system_detail" Old name: 'misc/Lever_arm_Charging_energy/14-28-52_qtt_scan2Dfast'
"charge_stability_diagram_double_dot_system_bias_triangle" Old name: 'misc/Lever_arm_Charging_energy/14-33-26_qtt_scan2Dfast'
"pinchoff_scan_barrier_gate" Old name: 'misc/pinchoff/2018-06-28/13-24-22_qtt_scan1D'
"2electron_pat_background" Old name: 'misc/2electron_pat_26102017/background'
"2electron_pat" Old name: 'misc/2electron_pat_26102017/pat'
"elzerman_detuning_scan" Old name: 'elzerman_detuning_scan/2019-05-28_11-39-39_qtt_generic.json'
'charge_stability_diagram_dac_vs_awg' Old name 'misc/P5_y_P5_x.dat'
'rts_signal' Old name '2018-08-31/19-05-18_qtt_save_segments'

Name already good:
'time_rabi_scan.json'
'frequency_rabi_scan.json'
'double_gaussian_dataset.json'

To be removed from repository:
'misc/awg_to_plunger_scan.pickle'

@eendebakpt, I've extracted the renaming from your email.

To complete:

• Multi-processing
• Current through dots
• Output of SD
• Ohmics
• Temperature parameter, won't implement

@peendebak If times allows, I will have a look... Just posting it here as a reminder 📓.

What I did?:
Checked out the master branch. Installed using pip and ran the unittests with pytest.

What should have happend?:
All unittests should have been executed without warnings.

What happend?:
See the following output.

(spinqubits) D:\***>pytest
============================= test session starts =============================
platform win32 -- Python 3.6.5, pytest-3.6.1, py-1.5.3, pluggy-0.6.0
rootdir: D:\***\qtt, inifile: pytest.ini
collected 63 items

qtt\data.py .........                                                    [ 14%]
blablabla...
qtt\utilities\tools.py .....                                             [100%]
============================== warnings summary ===============================
qtt/measurements/storage.py::test_load_save_state
  C:\Users\tud278322\AppData\Local\conda\conda\envs\lcblom\lib\site-packages\hic
kle\hickle.py:409: SerializedWarning: <class 'collections.OrderedDict'> type not
 understood, data have been serialized
    warnings.warn("%s type not understood, data have been serialized" % type(py_
obj), SerializedWarning)

-- Docs: http://doc.pytest.org/en/latest/warnings.html
=================== 63 passed, 1 warnings in 17.28 seconds ====================
Failed to close virt_gates, ignored
Traceback (most recent call last):
  File "d:\users\lcblom\git\qcodes\qcodes\instrument\base.py", line 511, in clos
e_all
    inst = cls.find_instrument(inststr)
  File "d:\users\lcblom\git\qcodes\qcodes\instrument\base.py", line 608, in find
_instrument
    raise KeyError('Instrument {} has been removed'.format(name))
KeyError: 'Instrument virt_gates has been removed'
Failed to close testivvi, ignored
Traceback (most recent call last):
  File "d:\users\lcblom\git\qcodes\qcodes\instrument\base.py", line 511, in clos
e_all
    inst = cls.find_instrument(inststr)
  File "d:\users\lcblom\git\qcodes\qcodes\instrument\base.py", line 608, in find
_instrument
    raise KeyError('Instrument {} has been removed'.format(name))
KeyError: 'Instrument testivvi has been removed'
Failed to close testvgates, ignored
Traceback (most recent call last):
  File "d:\users\lcblom\git\qcodes\qcodes\instrument\base.py", line 511, in clos
e_all
    inst = cls.find_instrument(inststr)
  File "d:\users\lcblom\git\qcodes\qcodes\instrument\base.py", line 608, in find
_instrument
    raise KeyError('Instrument {} has been removed'.format(name))
KeyError: 'Instrument testvgates has been removed'

Currently, there's an inconsistency with appending phys_gates_vals to the dataset at the end of the scans that use virtual gates. scan1D and scan2D appends them, while in the fast and turbo versions it is not appended. This is my fault, because at some point I commented those lines of code and it seeped through one of the PRs.

Should we make a decision on whether to include them or not? Or perhaps have it as an option? I think the phys_gates_vals could be useful to record, but it's annoying to have to record them as a Measured array type. That's why I've been commenting them out. Could we perhaps add them as part of the metadata instead?

@peendebak @QFer

Is the module Polygon3 only used here:

?

The issue is that due to rounding errors the length of the sweepvalues is not consistent. @CJvanDiepen

A minimal example:

from qtt.scans import *
from qcodes import ManualParameter
from qcodes.instrument_drivers.devices import VoltageDivider
from qtt.instrument_drivers.virtual_instruments import VirtualIVVI
import qcodes
import qtt.measurements.scans
from qtt.measurements.scans import scanjob_t

p = ManualParameter('p'); q = ManualParameter('q')
R=VoltageDivider(p, 4)
gates=VirtualIVVI(name=qtt.measurements.scans.instrumentName('gates'), model=None)
station = qcodes.Station(gates)
station.gates=gates

scanjob = scanjob_t({'minstrument': [R]})
scanjob['sweepdata']={'start': 0, 'range': 100, 'end': 100, 'param': {'dac1': 1.0, 'dac2': 0}, 'wait_time': 0.2, 'step': 0.10775862068965517}
scanjob['stepdata'] = dict({'param': 'dac3', 'start': 24, 'end': 30, 'step': 1.})
data = scan2D(station, scanjob, liveplotwindow=False, verbose=0)

• Installing dulwich from condalist.yml attempts to install the version that includes C bindings, which require Microsoft Visual C++, which is quite heavy and not straight forward to install. This could be fixed by adding --global-option="--pure". I'm not sure if this is easy to do in the condalist.

• qtt.measurements.ttrace requires pycqed, which is now optional for us. But ttrace gets imported in qtt.tools.addPPTslide eventhough I don't really need it. I could just install pycqed but this is a bit annoying.

• Consider adding spirack to condalist.yml (needs to go in the pip list)

@peendebak @lucblom

I would like to run scan2Dfast sweeping with a vector parameter and stepping with a generic instrument (in my case with a MW source for PAT measurements).
Currently, once the scantype gets assigned as scan2dfastvec, the code assumes that both parameters are vector, assigning the gate names to the AWG (for the sweep) and the DAC gates (for the step). By definition, scan2dfast should sweep with AWG, but I think it should be able to step with a generic instrument.

I modified some functions in scans (multi_instruments branch) so that I could run the PAT measurement, but I'm not sure it's a very elegant way to do it. Could you guys have a look at the branch and tell me what you think?

@peendebak @CJvanDiepen

NB: if you have the external module 'mock' installed, you will probably not see this error.

Running pytest on the dev branch in a fresh environment gives the following import errors:
___________________________________________________________ ERROR collecting src/tests/unittests/measurements/acquisition/test_uhfli_stimulus.py ____________________________________________________________ ImportError while importing test module '<workdir>\qtt\src\tests\unittests\measurements\acquisition\test_uhfli_stimulus.py'. Traceback: src\tests\unittests\measurements\acquisition\test_uhfli_stimulus.py:3: in <module> from mock import patch, MagicMock, call E ModuleNotFoundError: No module named 'mock' _______________________________________________________ ERROR collecting src/tests/unittests/measurements/processing/test_signal_processor_runner.py ________________________________________________________ ImportError while importing test module '<workdir>\qtt\src\tests\unittests\measurements\processing\test_signal_processor_runner.py'. Traceback: src\tests\unittests\measurements\processing\test_signal_processor_runner.py:3: in <module> from mock import patch E ModuleNotFoundError: No module named 'mock'

The solution is described here:
https://stackoverflow.com/questions/11501520/importerror-no-module-named-mock
It seems that mock is now part of unittest, so:
from unittest.mock import bla
should work out-of-the-box and should not require the external module mock (I have not tried that though).

[NbConvertApp] Converting notebook docs/notebooks/analysis/example_coulomb_peak.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[IPKernelApp] ERROR | Exception in control handler:
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 201, in dispatch_control
    yield gen.maybe_future(handler(self.control_stream, idents, msg))
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/tornado/gen.py", line 209, in wrapper
    yielded = next(result)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 680, in shutdown_request
    content = yield gen.maybe_future(self.do_shutdown(parent['content']['restart']))
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 444, in do_shutdown
    self.shell.exit_now = True
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/traitlets/traitlets.py", line 585, in __set__
    self.set(obj, value)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/traitlets/traitlets.py", line 574, in set
    obj._notify_trait(self.name, old_value, new_value)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/traitlets/traitlets.py", line 1139, in _notify_trait
    type='change',
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/traitlets/traitlets.py", line 1176, in notify_change
    c(change)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 476, in _update_exit_now
    exit_hook(self.kernel)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/ipykernel/eventloops.py", line 136, in loop_qt_exit
    kernel.app.exit()
AttributeError: 'IPythonKernel' object has no attribute 'app'
[NbConvertApp] Writing 55405 bytes to docs/notebooks/analysis/example_coulomb_peak.nbconvert.ipynb

With link here: https://travis-ci.com/QuTech-Delft/qtt/jobs/220528445

Doing an install from scratch using:
python setup.py develop
I encounter the following error message:
pyqtgraph plotting not supported, try "from qcodes.plots.pyqtgraph import QtPlot" to see the full error

When I do as suggested I get:

>>> from qcodes.plots.pyqtgraph import QtPlot pyqtgraph plotting not supported, try "from qcodes.plots.pyqtgraph import QtPlot" to see the full error Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<envdir>\lib\site-packages\qcodes\plots\pyqtgraph.py", line 6, in <module> import pyqtgraph as pg File "<envdir>\lib\site-packages\pyqtgraph-0.10.0-py3.6.egg\pyqtgraph\__init__.py", line 13, in <module> from .Qt import QtGui File "<envdir>\lib\site-packages\pyqtgraph-0.10.0-py3.6.egg\pyqtgraph\Qt.py", line 153, in <module> from PyQt5 import QtGui, QtCore, QtWidgets, uic ModuleNotFoundError: No module named 'PyQt5.sip'

Looking around on the internet I found that the issue is that PyQt5 is now split into 2 packages: pyqt5 and pyqt5-sip, which are not working together as they should. A workaround is to install PyQt5 with:
pip install -I pyqt5
before installing qtt. Ideally we incorporate this in setup.py, but I don't know how.

The fit_double_gaussian can fail with an error message reporting nan values. A minimal example:

import numpy as np
import qtt

xx=np.arange(10.)
yy=0*np.arange(10)
yy[3]=80.
yy[6]=.1

qtt.algorithms.fitting.fit_double_gaussian(xx, yy)

Notes

A solution might be to redefine the gaussian used in the model to fit a exp(-alpha x+b) instead of a exp(x/sigma + b)

Currently not working. Needs qcodes source code and jsonschema 2.6.0

Recent PR: microsoft/Qcodes#1633 changes the version name to something like 0.4.0+30.g160bfbc45. This is not working with the QTT setup.py installer because the StrictVersion function cannot handle these type of versions.

How to resolve?

After updating my environment and trying to initialize my station I got this error. I updated numpy to the latest version and it fixed it. Consider removing the forced numpy version from condalist.yml

@peendebak @lucblom

On the qubyte setup I have measured a difference in clock speeds between the digitizer and the AWG of 1.002. For the ttraces code we have traces of 5ms with 8 blocks of data. For the last block of data the difference in clocks is clearly visible. For the ttraces code there is a PR #245 to compensate for this.

We need to check whether this makes a difference for the videomode and scan2Dturbo.

@lucblom @Christian-Volk @CJvanDiepen

Issues to be solved:

• Serialization
• Binary storage
• Missing units in current GNUPlotFormatter

If I try to run the GUI from a Jupyter notebook, the Python kernel crashes.
Issue introduced with commit 185eabc ; previously the ParameterViewer and the DataBrowser were just frozen, while LivePlot was correctly working

In the requires list in setup.py there are two (potential) sources of errors when running python setup.py develop:

• qcodes is in this list but it does not appear as an installed package if cloned from git, so I think this will cause an error if qcodes was not installed as a package (note the version requirement for qcodes mentioned a few lines lower)
• the requirement for the version of scipy: 'scipy>=0.18 caused the following error for me:

Traceback (most recent call last):
  File "setup.py", line 55, in <module>
    zip_safe=False,
  File "C:\Users\tud205859\AppData\Local\conda\cond
s\core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "C:\Users\tud205859\AppData\Local\conda\cond
kages\setuptools-27.2.0-py3.6.egg\setuptools\dist.p
  File "C:\Users\tud205859\AppData\Local\conda\cond
s\dist.py", line 253, in __init__
    getattr(self.metadata, "set_" + key)(val)
  File "C:\Users\tud205859\AppData\Local\conda\cond
s\dist.py", line 1207, in set_requires
    distutils.versionpredicate.VersionPredicate(v)
  File "C:\Users\tud205859\AppData\Local\conda\cond
s\versionpredicate.py", line 114, in __init__
    raise ValueError("expected parenthesized list:
ValueError: expected parenthesized list: '>=0.18'

The latest version of qcodes requires the package tqdm. Consider adding it to condalist.yml

Current issues:

• slow start up of the video mode window (with keysight system, upload should be fast?)

• digitizer cashes when switching from 1D to 2D window. Or when doing a measurement in beween that is not video mode.

• video mode often crashes the whole python session, unclear why this happens (could be related to the triggering).

• resolution argument for 1D scans does not work.

• No support for virtual gates?

• no clear way of setting time per point. setting the sample rate of the digitizer is not the way

• Document which instruments are needed
• Document output of VirtualAwg and measuresegment so users can implement their own backend

After the fix applied in pull request "OP-61 Improve qtt installation." installation of pyqtgraph still fails, but for a different reason.
pip install -e . fails because pyqtgraph needs PyQt5, which is not in the requirements list. This can be solved by adding pyqt5 to the install_requires list in setup.py.

I installed qtt as a site-package part of another package.

In my setup.py I have:

qtt = 'qtt @ git+ssh://[email protected]/QuTech-Delft/qtt#egg=qtt'
...
setup(
    ...
    install_requires = [ ..., qtt, ...]
    ...
    )

However, it seems that after the installation completes successfully (after a pip install .)the qtt/exampledata/ folder is not part of my qtt site-packages.

Can someone please have a look at this?
Thank you!

Dulwich is marked in setup.py under install_requires which makes it a mandatory dependency. It is used under qtt.tools for controlling information of the git repository, but is not required to run for normal operation.

When installing qtt via pip install git+https://github.com/vandersypenqutech/qtt, dulwich blocks installation. The normal installation of dulwich requires c++ build tools. An alternative pure-python installation is possible, but since dulwich is not strictly required for running qtt, it is better not to mark it under install_requires.

@eendebakpt What do you think?

There is a memory leak in the VideoMode object. The issue is that closing the VideoMode by closing the window does not destroy the object. A minimal script to show the issue is given below.

For some reason both closing the GUI window and performing a del on the object does not allow the garbage collector to remove the object.

@Christian-Volk For now the only solution is to restart the session after a high number of videomodes have been created

@lucblom Can you look into this?

# -*- coding: utf-8 -*-
""" Example script to show QTT capabilities

@author: eendebakpt
"""

import sys, os
import numpy as np
import pyqtgraph as pg
from collections import OrderedDict
from qtt.measurements.videomode import SimulationDigitizer, simulation_awg
from qtt.pgeometry import memory
import gc
import qtt
import virtualDot

nr_dots = 3
station = virtualDot.initialize(reinit=True, nr_dots=nr_dots)
gates = station.gates
model = station.model

#%% Simple 1D scan loop

gv={'B0': -300, 'B1': 0.13, 'B2': 0.19234, 'B3': -300,
 'D0': 0.022657216955232817, 'O1': -0.2664861053052888,
 'O2': -0.2605, 'O3': -0.19381, 'O4': -0.4512742136797938,
 'O5': -0.13197864131798487, 'P1': 0.23433899666854408,
 'P2': 0.24259605542352514, 'P3': -0.45958107506215873,
 'SD1a': -0.12825567573098984, 'SD1b': -0.27948444495176294,
 'SD1c': 0.41226866804982043, 'bias_1': -0.3937390516145264, 'bias_2': 0.20786852176756687}
gates.resetgates(gv, gv)

#%%


verbose = 1
multiprocess = False

digitizer = SimulationDigitizer(
    qtt.measurements.scans.instrumentName('sdigitizer'), model=station.model)
station.components[digitizer.name] = digitizer

station.awg = simulation_awg(qtt.measurements.scans.instrumentName('vawg'))
station.components[station.awg.name] = station.awg
sweepparams = ['B0', 'B3']
sweepranges = [160, 80]
resolution = [80, 48]
minstrument = (digitizer.name, [0, 1])
station.model.sdnoise = .1
vm = qtt.measurements.videomode.VideoMode(station, sweepparams, sweepranges, minstrument, Naverage=25,
               resolution=resolution, sample_rate='default', diff_dir=None,
               verbose=1, nplots=None, dorun=True)

self = vm
vm.setGeometry(1310, 100, 800, 800)

#%%
for ii in range(10):
    print('loop %d: memory usage: %.1f MB' %(ii, memory() ) )
    gc.collect()
    
    vm = qtt.measurements.videomode.VideoMode(station, sweepparams, sweepranges, minstrument,verbose=0,
               resolution=resolution, sample_rate='default',  dorun=True)
    vm.stopreadout()
    #time.sleep(0.05)
    vm.mainwin.close()
    
    ll=qtt.pgeometry.list_objects(objectclassname='VideoMode', verbose=0)
    print('there are %d VideoMode objects in memory '  % len(ll))
#%% Solve?
if 0:
    ll=qtt.pgeometry.list_objects(objectclassname='VideoMode', verbose=0)
    for ii,l in enumerate(ll):    
        print('delete %s'  % l)
        del l
        print('loop %d: memory usage: %.1f MB' %(ii, memory() ) )
        gc.collect()

We should change the VirtualDAC to allow input like gate_map={'P0': (0,'DAC_0'), 'P1': (0,'DAC_1')}. This means that the VirtualDAC can be used with DACs other than the IVVI and SPI D5a.

The parater viewer is quite a handy tool to manage the voltages on the samples, but sometimes there are a few issues that make it harder to work with. The following things are missing a bit for our measurements at the moment (these should be quite useful for all users):

• are you sure notification if you make big jumps (e.g. > 200mV) (example scenario if you wanted to go to 200 mV but accidentally pressed 2000)
• true multithreading/processing (so no hanging of the parameter viewer anymore), set voltages during the measurement.
• Tab with virtual gates (one for every real one)
• Tab with settings (e.g. step taken when pressing up a voltage, stepsize of the dacs, boundaries, ...)

A minimal example:

import qcodes.data
import qtt
import numpy as np


#%% Fails for default formatter, but not for HDF5 formatter
print(qcodes.DataSet.default_formatter)
fractions_dataset=qtt.data.makeDataSet1Dplain('x with space', np.arange(4.), 'y', np.arange(4.)+.5)
fractions_dataset.write(write_metadata=True)
fractions_dataset.finalize() # works!
ds=qtt.data.load_dataset(fractions_dataset.location, verbose=2) # fails!
print(ds)


#%% Always fails
fractions_dataset=qtt.data.makeDataSet1Dplain('x with pi/2', np.arange(4.), 'y', np.arange(4.)+.5)
fractions_dataset.write(write_metadata=True)
fractions_dataset.finalize()
ds=qtt.data.load_dataset(fractions_dataset.location, verbose=2)  # fails!
print(ds)

The formatter should give an error when writing with an invalid name, or the makeDataSet method should already raise an error.

@sldesnoo-Delft @QFer

Maybe by including a dropdown menu for all measured datasets? Showing the metadata would also be a nice feature...
@peendebak what are the chances the of the qcodes data management changing completely in the near future? should we even bother?

@lpetitTUD @NicoHendrickx

I'm getting the following error when pip tries to collect pyzmqrc:
Could not find a version that satisfies the requirement pyzmqrc (from versions: ) No matching distribution found for pyzmqrc

@peendebak

The addPPT_dataset function creates a new qtplot which is added to the .plots list and hence not garbage collected. Therefore up to 100 plots are kept in memory

For a scan1Dfast the following scanjob is accepted without warning:

{'minstrument': 2,
 'minstrumenthandle': "('digitizer', 2)",
 'scantype': 'scan1Dfast',
 'sweepdata': {'end': -900,
  'param': 'X1',
  'start': 100,
  'step': -0.5393743257820928,
  'wait_time': 0.5}}

The range (1000 mV) is too large for the awg, so a warning should be emitted and/or the range reduced. In the resulting dataset the range is not reduced.

@CJvanDiepen

Thins to do before we can open source qtt:

• Check on the example notebooks (including the code) on correctness and sufficient documentation. Remove any notebooks that are not up to standards from the documentation
• Finish documentation
• Make apidoc links work...
• Write announcement for QuTech update
• Transfer to new organisation https://github.com/qutech-delft [done]
• Cleanup legacy code #359
• Add build system for read-the-docs [running on master]
• Add unit testing to Travis [done]
• Run selected notebooks as part of testing [done on Jenkins]
• Fix example notebook titles, order into subgroups
• Go through the source and remove any third-party code
• Email to spin-qubits to announce
• Good example of virtual gate matrices
• Create pip package on pypi
• Add check on Polygon3
• Build RTD for dev branch as well
• Tuning the sensing dot -> rerun on setup and record data

Optional

• Add coverage webhook to github
• Make opencv optional again
• Write joss paper
• Fix testdata (where to install them?)

Can we separate the VideoMode GUI from the commands that program the AWG for the fast maps? I would like to use it for our transport maps in the Ge system, but would prefer to keep our own upload code. In general, I am reluctant to run any code that talks to the AWG and/ or DACs without exactly knowing what it will do. Therefore, it would be great if the more 'dangerous', hardware specific code could be put together more clearly. For example, the preparation / uploading / running of the AWG could be separated into a few functions such that I can define a child class and overwrite those? That would also make the code more readable, imho. Even nicer would be the definition of a generic videomode class, with subclass-defined prepare, run, read etc functions...

@peendebak @lucblom @fvanriggelen

For most of the users opencv is useless, but it's mandatory to load qtt. It should be treated as optional dependency.

The VirtualAwg.__volt_to_millivolt is set to 0.001 which seems confusing. To convert a volt to millitvolt one should multiply by a factor 1000.