Skip to content
Snippets Groups Projects
Commit 70b1eb46 authored by Günter Quast's avatar Günter Quast
Browse files

removed mimoCoRB interface from this release - now in repo redpitaya-daq

parent 1eaafc62
No related branches found
No related tags found
No related merge requests found
...@@ -36,16 +36,9 @@ courses at the Faculty of Physics at Karlsruhe Institute of Technology. ...@@ -36,16 +36,9 @@ courses at the Faculty of Physics at Karlsruhe Institute of Technology.
- *README.md* this documentation - *README.md* this documentation
- *mchpa.py* client program - *mchpa.py* client program
- *redPdaq.py* client for data-acquisition using the mcpha server
- *examples/* recorded spectra - *examples/* recorded spectra
- *examples/peakFitter.py* code to find and fit peaks in spectum data - *examples/peakFitter.py* code to find and fit peaks in spectum data
- *read_npy.py* a simple example to read waveforms saved in *.npy* format
- *redP_mimocorb.py* runs *redPdaq* as a client of the buffer manager *mimoCoRB*
- *setup.yaml* coniguration script defining the *mimoCoRB* application
- *modules/* and *config/* contain code and configuration files for the *redP_mimoCoRB* application
- *mcpha.ui* qt5 graphical user interface for *mcpha* application - *mcpha.ui* qt5 graphical user interface for *mcpha* application
- *rpControl.ui* qt5 tab for *redPdaq* application
- *mcpha_daq.ui* qt5 tab for oscilloscope with daq mode
- *mcpha_gen.ui* qt5 tab for generator - *mcpha_gen.ui* qt5 tab for generator
- *mcpha_osc.ui* qt5 tab for oscilloscope - *mcpha_osc.ui* qt5 tab for oscilloscope
- *mcpha_hst.ui* qt5 tab for histogram display - *mcpha_hst.ui* qt5 tab for histogram display
...@@ -121,31 +114,6 @@ Note that spectra and waveforms are plotted with a very large number of channels ...@@ -121,31 +114,6 @@ Note that spectra and waveforms are plotted with a very large number of channels
the resolution of a computer display. It is therefore possible to use the looking-glass button the resolution of a computer display. It is therefore possible to use the looking-glass button
of the *matplotlib*window to mark regions to zoom in for a detailed inspection of the data. of the *matplotlib*window to mark regions to zoom in for a detailed inspection of the data.
## Oscilloscope and data recorder
The script *redPdaq.py* relies on the same server and FPGA image as the pulse-height analyzer,
providing the same functionality as *mcpha.py*. In addition, however, there is a button
"*Start DAQ*" in the oscilloscope tab to run the oscilloscope in data acquisition mode,
i.e. continuously. A subset of the data is shown in the oscilloscope display, together with
information on the trigger rate and the transferred data volume. A configurable user-defined
function may also be called to analyse and store the recorded waveforms.
It is possible to transfer data over a one-Gbit network from the RedPitaya with a rate of 50 MB/s
or about 500 waveforms/s.
An examples of call-back functions callable from within redPdaq is provided with the package
- redP_consumer()
calculates and displays statistics on trigger rate and data volume
*redP_mimocorb.py* is a script containing code to be started from the command line and
a function defined in the script, *redP_to_rb*, is called as sub-process within the
*mimiCoRB* buffer manager frame-work for more advanced data analysis tasks requiring
multiple processes running in parallel. A *mimoCoRB* setup-file is also provided and can
be started by running typing `redP_mimoCoRB.py setup.yaml` on the command line. Modules
and configuration files for a pulse-height analysis of recorded signals are contained
as exampless in the sub-directories *modules/* and *config/*, respectively.
## Installation ## Installation
...@@ -206,9 +174,6 @@ Then, on the client side: ...@@ -206,9 +174,6 @@ Then, on the client side:
the *oscilloscpe* tab; the *oscilloscpe* tab;
- now click the tab *spectrum histogram 1*; adjust the amplitude threshold and time - now click the tab *spectrum histogram 1*; adjust the amplitude threshold and time
of exposure, then click the *Start* button and watch the spectrum building up; of exposure, then click the *Start* button and watch the spectrum building up;
- if running `redPdaq.py`, threre is a buttton "StartDQQ"; click it to cintinuously transfer
waveform data to the client computer. If a filename was specified, data is recorded to disk
in *.npy* format; note that any active spectrum rab is put in paused mode if DAQ is active.
- when finished, use the *Save* button to save the spectrum to a file with a - when finished, use the *Save* button to save the spectrum to a file with a
meaningful name. meaningful name.
......
# Dict with uid's as key and a nested dict with configuration variables
general:
runtime: 600 # desired runtime in seconds
runevents: &number_of_events 100000
number_of_samples: &number_of_samples 1000
analogue_offset: &analogue_offset 0
# special settings for RedPitaya
decimation_index: &decimation 4 # decimation 0:1, i:2^(i+1) # data decimation factor, 0 is 8 ns/sample
sample_time_ns: &sample_time_ns 256 # for decimation 4
invert_channel1: &invert1 0
invert_channel2: &invert2 0
trigger_level: &trigger_level 50
trigger_channel: &trigger_channel '1'
trigger_direction: &trigger_direction "rising" # or "falling"
pre_trigger_samples: &pre_trigger_samples 103 # 5%
# dict for spectrum_filter, function find_peaks
find_peaks:
# signal_characteristics: 256ns/sample, duration 50µs ( ~200 samples)
sample_time_ns: *sample_time_ns
analogue_offset: *analogue_offset
number_of_samples: *number_of_samples
pre_trigger_samples: *pre_trigger_samples
peak_minimal_prominence: 200 # has to be positive and higher than avg. noise peaks to not cause havoc!
trigger_channel: *trigger_channel
peak_minimal_distance: 400 # minimal distance between two peaks in number of samples
peak_minimal_width: 100 # in number of samples
trigger_channel: *trigger_channel
trigger_position_tolerance: 20 # in number of samples
# dict for RedPitaya redPoscdaq
redP_to_rb:
ip_address: '192.168.0.103'
eventcount: *number_of_events
sample_time_ns: *sample_time_ns
number_of_samples: *number_of_samples
pre_trigger_samples: *pre_trigger_samples
trigger_channel: *trigger_channel
trigger_level: *trigger_level
trigger_mode: "norm" # or "auto"
# special settings for RedPitaya
decimation_index: *decimation
invert_channel1: *invert1
invert_channel2: *invert2
# Dict for simul_source.py
simul_source:
sample_time_ns: *sample_time_ns
number_of_samples: *number_of_samples
pre_trigger_samples: *pre_trigger_samples
analogue_offset: *analogue_offset
eventcount: *number_of_events
sleeptime: 0.03
random: true
# Dict for push_simul
push_simul:
sample_time_ns: *sample_time_ns
number_of_samples: *number_of_samples
pre_trigger_samples: *pre_trigger_samples
analogue_offset: *analogue_offset
eventcount: *number_of_events
sleeptime: 0.03
random: true
save_to_txt:
filename: "spectrum"
save_parquet:
filename: "spectrum"
plot_waveform:
title: "Muon waveform"
min_sleeptime: 0.5 # time to wait between graphics updates
number_of_samples: *number_of_samples
sample_time_ns: *sample_time_ns
analogue_offset: *analogue_offset # analogue offset in V
pre_trigger_samples: *pre_trigger_samples
channel_range: 4096 # channel range in mV
trigger_channel: *trigger_channel # Channel name in the PicoScope. Valid values are 'A', 'B', 'C' or 'D'
trigger_level: *trigger_level # value in mV, take account of analogue_offset, which is added to input voltage !
plot_histograms:
title: "on-line histograms"
# define histograms
histograms:
# name min max nbins ymax name lin/log
ch1_height: [50., 3000., 250, 20., "ph 1A", 0]
ch2_height: [50., 3000., 250, 20., "ph 1B", 0]
from mimocorb import mimo_buffer as bm
from scipy import signal
import numpy as np
from numpy.lib import recfunctions as rfn
import sys
import os
def normed_pulse(ch_input, position, prominence, analogue_offset):
# > Compensate for analogue offset
ch_data = ch_input - analogue_offset
# > Find pulse area
# rel_height is not good because of the quantized nature of the picoscope data
# so we have to "hack" a little bit to always cut 10mV above the analogue offset
width_data = signal.peak_widths(ch_data, [int(position)], rel_height=(ch_data[int(position)] - 10) / prominence)
left_ips, right_ips = width_data[2], width_data[3]
# Crop pulse area and normalize
pulse_data = ch_data[int(np.floor(left_ips)) : int(np.ceil(right_ips))]
pulse_int = sum(pulse_data)
pulse_data *= 1 / pulse_int
return pulse_data, int(np.floor(left_ips)), pulse_int
def correlate_pulses(data_pulse, reference_pulse):
correlation = signal.correlate(data_pulse, reference_pulse, mode="same")
shift_array = signal.correlation_lags(data_pulse.size, reference_pulse.size, mode="same")
shift = shift_array[np.argmax(correlation)]
return shift
def tag_peaks(input_data, prominence, distance, width):
peaks = {}
peaks_prop = {}
for key in input_data.dtype.names:
peaks[key], peaks_prop[key] = signal.find_peaks(
input_data[key], prominence=prominence, distance=distance, width=width
)
return peaks, peaks_prop
def correlate_peaks(peaks, tolerance):
m_dtype = []
for key in peaks.keys():
m_dtype.append((key, np.int32))
next_peak = {}
for key, data in peaks.items():
if len(data) > 0:
next_peak[key] = data[0]
correlation_list = []
while len(next_peak) > 0:
minimum = min(next_peak.values())
line = []
for key, data in peaks.items():
if key in next_peak:
if abs(next_peak[key] - minimum) < tolerance:
idx = data.tolist().index(next_peak[key])
line.append(idx)
if len(data) > idx + 1:
next_peak[key] = data[idx + 1]
else:
del next_peak[key]
else:
line.append(-1)
else:
line.append(-1)
correlation_list.append(line)
array = np.zeros(len(correlation_list), dtype=m_dtype)
for idx, line in enumerate(correlation_list):
array[idx] = tuple(line)
return array
def match_signature(peak_matrix, signature):
if len(signature) > len(peak_matrix):
return False
# Boolean array with found peaks
input_peaks = rfn.structured_to_unstructured(peak_matrix) >= 0
must_have_peak = np.array(signature, dtype=np.str0) == "+"
must_not_have_peak = np.array(signature, dtype=np.str0) == "-"
match = True
# Check the signature for each peak (1st peak with 1st signature, 2nd peak with 2nd signature, ...)
for idx in range(len(signature)):
# Is everywhere a peak, where the signature expects one -> Material_conditial(A, B): (not A) OR B
first = (~must_have_peak[idx]) | input_peaks[idx]
# Is everywhere no peak, where the signature expects no peak -> NAND(A, B): not (A and B)
second = ~(must_not_have_peak[idx] & input_peaks[idx])
match = match & (np.all(first) & np.all(second))
return match
if __name__ == "__main__":
print("Script: " + os.path.basename(sys.argv[0]))
print("Python: ", sys.version, "\n".ljust(22, "-"))
print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")
"""
**plot_histograms**: histogram variable(s) from buffer using mimoCoRB.histogram_buffer
"""
import sys
import os
from mimocorb.histogram_buffer import histogram_buffer
import matplotlib
# select matplotlib frontend if needed
matplotlib.use("TkAgg")
def plot_histograms(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
"""
Online display of histogram(s) of variable(s) from mimiCoRB buffer
:param input: configuration dictionary
"""
histbuf = histogram_buffer(source_list, sink_list, observe_list, config_dict, **rb_info)
histbuf()
if __name__ == "__main__":
print("Script: " + os.path.basename(sys.argv[0]))
print("Python: ", sys.version, "\n".ljust(22, "-"))
print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")
"""
**plot**: plotting waveforms from buffer using mimoCoRB.buffer_control.OberserverData
"""
import sys
import os
from mimocorb.plot_buffer import plot_buffer
import matplotlib
# select matplotlib frontend if needed
matplotlib.use("TkAgg")
def plot_waveform(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
"""
Plot waveform data from mimiCoRB buffer
:param input: configuration dictionary
- plot_title: graphics title to be shown on graph
- min_sleeptime: time between updates
- sample_time_ns, channel_range, pretrigger_samples and analogue_offset
describe the waveform data as for oscilloscope setup
"""
pltbuf = plot_buffer(source_list, sink_list, observe_list, config_dict, **rb_info)
pltbuf()
if __name__ == "__main__":
print("Script: " + os.path.basename(sys.argv[0]))
print("Python: ", sys.version, "\n".ljust(22, "-"))
print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")
"""
**simul_source**: a simple template for a mimoCoRB source to
enter simulated wave form data in a mimoCoRB buffer.
Input data is provided as numpy-arry of shape (number_of_channels, number_of_samples).
"""
from mimocorb.buffer_control import rbImport
import numpy as np
import sys, time
from pulseSimulator import pulseSimulator
def simul_source(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
"""
Generate simulated data and pass data to buffer
The class mimocorb.buffer_control/rbImport is used to interface to the
newBuffer and Writer classes of the package mimoCoRB.mimo_buffer
This example may serve as a template for other data sources
:param config_dict: configuration dictionary
- events_required: number of events to be simulated or 0 for infinite
- sleeptime: (mean) time between events
- random: random time between events according to a Poission process
- number_of_samples, sample_time_ns, pretrigger_samples and analogue_offset
describe the waveform data to be generated (as for oscilloscope setup)
Internal parameters of the simulated physics process (the decay of a muon)
are (presently) not exposed to user.
"""
events_required = 1000 if "eventcount" not in config_dict else config_dict["eventcount"]
def yield_data():
"""generate simulated data, called by instance of class mimoCoRB.rbImport"""
event_count = 0
while events_required == 0 or event_count < events_required:
pulse = dataSource(number_of_channels)
# deliver pulse data and no metadata
yield (pulse, None)
event_count += 1
dataSource = pulseSimulator(config_dict)
simulsource = rbImport(config_dict=config_dict, sink_list=sink_list, ufunc=yield_data, **rb_info)
number_of_channels = len(simulsource.sink.dtype)
# possibly check consistency of provided dtype with simulation !
# TODO: Change to logger!
# print("** simul_source ** started, config_dict: \n", config_dict)
# print("?> sample interval: {:02.1f}ns".format(osci.time_interval_ns.value))
simulsource()
"""Module save_files to handle file I/O for data in txt and parquet format
This module relies on classes in mimocorb.buffer_control
"""
import sys
import os
from mimocorb.buffer_control import rb_toTxtfile, rb_toParquetfile
# def save_to_txt(source_dict):
def save_to_txt(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
sv = rb_toTxtfile(source_list=source_list, config_dict=config_dict, **rb_info)
sv()
# print("\n ** save_to_txt: end seen")
def save_parquet(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
sv = rb_toParquetfile(source_list=source_list, config_dict=config_dict, **rb_info)
sv()
if __name__ == "__main__":
print("Script: " + os.path.basename(sys.argv[0]))
print("Python: ", sys.version, "\n".ljust(22, "-"))
print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")
"""Module **pulse_filter**
This (rather complex) module filters waveform data to search for valid signal pulses.
The code first validates the trigger pulse, identifies coincidences of signals in
different layers (indiating the passage of a cosmic ray particle, a muon) and finally
searches for double-pulse signatures indicating that a muon was stopped in or near
a detection layer where the resulting decay-electron produced a delayed pulse.
The time difference between the initial and the delayed pulses is the individual
lifetime of the muon.
The decay time and the properties of the signal pulses (height, integral and
postition in time) are written to a buffer; the raw wave forms are optionally
also written to another buffer.
The callable functions *find_peaks()* and *calulate_decay_time()* depend on the
buffer manager *mimoCoRB* and provide the filter functionality described above.
These functions support multiple sinks to be configured for output.
The relevant configuration parameters can be found in the section *find_peaks:*
and *calculate_decay_time:* in the configuration file.
"""
from mimocorb.buffer_control import rbTransfer
import numpy as np
import pandas as pd
import os, sys
from filters import *
def find_peaks(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
"""filter client for mimoCoRB: Find valid signal pulses in waveform data
Input:
- wave form data from source buffer defined in source_list
Returns:
- None if filter not passed
- list of list(s) of pulse parameters, written to sinks defined in sink_list
"""
if config_dict is None:
raise ValueError("ERROR! Wrong configuration passed (in lifetime_modules: calculate_decay_time)!!")
# Load configuration
sample_time_ns = config_dict["sample_time_ns"]
analogue_offset = config_dict["analogue_offset"]*1000
peak_minimal_prominence = config_dict["peak_minimal_prominence"]
peak_minimal_distance = config_dict["peak_minimal_distance"]
peak_minimal_width = config_dict["peak_minimal_width"]
pre_trigger_samples = config_dict["pre_trigger_samples"]
trigger_channel = config_dict["trigger_channel"]
# if trigger_channel not in ['A','B','C','D']:
if trigger_channel not in ['1','2']:
trigger_channel = None
trigger_position_tolerance = config_dict["trigger_position_tolerance"]
pulse_par_dtype = sink_list[-1]['dtype']
def tag_pulses(input_data):
"""find all valid pulses
This function to be called by instance of class mimoCoRB.rbTransfer
Args: input data as structured ndarray
Returns: list of parameterized pulses
"""
# Find all the peaks and store them in a dictionary
peaks, peaks_prop = tag_peaks(input_data, peak_minimal_prominence, peak_minimal_distance, peak_minimal_width)
# identify trigger channel, validate trigger pulse and get time of trigger pulse
if trigger_channel is not None:
trigger_peaks = peaks['ch' + trigger_channel]
if len(trigger_peaks) == 0:
return None
reference_position = trigger_peaks[np.argmin(np.abs(trigger_peaks - pre_trigger_samples))]
else: # external or no trigger: set to nominal position
reference_position = pre_trigger_samples
peak_data= np.zeros( (1,), dtype=pulse_par_dtype)
for key in peaks.keys():
for position, height, left_ips, right_ips in zip(
peaks[key], peaks_prop[key]['prominences'],
peaks_prop[key]['left_ips'], peaks_prop[key]['right_ips']):
if np.abs(reference_position - position) < trigger_position_tolerance:
peak_data[0][key+'_position'] = position
peak_data[0][key+'_height'] = input_data[key][position] - analogue_offset #height
left = int(np.floor(left_ips))
right = int(np.ceil(right_ips))
peak_data[0][key+'_integral'] = \
sum(input_data[key][left:right] - analogue_offset) * sample_time_ns * 1e-9/50/5
return [peak_data]
p_filter = rbTransfer(source_list=source_list, sink_list=sink_list, config_dict=config_dict,
ufunc=tag_pulses, **rb_info)
p_filter()
if __name__ == "__main__":
print("Script: " + os.path.basename(sys.argv[0]))
print("Python: ", sys.version, "\n".ljust(22, '-'))
print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")
#!/usr/bin/env python3
"""read file from redPoscdaq (in npy format) and display data
"""
from npy_append_array import NpyAppendArray
import numpy as np
import sys
import matplotlib.pyplot as plt
data = np.load(sys.argv[1], mmap_mode='r')
print("data read sucessfully, shape = ", data.shape)
n_samples = len(data[0,0])
xplt = 0.5 + np.linspace(0, n_samples, num=n_samples, endpoint=True)
fig = plt.figure("Oscillogram", figsize=(8,6))
for d in data:
plt.plot(xplt, d[0], '-')
plt.plot(xplt, d[1], '-')
plt.xlabel("time bin")
plt.ylabel("Voltage")
plt.show()
#!/usr/bin/env python3
"""
**redP_mimoCoRB**: use mimoCoRB with the RedPitaya and redPoscdaq.py
Input data is provided as numpy-arry of shape (number_of_channels, number_of_samples)
via callback of the __call__() function in class redP_mimoCoRB.
This script depends on redPdaq.py and is started as a sup-process within the mimoCoRB
framework. The detailed set-up of ring buffers and the associated funtions is specified
in a configuration file, *setup.yaml*. The process suite is started by running this
script from the command line, possibly specifying the name of the configutation file
as an argument.
As a demonstration, a configuration *setup.yaml* is contained in this package to
import waveforms from the RedPitaya, display a sub-set of the waveforms and perform
a pulse-height analysis with updating results shown as histograms.
To run this example, connect the out1 of the RedPitaya to one or both of the inputs,
type "./redP_mimoCoRB.py* to run the example and use the graphical interface to connect
the RedPitaya to the network, start the pulse generator, and finally press the *StartDAQ"
button in the oscilloscope tag to start data transfer to the *mimiCoRB* input buffer.
Stop data taking with the button "End run" in the *mimoCoRB* conotrol window to
cleanly shut down all processes.
Note that this script depends on additional code for the *mimoCoRB* peak finder
in the sub-directory *modules/* and a corresponding configuration file in
subdirectory *config/*.
"""
import time
import sys
import redPdaq as rp
from mimocorb.buffer_control import rbPut
class redP_mimocorb():
""" Interface for redPoscdaq.py to the daq rinbuffer mimoCoRB
"""
def __init__(self, source_list=None, sink_list=None, observe_list=None, config_dict=None,
**rb_info):
# initialize mimoCoRB interface
self.rb_exporter = rbPut(config_dict=config_dict, sink_list=sink_list, **rb_info)
self.number_of_channels = len(self.rb_exporter.sink.dtype)
self.events_required = 1000 if "eventcount" not in config_dict else config_dict["eventcount"]
self.event_count = 0
self.active=True
def __call__(self, data):
"""function called by redPoscdaq
"""
if (self.events_required == 0 or self.event_count < self.events_required) and self.active:
# deliver pulse data and no metadata
active = self.rb_exporter(data, None) # send data
self.event_count += 1
else:
active = self.rb_exporter(None, None) # send None when done
print("redPoscdaq exiting")
sys.exit()
def redP_to_rb(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
"""Main function,
executed as a multiprocessing Process, to pass data from the RedPitaya to a mimoCoRB buffer
:param config_dict: configuration dictionary
- events_required: number of events to be taken
- number_of_samples, sample_time_ns, pretrigger_samples and analogue_offset
- decimation index, invert flags, trigger mode and trigger direction for RedPitaya
"""
# initialize mimocorb class
rb_source= redP_mimocorb(config_dict=config_dict, sink_list=sink_list, **rb_info)
#print("data source initialized")
# start oscilloscope in callback mode
#print("starting osci")
rp.run_rpControl(callback=rb_source, conf_dict=config_dict)
if __name__ == "__main__": # --------------------------------------
#run mimoCoRB data acquisition suite
# the code below is idenical to the mimoCoRB script run_daq.py
import argparse
import sys, time
from mimocorb.buffer_control import run_mimoDAQ
# define command line arguments ...
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('filename', nargs='?', default = "setup.yaml",
help = "configuration file")
parser.add_argument('-v','--verbose', type=int, default=2,
help="verbosity level (2)")
parser.add_argument('-d','--debug', action='store_true',
help="switch on debug mode (False)")
# ... and parse command line input
args = parser.parse_args()
print("\n*==* script " + sys.argv[0] + " running \n")
daq = run_mimoDAQ(args.filename, verbose=args.verbose, debug=args.debug)
daq.setup()
daq.run()
print("\n*==* script " + sys.argv[0] + " finished " + time.asctime() + "\n")
This diff is collapsed.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment