CHANGELOG

-3-Feb-2024 initial commit 
+03-Feb-2024 initial commit
+11-Apr-2024 added helper script peakFitter.py to fit peaks to spectrum
+02-Jun-2024 added callback mechanism to analyse and store oscillogram data,
+            helper script read_npy.py to read exported data
+10-Jun-2024 added mimoCoRB interface redP_mimoCoRB.py
+16-Jun-2024 new app redPdaq.py for MCPHA and DAQ

README.md

@@ -7,7 +7,7 @@ for physics laboratory courses
 
 The RedPitaya is a small, credit-card sized single board computer with a dual-core ARM Cortex-A processor 
 and a XILINX Zynq 7010 FPGA. The board contains two fast ADCs and two DACs with 14 bit resolution running
-at a sampling frequency of 125 MHz. Extension connector provide general purpose IO pins with
+at a sampling frequency of 125 MHz. Extension connectors provide general purpose IO pins with
 slow analog inputs and outputs and support for serial bus interfaces like I²C, SPI and UART. 
 The system runs under Ubuntu Linux, which provides network access and supports a wide range of
 applications running on the board. 
@@ -17,22 +17,27 @@ analyzer can be realized on this board by simply changing the FPGA image and the
 
 ![Image of the RedPitaya board (source: `redpitaya.com`).](images/RedPitayaBoard-1024x526.png){width=800px}
  
-The MCPHA application for the RedPitaya by Pavel Demin provides a multi-channel pulse-height analyzer and
-consists of an FPGA image and a server process running on the RedPitaya board. A client script controls
-the server and pulls the data to the client computer. 
+The MCPHA application by Pavel Demin provides a multi-channel pulse-height analyzer as
+well as an oscilloscope capable of transferring large data rates reaching the theoretical
+limit of the Gbit ethernet port. The package consists of an FPGA image and a server process
+running on the RedPitaya board. A client script controls the server and pulls the data
+to the client computer.
+
+The package provided here extends the original one by a possibility to record or export
+waveform data and provides helper scripts to analyze recorded spectra or to read exported
+waveform data. An interface to the buffer manager *mimoCoRB* for buffering and parallel
+processing of large date volumes is also provided.
+
+This package is in use for gamma-ray spectrography experiments in physics laboratory
+courses at the Faculty of Physics at Karlsruhe Institute of Technology.
 
 
 ### Files:
 
   - *README.md*         this documentation 
   - *mchpa.py*          client program 
-  - *rePoscidaq.py*     a simple oscilloscope and daq client using the mcpha server 
-  - *rpControl.ui*      qt5 tab for *redPosci* application   
-  - *mcpha_daq.ui*      qt5 tab for oscilloscope with daq mode
-  - *redP_mimocorb.py*  runs *redPoscdaq* as a client of the buffer manager *mimoCoRB*
-  - *setup.yaml*        coniguration script defining the *mimoCoRB* application
-  - *run_daq.py*        script to strat the *mimoCoRB* application
-  - *modules/* and *config/* contain code and configuration files for the *redP_mimoCoRB* application
+  - *examples/*         recorded spectra
+  - *examples/peakFitter.py* code to find and fit peaks in spectum data
   - *mcpha.ui*          qt5 graphical user interface for *mcpha* application 
   - *mcpha_gen.ui*      qt5 tab for generator 
   - *mcpha_osc.ui*      qt5 tab for oscilloscope
@@ -40,46 +45,60 @@ the server and pulls the data to the client computer.
   - *mcpha_log.ui*      qt5 tab for message display
   - RP-image  directory with all files necessary to boot a RedPitaya and start the server
     application based on the "small, simple and secure" linux distribution
-    [alpine-3.18-armv7-20240204](https://github.com/pavel-demin/red-pitaya-notes/releases/tag/20240204)
-
+    [alpine-3.18-armv7-20240204](https://github.com/pavel-demin/red-pitaya-notes/releases/tag/20240204) 
   - utility scripts in the sub-directory *helpers/*
 
 
 ## Credit:
 
-This project essentially is a fork of the sub-directory *projects/mcpha* in a project by
-[Pavel Demin, [red-pitaya-notes](https://pavel-demin.github.io/red-pitaya-notes).
-
-
-## Multi-Channel Pulse-Height Analyzer for the RedPitaya FPGA board
+*mcpha.py* is a fork of the sub-directory *projects/mcpha* in a project by
+Pavel Demin, [red-pitaya-notes](https://pavel-demin.github.io/red-pitaya-notes).
+*redPdaq.py* is an extension of the oscilloscope class enabling fast restart
+and data export.  
 
-A multi-channel pulse-height analyzer produces a histogram of the heights of pulses present in a signal supplied to the input. The MCPHA project uses the FPGA on the RedPitaya board to process the digitized input signal at very high rates. A server process on the ARM processor of the RedPitaya communicates with a client process via network. The client communicates with the server, starts and stops data recording and receives and displays the data. The client is also responsible for saving data to files. 
 
-The original version by Pavel Demin has been modified to better meet the usual standards for graphics
-displays in physics. A command line interface has also been added to allow easy control of important
-parameters  at program start.
+# Users' Guide
 
-A special version of the original oscilloscope display, *redPosci.py*, with fast transfer of data to the client for data acquisition applications is also contained in  this extended package.  
-
-### Basic functionality 
-
-The *mcpha* application uses a rather simple, but straight-forward algorithm to determine the
-height of pulses.  When the signal voltage of a supplied input signal starts rising, the corresponding 
-ADC count is  stored. A second ADC value is stored when the signal level starts falling again, and the difference of these two ADC values is histogrammed. The histogram is transferred to the client upon request. 
-
-The *mcpha* application also contains a signal generator that runs independently and parallel to the
-pulse-height analyzer. It provides configurable signal shapes and signal rates at the *out1* connector
-of the RedPitaya board. Connecting *out1* with a (short) cable to one of the inputs *in1* or *in2*
-provides input signals that can be be used to familiarize with the functionality and to benchmark
-the performance. 
+## Multi-Channel Pulse-Height Analyzer for the RedPitaya FPGA board
 
-An oscilloscope with very basic functionality to set the trigger level and direction is also provided.
-The timing is controlled by the so-called decimation factor that can be adjusted using the control
-in the upper right corner of the graphical window. The RedPitaya samples data at a constant rate
-of 125 MHz, and the decimation factor determines how many samples are averaged over and stored
-in the internal ring buffer. This reduces the effective sampling rate accordingly. Only decimation
-factors corresponding to powers of two are allowed. An example of randomly occurring exponential
-signal pulses at an average rate of 10 kHz with a fall time of 10 µs is shown below; there is significant 
+A multi-channel pulse-height analyzer produces a histogram of the heights of pulses present
+in a signal supplied to the input. The MCPHA project uses the FPGA on the RedPitaya board
+to process the digitized input signal at very high rates. A server process on the ARM
+processor of the RedPitaya communicates with a client process via network. The client
+communicates with the server, starts and stops data recording and receives and displays
+the data. The client is also responsible for saving data to files. 
+
+The original version by Pavel Demin has been modified to better meet the usual standards
+for graphics displays in physics. A command line interface has also been added to allow
+easy control of important parameters at program start.
+
+A special version of the original oscilloscope display, *redPdaq.py*, with fast transfer
+of data to the client for data acquisition applications is also contained in  this extended
+package.
+
+
+### Basic functionality
+
+The *mcpha* application uses a rather simple, but straight-forward algorithm to determine
+the height of pulses.  When the signal voltage of a supplied input signal starts rising,
+the corresponding ADC count is  stored. A second ADC value is stored when the signal level
+starts falling again, and the difference of these two ADC values is histogrammed. The
+histogram is transferred to the client upon request. 
+
+The *mcpha* application also contains a signal generator that runs independently and parallel
+to the pulse-height analyzer. It provides configurable signal shapes and signal rates at the
+*out1* connector of the RedPitaya board. Connecting *out1* with a (short) cable to one or both
+of the inputs *in1* or *in2* provides input signals that can be be used to familiarize with
+the functionality and to benchmark the performance. 
+
+An oscilloscope with very basic functionality to set the trigger level and direction is also
+provided. The rate of samples transferred to the client is controlled by the so-called decimation
+factor that can be adjusted using the control in the upper right corner of the graphical window.
+The RedPitaya board samples data at a constant rate of 125 MHz, and the decimation factor determines
+how many samples are averaged over and stored in the internal ring buffer. This reduces the e
+ffective sampling rate accordingly. Only decimation factors corresponding to powers of two are
+allowed. An example of randomly occurring exponential signal pulses at an average rate of 10 kHz 
+with a fall time of 10 µs is shown below; there is significant 
 signal overlap in this case, making pulse-height detection more complex.
 
 ![Oscilloscope display of signals with a fall time of 10µs at an average rate of 10 kHz](images/oscilloscope_10mus10kHz.png){width=800px}
@@ -95,36 +114,14 @@ Note that spectra and waveforms are plotted with a very large number of channels
 the resolution of a computer display. It is therefore possible to use the looking-glass button
 of the *matplotlib*window to mark regions to zoom in for a detailed inspection of the data. 
 
-## Oscilloscope and data recorder
-
-The script *redPosci.py* relies on the same server and FPGA image as the pulse-height analyzer.
-The *oscilloscope* and *generator* tabs provide the same functionality as in *mcpha.py*.
-In addition, however,  there is a button "*Start DAQ*" to run the oscilloscope in data acquisition
-mode, i.e. continuously. A subset of the data is shown in the oscilloscope display,together with
-information on the trigger rate and the transferred data volume. A configurable user-defined function
-may also be called to analyse and store the recorded waveforms. 
-It is possible to transfer data over a one-Gbit network from the RedPitaya with a rate of 50 MB/s
-or about 500 waveforms/s.
-
-Two examples of call-back functions callable by redPoscdaq are provided with the package
-
-  - redP_consumer()            
-      calculates and displays statistics on trigger rate and data volume
-
-  - redP_mimocorb()            
-      provides an interface to the buffer manager *mimiCoRB* for more advanced data analysis tasks
-      requiring multiple processes running in parallel. A simple *mimoCoRB* setup is also provided
-      and can be started by *./run_daq setup.yaml*; modules and configuration files for a pulse-height
-      analysis of signals are contained in the subdirectories *modules/* and *config/*, respectively. 
-
 
 ## Installation
 
 The sub-directory *RP-image* contains files to be transferred to a SD card for the RedPitaya board.
 Proceed as follows:
 
-  - copy the contents of the directory *RP-image* to an empty SD card formatted as VFAT32.
-  - connect the RedPitaya to the network via the LAN port 
+  - copy the contents of the directory *RP-image* to an empty SD card formatted as VFAT32;
+  - connect the RedPitaya to the network via the LAN port; 
   - insert the SD card in the RedPitaya and connect the power. 
 
 The RepPitaya directly starts the *mcpha* server application, requests an IP address via DHCP
@@ -132,15 +129,15 @@ and waits for the client program to connect via network.
 
 On the client computer, download the client software:  
 
-  - clone the *mcpha* repository via `git clone https://gitlab.kit.edu/guenter.quast/redpitaya-mcpha`
-  - change directory to the installation directory and start the graphical interface of the client
-    software via `python3 mcpha.py` on the command line. 
+  - clone the *redpitaya-mcpha* repository via `git clone https://gitlab.kit.edu/guenter.quast/redpitaya-mcpha`;
+  - change directory to the installation directory and start the graphical interface of the desired Python
+    client script on the command line. 
  
-The application program *mcpha.py* takes care of initializing the processes on the RedPitaya board 
-through the server process, initiates data transfers from the RedPitaya board to 
-the client computer and provides several tabs to visualize data, generate test data and 
+The application programs *mcpha.py* or *redPdaq.py* take care of initializing the processes on the
+RedPitaya board through the server process, initiate data transfers from the RedPitaya board to 
+the client computer and provide several tabs to visualize data, generate test pulses and 
 to store the acquired spectra. 
- 
+
 
 ### Network connection to the RedPitaya Board
 
@@ -162,21 +159,21 @@ and the server process on the RedPitaya board.
 
 Then, on the client side:
 
-  - start the client program via `python3 mcpha.py`
+  - start the client program from within a terminal via `python3 mcpha.py` or `python3 redPdaq.py`; 
   - in the graphical interface, enter the network address of the RedPitaya in the 
     field next to the orange button and click *connect*; 
     watch out for connection errors in the *Messages* tab!
-     The message "*IO started*" is displayed if everything is ok, and the address turns green. 
+    The message "*IO started*" is displayed if everything is ok, and the address turns green; 
   - click the *oscilloscope* tab, check the trigger level and then start the oscilloscope
-    to see whether signals are arriving at one or both of the RedPitaya inputs. 
-    Adjust the *decimation factor* in the top-right corner of the main display to ensure
+    to see whether signals are arriving at one or both of the RedPitaya inputs;
+    adjust the *decimation factor* in the top-right corner of the main display to ensure
     that the sampling rate is high enough for about 50 samples over the pulse duration. 
   - if no signal source is available, you may click the *generator* tab, set the desired
-    signal parameters and start the generator; connect *out1* of the RedPitaya to the  
-    input *in1* with a (short) cable and then check for the presence of signals in
-    the *oscilloscpe* tab. 
+    signal parameters and start the generator; connect *out1* of the RedPitaya to one of  
+    rhe inputs with a (short) cable and then check for the presence of signals in
+    the *oscilloscpe* tab;
   - now click the tab *spectrum histogram 1*; adjust the amplitude threshold and time
-    of exposure, then click the *Start* button and watch the spectrum building up.
+    of exposure, then click the *Start* button and watch the spectrum building up;
   - when finished, use the *Save* button to save the spectrum to a file with a 
     meaningful name.
 
@@ -186,22 +183,32 @@ Then, on the client side:
 The directory *helpers/* contains helper scripts to read and visualize data from
 files written by *mcpha.py*, for both spectrum histograms and exported waveforms.
 Note that presently mcpha.py exports data in human-readable format using 
-*numpy.savetxt()* 
+*numpy.savetxt()*.
 
  >  *generate_spectrum_input.py*  is a script to to generate input spectra for the signal 
     generator of mcpha.py The convention is to use 4096 channels for a range from 0 to 500 mV.
     Pulse heights are drawn randomly from this spectrum, and pulses are formed according to the
     frequency and the rise and fall times specified in the graphical interface. The signals are 
     available at the *out1*  connector of the RedPitaya board.   
-    
-    Pulses can be generated at a fixed frequency, or with random timing corresponding to a 
+ >    
+ >  Pulses can be generated at a fixed frequency, or with random timing corresponding to a 
     Poisson process with a mean pulse rate given by the chosen frequency. The latter option is 
     useful to study "pile-up" effects from overlapping pulses. 
 
->  *read_hst.py*  illustrates how to read and plot spectrum data exported by mcpha.py.
+ >  *read_hst.py*  illustrates how to read and plot spectrum data exported by mcpha.py.
+
+ >  *read_osc.py* demonstrates how to read and plot waveform data exported from the 
+    oscilloscope tab of mcpha.py.
+
+>  *read_npy.py*  illustrates how to read waveform recorded by `redPdaq.py`.
+
+## Examples
+
+The directory *examples/* contains some spectra recorded with *mchph.py* and the Python
+program *peakFitter.py* to find and precisely fit peaks in recorded spectra. An example is
+shown here:
 
->  *read_osc.py* demonstrates how to read and plot waveform data exported from the 
-   oscilloscope tab of mcpha.py.
+![Spectrum with fitted peaks](images/Spectrum.png){width=800px}
 
 
 ## License
@@ -211,6 +218,5 @@ Like the original code by Pavel Demin, this open-source code is provided under t
 
 ## Project status
 
-This project has been developed for experiments in the physics lab courses at the Faculty of 
-Physics at Karlsruhe Institute of Technology. Presently, the code is under test. A public
-β-release is in preparation. 
+This project has been developed for experiments in the physics lab courses at the Faculty of Physics  
+ at Karlsruhe Institute of Technology. The code is already public, but presently still under test. 

config/spectrum_config.yaml

-# Dict with uid's as key and a nested dict with configuration variables
-general:
-  runtime: 600  # desired runtime in seconds
-  runevents: &number_of_events 100000
-  number_of_samples: &number_of_samples 1000
-  analogue_offset: &analogue_offset 0
-  # special settings for RedPitaya
-  decimation_index: &decimation 4 # decimation 0:1, i:2^(i+1) # data decimation factor, 0 is 8 ns/sample
-  sample_time_ns: &sample_time_ns 256 #  for decimation 4
-  invert_channel1: &invert1 0 
-  invert_channel2: &invert2 0
-
-  trigger_level: &trigger_level 50
-  trigger_channel: &trigger_channel '1'
-  trigger_direction: &trigger_direction "rising" # or "falling"
-  pre_trigger_samples: &pre_trigger_samples 103 # 5%
-
-# dict for spectrum_filter, function find_peaks
-find_peaks:
-  # signal_characteristics: 256ns/sample, duration 50µs ( ~200 samples)
-  sample_time_ns: *sample_time_ns
-  analogue_offset: *analogue_offset
-  number_of_samples: *number_of_samples
-  pre_trigger_samples: *pre_trigger_samples
-  peak_minimal_prominence: 200  # has to be positive and higher than avg. noise peaks to not cause havoc!
-  trigger_channel: *trigger_channel
-  peak_minimal_distance: 400  # minimal distance between two peaks in number of samples
-  peak_minimal_width: 100  # in number of samples
-  trigger_channel: *trigger_channel
-  trigger_position_tolerance: 20  # in number of samples
-
-# dict for RedPitaya redPoscdaq
-redP_to_rb:
-  ip_address: '192.168.0.103'
-  eventcount: *number_of_events
-  sample_time_ns: *sample_time_ns
-  number_of_samples: *number_of_samples
-  pre_trigger_samples: *pre_trigger_samples
-  trigger_channel: *trigger_channel
-  trigger_level: *trigger_level
-  trigger_mode: "norm" # or "auto"
-  # special settings for RedPitaya
-  decimation_index: *decimation
-  invert_channel1: *invert1
-  invert_channel2: *invert2
-  
-# Dict for simul_source.py
-simul_source:
-  sample_time_ns: *sample_time_ns
-  number_of_samples: *number_of_samples
-  pre_trigger_samples: *pre_trigger_samples
-  analogue_offset: *analogue_offset
-  eventcount: *number_of_events
-  sleeptime: 0.03
-  random: true
-
-# Dict for push_simul
-push_simul:
-  sample_time_ns: *sample_time_ns
-  number_of_samples: *number_of_samples
-  pre_trigger_samples: *pre_trigger_samples
-  analogue_offset: *analogue_offset
-  eventcount: *number_of_events
-  sleeptime: 0.03
-  random: true
-
-save_to_txt:
-  filename: "spectrum"
-
-save_parquet:
-  filename: "spectrum"
-
-plot_waveform:
-  title: "Muon waveform"
-  min_sleeptime: 0.5                # time to wait between graphics updates
-  number_of_samples: *number_of_samples
-  sample_time_ns: *sample_time_ns
-  analogue_offset: *analogue_offset # analogue offset in V
-  pre_trigger_samples: *pre_trigger_samples
-  channel_range: 4096                # channel range in mV
-  trigger_channel: *trigger_channel  # Channel name in the PicoScope. Valid values are 'A', 'B', 'C' or 'D'  
-  trigger_level: *trigger_level  # value in mV, take account of analogue_offset, which is added to input voltage !
-
-plot_histograms:
-  title: "on-line histograms"
-  # define histograms 
-  histograms:  
-    #  name        min   max   nbins  ymax   name   lin/log
-    ch1_height:   [50., 3000.,   250, 20., "ph 1A", 0]
-    ch2_height:   [50., 3000.,   250, 20., "ph 1B", 0]

mcpha_daq.ui

@@ -305,14 +305,14 @@
       <item>
        <widget class="QPushButton" name="startButton">
         <property name="text">
-         <string>Start</string>
+         <string>Start monitor</string>
         </property>
        </widget>
       </item>
       <item>
        <widget class="QPushButton" name="startDAQButton">
         <property name="text">
-         <string>Start DAQ</string>
+         <string>Start DAQ mode</string>
         </property>
        </widget>
       </item>

modules/#redPitaya_source.py#

-"""
-**redPitaya_source**: mimoCoRB source compatible to redPoscdaq
-
-Input data is provided as numpy-arry of shape (number_of_channels, number_of_samples).
-"""
-
-from mimocorb.buffer_control import rbImport
-import numpy as np
-import sys, time
-from mutiprocessing import Event
-
-def simul_source(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
-    """
-    Generate simulated data and pass data to buffer
-
-    The class mimocorb.buffer_control/rbImport is used to interface to the
-    newBuffer and Writer classes of the package mimoCoRB.mimo_buffer
-
-    This example may serve as a template for other data sources
-
-    :param config_dict: configuration dictionary
-
-      - events_required: number of events to be simulated or 0 for infinite
-      - sleeptime: (mean) time between events
-      - random: random time between events according to a Poission process
-      - number_of_samples, sample_time_ns, pretrigger_samples and analogue_offset
-        describe the waveform data to be generated (as for oscilloscope setup)
-
-    Internal parameters of the simulated physics process (the decay of a muon)
-    are (presently) not exposed to user.
-    """
-
-    global databuffer
-    data_ready = Event()
-    data_ready.clear()
-
-    run_rpControl(callback=rp_data)
-
-    def rp_data(data):
-        while data_ready.is_set():
-            time.sleep(0.01)
-        databuffer = data    
-        data_ready.set()
-
-    events_required = 1000 if "eventcount" not in config_dict else config_dict["eventcount"]
-
-    def yield_data():
-        """generate simulated data, called by instance of class mimoCoRB.rbImport"""
-
-        event_count = 0
-        while events_required == 0 or event_count < events_required:
-            data_ready.wait()
-            # deliver pulse data and no metadata
-            yield (databuffer, None)
-            data_ready.clear()
-            event_count += 1
-            
-    datasource = rbImport(config_dict=config_dict, sink_list=sink_list, ufunc=yield_data, **rb_info)
-    # possibly check consistency of provided dtype with simulation !
-
-    # TODO: Change to logger!
-    # print("** simul_source ** started, config_dict: \n", config_dict)
-    # print("?> sample interval: {:02.1f}ns".format(osci.time_interval_ns.value))
-    datasource()

modules/filters.py

-from mimocorb import mimo_buffer as bm
-from scipy import signal
-import numpy as np
-from numpy.lib import recfunctions as rfn
-import sys
-import os
-
-
-def normed_pulse(ch_input, position, prominence, analogue_offset):
-    # > Compensate for analogue offset
-    ch_data = ch_input - analogue_offset
-    # > Find pulse area
-    #       rel_height is not good because of the quantized nature of the picoscope data
-    #       so we have to "hack" a little bit to always cut 10mV above the analogue offset
-    width_data = signal.peak_widths(ch_data, [int(position)], rel_height=(ch_data[int(position)] - 10) / prominence)
-    left_ips, right_ips = width_data[2], width_data[3]
-    # Crop pulse area and normalize
-    pulse_data = ch_data[int(np.floor(left_ips)) : int(np.ceil(right_ips))]
-    pulse_int = sum(pulse_data)
-    pulse_data *= 1 / pulse_int
-    return pulse_data, int(np.floor(left_ips)), pulse_int
-
-
-def correlate_pulses(data_pulse, reference_pulse):
-    correlation = signal.correlate(data_pulse, reference_pulse, mode="same")
-    shift_array = signal.correlation_lags(data_pulse.size, reference_pulse.size, mode="same")
-    shift = shift_array[np.argmax(correlation)]
-    return shift
-
-
-def tag_peaks(input_data, prominence, distance, width):
-    peaks = {}
-    peaks_prop = {}
-    for key in input_data.dtype.names:
-        peaks[key], peaks_prop[key] = signal.find_peaks(
-            input_data[key], prominence=prominence, distance=distance, width=width
-        )
-    return peaks, peaks_prop
-
-
-def correlate_peaks(peaks, tolerance):
-    m_dtype = []
-    for key in peaks.keys():
-        m_dtype.append((key, np.int32))
-    next_peak = {}
-    for key, data in peaks.items():
-        if len(data) > 0:
-            next_peak[key] = data[0]
-    correlation_list = []
-    while len(next_peak) > 0:
-        minimum = min(next_peak.values())
-        line = []
-        for key, data in peaks.items():
-            if key in next_peak:
-                if abs(next_peak[key] - minimum) < tolerance:
-                    idx = data.tolist().index(next_peak[key])
-                    line.append(idx)
-                    if len(data) > idx + 1:
-                        next_peak[key] = data[idx + 1]
-                    else:
-                        del next_peak[key]
-                else:
-                    line.append(-1)
-            else:
-                line.append(-1)
-        correlation_list.append(line)
-    array = np.zeros(len(correlation_list), dtype=m_dtype)
-    for idx, line in enumerate(correlation_list):
-        array[idx] = tuple(line)
-    return array
-
-
-def match_signature(peak_matrix, signature):
-    if len(signature) > len(peak_matrix):
-        return False
-    # Boolean array with found peaks
-    input_peaks = rfn.structured_to_unstructured(peak_matrix) >= 0
-    must_have_peak = np.array(signature, dtype=np.str0) == "+"
-    must_not_have_peak = np.array(signature, dtype=np.str0) == "-"
-    match = True
-    # Check the signature for each peak (1st peak with 1st signature, 2nd peak with 2nd signature, ...)
-    for idx in range(len(signature)):
-        # Is everywhere a peak, where the signature expects one -> Material_conditial(A, B): (not A) OR B
-        first = (~must_have_peak[idx]) | input_peaks[idx]
-        # Is everywhere no peak, where the signature expects no peak -> NAND(A, B): not (A and B)
-        second = ~(must_not_have_peak[idx] & input_peaks[idx])
-        match = match & (np.all(first) & np.all(second))
-    return match
-
-
-if __name__ == "__main__":
-    print("Script: " + os.path.basename(sys.argv[0]))
-    print("Python: ", sys.version, "\n".ljust(22, "-"))
-    print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")

modules/plot_histograms.py

-"""
-**plot_histograms**: histogram variable(s) from buffer using mimoCoRB.histogram_buffer 
-"""
-import sys
-import os
-from mimocorb.histogram_buffer import histogram_buffer
-import matplotlib
-
-# select matplotlib frontend if needed
-matplotlib.use("TkAgg")
-
-
-def plot_histograms(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
-    """
-    Online display of histogram(s) of variable(s) from mimiCoRB buffer
-
-    :param input: configuration dictionary
-
-    """
-    histbuf = histogram_buffer(source_list, sink_list, observe_list, config_dict, **rb_info)
-    histbuf()
-
-
-if __name__ == "__main__":
-    print("Script: " + os.path.basename(sys.argv[0]))
-    print("Python: ", sys.version, "\n".ljust(22, "-"))
-    print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")

modules/plot_waveform.py

-"""
-**plot**: plotting waveforms from buffer using mimoCoRB.buffer_control.OberserverData 
-"""
-
-import sys
-import os
-from mimocorb.plot_buffer import plot_buffer
-import matplotlib
-
-# select matplotlib frontend if needed
-matplotlib.use("TkAgg")
-
-
-def plot_waveform(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
-    """
-    Plot waveform data from mimiCoRB buffer
-
-    :param input: configuration dictionary
-
-      - plot_title: graphics title to be shown on graph
-      - min_sleeptime: time between updates
-      - sample_time_ns, channel_range, pretrigger_samples and analogue_offset
-        describe the waveform data as for oscilloscope setup
-    """
-
-    pltbuf = plot_buffer(source_list, sink_list, observe_list, config_dict, **rb_info)
-    pltbuf()
-
-
-if __name__ == "__main__":
-    print("Script: " + os.path.basename(sys.argv[0]))
-    print("Python: ", sys.version, "\n".ljust(22, "-"))
-    print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")

modules/redPitaya_source.py

-"""
-**simul_source**: a simple template for a mimoCoRB source to 
-enter simulated wave form data in a mimoCoRB buffer.
-
-Input data is provided as numpy-arry of shape (number_of_channels, number_of_samples).
-"""
-
-from mimocorb.buffer_control import rbImport
-import numpy as np
-import sys, time
-from pulseSimulator import pulseSimulator
-
-def simul_source(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
-    """
-    Generate simulated data and pass data to buffer
-
-    The class mimocorb.buffer_control/rbImport is used to interface to the
-    newBuffer and Writer classes of the package mimoCoRB.mimo_buffer
-
-    This example may serve as a template for other data sources
-
-    :param config_dict: configuration dictionary
-
-      - events_required: number of events to be simulated or 0 for infinite
-      - sleeptime: (mean) time between events
-      - random: random time between events according to a Poission process
-      - number_of_samples, sample_time_ns, pretrigger_samples and analogue_offset
-        describe the waveform data to be generated (as for oscilloscope setup)
-
-    Internal parameters of the simulated physics process (the decay of a muon)
-    are (presently) not exposed to user.
-    """
-
-    events_required = 1000 if "eventcount" not in config_dict else config_dict["eventcount"]
-
-    def yield_data():
-        """generate simulated data, called by instance of class mimoCoRB.rbImport"""
-
-        event_count = 0
-        while events_required == 0 or event_count < events_required:
-            pulse = dataSource(number_of_channels)
-            # deliver pulse data and no metadata
-            yield (pulse, None)
-            event_count += 1
-
-    dataSource = pulseSimulator(config_dict)
-    simulsource = rbImport(config_dict=config_dict, sink_list=sink_list, ufunc=yield_data, **rb_info)
-    number_of_channels = len(simulsource.sink.dtype)
-    # possibly check consistency of provided dtype with simulation !
-
-    # TODO: Change to logger!
-    # print("** simul_source ** started, config_dict: \n", config_dict)
-    # print("?> sample interval: {:02.1f}ns".format(osci.time_interval_ns.value))
-    simulsource()

modules/save_files.py

-"""Module save_files to handle file I/O for data in txt and parquet format
-
-   This module relies on classes in mimocorb.buffer_control
-"""
-
-import sys
-import os
-from mimocorb.buffer_control import rb_toTxtfile, rb_toParquetfile
-
-
-# def save_to_txt(source_dict):
-def save_to_txt(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
-    sv = rb_toTxtfile(source_list=source_list, config_dict=config_dict, **rb_info)
-    sv()
-    # print("\n ** save_to_txt: end seen")
-
-
-def save_parquet(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
-    sv = rb_toParquetfile(source_list=source_list, config_dict=config_dict, **rb_info)
-    sv()
-
-
-if __name__ == "__main__":
-    print("Script: " + os.path.basename(sys.argv[0]))
-    print("Python: ", sys.version, "\n".ljust(22, "-"))
-    print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")

modules/spectrum_filter.py

-"""Module **pulse_filter** 
-
-This (rather complex) module filters waveform data to search for valid signal pulses. 
-The code first validates the trigger pulse, identifies coincidences of signals in 
-different layers (indiating the passage of a cosmic ray particle, a muon) and finally
-searches for double-pulse signatures indicating that a muon was stopped in or near 
-a detection layer where the resulting decay-electron produced a delayed pulse. 
-The time difference between the initial and the delayed pulses is the individual 
-lifetime of the muon.
-
-The decay time and the properties of the signal pulses (height, integral and 
-postition in time) are written to a buffer; the raw wave forms are optionally
-also written to another buffer. 
-
-The callable functions *find_peaks()* and *calulate_decay_time()* depend on the 
-buffer manager *mimoCoRB* and provide the filter functionality described above. 
-These functions support multiple sinks to be configured for output. 
-
-The relevant configuration parameters can be found in the section *find_peaks:* 
-and *calculate_decay_time:* in the configuration file. 
-
-""" 
-
-from mimocorb.buffer_control import rbTransfer
-import numpy as np
-import pandas as pd
-import os, sys
-
-from filters import *
-
-def find_peaks(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
-    """filter client for mimoCoRB: Find valid signal pulses in waveform data
-
-       Input: 
-
-        - wave form data from source buffer defined in source_list
-
-       Returns: 
-  
-         - None if filter not passed
-         - list of list(s) of pulse parameters, written to sinks defined in sink_list
-
-    """
-    
-    if config_dict is None:
-        raise ValueError("ERROR! Wrong configuration passed (in lifetime_modules: calculate_decay_time)!!")
-
-    # Load configuration
-    sample_time_ns = config_dict["sample_time_ns"]
-    analogue_offset = config_dict["analogue_offset"]*1000
-    peak_minimal_prominence = config_dict["peak_minimal_prominence"]
-    peak_minimal_distance = config_dict["peak_minimal_distance"]
-    peak_minimal_width = config_dict["peak_minimal_width"]
-    pre_trigger_samples = config_dict["pre_trigger_samples"]
-    trigger_channel = config_dict["trigger_channel"]
-#    if trigger_channel not in ['A','B','C','D']:
-    if trigger_channel not in ['1','2']:
-         trigger_channel = None    
-    trigger_position_tolerance = config_dict["trigger_position_tolerance"]
-
-    pulse_par_dtype = sink_list[-1]['dtype']
-
-    
-    def tag_pulses(input_data):   
-        """find all valid pulses 
-
-        This function to be called by instance of class mimoCoRB.rbTransfer
-
-           Args:  input data as structured ndarray
-    
-           Returns: list of parameterized pulses
-        """
-
-        # Find all the peaks and store them in a dictionary
-        peaks, peaks_prop = tag_peaks(input_data, peak_minimal_prominence, peak_minimal_distance, peak_minimal_width)
-
-        # identify trigger channel, validate trigger pulse and get time of trigger pulse
-        if trigger_channel is not None:
-           trigger_peaks = peaks['ch' + trigger_channel]
-           if len(trigger_peaks) == 0:
-               return None
-           reference_position = trigger_peaks[np.argmin(np.abs(trigger_peaks - pre_trigger_samples))]
-        else: # external or no trigger: set to nominal position 
-           reference_position = pre_trigger_samples
-
-        peak_data= np.zeros( (1,), dtype=pulse_par_dtype)
-        for key in peaks.keys():
-            for position, height, left_ips, right_ips in zip(
-                    peaks[key], peaks_prop[key]['prominences'],
-                    peaks_prop[key]['left_ips'], peaks_prop[key]['right_ips']):
-                if np.abs(reference_position - position) < trigger_position_tolerance:
-                    peak_data[0][key+'_position'] = position
-                    peak_data[0][key+'_height'] = input_data[key][position] - analogue_offset #height
-                    left = int(np.floor(left_ips))
-                    right = int(np.ceil(right_ips))
-                    peak_data[0][key+'_integral'] = \
-                    sum(input_data[key][left:right] - analogue_offset) * sample_time_ns * 1e-9/50/5
-        return [peak_data]    
-        
-    p_filter = rbTransfer(source_list=source_list, sink_list=sink_list, config_dict=config_dict,
-                        ufunc=tag_pulses, **rb_info)
-    p_filter()
-
-if __name__ == "__main__":
-    print("Script: " + os.path.basename(sys.argv[0]))
-    print("Python: ", sys.version, "\n".ljust(22, '-'))
-    print("THIS IS A MODULE AND NOT MEANT FOR STANDALONE EXECUTION")

read_npy.py

-#!/usr/bin/env python3
-"""read file from redPoscdaq (in npy format) and display data
-"""
-
-from npy_append_array import NpyAppendArray
-import numpy as np
-import sys
-import matplotlib.pyplot as plt
-
-data = np.load(sys.argv[1], mmap_mode='r')
-print("data read sucessfully, shape = ", data.shape)
-
-n_samples = len(data[0,0])
-xplt = 0.5 + np.linspace(0, n_samples, num=n_samples, endpoint=True)
-fig = plt.figure("Oscillogram", figsize=(8,6))
-
-for d in data:
-    plt.plot(xplt, d[0], '-')
-    plt.plot(xplt, d[1], '-')
-    plt.xlabel("time bin")
-    plt.ylabel("Voltage")    
-    plt.show()
-                  

redP_mimoCoRB.py

-"""
-**redP_mimoCoRB**: a simple template to use mimoCoRB with the RedPitaya and redPoscdaq.py 
-
-Input data is provided as numpy-arry of shape (number_of_channels, number_of_samples).
-"""
-
-import time
-import sys
-
-import redPoscdaq as rp
-
-
-def redP_to_rb(source_list=None, sink_list=None, observe_list=None, config_dict=None, **rb_info):
-    """
-    Get data from RedPitaya and pass data to buffer
-
-    The class mimocorb.buffer_control/rbImport is used to interface to the
-    newBuffer and Writer classes of the package mimoCoRB.mimo_buffer
-
-    This example may serve as a template for other data sources
-
-    :param config_dict: configuration dictionary
-
-      - events_required: number of events to be simulated or 0 for infinite
-      - sleeptime: (mean) time between events
-      - random: random time between events according to a Poission process
-      - number_of_samples, sample_time_ns, pretrigger_samples and analogue_offset
-        describe the waveform data to be generated (as for oscilloscope setup)
-
-    Internal parameters of the simulated physics process (the decay of a muon)
-    are (presently) not exposed to user.
-    """
-
-    # initialize mimocorb class inside redPoscidaq
-    datasource= rp.redP_mimocorb(config_dict=config_dict, sink_list=sink_list, **rb_info)
-    #print("data source initialized")
-
-    # start oscilloscope
-    #print("starting osci")
-    rp.run_rpControl(callback=datasource.data_sink, conf_dict=config_dict)
-    
-
-#daq = run_mimoDAQ('redP_mimoCoRB.yaml')
-#daq.setup()
-#RB_1 = daq.ringbuffers['RB_1']
-#sink_dict = RB_1.new_writer()
-#datasource= rp.redP_mimocorb(config_dict={}, sink_list=[sink_dict], RB_1='write')
-#print("data source initialized")
-#rp.run_rpControl(callback=datasource.data_sink)
-#print("starting DAQ")
-#daq.run()

setup.yaml

@@ -23,14 +23,14 @@ RingBuffer:
   # define ring buffers
   - RB_1:
       # raw input data buffer (from picoScope, file or simulation)
-      number_of_slots: 16
+      number_of_slots: 100
       channel_per_slot: 1000
       data_type:
           1: ['ch1', "float32"]
           2: ['ch2', "float32"]
   - RB_2:
       # buffer with correct signature double pulse parameters
-      number_of_slots: 16
+      number_of_slots: 100
       channel_per_slot: 1
       data_type:
       data_type: