OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Extracting values from multiple dataframes after mathematical calculations

  • Thread starter Thread starter S N B
  • Start date Start date
S

S N B

Guest
Currently, I have ECG sensor data, stored in dataframe df_data with column names ‘slno’ (x axis) and ‘data’(y axis). I detect peaks and store the location of peak as ‘peak_at’(xaxis) and ‘peak_marked’ (y-axis) in df_BFI.

So, I have two dataframes

  1. df_data: This has ‘slno’ and ‘data’ as two columns
  2. df_BFI: This has column ‘peak_at’ and ‘peak_marked’

Code:
import pandas as pd
import numpy as np
from scipy.signal import argrelextrema

# input -  two column: 'slno' (x axis), 'data'(y axis).
data = [[1, 2],[2, 3],[3, 2.5],[4, 1.9],[5, 2.8],[6, 15],[7, 17],[8, 20],[9, 18],[10, 16],
        [11,14],[12, 12],[13, 10],[14, 4],[15, 3], [16, 22], [17, 1],[18, 5],[19, 8], [20, 10],
        [21,12],[22, 9],[23, 6],[24, 7],[25, 10],[26, 15],[27, 18],[28, 21],[29, 25],[30, 22],
        [31, 19], [32, 27], [33, 24], [34, 21], [35, 18], [36, 7], [37, 3], [38, 2], [39, 1], [40, 3],
        [41, 1], [42, 4],[43, 5],[44, 7], [45, 10],[46, 18], [47, 22], [48, 14], [49,12], [50,10],
        [51, 6], [52,9], [53, 7], [54, 5], [55, 28]]
df_data = pd.DataFrame(data, columns=['slno', 'data'])

#few peak points are marked and stored in another dataframe df_bfi. here I have manually entered those by defining 'bfi'.
#'peak_at' is x axis coordinate where there is a peak,
#'peak_marked' is y axis coordinate at that point. here you can cross verify, each row of df_bfi is a row in df_data.

bfi =[[16, 22],[32, 27],[55, 28]]
df_bfi = pd.DataFrame(bfi,columns=['peak_at', 'peak_marked'])

Hereafter I have to find if there exists any turning point/peak to left of marked peak. condition is it has to be within 10 data points and its value must be 75% more than the peak marked.

In the above case discussed, 3 peaks are identified at 16, 32 and 55. Its marked peak values are 22, 27 and 28 respectively. Now consider peak at 55 which is 28. We need to find if there exists another peak to left 10 samples of this point such that its value is greater than 21(>75% of 28).

10 values to left of [55,28] is [10, 18, 22, 14, 12, 10, 6, 9,7, 5] referring to df_data.data. Here we have two turning points. I want to validate only 22.0 present at position p-8 as a valid data as its value is >75% of 28. if validated, I have to update df_bfi.

Updated df_bfi entries would look like:This is the expected output. [8, 20], [29, 25], [47, 22].

I have tried doing the following.

#picking 10 'data' values from dataframe df_data df_bfi['data_left_10'] = df_bfi['peak_at'].apply(lambda p:df_data.loc[p-11:p-2, 'data'].to_list())#Make array of 10 samples to left of peak found.

So I have list of 10 'data' values from 'df_data' which lies to the left of the marked peak ('peak_at').

I find the turning points using argrelextrema. But i don know how to validate the value to >75% of peak_marked and also update dataframe values.

Code:
df_bfi['turning_pts'] = df_bfi.data_left_10.apply(lambda p: np.array(argrelextrema(np.array(p),np.greater)))#find the turnng points in 10 samples.

How to handle this?
<p>Currently, I have ECG sensor data, stored in dataframe df_data with column names ‘slno’ (x axis) and ‘data’(y axis). I detect peaks and store the location of peak as ‘peak_at’(xaxis) and ‘peak_marked’ (y-axis) in df_BFI.</p>
<p>So, I have two dataframes</p>
<ol>
<li>df_data: This has ‘slno’ and ‘data’ as two columns</li>
<li>df_BFI: This has column ‘peak_at’ and ‘peak_marked’</li>
</ol>
<pre><code>import pandas as pd
import numpy as np
from scipy.signal import argrelextrema

# input - two column: 'slno' (x axis), 'data'(y axis).
data = [[1, 2],[2, 3],[3, 2.5],[4, 1.9],[5, 2.8],[6, 15],[7, 17],[8, 20],[9, 18],[10, 16],
[11,14],[12, 12],[13, 10],[14, 4],[15, 3], [16, 22], [17, 1],[18, 5],[19, 8], [20, 10],
[21,12],[22, 9],[23, 6],[24, 7],[25, 10],[26, 15],[27, 18],[28, 21],[29, 25],[30, 22],
[31, 19], [32, 27], [33, 24], [34, 21], [35, 18], [36, 7], [37, 3], [38, 2], [39, 1], [40, 3],
[41, 1], [42, 4],[43, 5],[44, 7], [45, 10],[46, 18], [47, 22], [48, 14], [49,12], [50,10],
[51, 6], [52,9], [53, 7], [54, 5], [55, 28]]
df_data = pd.DataFrame(data, columns=['slno', 'data'])

#few peak points are marked and stored in another dataframe df_bfi. here I have manually entered those by defining 'bfi'.
#'peak_at' is x axis coordinate where there is a peak,
#'peak_marked' is y axis coordinate at that point. here you can cross verify, each row of df_bfi is a row in df_data.

bfi =[[16, 22],[32, 27],[55, 28]]
df_bfi = pd.DataFrame(bfi,columns=['peak_at', 'peak_marked'])
</code></pre>
<p>Hereafter I have to find if there exists any turning point/peak to left of marked peak. condition is it has to be within 10 data points and its value must be 75% more than the peak marked.</p>
<p>In the above case discussed, 3 peaks are identified at 16, 32 and 55. Its marked peak values are 22, 27 and 28 respectively. Now consider peak at 55 which is 28. We need to find if there exists another peak to left 10 samples of this point such that its value is greater than 21(>75% of 28).</p>
<p>10 values to left of [55,28] is [10, 18, 22, 14, 12, 10, 6, 9,7, 5] referring to df_data.data. Here we have two turning points. I want to validate only 22.0 present at position p-8 as a valid data as its value is >75% of 28. if validated, I have to update df_bfi.</p>
<p>Updated df_bfi entries would look like:This is the expected output.
[8, 20], [29, 25], [47, 22].</p>
<p>I have tried doing the following.</p>
<p>#picking 10 'data' values from dataframe df_data
df_bfi['data_left_10'] = df_bfi['peak_at'].apply(lambda p:df_data.loc[p-11:p-2, 'data'].to_list())#Make array of 10 samples to left of peak found.</p>
<p>So I have list of 10 'data' values from 'df_data' which lies to the left of the marked peak ('peak_at').</p>
<p>I find the turning points using argrelextrema. But i don know how to validate the value to >75% of peak_marked and also update dataframe values.</p>
<pre><code>df_bfi['turning_pts'] = df_bfi.data_left_10.apply(lambda p: np.array(argrelextrema(np.array(p),np.greater)))#find the turnng points in 10 samples.
</code></pre>
<p>How to handle this?</p>
 
Top