OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Python Polars Groupby_rolling() start from window size

  • Thread starter Thread starter user25735121
  • Start date Start date
U

user25735121

Guest
Code:
import polars as pl
import numpy as np
import pandas as pd
from functime.feature_extractors import FeatureExtractor, binned_entropy

Data_Test = pl.read_csv('./DataDemo.csv')

Calc_T = 200
Data_Test.with_columns(
    pl.col('datetime').cumcount().cast(pl.Int32).alias('Index')
).group_by_rolling('Index', period = '{}i'.format(Calc_T)).agg(
    pl.col('datetime').last(),
    pl.col('*').exclude(['datetime', 'order_book_id', 'underlying_symbol', 'Index']).ts.binned_entropy(bin_count=10).suffix('_binned_entropy'),
    pl.col('*').exclude(['datetime', 'order_book_id', 'underlying_symbol', 'Index']).ts.absolute_energy().suffix('_absolute_energy'),
    # pl.col('*').exclude(['datetime', 'order_book_id', 'underlying_symbol', 'Index']).ts.autocorrelation(n_lags = 20).suffix('_autocorrelation')
    
)

In the above code, the line that is commented out will result in an error:PanicException: Cannot apply operation on arrays of different lengths

In polars, DF.groupby_rolling('Index', period = '20i'). the first group will be the first row, and the second group will be the first two rows. So the length of the window is less than the length we specified.

I think the reason for the error is that the algorithm in the commented line requires a time series of length greater than 1, and in practice the first group has only one piece of data. I've tried usegroupby_dynamic('Index', every = '1i', period = '{}i'.format(Calc_T), start_by = 'datapoint'),But in this case, the length of the last group becomes 1..

Is there a way for Polars to skip the first n-1 rows of data when scrolling through the window, where N is the window length we specified?

Thanks in advance!
<pre><code>import polars as pl
import numpy as np
import pandas as pd
from functime.feature_extractors import FeatureExtractor, binned_entropy

Data_Test = pl.read_csv('./DataDemo.csv')

Calc_T = 200
Data_Test.with_columns(
pl.col('datetime').cumcount().cast(pl.Int32).alias('Index')
).group_by_rolling('Index', period = '{}i'.format(Calc_T)).agg(
pl.col('datetime').last(),
pl.col('*').exclude(['datetime', 'order_book_id', 'underlying_symbol', 'Index']).ts.binned_entropy(bin_count=10).suffix('_binned_entropy'),
pl.col('*').exclude(['datetime', 'order_book_id', 'underlying_symbol', 'Index']).ts.absolute_energy().suffix('_absolute_energy'),
# pl.col('*').exclude(['datetime', 'order_book_id', 'underlying_symbol', 'Index']).ts.autocorrelation(n_lags = 20).suffix('_autocorrelation')

)
</code></pre>
<p>In the above code, the line that is commented out will result in an error:<strong>PanicException: Cannot apply operation on arrays of different lengths</strong></p>
<p>In polars, <code>DF.groupby_rolling('Index', period = '20i')</code>. the first group will be the first row, and the second group will be the first two rows. So the length of the window is less than the length we specified.</p>
<p>I think the reason for the error is that the algorithm in the commented line requires a time series of length greater than 1, and in practice the first group has only one piece of data.
I've tried use<code>groupby_dynamic('Index', every = '1i', period = '{}i'.format(Calc_T), start_by = 'datapoint')</code>,But in this case, the length of the last group becomes 1..</p>
<p>Is there a way for Polars to skip the first n-1 rows of data when scrolling through the window, where N is the window length we specified?</p>
<p>Thanks in advance!</p>
 

Latest posts

I
Replies
0
Views
1
impact christian
I
Top