OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Getting *nan* values when doing inverse box-Cox transformation

  • Thread starter Thread starter mani mohtasham
  • Start date Start date
M

mani mohtasham

Guest
I have transformed my entire data frame(dependent and independent) with box-Cox transformation and I have array below as coefficients:

Code:
array([-0.12050315, -0.07470903, -0.11079292, -0.22158582, -0.06629854,
       -0.14941804, -0.11161256, -0.08786566, -0.17451716, -0.10773153,
       -0.09787049, -0.19569526, -0.21546309, -0.12817391, -0.03621895,
       -0.22004721, -0.10557047, -0.06584703, -0.13174353,  0.16621695,
        0.01901815, -0.1635233 ,  0.15919524,  0.05952151,  0.06084785,
        0.01126627,  0.01126626, -0.87299313])

After training my Keras regression model, I want to do an inverse transform to both test-targets(y_test) and predicted values(y_pred). The code I used is from sklearn package as below: for y_test as below:

Code:
array([ 0.98333783,  1.50200139,  1.83218911, -0.15996774,  0.78260773,
        0.74395122, -1.03379222, -1.49198108,  1.15562537,  1.12287162,
       -0.73178092,  1.81372947, -1.07406015,  0.33985499,  1.03716673,
       -1.57230688,  1.47911475,  0.84982504, -1.44474193, -0.56389331,
       -0.20603036,  0.1266229 ,  0.34537935, -0.91948754, -0.0859256 ,
       -1.04186052, -0.46334846, -0.60194859,  1.49238056,  0.05740751,
        0.00789972,  1.72823035, -0.60434219, -1.53115573,  1.65947602,
        1.55132413, -1.63633904,  1.50525545, -1.41644302,  1.18606998,
       -0.97952481,  1.791517  ,  0.74122464,  1.35466604, -0.91996944,
       -0.66993356, -0.27276727, -0.70785461,  0.97178584, -0.26121448,
       -1.38664819,  1.10732533, -1.41707219,  1.9831894 , -1.13688763,
        0.8281032 , -0.21455527, -0.42318826, -0.10473893, -0.2152237 ,
       -1.47820045,  0.94680937, -0.17741741,  0.63605613, -0.39629346,
        1.09077037, -0.51547212, -0.67354063, -1.57748571, -0.79634151,
        0.15821278,  0.46977151,  0.27850129,  0.07907392, -0.93708215,
        1.53871748, -1.21563852,  0.79040892,  0.21197071, -0.66894071,
        0.27635132,  1.28436786,  0.48883812, -0.05657853,  1.28865163,
       -0.54660271,  1.11260471, -1.24212802,  1.71250636,  0.6430331 ,
        0.71056295,  1.09364413, -0.43520344, -0.51681466,  0.54017086,
        0.61223984,  0.05203929,  0.83618294, -0.55494482, -0.04054612,
       -0.0886779 ,  1.44277091,  1.77569744,  0.86381339,  0.15509978,
        0.65514589, -0.39788517, -0.91833687,  0.42345815, -0.86787368,
        0.20091955,  0.59593155,  1.69567572,  0.51849197,  0.60554069,
        1.32229222, -1.57268445, -0.19908104,  0.04183203, -0.18814032,
       -0.33261663, -0.2595337 , -0.74548164,  1.29234243,  1.52822498,
        0.04565854,  2.08218297])


def inverse_bc(vector , lam):
    if lam == 0:
        inv = exp(vector)
    
    else:
        inv = ((np.sign(vector)*np.abs(vector) * lam) + 1) ** (1 / lam)

    return inv

y_lambda = power.lambdas_[-1]   #-0.87299313
inverse_bc(y_test , y_lambda)

I am getting some nan values in the returned array like below:

Code:
*array([0.10610358,        nan,        nan, 1.16152319, 0.26775244,
   0.30072773, 2.08877834, 2.5988645 ,        nan, 0.0106793 ,
   1.76078457,        nan, 2.13303091, 0.66815944, 0.0666456 ,
   2.68968613,        nan, 0.2116487 , 2.54563854, 1.58165874,
   1.20858382, 0.8744233 , 0.66291199, 1.96381267, 1.0863822 ,
   2.09763562, 1.47558096, 1.62204719,        nan, 0.94280366,
   0.99210423,        nan, 1.62459181, 2.6431087 ,        nan,
          nan, 2.76236425,        nan, 2.51382021,        nan,
   2.02932817,        nan, 0.30307155,        nan, 1.96433746,
   1.69451594, 1.27717759, 1.7351098 , 0.11484951, 1.26526928,
   2.48037479, 0.01982524, 2.51452707,        nan, 2.20230692,
   0.22959532, 1.21731914, 1.43347482, 1.10541436, 1.21800441,
   2.58332304, 0.13404354, 1.17932312, 0.39511208, 1.40536365,
   0.03020463, 1.53045691, 1.69837201, 2.69555511, 1.83029795,
   0.84343459, 0.54620617, 0.72678135, 0.92132898, 1.98298456,
          nan, 2.28953023, 0.26115763, 0.79103097, 1.69345474,
   0.72884676,        nan, 0.52857803, 1.0567779 ,        nan,
   1.56335081, 0.01664213, 2.31896501,        nan, 0.38891339,
   0.32958372, 0.0283648 , 1.44605602, 1.53187367, 0.48149097,
   0.41636353, 0.94813399, 0.22289834, 1.57218039, 1.0406489 ,
   1.0891639 ,        nan,        nan, 0.2001915 , 0.84648205,
   0.37818144, 1.40702539, 1.96255967, 0.58932261, 1.90770871,
   0.8017689 , 0.43099541,        nan, 0.50130903, 0.42236631,
          nan, 2.69011397, 1.20146886, 0.95827965, 1.19027816,
   1.3390913 , 1.26353796, 1.77550785,        nan,        nan,
   0.95447464,        nan])
*

My first question is, should the transformation return just one value for each column as lambda? and please help me understand what are these nan values?

thanks
<p>I have transformed my entire data frame(dependent and independent) with box-Cox transformation and I have array below as coefficients:</p>
<pre><code>array([-0.12050315, -0.07470903, -0.11079292, -0.22158582, -0.06629854,
-0.14941804, -0.11161256, -0.08786566, -0.17451716, -0.10773153,
-0.09787049, -0.19569526, -0.21546309, -0.12817391, -0.03621895,
-0.22004721, -0.10557047, -0.06584703, -0.13174353, 0.16621695,
0.01901815, -0.1635233 , 0.15919524, 0.05952151, 0.06084785,
0.01126627, 0.01126626, -0.87299313])
</code></pre>
<p>After training my Keras regression model, I want to do an inverse transform to both test-targets(y_test) and predicted values(y_pred). The code I used is from sklearn package as below:
for y_test as below:</p>
<pre><code>array([ 0.98333783, 1.50200139, 1.83218911, -0.15996774, 0.78260773,
0.74395122, -1.03379222, -1.49198108, 1.15562537, 1.12287162,
-0.73178092, 1.81372947, -1.07406015, 0.33985499, 1.03716673,
-1.57230688, 1.47911475, 0.84982504, -1.44474193, -0.56389331,
-0.20603036, 0.1266229 , 0.34537935, -0.91948754, -0.0859256 ,
-1.04186052, -0.46334846, -0.60194859, 1.49238056, 0.05740751,
0.00789972, 1.72823035, -0.60434219, -1.53115573, 1.65947602,
1.55132413, -1.63633904, 1.50525545, -1.41644302, 1.18606998,
-0.97952481, 1.791517 , 0.74122464, 1.35466604, -0.91996944,
-0.66993356, -0.27276727, -0.70785461, 0.97178584, -0.26121448,
-1.38664819, 1.10732533, -1.41707219, 1.9831894 , -1.13688763,
0.8281032 , -0.21455527, -0.42318826, -0.10473893, -0.2152237 ,
-1.47820045, 0.94680937, -0.17741741, 0.63605613, -0.39629346,
1.09077037, -0.51547212, -0.67354063, -1.57748571, -0.79634151,
0.15821278, 0.46977151, 0.27850129, 0.07907392, -0.93708215,
1.53871748, -1.21563852, 0.79040892, 0.21197071, -0.66894071,
0.27635132, 1.28436786, 0.48883812, -0.05657853, 1.28865163,
-0.54660271, 1.11260471, -1.24212802, 1.71250636, 0.6430331 ,
0.71056295, 1.09364413, -0.43520344, -0.51681466, 0.54017086,
0.61223984, 0.05203929, 0.83618294, -0.55494482, -0.04054612,
-0.0886779 , 1.44277091, 1.77569744, 0.86381339, 0.15509978,
0.65514589, -0.39788517, -0.91833687, 0.42345815, -0.86787368,
0.20091955, 0.59593155, 1.69567572, 0.51849197, 0.60554069,
1.32229222, -1.57268445, -0.19908104, 0.04183203, -0.18814032,
-0.33261663, -0.2595337 , -0.74548164, 1.29234243, 1.52822498,
0.04565854, 2.08218297])


def inverse_bc(vector , lam):
if lam == 0:
inv = exp(vector)

else:
inv = ((np.sign(vector)*np.abs(vector) * lam) + 1) ** (1 / lam)

return inv

y_lambda = power.lambdas_[-1] #-0.87299313
inverse_bc(y_test , y_lambda)
</code></pre>
<p>I am getting some <strong>nan</strong> values in the returned array like below:</p>
<pre><code>*array([0.10610358, nan, nan, 1.16152319, 0.26775244,
0.30072773, 2.08877834, 2.5988645 , nan, 0.0106793 ,
1.76078457, nan, 2.13303091, 0.66815944, 0.0666456 ,
2.68968613, nan, 0.2116487 , 2.54563854, 1.58165874,
1.20858382, 0.8744233 , 0.66291199, 1.96381267, 1.0863822 ,
2.09763562, 1.47558096, 1.62204719, nan, 0.94280366,
0.99210423, nan, 1.62459181, 2.6431087 , nan,
nan, 2.76236425, nan, 2.51382021, nan,
2.02932817, nan, 0.30307155, nan, 1.96433746,
1.69451594, 1.27717759, 1.7351098 , 0.11484951, 1.26526928,
2.48037479, 0.01982524, 2.51452707, nan, 2.20230692,
0.22959532, 1.21731914, 1.43347482, 1.10541436, 1.21800441,
2.58332304, 0.13404354, 1.17932312, 0.39511208, 1.40536365,
0.03020463, 1.53045691, 1.69837201, 2.69555511, 1.83029795,
0.84343459, 0.54620617, 0.72678135, 0.92132898, 1.98298456,
nan, 2.28953023, 0.26115763, 0.79103097, 1.69345474,
0.72884676, nan, 0.52857803, 1.0567779 , nan,
1.56335081, 0.01664213, 2.31896501, nan, 0.38891339,
0.32958372, 0.0283648 , 1.44605602, 1.53187367, 0.48149097,
0.41636353, 0.94813399, 0.22289834, 1.57218039, 1.0406489 ,
1.0891639 , nan, nan, 0.2001915 , 0.84648205,
0.37818144, 1.40702539, 1.96255967, 0.58932261, 1.90770871,
0.8017689 , 0.43099541, nan, 0.50130903, 0.42236631,
nan, 2.69011397, 1.20146886, 0.95827965, 1.19027816,
1.3390913 , 1.26353796, 1.77550785, nan, nan,
0.95447464, nan])
*
</code></pre>
<p>My first question is, should the transformation return just one value for each column as lambda? and please help me understand what are these nan values?</p>
<p>thanks</p>
 

Latest posts

Top