I have two templates created in Keras to solve a multivariate regression problem. The first one has 1 entrance and 5 exits. The second has an additional input, ie it has two inputs and the same 5 outputs. Intuitively, the second model would have a smaller MSE than the first one because of the additional input, but that is not the case. The first model has the MSE smaller than the second one, that is, it performs better. Do you think this intuition is correct? How do I make sure the second model has a smaller MSE?
multivariate regression in keras
0
asked by anonymous 12.09.2018 / 02:52
2 answers
0
It makes sense to think that the more information you put into the model, the better it will be. But this is not always true when we are talking about forecast error.
It may be that including this information has left your model more specific to the training base and less general to the validation basis.
18.09.2018 / 00:50
0
The complex mathematics of neural networks can create confusion in the model. If this input does not contribute significantly to a better result, it may be that it will disrupt and "consume" part of the neurons in calculations that lead to nothing.
This is not uncommon: adding not very meaningful inputs and making the models worse.
But something else may always be happening, check the bugs, see if the final activations are compatible, etc.
It's also worth considering overfitting, as mentioned by @Daniel Falbel in his response.
09.11.2018 / 20:19