smith_eda asked . 2021-07-30

Why is my neural network overfitting?

Hello! I'm trying to make a forecast program using neural networks. The training function I'm using is the Bayesian Regularization.

Results are pretty good, but when I see the performance, I notice that the training error has decreased but the Test values didn't.

In fact, when I test the network with additional new values, the results are pretty awful. I believe that this was because the network became overfitted.

My question is,how can I prevent the 'trainbr' function from overfitting? Every time I train the network the error of the values assigned for testing does not decrease.

inputs = tonndata(x,false,false);
targets = tonndata(t,false,false);

net = feedforwardnet([15,13],'trainbr');

net.trainParam.lr = 0.05; 
net.trainParam.mc = 0.9; 

net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
net.outputs{2}.processFcns = {'removeconstantrows','mapminmax'};

net.divideFcn = 'dividerand';  

net.divideMode = 'time'; 
net.divideParam.trainRatio = 85/100;
net.divideParam.valRatio = 0/100;
net.divideParam.testRatio = 15/100;
net.layers{1}.transferFcn = 'logsig';
net.layers{2}.transferFcn = 'logsig';

net.performFcn = 'mse';

net = train(net,inputs,targets);
outputs = net(inputs);
errors = gsubtract(targets,outputs);
performance = perform(net,targets,outputs)

neural networks , bayesian regulariz...overfittingclassification

Expert Answer

Prashant Kumar answered . 2024-12-21 18:00:03

The best approach for regression is to start with FITNET using as many defaults as possible. The default I-H-O node topology contains Nw = (I+1)*H+(H+1)*O unknown weights. Ntrn training examples yields Ntrneq = Ntrn*O training equations with Ntrndof = Ntrneq-Nw training degrees of freedom. The average variance in the training target examples is MSEtrn00 = mean(var(target')). Obtaining a mean-square-error lower than MSEtrngoal = 0.01*Ntrndof*MSEtrn00a/Ntrneq for Ntrndof > 0 results in a normalized DOF adjusted MSE of NMSEtrna <= 0.01 and the corresponding adjusted training Rsquared R2trna = 1-NMSEtrna >= 0.99. That is interpreted as the successful modeling of at least 99% of the variation in the target.

The training objective is to try and minimize H with the constraint R2trna >=0.99. This is usually achieved by trial and error over a double for loop with the outer loop of hidden node candidate values h = Hmin:dH:Hmax and an inner loop of i = 1:Ntrials random weight initializations. I have posted many, many examples. Search NEWSGROUP and ANSWERS using

 greg fitnet Ntrials

If Ntrneq < ~2*Nw, validation stopping and/or regularization should be used to mitigate the problem of overtraining and overfit net.

The best approach to avoid overtraining is to use BOTH validation set stopping AND regularization.

HOWEVER FOR SOME STRANGE REASON, using validation stopping with TRAINBR is NOT AVAILABLE IN THE NNTOOLBOX !!!

Your choice of TRAINBR instead of FITNET is not wrong. However, you have made numerous errors, especially by not accepting as many defaults as possible.

Why not just use the syntax in

 help trainbr

 doc trainbr

with the double loop approach?

Don't forget to initialize the RNG before the first loop so that you can duplicate results.

Not satisfied with the answer ?? ASK NOW

Recent Matlab Projects

Classification of Covid and Non-Covid Lungs CT-Scan using Deep Learning with MATLAB

Matlab simulation on Wind Energy system

Frequently Asked Questions

MATLAB offers tools for real-time AI applications, including Simulink for modeling and simulation. It can be used for developing algorithms and control systems for autonomous vehicles, robots, and other real-time AI systems.

MATLAB Online™ provides access to MATLAB® from your web browser. With MATLAB Online, your files are stored on MATLAB Drive™ and are available wherever you go. MATLAB Drive Connector synchronizes your files between your computers and MATLAB Online, providing offline access and eliminating the need to manually upload or download files. You can also run your files from the convenience of your smartphone or tablet by connecting to MathWorks® Cloud through the MATLAB Mobile™ app.

Yes, MATLAB provides tools and frameworks for deep learning, including the Deep Learning Toolbox. You can use MATLAB for tasks like building and training neural networks, image classification, and natural language processing.

MATLAB and Python are both popular choices for AI development. MATLAB is known for its ease of use in mathematical computations and its extensive toolbox for AI and machine learning. Python, on the other hand, has a vast ecosystem of libraries like TensorFlow and PyTorch. The choice depends on your preferences and project requirements.

You can find support, discussion forums, and a community of MATLAB users on the MATLAB website, Matlansolutions forums, and other AI-related online communities. Remember that MATLAB's capabilities in AI and machine learning continue to evolve, so staying updated with the latest features and resources is essential for effective AI development using MATLAB.

Without any hesitation the answer to this question is NO. The service we offer is 100% legal, legitimate and won't make you a cheater. Read and discover exactly what an essay writing service is and how when used correctly, is a valuable teaching aid and no more akin to cheating than a tutor's 'model essay' or the many published essay guides available from your local book shop. You should use the work as a reference and should not hand over the exact copy of it.