Trial and Error Method for Optimal Hidden Neurons

The trial and error method trains multiple neural network architectures across a range of neuron counts and selects the smallest model with the lowest validation error.

Step-by-Step MATLAB Script

This script sweeps through hidden neuron counts, averages 5 runs per candidate to remove random weight bias, and finds the best architecture.

% Generate sample dataset
rng(100);
X = linspace(-3, 3, 500);
Y = sin(X) + 0.2 * randn(1, 500);

% Define search range and repeats
neuronCandidates = 1:2:25;  % Test 1, 3, 5, ..., 25 neurons
numRepeats = 5;
trainMSE = zeros(length(neuronCandidates), 1);
valMSE   = zeros(length(neuronCandidates), 1);
for i = 1:length(neuronCandidates)
    nHidden = neuronCandidates(i);
    tMseTemp = zeros(numRepeats, 1);
    vMseTemp = zeros(numRepeats, 1);
    
    for r = 1:numRepeats
        net = fitnet(nHidden, 'trainlm');
        net.trainParam.showWindow = false;
        net.divideParam.trainRatio = 0.70;
        net.divideParam.valRatio   = 0.15;
        net.divideParam.testRatio  = 0.15;
        
        [net, tr] = train(net, X, Y);
        pred = net(X);
        
        tMseTemp(r) = mean((Y(tr.trainInd) - pred(tr.trainInd)).^2);
        vMseTemp(r) = mean((Y(tr.valInd)   - pred(tr.valInd)).^2);
    end
    
    trainMSE(i) = mean(tMseTemp);
    valMSE(i)   = mean(vMseTemp);
    fprintf('Neurons: %2d | Val MSE: %.5f\n', nHidden, valMSE(i));
end
% Identify optimal neuron count
[minValError, bestIdx] = min(valMSE);
bestNeurons = neuronCandidates(bestIdx);
fprintf('\nBest Hidden Neurons: %d (MSE = %.5f)\n', bestNeurons, minValError);
% Plot error curves
figure('Color', 'w');
plot(neuronCandidates, trainMSE, 'b-o', 'LineWidth', 1.2, 'DisplayName', 'Train MSE');
hold on;
plot(neuronCandidates, valMSE, 'r-s', 'LineWidth', 1.5, 'DisplayName', 'Val MSE');
xline(bestNeurons, 'g--', 'LineWidth', 1.2, 'DisplayName', 'Optimal');
xlabel('Number of Hidden Neurons');
ylabel('Mean Squared Error (MSE)');
title('Error vs. Hidden Neuron Count');
legend('Location', 'northeast');
grid on;

Decision Criteria