ews asked . 2021-08-24
System Identification Loss Function
Hi guys !
I have successfully estimated several linear Models for my system including ARX, SS and OE Models. For Model Evaluation and a little bit of presentation I would like to plot the Loss Function Progress. However, searching through the Matlab Doc and the extensive System Identification Toolbox Guide did not help. I can solely extract the last Loss Function Value via
loss_fcn_value = estimated_model.Report.Fit.LossFcn;
% Analogously for AIC, BIC and other measures
What I would like to see is a Loss Function Progress similar to the Neural Network Toolbox where I can see a distinct decline.
system identification , regression , machine learning , optimization , loss function , mse , aic , b
John Williams answered . 2024-11-14 18:28:16
You haven't mentioned it but I guess you have been fitting your models with Suspace method (n4sid). If so, that is a noniterative method. It supports several algorithms that can be selected as values of the N4Weight. What you get with your loss_fcn_value is the final fit.
If you select an iterative method, like the Prediction Error Minimization, you can see the the progress of the iterations, including the cost function value, as is shown below with an example.
%-------------------------------------------------------------
Initializing model parameters...
Estimating parameters using subspace algorithm...
Initialization complete.
Algorithm: Nonlinear least squares with automatically chosen line search method
Norm of First-order Improvement (%)
Iteration Cost Step opti mality Expected Achieved Bisections --------------------------------------------------------------
0 2.58018e-11 - 7.45e+04 13.7 - -
1 2.55764e-11 119 7.71e+05 13.7 0.873 1
2 2.47818e-11 25.7 9.19e+05 12 3.11 2
3 2.44525e-11 9.96 9.61e+05 12.6 1.33 3
4 2.41906e-11 9.15 1.01e+06 12.1 1.07 3
5 2.39705e-11 8.55 1.06e+06 11.6 0.91 3
6 2.37691e-11 8.11 1.1e+06 12.4 0.841 3
7 2.35665e-11 7.78 1.14e+06 12.2 0.852 3
8 2.33486e-11 7.53 1.15e+06 11.8 0.924 3
9 2.31081e-11 7.36 1.16e+06 12.1 1.03 3
10 2.28436e-11 7.23 1.15e+06 11.3 1.14 3
11 2.25592e-11 7.15 1.14e+06 11.8 1.25 3
12 2.25243e-11 14.2 1.28e+06 11.1 0.155 2
13 2.21484e-11 14.3 1.29e+06 12 1.67 2
14 2.15816e-11 14.3 1.23e+06 11.1 2.56 2
15 2.11078e-11 29 1.34e+06 10.6 2.2 1
16 2.00417e-11 60 1.41e+06 9.59 5.05 0
17 1.85941e-11 59 1.84e+05 7.19 7.22 0
18 1.83451e-11 46.9 5.52e+04 1.31 1.34 0
19 1.82854e-11 31.7 1.17e+04 0.532 0.326 0
20 1.8271e-11 16.5 4.41e+03 1.03 0.0789 0
------------------------------------------------------------------------------------------
Estimating parameter covariance...
Not satisfied with the answer ?? ASK NOW