nawab_faruk asked . 2022-04-21

Limit divergence of polynomial fitting

I am approximating a set of data with a polynomial to later find the cumulative density function. I am using a polynomial since I do not know the distribution of the set of data, that may even vary, and because it should be fully automatized. The problem is thatat the end of my set of data, once the polynomial reaches the last data, divergest to infinity. Since it should approximate a probability density function it shouldn't happen, because like so it influence all the results. How can I fix that?

      p=polyfit(X,N,21);
      x=linspace(0,4000,1000);
      y=polyval(p,x);

matlab , AI, Data Science, and Statistics , Statistics and Machine Learning Toolbox

Expert Answer

Prashant Kumar answered . 2024-12-21 17:56:41

The simple answer for this is to ask a question. WHY IN THE NAME OF GOD AND LITTLE GREEN APPLES ARE YOU USING A POLYNOMIAL????? Ok, maybe I'm being too hard on you. Polyfit is there for you to use. Nothing in the documentation tells you it does silly things, or that this is the fundamental nature of polynomials. Somewhere teachers should teach you, right after they tell you that you CAN use a polynomial, that you probably, usually, should not do so. And then explain why not.

Seriously, DON'T USE A POLYNOMIAL. And NEVER use a high order polynomial. They go to hell faster than a low order one. A polynomial order of 21? WAY TOO HIGH. But then you should not use a polynomial at all here.

Polynomials do exactly what you saw. They go to infinity. Think about it. what other behavior can you possibly expect? Can a polynomial approximate a function that does non-trivial (non-constant) stuff within some region, then goes constant as x goes to plus or minus infinity? Think about it. A polynomial can never be a model for a process that behaves like a CDF. You have this really high order polynomial, with powers as high as 21. X is as large as 4000?

What is 4000^21? How about other numbers near there?

4000^21
ans =
    4.398e+75

3950^21
ans =
   3.3771e+75

3500^21
ans =
   2.6634e+74

The highest power of X with a non-zero coefficient will dominate the behavior of a polynomial as X goes to +/-inf. So your high order polynomial produces complete and utter garbage out there. You know this is the behavior that you see. And simple mathematics tells you that you cannot expect any other behavior from a polyomial model. That is exactly the behavior you expect from a polynomial, the innate, fundamental behavior that exists in any polynomial, and something you cannot change.

The obvious answer to my original question is you have no idea what else you might use. Sorry, but that is clearly true.

And, the answer to that is to use a spline. There are several choices you might use there. The simplest is pchip, which will interpolate your data. (Of course, you could have just tried interp1, which can use pchip, or various options as you would direct it to use.)

pp = pchip(X,N);
x = linspace(0,4000,1000);
y = ppval(pp,x);

DON'T use the function spline, as it will show oscillatory behavior in the tails. If you use interp1 instead, then the 'linear', 'pchip', or 'cubic' methods should all be ok. While 'linear' will just do piecewise linear interpolation, that is probably entirely adequate for this purpose.

IF you feel that you needed to smooth the curve instead of interpolate it, then download my SLM toolbox from the file exchange. It will require the optimization toolbox. But pchip should be entirely adequate here.

Not satisfied with the answer ?? ASK NOW

Recent Matlab Projects

Classification of Covid and Non-Covid Lungs CT-Scan using Deep Learning with MATLAB

Matlab simulation on Wind Energy system

Frequently Asked Questions

MATLAB offers tools for real-time AI applications, including Simulink for modeling and simulation. It can be used for developing algorithms and control systems for autonomous vehicles, robots, and other real-time AI systems.

MATLAB Online™ provides access to MATLAB® from your web browser. With MATLAB Online, your files are stored on MATLAB Drive™ and are available wherever you go. MATLAB Drive Connector synchronizes your files between your computers and MATLAB Online, providing offline access and eliminating the need to manually upload or download files. You can also run your files from the convenience of your smartphone or tablet by connecting to MathWorks® Cloud through the MATLAB Mobile™ app.

Yes, MATLAB provides tools and frameworks for deep learning, including the Deep Learning Toolbox. You can use MATLAB for tasks like building and training neural networks, image classification, and natural language processing.

MATLAB and Python are both popular choices for AI development. MATLAB is known for its ease of use in mathematical computations and its extensive toolbox for AI and machine learning. Python, on the other hand, has a vast ecosystem of libraries like TensorFlow and PyTorch. The choice depends on your preferences and project requirements.

You can find support, discussion forums, and a community of MATLAB users on the MATLAB website, Matlansolutions forums, and other AI-related online communities. Remember that MATLAB's capabilities in AI and machine learning continue to evolve, so staying updated with the latest features and resources is essential for effective AI development using MATLAB.

Without any hesitation the answer to this question is NO. The service we offer is 100% legal, legitimate and won't make you a cheater. Read and discover exactly what an essay writing service is and how when used correctly, is a valuable teaching aid and no more akin to cheating than a tutor's 'model essay' or the many published essay guides available from your local book shop. You should use the work as a reference and should not hand over the exact copy of it.