Do you want to get informed about new posts via Email?

How to do Fourier Transform to find out the cycle frequencies ?

Author: JohnTong

Creation Date: 1/13/2018 6:34 AM

Probably too complex of a discussion for me anymore, but think of it this way... Unless a signal is a pure sine wave, it probably has many frequencies. The output of a Fourier Transform (the frequency spectrum of a signal) is the amplitude of each of the sine waves that compose the signal. The sine wave (a single frequency) with the greatest power (amplitude) would be considered the cycle frequency.

For a demonstration, check out TASC 2008-03 | Measuring Cycle Periods (Ehlers)

For a demonstration, check out TASC 2008-03 | Measuring Cycle Periods (Ehlers)

I've wondered about the best way to do an FFT (or DFT) in Wealth Lab too. If you're running the Visual Studio IDE, then you could develop a WL Performance Visualizer specifically for this purpose. http://personal.fidelity.com/products/trading/Trading_Platforms_Tools/pdf/Creating-Performance-Visualizers-in-Wealth.pdf

The main reason for implementing this with a Performance Visualizer is because you'll be plotting complex number results (or a power spectrum) on a horizontal frequency scale, and you can only open a Windows.Forms chart window inside a WL Performance Visualizer. You can't do that (easily) within a WL strategy.

Math.Net provides some C# signal processing libraries. https://numerics.mathdotnet.com/

Be sure the read the section on Matrices and Vectors first so you declare your data types right.

From__inside__ your Visual Studio *Performance Visualizer* project (use the VS Tools menu), you'll need to NuGet (download) "MathNet.Numerics" into your Performance Visualizer project. Now you can add the C# code below to your VS Performance Visualizer project for WL.

Of course, you'll want the windowing functions too (You'll introduce unwanted sampling harmonics if you skip preprocessing the input with a windowing function.), but they are in a different download. From your Visual Studio*Performance Visualizer* project, you'll need to NuGet (download) "MathNet.Filtering". Now you can add the C# code below to your VS Performance Visualizer project for WL.

There's a nice example coding the FFT with windowing at https://stackoverflow.com/questions/41023808/how-to-properly-run-an-fft-on-a-windowed-set-of-data-from-a-pure-sine-waved

If you get all this working let me know. I haven't had time to explore it yet, but I have thought about it.

---

On a separate note, I see Ehler's solution only explores the dominant cycle (or frequency) and disregards all the components. I do think most of the less significant components are just noise and should be excluded. I'm not sure if only the dominant frequency is important though. But it's an interesting approach.

The main reason for implementing this with a Performance Visualizer is because you'll be plotting complex number results (or a power spectrum) on a horizontal frequency scale, and you can only open a Windows.Forms chart window inside a WL Performance Visualizer. You can't do that (easily) within a WL strategy.

Math.Net provides some C# signal processing libraries. https://numerics.mathdotnet.com/

Be sure the read the section on Matrices and Vectors first so you declare your data types right.

From

CODE:

Please log in to see this code.

Of course, you'll want the windowing functions too (You'll introduce unwanted sampling harmonics if you skip preprocessing the input with a windowing function.), but they are in a different download. From your Visual Studio

CODE:

Please log in to see this code.

There's a nice example coding the FFT with windowing at https://stackoverflow.com/questions/41023808/how-to-properly-run-an-fft-on-a-windowed-set-of-data-from-a-pure-sine-waved

If you get all this working let me know. I haven't had time to explore it yet, but I have thought about it.

---

On a separate note, I see Ehler's solution only explores the dominant cycle (or frequency) and disregards all the components. I do think most of the less significant components are just noise and should be excluded. I'm not sure if only the dominant frequency is important though. But it's an interesting approach.

I did it. Several times. The results:

If you understand a price series as a time signal and do a Fourier Analysis, you'll find a nearly perfect pink noise signal.

If you analyze just very short intervals, in some circumstances some "frequencies" may seem to be stronger than others. This is "normal" for a noise signal.

If you try to be clever (I did) and say: Well, if I find some prominent cycle (A strong line in the FFT) I'll "prolong" this curve to predict future prices.

This is quite some effort. The result:

Prediction is not better than random predictions.

In other words: Any "cycle" which is found by an Fourier Transform is just a statistical fluctuation and says nothing about future movements - nothing.

If you understand a price series as a time signal and do a Fourier Analysis, you'll find a nearly perfect pink noise signal.

If you analyze just very short intervals, in some circumstances some "frequencies" may seem to be stronger than others. This is "normal" for a noise signal.

If you try to be clever (I did) and say: Well, if I find some prominent cycle (A strong line in the FFT) I'll "prolong" this curve to predict future prices.

This is quite some effort. The result:

Prediction is not better than random predictions.

In other words: Any "cycle" which is found by an Fourier Transform is just a statistical fluctuation and says nothing about future movements - nothing.

QUOTE:Yes, I was kind of afraid of that. Be sure to try several windowing types because some will create less sampling artifact than others. I kind of like the Gaussian one (shown above) as a good initial choice.

[Conclusion:] Any "cycle" which is found by an Fourier Transform is just a statistical fluctuation and says nothing about future movements - nothing.

It's really going to depend on the stocks involved. Some stocks are going to have seasonal effects (cycles), but most probably won't. Of the few that are cyclic, they will probably have phase-lag properties. For example, restaurants may trade highest on Fridays than on other days of the week.

I feel kind of bad suggesting this (seems backwards), but maybe this problem should be turned upside down. Identify the predominate noise ranges, then notch filter those bands out (band rejection). Then the residuals are left with only the meaningful signal energies to study.

---

This is off topic, but I think employing some different kinds of nonlinear filters (Median is the most common example) is the best choice here. I don't think linear filters based on complex exponentials (i.e. sinusoids) alone will do justice to this problem. If there's some forum to discuss time series analysis on, I can share more of my thoughts. But this isn't really a Wealth Lab topic.

While Dr. Koch without any doubt is one of the most competent contributors here in the forum, I would like to add the following:

The interesting thing about spectrum-extracted cycles in market data is that they tend to cluster around certain number of daycounts, such as e.g. 28-29 days, 120-140 days, 180 days, shift slightly over time (e.g. from 122 days to 127 days etc,), and then disappear for 2-4 years just to reemerge with very similar periods again once a trending phase of the market has terminated.

Interestingly, other "daycount periods" that seem to exhibit more frequent appearances than what would be statistically deemed normal are 1020 to 1080 days, present in many different assets

If someone has an idea how to deal with this, please kindly get back to me. :-)

The interesting thing about spectrum-extracted cycles in market data is that they tend to cluster around certain number of daycounts, such as e.g. 28-29 days, 120-140 days, 180 days, shift slightly over time (e.g. from 122 days to 127 days etc,), and then disappear for 2-4 years just to reemerge with very similar periods again once a trending phase of the market has terminated.

Interestingly, other "daycount periods" that seem to exhibit more frequent appearances than what would be statistically deemed normal are 1020 to 1080 days, present in many different assets

If someone has an idea how to deal with this, please kindly get back to me. :-)

I agree with DrKoch. You can (over)fit cycles to back periods of almost any duration. IMHO, there is no predictive power, however.

QUOTE:A Fourier Transform (FFT/DFT) is going to require about

... spectrum-extracted cycles in market data ... tend to cluster around certain number of day counts, such as 28-29 days, 120-140 days, 180 days, shift slightly over time (e.g. from 122 days to 127 days etc,), and thendisappear for 2-4 years

Perhaps some type of phase-tracking methodology (e.g. phase-lock loop) might track and flag such an infrequent wave pattern if it fell within its frequency ranges. Phase-lock loops have a front-end low-pass filter.

The Kalman filter for financial apps uses a 3-term 0th-order, 1st-order, and 2nd-order Taylor series to fit a dynamic trend. Have you looked into that? The WL implementation of the Kalman indicator won't do forecasting, which I think is a big omission. But it's still worth a look.

When using the Kalman, if you're working with a