Adaptive Filtering for Stochastic System Identification: This project implements and compares adaptive filtering algorithms for identifying unknown systems under white-noise and coloured-noise input conditions. The simulations evaluate convergence behavior, tracking behavior, ensemble-averaged mean-squared error, and adaptive weight trajectories for both time-invariant and time-varying systems. It also shows the importance of selecting appropriate step-size and forgetting-factor parameters for LMS-based and RLS-based adaptive algorithms. These parameters affect convergence speed, tracking ability, steady-state error, and computational complexity tradeoffs.
This repository presents a proof of concept MATLAB implementation of adaptive filtering algorithms for stochastic system identification. Current simulations demonstrate convergence, tracking, MSE behavior, and algorithm tradeoffs. Future work includes additional unit tests and stochastic parameter tests, fixed-point implementation, and hardware-oriented optimization.
Simulation Structure: The adaptive filters used in this project are Normalized Least Mean Square (NLMS), Recursive Least Square (RLS), and Discrete Fourier Transform Least Mean Square (DFT-LMS). These adaptive filters follow the same basic system identification workflow, as shown in Figure 1. However, DFT-LMS includes a preprocessing block that transforms the input signal before adaptive filtering. This helps reduce input correlation and improves convergence behavior, especially for coloured noise inputs, as shown in Figure 2.
Figure 1: System identification block diagram using adaptive filters.
Figure 2: System identification block diagram using DFT-LMS.
Simulation:
- Step-size Impact on Tradeoffs: A 5-tap DFT-LMS adaptive filter was used to estimate the impulse response of a time-varying unknown system modeled as a first-order Markov process. Three different step sizes were evaluated to demonstrate how the step size affects convergence speed, tracking ability, and steady-state error, as shown in the table and figures below.
Figure 3: Time-varying estimate & tracking of the unknown system excited by white noise
Figure 4: Time-varying estimate & tracking of the unknown system excited by coloured noise
Figure 5: mu = 0.01, ensembled average of the time-varying estimate & tracking of the unknown system
Figure 6: Time-varying estimate & tracking of the unknown system excited by white noise
Figure 7: Time-varying estimate & tracking of the unknown system excited by coloured noise
Figure 8: mu = 0.05, ensembled average error of the time-varying estimate & tracking of the unknown system
Figure 9: Time-varying estimate & tracking of the unknown system excited by white noise
Figure 10: Time-varying estimate & tracking of the unknown system excited by coloured noise
Figure 11: mu = 1, ensembled average error of the time-varying estimate & tracking of the unknown system
Table 1: Effect of step size on convergence speed, tracking ability, and steady-state error.
Conclusion Based on the MSE results, a step size of 0.05 provided the best balance between convergence speed, tracking accuracy, and steady-state error. A convergence factor of 0.01 produced slower convergence and higher tracking lag, while a step size of 1 was too large and caused divergence. Also, with the exception of the divergent case where μ = 1, the TD-LMS adaptive filter performed better in white-noise environments than in coloured-noise environments. This is because white noise has lower input correlation, allowing faster and more reliable adaptation, while coloured noise increases correlation and can slow convergence and reduce tracking accuracy.
- Input Signal Impact Tradeoffs: White-noise and coloured-noise inputs were used with the 5-tap variable-RLS and 5-tap NLMS adaptive filters to demonstrate the impact of input-signal correlation on convergence speed and steady-state error during the identification of the impulse response of a time-invariant unknown system.
Figure 12: Ensembled average error of the time-invariant estimate of the unknown system impulse response excited by coloured noise.
Figure 13: Ensembled average error of the time-invariant estimate of the unknown system impulse response excited by white noise.
Conclusion For highly correlated input signals such as speech, careful adaptive-filter selection is important when identifying a time-invariant unknown system. Figures 12 and 13 show that both NLMS and variable-RLS can identify the system coefficients, but they differ in convergence speed, steady-state MSE, and computational cost.Where variable-RLS provides faster convergence and reaches its steady-state MSE in fewer samples compared to NLMS but with higher computational cost. Although these tradeoff considerations were demonstrated using a time-invariant system, the same principles apply to tracking time-varying systems. A variable-RLS algorithm will generally track time-varying system changes much faster than NLMS due to its superior convergence properties, though at the expense of significantly higher computational complexity.
Table 2: Tradeoff Considerations
Adaptive-Filters/
|-- Time-invariant System/ # Time-invariant system identification
|-- Time-varying System/ # Time-varrying system identification
MATLAB
Clone the repository:
git clone https://github.com/DamiProject/Adaptive-Filters.gitOpen MATLAB and navigate to the project root directory.
Individual modules can be explored and run from the Time-invariant System/ and Time-varying System/ folder.
Damilola Awotunde
MEng, Communications & Signal Processing - Western University | LinkedIn