Optimal feedback value for the SDS controller

By Corollary 4.3, we can expect that the time needed for convergence decreases by increasing the gain factor $\Lambda$. Indeed, Figure 6 shows that an optimal $\Lambda$ exists. At higher gain factors, the discretization introduces instabilities: The SDS ``overshoots'' within discretization domains. Therefore performance quickly deteriorates for large $\Lambda$ values. Finer discretization and/or more frequent observations are needed to improve performance: for larger $\Lambda$ values the update rate needs to be increased.

Figure 6: Choosing an Optimal Feedback Gain.
The figure demonstrates that an optimal feedback gain exists for SDS. Because of the stochastic nature of the process, results depend on random factors. Therefore we calculated every result for lower $\Lambda$ values 3 times with different random seeds. In experiments with coarser discretizations, RL was able to learn the task only for non-zero $\Lambda$ values.