0👍
I found the answer myself, wanted to update the post for others just in case if someone having the same issue.
Apparently that was an issue that is resolved with the latest version regarding auto scaling the charts. I updated to https://github.com/chartjs/Chart.js/releases/tag/v2.4.0 and everything works fine.
Source:stackexchange.com