Defining y-starting point for floating horizontal bar charts in Chart.js?

The only way as of now seems to be stacking the bar chart with invisible charts as the padding on the x-axis (as suggested here), it works pretty well.

There is possibly a multi-axis solution as well, as suggested by @Arathi in the comments.