Get ytick label distance in axis coordinates

I would like to place the ylabel in a matplotlib plot horizontally and with the same distance to the y-axis as the y-tick labels. When using axis coordinate 0.0 and "right" horizontal alignment, they are (almost) snug with the axis:

import matplotlib.pyplot as plt
import numpy as np


x = np.array([0.0, 1.0])
y = x
plt.plot(x, y)

ax = plt.gca()
ylabel = plt.ylabel("xxxx\nxxx", horizontalalignment="right", multialignment="right")
plt.gca().yaxis.set_label_coords(0.0, 0.5)
ylabel.set_rotation(0)

plt.show()

img

I want the xs to have the same distance from the axis as the 0.6 in the plot.

Any hints?

Offset them by the tick length?