padding when saving to a file

Is it possible to control (actually remove) the amount of white space
padded to a figure when saving to a file? For example, the white
border found on the output of something like the following:

import matplotlib.pyplot as plt
import matplotlib.patches as mpatches

fig = plt.figure()
ax = fig.add_subplot(111,frameon=False)
ax.add_artist(mpatches.Rectangle((0,0), 1, 1, fc='gray', ec='r'))

ax.set_xticks([])
ax.set_yticks([])

ax.set_xbound([0,1])
ax.set_ybound([0,1])

fig.set_size_inches(3,3)
fig.savefig("test.eps")