Question about backends and X

I have a matplotlib application which can display a plot or write it
to a file based on command line flags. I have discovered that if it is
run without DISPLAY set (say, from a crontab file) that the default
backend (I have no matplotlibrc file) attempts to chat with X, causing
crashes. I've worked around this like so:

import os
import matplotlib
if not os.environ.get("DISPLAY"):
    # Allow non-interactive use (e.g. running with -p from cron)

It seems kind of kludgy to have to do this in all matplotlib
applications. Could this be done within matplotlib itself?