Add some caching to texmanager.py?

       TEXMFOUTPUT Normally, TeX puts its output files in

    > the current directory. If any output file cannot be
    > opened there, it tries to open it in the directory
    > specified in the environment variable TEXM- FOUTPUT.
    > There is no default value for that variable. For
    > example, if you say tex paper and the current directory
    > is not writable, if TEXMFOUTPUT has the value /tmp, TeX
    > attempts to create /tmp/paper.log (and /tmp/paper.dvi, if
    > any output is produced.)

Hmm, that looks familiar -- I think I did come across that. But this
will be a bit of a pain to set from matplotlib, since we are making
calls to os.popen. We might have to do something like

  os.popen('export TEXMFOUTPUT=/some/path; run some command')

and then deal with platform and shell dependent differences in how to
set environment variables. Is there a better way? Me thinks it is
better to make this a FAQ and advise people working in non-writable
dirs to set this var themselves.

Thanks for the help...

JDH

John Hunter wrote:

We might have to do something like

os.popen('export TEXMFOUTPUT=/some/path; run some command')

and then deal with platform and shell dependent differences in how to
set environment variables. Is there a better way? Me thinks it is
better to make this a FAQ and advise people working in non-writable
dirs to set this var themselves.

I agree with this last. On Unix, you could do

os.popen('/bin/sh -c "export TEXMFOUTPUT=/some/path; run some command"')

and be pretty sure it would work on all systems. But Windows would be a pain. I agree with the FAQ solution. This seems like a very rare problem to me, to be honest.

Stephen Walton wrote:

John Hunter wrote:

We might have to do something like

os.popen('export TEXMFOUTPUT=/some/path; run some command')

and then deal with platform and shell dependent differences in how to
set environment variables. Is there a better way? Me thinks it is
better to make this a FAQ and advise people working in non-writable
dirs to set this var themselves.

I agree with this last. On Unix, you could do

os.popen('/bin/sh -c "export TEXMFOUTPUT=/some/path; run some command"')

and be pretty sure it would work on all systems. But Windows would be a pain. I agree with the FAQ solution. This seems like a very rare problem to me, to be honest.

It may be simpler than that:

planck[~/test]> cat env.py
#!/usr/bin/env python
import os

print 'before'
os.system('env | grep TEXMFOUTPUT')

os.environ['TEXMFOUTPUT'] = '/some/path'

print 'after'
os.system('env | grep TEXMFOUTPUT')
planck[~/test]> ./env.py
before
after
TEXMFOUTPUT=/some/path
planck[~/test]>

Cheers,

f

Fernando Perez wrote:

It may be simpler than that:

So os.environ does change the environment under Python and its children. The remaining question is what should TEXMFOUTPUT be? /home/user/.tex.cache will not work across all platforms.

Stephen Walton wrote:

Fernando Perez wrote:

It may be simpler than that:

So os.environ does change the environment under Python and its children. The remaining question is what should TEXMFOUTPUT be? /home/user/.tex.cache will not work across all platforms.

Well, it could be something like $HOME/.tex.cache, where $HOME can be determined via a routine like the below (this is what ipython uses to try and guess a sensible value for $HOME):

···

#----------------------------------------------------------------------------
class HomeDirError(Error):
     pass

def get_home_dir():
     """Return the closest possible equivalent to a 'home' directory.

     We first try $HOME. Absent that, on NT it's $HOMEDRIVE\$HOMEPATH.

     Currently only Posix and NT are implemented, a HomeDirError exception is
     raised for all other OSes. """ #'

     try:
         return os.environ['HOME']
     except KeyError:
         if os.name == 'posix':
             raise HomeDirError,'undefined $HOME, IPython can not proceed.'
         elif os.name == 'nt':
             # For some strange reason, win9x returns 'nt' for os.name.
             try:
                 return os.path.join(os.environ['HOMEDRIVE'],os.environ['HOMEPATH'])
             except:
                 try:
                     # Use the registry to get the 'My Documents' folder.
                     import _winreg as wreg
                     key = wreg.OpenKey(wreg.HKEY_CURRENT_USER,
"Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders")
                     homedir = wreg.QueryValueEx(key,'Personal')[0]
                     key.Close()
                     return homedir
                 except:
                     return 'C:\\'
         elif os.name == 'dos':
             # Desperate, may do absurd things in classic MacOS. May work under DOS.
             return 'C:\\'
         else:
             raise HomeDirError,'support for your operating system not implemented.'

Cheers,

f

Fernando Perez wrote:

Well, it could be something like $HOME/.tex.cache, where $HOME can be determined via a routine like the below (this is what ipython uses to try and guess a sensible value for $HOME):

I *like* it.

Stephen Walton wrote:

Fernando Perez wrote:

Well, it could be something like $HOME/.tex.cache, where $HOME can be determined via a routine like the below (this is what ipython uses to try and guess a sensible value for $HOME):

I *like* it.

Though I'd personally vote for matplotlib holding $HOME/.matplotlib/ as a directory, and putting in there a tex.cache dir, the matplotlibrc file, and anything else it may need in the future.

I *really* don't like the idea that matplotlib will begin to put a bunch of differently named things under $HOME with various .foo names. Ipython also started with .ipythonrc, and I quickly moved to an .ipython/ directory, where I stuff anything I need. It's future-proof, clean, and gives an easy way for users to clean up if needed, without having to guess 'how many dot files/directories does this thing create'?

So if you were to ask for my opinion, I'd vote +100 on matplotlib moving to a single directory for holding *ALL* user and configuration data, which would default to $HOME/.matplotlib, and which could be overridden if the $MATPLOTLIBDIR environment variable is defined. That's exactly how ipython works, so it must be the perfect solution :slight_smile:

Cheers,

f

Stephen Walton wrote:
> Fernando Perez wrote:
>>Well, it could be something like $HOME/.tex.cache, where $HOME can be
>>determined via a routine like the below (this is what ipython uses to
>>try and guess a sensible value for $HOME):
>
> I *like* it.

Though I'd personally vote for matplotlib holding $HOME/.matplotlib/ as a
directory, and putting in there a tex.cache dir, the matplotlibrc file, and
anything else it may need in the future.

I second that notion. In fact, I have been meaning to suggest this on the list
for some time now. Thank for reminding me.

I *really* don't like the idea that matplotlib will begin to put a bunch of
differently named things under $HOME with various .foo names. Ipython also
started with .ipythonrc, and I quickly moved to an .ipython/ directory,
where I stuff anything I need. It's future-proof, clean, and gives an easy
way for users to clean up if needed, without having to guess 'how many dot
files/directories does this thing create'?

So if you were to ask for my opinion, I'd vote +100 on matplotlib moving to
a single directory for holding *ALL* user and configuration data, which
would default to $HOME/.matplotlib, and which could be overridden if the
$MATPLOTLIBDIR environment variable is defined. That's exactly how ipython
works, so it must be the perfect solution :slight_smile:

+101

Darren

···

On Wednesday 22 June 2005 06:35 pm, Fernando Perez wrote:

According to the online docs
(http://docs.python.org/lib/os-procinfo.html) setting os.environ isn't
safe/available for all platforms. You can use the subprocess module to
set the environment of a subprocess under python 2.4 but I don't think
there's a simple way to do this and capture the output for earlier
versions.

Nick

···

On Wed, 2005-06-22 at 11:45 -0600, Fernando Perez wrote:

os.environ['TEXMFOUTPUT'] = '/some/path'

Nicholas Young wrote:

os.environ['TEXMFOUTPUT'] = '/some/path'

According to the online docs
(http://docs.python.org/lib/os-procinfo.html) setting os.environ isn't
safe/available for all platforms. You can use the subprocess module to
set the environment of a subprocess under python 2.4 but I don't think
there's a simple way to do this and capture the output for earlier
versions.

Well, after reading that I get that os.environ _is_ writable everywhere, it's just that it may leak memory in OSX/BSD. What's not always available is the putenv() call, but python will find its way around it if needed.

Since this would be a once-only call, I think that leaking a few bytes is an acceptable price to pay to prevent a crash if the user happens to be positioned on a non-writable dir.

Cheers,

f

···

On Wed, 2005-06-22 at 11:45 -0600, Fernando Perez wrote:

Nicholas Young wrote:
>
>>os.environ['TEXMFOUTPUT'] = '/some/path'
>
>
> According to the online docs
> (http://docs.python.org/lib/os-procinfo.html) setting os.environ isn't
> safe/available for all platforms. You can use the subprocess module to
> set the environment of a subprocess under python 2.4 but I don't think
> there's a simple way to do this and capture the output for earlier
> versions.

Well, after reading that I get that os.environ _is_ writable everywhere, it's
just that it may leak memory in OSX/BSD. What's not always available is the
putenv() call, but python will find its way around it if needed.

To quote "If putenv() is not provided, this mapping may be passed to the
appropriate process-creation functions to cause child processes to use a
modified environment.". To me this implies that you have to pass
os.environ to a process-creation function supporting the env keyword
(os.execve, etc.) none of which seem to support capturing output. On
the other hand does anyone actually run mpl on a platform which doesn't
support putenv?

Since this would be a once-only call, I think that leaking a few bytes is an
acceptable price to pay to prevent a crash if the user happens to be
positioned on a non-writable dir.

After reading an online copy of the freebsd man putenv it actually reads
"Successive calls to setenv() or putenv() assigning a differently sized
value to the same name will result in a memory leak." so setting this
once wouldn't be a problem.

So I was wrong and setting os.environ seems reasonable in this case -
but it seems sensible to be aware of the potential for causing problems
if anyone trys to get mpl working on an odd platform.

Nick

···

On Thu, 2005-06-23 at 10:58 -0600, Fernando Perez wrote:

> On Wed, 2005-06-22 at 11:45 -0600, Fernando Perez wrote: