How matplotlib got me a job

Alternate title: “How I finally convinced my Dad that open-source can put food on the table”. Since this entire story got started on this mailing list, I figured it would be appropriate to end it here.

Last Friday, I signed a contract to begin working as a “Senior Scientific Programmer” for a research company. The company has recently begun making Python the “preferred language for new development” and has become heavily dependent upon NumPy, SciPy and matplotlib. They have been doing fairly well for a while now, but a few months ago, they ran into a problem with matplotlib. After spending a few weeks butting heads on it, they finally decided to post a question about it to the matplotlib-users list. After reading the question and seeing the code example, I replied with a one-line fix within half an hour of its posting and moved on.

About a week later, I got a personal email from the original poster informing me that my solution worked perfectly. He also noticed that I was working in a neighboring building on campus and wondered just how much longer my PhD was going to take and if I had any interest in going into the private sector. (The company happened to deal with atmospheric science and my PhD is in meteorology).

It turned out that the company realized the value of having on-staff a “SciPy Guru” (I still consider myself a beginner). After the usual visits and interviews, I was offered a position. At multiple times throughout the process, it was obvious to me that while it was good that I was an atmospheric scientist, what was most valuable to them was my knowledge, insight and expertise with Python and its tools.

The lesson I hope everyone here can take in is that there are many companies out there that are using open-source tools and libraries for their purposes. Learning and using these tools for your own purposes not only solves your immediate needs, but also sets you up for future opportunities.

Therefore, I would like to thank John Hunter for making matplotlib available for the community, and a hearty thanks to the rest of the community for their contributions to matplotlib. Without this, I doubt I would have found this job opportunity, nor have the “value-added” skills to have them consider hiring me.

Lastly, a reminder to everyone on this list, I hope this encourages more of you to help each other out with answers. You never know if the person you help out is your future co-worker!

Cheers!
Ben Root

Benjamin Root, on 2012-02-06 13:59, wrote:

Alternate title: "How I finally convinced my Dad that open-source can put
food on the table". Since this entire story got started on this mailing
list, I figured it would be appropriate to end it here.

Last Friday, I signed a contract to begin working as a "Senior Scientific
Programmer" for a research company.

Congrats on the new job, Ben! Great story, I could say that it
had me "Root-ing" for you - but that would make you groan from
the number of times you've probably heard it before, so I'm not
gonna do that :wink:

Lastly, a reminder to everyone on this list, I hope this encourages more of
you to help each other out with answers. You never know if the person you
help out is your future co-worker!

Hope this doesn't mean you'll be posting less, now :slight_smile:

I want to second Ben's comments: I learned (and continue to
learn) quite a bit about matplotlib by trying to answer the
questions others have (with my trusty IPython tab-completion, and
when necessary, doing what every Python Jedi does, and use the
Source) - and by following along with the answers others provide.

best,

···

--
Paul Ivanov
314 address only used for lists, off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7

Inspiring and uplifting story, Ben. I’m glad you posted it. Congratulations on your new job! (That’s also interesting that your father even knows what open-source is).

Che

···

On Mon, Feb 6, 2012 at 2:59 PM, Benjamin Root <ben.root@…1304…> wrote:

Alternate title: “How I finally convinced my Dad that open-source can put food on the table”. Since this entire story got started on this mailing list, I figured it would be appropriate to end it here.

Alternate title: "How I finally convinced my Dad that open-source can put
food on the table". Since this entire story got started on this mailing
list, I figured it would be appropriate to end it here.

Love the alternate title. I'm sure we can all substitute
dad>mom>wife>husband>significant_other in that one.

About a week later, I got a personal email from the original poster
informing me that my solution worked perfectly. He also noticed that I was
working in a neighboring building on campus and wondered just how much
longer my PhD was going to take and if I had any interest in going into the
private sector. (The company happened to deal with atmospheric science and
my PhD is in meteorology).

I love that a tiny bit of altruism turned into a good job for you.
Recently my wife, who is a criminal defense attorney, decided to
transition from criminal law to family law, and took on a pro-bono
case of a friend who was in a tough spot (he was accused of spousal
battery by a mentally ill wife and they had several young kids in the
middle). At a Halloween party, she met someone who worked at a family
law firm and began telling him about her case, and that led to a job
interview and soon she'll be having her third interview with the firm.
I don't know how it will turn out, but I'm pretty sure that she
wouldn't have gotten this opportunity had she not taken on this case
pro-bono.

I grew up pretty much accepting the US ethos that 'there is no such
thing as free lunch" and "no one works for free". So it came as a
great surprise to me, sometime in 1994-1995, when I posted a question
on comp.lang.awk about a script I was developing mixing sed and awk
which parsed BibTeX. Some kind soul responded withing 12 hours, "you
should really be using Perl for this", *and* wrote a non-trivial,
several hundred line piece of Perl to solve my problem. I was
dumbstruck that someone would stay up all night solving a problem for
me, not looking for anything except perhaps for "credit". What I
learned next was that altruism is infectious. I began diving deeply
into Perl, mastering it, and answering other people's questions on the
Perl mailing list. At one point, I was one of the top ten posters on
the Perl mailing list -- no mean feat at the time -- mainly
obsessively answering people's questions. Of course when I discovered
Python, I dropped Perl faster than a hot potato, but that spirit of
contributing to and benefiting from a community of people motivated
not by a payback but by contributing to and participating in something
excellent persisted. That free help that guy gave me on comp.lang.awk
probably caused me to spend 8,000 hours over the next decade helping
other people. I guess there is no such thing as free lunch.

It turned out that the company realized the value of having on-staff a
"SciPy Guru" (I still consider myself a beginner). After the usual visits
and interviews, I was offered a position. At multiple times throughout the
process, it was obvious to me that while it was good that I was an
atmospheric scientist, what was most valuable to them was my knowledge,
insight and expertise with Python and its tools.

The lesson I hope everyone here can take in is that there are many companies
out there that are using open-source tools and libraries for their
purposes. Learning and using these tools for your own purposes not only
solves your immediate needs, but also sets you up for future opportunities.

No doubt about this one. I have tried with mixed success on a number
of occasions to hire people for a job in quantitative finance who
possess skills in scientific python tools as well as statistics, and
it is hard to find good matches. Whenever I meet other people like me
who are trying to hire people, they all tell the same tale: it's hard
to find talent. So if you have these skills and would like a job,
contact me :slight_smile:

I've been astounded by the degree of uptake of the scientific python
toolset, and it is accelerating. As more and more people use these
tools, more and more companies require them and most importantly, more
and more talented developers put their energies into them. The amount
of productivity being poured into not only the core tools but also
pandas, scikits-learn, scikits-image, pystatsmodels and others is
awesome, and is definitely taking the tool chain to the next level.

Therefore, I would like to thank John Hunter for making matplotlib available
for the community, and a hearty thanks to the rest of the community for
their contributions to matplotlib. Without this, I doubt I would have found
this job opportunity, nor have the "value-added" skills to have them
consider hiring me.

You're welcome, but I owe you a significant thanks as well. As my
time for significant development has dwindled, the major contributions
by you and the other developers has enabled the project to thrive. I
hope that in your new position, you can continue to devote some time
to core development. Congrats on the new job, and thanks for sharing
the story.

JDH

···

On Mon, Feb 6, 2012 at 1:59 PM, Benjamin Root <ben.root@...1304...> wrote:

2012/2/7 John Hunter <jdh2358@...287...>:

[...]

[...]

About a week later, I got a personal email from the original poster
informing me that my solution worked perfectly. He also noticed that I was
working in a neighboring building on campus and wondered just how much
longer my PhD was going to take and if I had any interest in going into the
private sector. (The company happened to deal with atmospheric science and
my PhD is in meteorology).

I love that a tiny bit of altruism turned into a good job for you.
[...]

[...]

John, I agree fully with you on the altruism, but I think there is
more to Open Source than just altruism.

I'm currenty trying to convice me to work on publishing my
matplotlayers project, which is a mature but in some parts incomplete
project to add layers on top of matplotlib, and connect layers with
colorbars, all intended to make the data more dynamic and take the
update process off the user. But that as a side note, it might be
necessary to understand the procrastination character of this email.

I just have trouble understanding *why the hell I should do that*. I
think that's a rather general question of life. And the answer of OSS
to this is, AISI: Good programming is like good art. It shows you, in
the case of programming, that things just work, instead of having easy
talk, instead of dwelling on the only shallow and only apparently deep
"wisdom" that is too tempting – the ideas, the planning, all that
things that are up in the air but never turn into truth. What I do
like of matplotlib, is that it just happened, that it works, at least
mostly, and that everyone with a basic understanding of Python can
verify fully independently that it really works, is truth, and to what
extent it works, and where it stops working. What I don't like about
matplotlib, is that so many details are solved not thoroughly. But
that's a matter of taste. And I guess, as talk is easy, that's the
reason why I wrote matplotlayers, just to help overcoming that part.
And this is what connects matplotlib and all OSS with art for me: Just
do it. Show and experience the deepness of life when it comes to the
kernel of it. There is more to it than altruism.

Altruism is directed to other people, or, let's say, when we stress
the altruistic character of OSS, we mean the mailing lists, where we
help each other, if we can, where the conversation partner is in mind.
As I said, what I mean, is that part of OSS that makes the code
happen, the dark or not dark rooms with the keyboards or nowadays the
laptops, where everyone dies on his own, where is no conversation
partner at first, where we all stand on our own, where we have to
overcome the procrastination. This is something much more personal,
something much more "egoistic", or better, avoiding the word "ego",
something much more selfless. To be a good OSS dev or try to become
one one needs to train one's selflessness. Maybe that is the reason
why the people doing it make it free of cost, just as a side effect of
this selflessness, when I did it just for me anyway, it doesn't matter
if I make it public in the end too; making people pay for it would
spoil the whole feeling.

Of course we are paid sometimes, but I see that as a side effect.
First the work, then the payment. The work is to make yourself a
name, to show you're capable and determined, and all this stuff which
comes for free. I see this effect of success as the ingredient of
fate to a good dev – fate rewards the selflessness, at least to my
experience. It just pays better. People will be interested in
investing in you if you show that you do it just for youself, that you
are really interested in what you do, etc. pp. – people will tend to
let you die alone if you make the appearance of doing it just for the
money. Of course that depends a bit on socialisation; meaning on the
people around you. But there's also an active component of
socialisation; one has some influence on it, let it be with or without
noticing it.

I'd like to thank John and Ben for this inspiring posts, which showed
them from a side I've never seen so far. Show your hands if you're
thinking the same. I hope I didn't bore you all :slight_smile:

Friedrich

···

On Mon, Feb 6, 2012 at 1:59 PM, Benjamin Root <ben.root@...1304...> wrote:

Well said Friedrich, though I'm a relative newcomer to python/matplotlib, I've been on the peripheral of a lot of open source, and I've enjoyed this thread.

Also, congrats Ben, both on finishing the PhD and on the job. If you ever find yourself in Boulder, CO (I'm at NCAR), let me know and I'll buy you a drink.

ethan

···

On Feb 7, 2012, at 1:42 AM, Friedrich Romstedt wrote:

I'd like to thank John and Ben for this inspiring posts, which showed
them from a side I've never seen so far. Show your hands if you're
thinking the same. I hope I didn't bore you all :slight_smile:

Indeed, congrats to Ben! And I'm very glad to see your many
contributions to the project finding acknowledgment and long-term
benefits for you.

BTW Ethan, it's been a few years since the last time that John Hunter
and I lectured at NCAR (Dec. 2007), but in early April Min
Ragan-Kelley and I will be teaching at a workshop at CU, focusing on
data analysis with the 'scipy stack' and ipython's parallel machinery.
This will be in the context of a genomics workshop on campus, but if
you are interested we might be able to meet up with some of the python
crowd at NCAR...

Just let me know if you are interested; best to write to
fernando.perez@...1016..., as I sometimes stop monitoring mailing
lists if I get swamped.

Cheers,

f

···

On Tue, Feb 7, 2012 at 9:02 AM, Ethan Gutmann <ethan.gutmann@...287...> wrote:

Also, congrats Ben, both on finishing the PhD and on the job. If you ever find yourself in Boulder, CO (I'm at NCAR), let me know and I'll buy you a drink.

Congratulations, Ben! That's an awesome story!

I am by no means at the level of competence that Ben (or Ryan May,
another matplotlib contributor and meteorology PhD student at OU) but
have been able to turn what I've learned from lurking on the MPL
listserv (as well as the Numpy/Scipy listservs) into an improved lot
in life. By no means is it a permanent job, but based on what I've
learned from everyone here, I was able to land what amounts to a
Post-Doc position with the National Severe Storms Laboratory, while I
work toward finishing my PhD. It's so much so that I spend more of my
time actually helping solve other's MPL/Numpy/Scipy problems than I do
working on my own stuff! I'll admit there are times where I think to
myself I should be more selfish and focus on finishing my degree, but
I believe that I owe it to the community to give back, even if it is
off-list. My knowledge is a product of your knowledge and thus I
believe I'm obligated to share it with those around me who lack it,
but desperately need it.

I have to admit, when I first came to the world of Python (from Java
and C), I was amazed at the tools readily available --- the
matplotlib, numpy, and scipys of the community. Specific to MPL, I
thought it was the greatest thing ever. I did all of my plotting with
it. However, as I got into more advanced Python programming and
needed more complex examples, I became frustrated and discouraged that
I couldn't do exactly what I wanted. So much so, that I temporarily
switched to R and the use of the ggplot2 package. However, as I
continued to advance I learned the beauty of matplotlib and realized
just how powerful it actually was. I began to see how to do all the
things I couldn't do before. It's powerful enough that I'm forcing
those around me to switch to Python and MPL, not because I'm telling
them they have to, but because they see what I'm doing with it and
realize that in order to keep up, they need to make the switch.

As I continue to mature as a Python person, I want to give back
explicitly to the community that's given so much to me. The problem
is, I don't know how. I'm intimidated by the awesomeness of what I see
being done around me that I don't even know where to begin. How does
one even begin to learn how to understand the deep intricacies of MPL,
Numpy, and Scipy so that I'd begin to develop a comfort level that
would allow me to begin to actively contribute? I know pretty much
everyone on these listservs, including myself, is busy. (I'm in the
midst of a 30-day PhD General Exam, and probably shouldn't even be
reading the listservs and/or typing this email! *wink*) But if there
are those out there that are willing to take a little time and invest
in me, and I'm sure there are others like me, I'd gladly become an
active contributor instead of a lurker.

Anyways, I know this email is a tad on the long side, and a little off
the original topic, so if you're still reading, thanks! This is
something that's been weighing on me for a few months now, and I
thought Ben's exultation of the benefits of the community might be a
good time to open up.

Cheers,
Patrick

···

---
Patrick Marsh
Ph.D. Student / Liaison to the HWT
School of Meteorology / University of Oklahoma
Cooperative Institute for Mesoscale Meteorological Studies
National Severe Storms Laboratory
http://www.patricktmarsh.com

On Tue, Feb 7, 2012 at 11:19 AM, Fernando Perez <fperez.net@...287...> wrote:

On Tue, Feb 7, 2012 at 9:02 AM, Ethan Gutmann <ethan.gutmann@...287...> wrote:

Also, congrats Ben, both on finishing the PhD and on the job. If you ever find yourself in Boulder, CO (I'm at NCAR), let me know and I'll buy you a drink.

Indeed, congrats to Ben! And I'm very glad to see your many
contributions to the project finding acknowledgment and long-term
benefits for you.

BTW Ethan, it's been a few years since the last time that John Hunter
and I lectured at NCAR (Dec. 2007), but in early April Min
Ragan-Kelley and I will be teaching at a workshop at CU, focusing on
data analysis with the 'scipy stack' and ipython's parallel machinery.
This will be in the context of a genomics workshop on campus, but if
you are interested we might be able to meet up with some of the python
crowd at NCAR...

Just let me know if you are interested; best to write to
fernando.perez@...1016..., as I sometimes stop monitoring mailing
lists if I get swamped.

Cheers,

f

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-d2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Patrick Marsh, on 2012-02-07 13:58, wrote:

As I continue to mature as a Python person, I want to give back
explicitly to the community that's given so much to me. The problem
is, I don't know how. I'm intimidated by the awesomeness of what I see
being done around me that I don't even know where to begin. How does
one even begin to learn how to understand the deep intricacies of MPL,
Numpy, and Scipy so that I'd begin to develop a comfort level that
would allow me to begin to actively contribute? I know pretty much
everyone on these listservs, including myself, is busy. (I'm in the
midst of a 30-day PhD General Exam, and probably shouldn't even be
reading the listservs and/or typing this email! *wink*) But if there
are those out there that are willing to take a little time and invest
in me, and I'm sure there are others like me, I'd gladly become an
active contributor instead of a lurker.

One simple, minimally intimidating way to contribute is by making
improvements to the documentation. Here's a relevant pitch I just
made on the IPython lists about how easy, yet valuable such
improvements can be.

http://mail.scipy.org/pipermail/ipython-user/2012-February/009428.html

Another would be to send the colleagues whom your helping here to
these lists, that way any effort you put in to help them has a
good chance of helping others, thanks to search engines and
archives.

Yet another would be to go through some the active issues on the
tracker and trying to make a test for them.

Yet another still would be to test the various active pull
requests - and confirm that the fixes or new functionality they
provide actually work - look through patches and ask questions -
we have a lot PRs that get very few eyes or comments on them
(Last [academic] year, I consciously made an effort to be more
active on this list, and though I haven't pitched in as much
lately, perhaps the rest of this year I should focus my efforts
on incoming PRs)

Anyways, I know this email is a tad on the long side, and a little off
the original topic, so if you're still reading, thanks! This is
something that's been weighing on me for a few months now, and I
thought Ben's exultation of the benefits of the community might be a
good time to open up.

I'm still rather enjoying this whole thread, thanks for opening
up. I got worried how relatively quiet it's been here for a few
months, and glad we're starting to buck that trend.

best,

···

--
Paul Ivanov
314 address only used for lists, off-list direct email at:
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7

Another big way we've seen new people contribute to the Sage community is to answer questions on the ask.sagemath.org site. Scipy has a similar ask.scipy.org site, but it seems relatively quiet.

Jason

···

On 2/8/12 2:53 AM, Paul Ivanov wrote:

Patrick Marsh, on 2012-02-07 13:58, wrote:

As I continue to mature as a Python person, I want to give back
explicitly to the community that's given so much to me. The problem
is, I don't know how. I'm intimidated by the awesomeness of what I see
being done around me that I don't even know where to begin. How does
one even begin to learn how to understand the deep intricacies of MPL,
Numpy, and Scipy so that I'd begin to develop a comfort level that
would allow me to begin to actively contribute? I know pretty much
everyone on these listservs, including myself, is busy. (I'm in the
midst of a 30-day PhD General Exam, and probably shouldn't even be
reading the listservs and/or typing this email! *wink*) But if there
are those out there that are willing to take a little time and invest
in me, and I'm sure there are others like me, I'd gladly become an
active contributor instead of a lurker.

One simple, minimally intimidating way to contribute is by making
improvements to the documentation. Here's a relevant pitch I just
made on the IPython lists about how easy, yet valuable such
improvements can be.

http://mail.scipy.org/pipermail/ipython-user/2012-February/009428.html

Another would be to send the colleagues whom your helping here to
these lists, that way any effort you put in to help them has a
good chance of helping others, thanks to search engines and
archives.

Yet another would be to go through some the active issues on the
tracker and trying to make a test for them.

Yet another still would be to test the various active pull
requests - and confirm that the fixes or new functionality they
provide actually work - look through patches and ask questions -
we have a lot PRs that get very few eyes or comments on them
(Last [academic] year, I consciously made an effort to be more
active on this list, and though I haven't pitched in as much
lately, perhaps the rest of this year I should focus my efforts
on incoming PRs)