From 2004 to 2006 I had a job finding, researching, writing/editing technology news stories about state and local governments for www.govtech.com. I would post at least 3 news stories a day. We also made up HTML newsletters of our stories every week and sent them to our readers. We were having some difficulties with our newsletter maker and since I had been learning PHP, MySQL, HTML, Javascript and CSS I thought I would make some utilities to make our lives easier and faster.
It wasn't my job to do any programming so I did it off work hours at first. I wrote a screen scrapper in PHP that pulled our new news off our website and formatted them into an HTML email newsletter. I started using it and I showed it to my boss who liked it and started using it too. This was the start of my professional programming career.
We had another problem. Everyday we spent hours looking through many websites looking for technology news. Same thing every day. This was the same time I had a growing interest in Web programming. In my spare time I was often reading something about programming. I came across an article called Why Python? by Eric Raymond. I loved the article and it got me really interested in Python. So I bought a book on Python and learned it like crazy.
Now back to my problem at work. So much time looking through the same websites over and over. I decided to write a Web crawler to find the news for us. I decided to write the Web crawler in Python! So I started working on this. I made the Web crawler/spider search the most common webpages and websites that we would look through. At first it would search about 25 websites, then 50 then maybe 75 or 100. It would search for technology related content. I showed it to my boss and he liked it. He said that I could spend part of my working hours developing this and I did. I kept tweaking and tweaking and developing it, making it better and better. We called this piece of software the Govbot.
The Govbot found news through a set of regular expressions that I wrote that would analyse the content: basically a complicated keyword search process. But different websites were built differently and I found that while the Govbot would do well searching some websites it would do poorly searching others. So I built a plugin system into the Govbot that made it easy to add code for searching specific websites. That way I could easily customize the Govbot for websites that we wanted to search but that were difficult because of the way they were built. The Govbot showed two categories of results: just new stuff it found and content that was likely to be news of interest to us. My boss continued to use the Govbot for 3 1/2 years after I left my job there.
During my time at govtech I installed and customized a WordPress blog for myself and another blog using Serendipity. I also created a directory of government blogs and a public web-based RSS reader for those blogs.
In 2006 I got a job as a graphics designer in Los Angeles. I mostly designed posters, post cards and emails. I also helped layout a magazine. I used InDesign and Photoshop. While I had this job I found that I missed programming. I found that I would actually prefer using a graphics library to program graphics directly rather than using an authoring tool like Photoshop or InDesign. But I ended up doing some programming anyway. I wrote a plugin in Javascript for InDesign. It would take a list of addresses and a design of a post card and automatically create the post card for each address. This made it much faster to print post cards for each address. I also created a database of contacts using Gadfly, a SQL database system written in Python.
I found out that I am really a programmer, not a graphics designer. But of course I will use GIMP or Inkscape from time to time when I need graphics.
Did I tell you that at this point I had done most of my programming on Linux and that I love Linux? It's true.
In my next post I talk about my first full time programming job.
