ENIGMA CURRY

Emacs ansi-term tricks

Being on vacation is soo nice. With some of my free time I'm planning on revamping my emacs environment, so hopefully that means a few more articles showing up here on that topic.

I read some great tips on ansi-term. Ansi-term is a terminal emulator written in emacs lisp that is as close to a real terminal as possible. That means you can run virtually all command line programs, even the ones that use ncurses like top or screen, all within emacs. You can still switch between line and char modes which means you can still edit the buffer as you could in a regular (emacs) shell too.

F2 Keybinding

In the above mentioned article Joseph wrote a nice little bit of elisp to get to a running ansi-term efficiently, by hitting F2. The nice thing about it is that it does what I mean:

1. If I'm already in an ansi-term, but it's called "*ansi-term*" rename it.
2. If I'm already in an ansi-term, but it's called something else, start a new ansi-term called "*ansi-term*"
3. If I'm in another non-terminal buffer, switch to a buffer called "*ansi-term*" or create a new one if it doesn't exist

There's one more catch though, as Joseph explains, an ansi-term can be considered "stopped" such that it is no longer running but the buffer still exists. In that case I don't want the third rule to switch me to a defunct terminal, so instead I want it to kill the buffer and create a new ansi-term. Here is my enhanced elisp:

(require 'term)
(defun visit-ansi-term ()
  "If the current buffer is:
     1) a running ansi-term named *ansi-term*, rename it.
     2) a stopped ansi-term, kill it and create a new one.
     3) a non ansi-term, go to an already running ansi-term
        or start a new one while killing a defunt one"
  (interactive)
  (let ((is-term (string= "term-mode" major-mode))
        (is-running (term-check-proc (buffer-name)))
        (term-cmd "/bin/bash")
        (anon-term (get-buffer "*ansi-term*")))
    (if is-term
        (if is-running
            (if (string= "*ansi-term*" (buffer-name))
                (call-interactively 'rename-buffer)
              (if anon-term
                  (switch-to-buffer "*ansi-term*")
                (ansi-term term-cmd)))
          (kill-buffer (buffer-name))
          (ansi-term term-cmd))
      (if anon-term
          (if (term-check-proc "*ansi-term*")
              (switch-to-buffer "*ansi-term*")
            (kill-buffer "*ansi-term*")
            (ansi-term term-cmd))
        (ansi-term term-cmd)))))
(global-set-key (kbd "<f2>") 'visit-ansi-term)

TRAMP Integration

Put the following inside of your .bash_profile on any computer that you ssh into frequently:

#Emacs ansi-term directory tracking
# track directory, username, and cwd for remote logons
if [ $TERM = eterm-color ]; then
    function eterm-set-cwd {
        $@
        echo -e "\033AnSiTc" $(pwd)
    }
    
    # set hostname, user, and cwd
    function eterm-reset {
        echo -e "\033AnSiTu" $(whoami)
        echo -e "\033AnSiTc" $(pwd)
        echo -e "\033AnSiTh" $(hostname)
    }
    
    for temp in cd pushd popd; do
        alias $temp="eterm-set-cwd $temp"
    done
    
    # set hostname, user, and cwd now
    eterm-reset
fi

Now when you ssh into a machine from within ansi-term and open a file with C-x C-f you'll be loading a file via TRAMP from the current working directory on the remote machine. Amazingly cool.

Wrangling my finances

I've gotten better at thinking about my finances over the years but I've never had anything better than a mental budget. I've never actually written anything down on paper. I've got a brand new job so I figure I've got a clean slate. Starting right now, I pledge to myself to keep a balanced checkbook at all times and to do monthly reviews of my finances looking for things to cut out of my budget.

Fiscal Conscience: Ha! How many times have you said that?
Me: Dozens of times!
Fiscal Conscience: And did you ever actually do it?
Me: No, but this time it will be different!
Fiscal Conscience: You're going to have do to better than that!
Me: Fine, I'll show you.

GnuCash is a wonderful piece of open-source accounting software. I have used it myself, dozens of times. But as it happens I inevitably stop using it. Not because the software is flawed per se, it does everything a good accounting application should do:

• Use Double-Entry accounting
• Support multiple currencies
• OFX import from online banking sites
• etc

The only thing they get wrong is the fundamental design choice they made when developing the application: they made it a GUI. Sure, GUIs are great, but in my opinion GUIs should be interfaces to a service oriented application rather than the application itself. Unless I'm sitting at my desk at home I can't use my application. I don't consider remote desktops a reasonable response to this problem due to the huge network latency issues as well as firewall problems.

I want to be able to record my transactions from anywhere. At home at my desk, at work at my desk, on my N800 when traveling, or on my phone when running around. With an application fundamentally written as a GUI this is essentially an insurmountable problem. An application written as a service can do all of these things quite easily with minimal amounts of programming.

Enter Ledger.

Ledger does not keep track of your accounts. You keep track of your accounts in a plain text file and Ledger helps you understand them better. I can update my checkbook register from any text editor, anywhere I am. Most often this is Emacs through SSH on my home computer. Because of the triviality of the file format (being plain text) I can write a simple application that takes text messages from my phone and adds them to the register automatically.

But what about all my automatic payments?

I have most of my monthly payments automatically debited from my account. This is nice since I don't have to spend the time submitting payments to X number of companies each month, but is a nightmare when it comes time to balancing my checkbook. What I have done in the past is to just import an OFX file from my bank directly into GnuCash. This works great until you realize that something is wrong with the import and your checkbook is no longer balancing correctly. For this reason alone I prefer to record each transaction I make by hand. My money is my responsibility after all.

So, I wanted to be instantly notified of any debit on my checking account as soon as it happens. This gives me the following:

• A text message to my phone describing the time, place and amount of the transaction
• The total balance of my account
• A way to constantly keep an eye on how I'm spending my money without the hassle of (remembering to) signing into my bank account
• A reminder to balance my checkbook. (tells me when my actual checking balance has been out of sync with ledger for over 48 hours.)

The central peice to this process is getting the transaction data from my bank account. My bank offers online banking but it does not offer an easy way to download my transactions. Sure it supports Quicken and MS Money exports, but its all protected behind a password protected javascript-enabled website. I wanted to use the wonderful mechanize library for python. However, it doesn't support javascript.

Enter Selenium RC.

Selenium RC is an application that can remote control a real web browser and submit and receive data. I run selenium which launches Firefox in a headless X-server (Xvfb) and continuously refreshes my online banking site and parses out my account activity. This is probably a bit memory intensive to always keep a browser open for this specific purpose. I may want to experiment with python-spidermonkey in the future and go back to using mechanize, but I think there's a lot of glue missing in that solution whereas Selenium RC works perfectly right now!

No code to share at the moment, it's all a giant hack. If a similar setup appeals to someone out there, let me know and I'll think about releasing something.

Shortened URLs in Emacs using is.gd (like tinyurl)

I've casually been teaching myself emacs lisp lately. Today I wrote a utility that shortens long urls within regions using the http://is.gd URL shortening service. There's plenty of existing code out there that is more lisp like, but this is supposed to be a learning experience for me so I did it myself. I like python and so I used python for most of the heavy lifting.

I created a directory to hold all of my emacs specific python functions: ~/.emacs.d/ryan-pymacs-extensions

I wrote the following python function, shorten_url.py in that directory:

!/usr/bin/env python
# -*- coding: utf-8 -*-

__author__ = "Ryan McGuire (ryan@enigmacurry.com)"
__date__   = "Mon Sep 15 12:27:14 2008"

import doctest
import urllib2
import re

def shorten_with_is_gd(url):
    """Shorten a URL with is.gd

    >>> shorten_with_is_gd('http://www.enigmacurry.com')
    'http://is.gd/FFP'

    """
    u = urllib2.urlopen("http://is.gd/api.php?longurl="+url)
    return u.read()

def shorten_in_text(text):
    """Shorten all the urls found inside some text

    >>> shorten_in_text('Hi from http://www.enigmacurry.com')
    'Hi from http://is.gd/FFP'
    
    """
    replacements = {} #URL -> is.gd URL
    #Only check for urls that start with "http://" for now
    for m in re.finditer("http://[^ \n\r]*", text):
        try:
            replacements[m.group()] = shorten_with_is_gd(m.group())
        except:
            replacements[m.group()] = m.group()
    for url,replacement in replacements.items():
        text = text.replace(url, replacement)
    return text

if __name__ == '__main__':
    doctest.testmod(verbose=True)

and the following lisp makes "M-x shorten-url" do the rest of the replacement work:

;add ~/.emacs.d/ryan-python-extensions to python path
(pymacs-exec "import sys, os")
(pymacs-exec "sys.path.append(os.path.join(os.path.expanduser('~'),'.emacs.d','ryan-pymacs-extensions'))")

;;Shorten URLs with is.gd
(pymacs-exec "import shorten_url")
(defun shorten-url (start end)
  (interactive "r")
  (let ((region (buffer-substring start end)))
    (let ((rt (pymacs-eval (format "shorten_url.shorten_in_text('''%s''')" region))))
      (kill-region start end)
      (insert rt)
      )
  ))

Emacs IRC (ERC) with Noticeable Notifications

I use ERC for all my IRC chatting. I finally got fed up with not noticing someones message because I didn't have emacs focused. So I spent my evening concocting a more noticeable messaging system through Pymacs and libnotify.

Half way though implementing this, wouldn't you know it, I found ErcPageMe which does exactly what I wanted. I figured I was learning quite a bit and I continued writing my own version. I expanded on their code and (at least for me) made some improvements. So kudos go to whoever wrote ErcPageMe :)

The following code will pop up a message on your gnome desktop alerting you whenever you receive a personal message or when someone mentions your nickname in a channel. It also avoids notification for the same user in the same channel if they triggered a message within the last 30 seconds.

Here is my lisp and embedded python/pymacs code:

(defun notify-desktop (title message &optional duration &optional icon)
  "Pop up a message on the desktop with an optional duration (forever otherwise)"
  (pymacs-exec "import pynotify")
  (pymacs-exec "pynotify.init('Emacs')")
  (if icon 
      (pymacs-exec (format "msg = pynotify.Notification('%s','%s','%s')"
                           title message icon))
    (pymacs-exec (format "msg = pynotify.Notification('%s','%s')" title message))
    ) 
  (if duration 
      (pymacs-exec (format "msg.set_timeout(%s)" duration))
    )
  (pymacs-exec "msg.show()")
  )

;; Notify me when someone wants to talk to me.
;; Heavily based off of ErcPageMe on emacswiki.org, with some improvements.
;; I wanted to learn and I used my own notification system with pymacs
;; Delay is on a per user, per channel basis now.
(defvar erc-page-nick-alist nil
  "Alist of 'nickname|target' and last time they triggered a notification"
  )
(defun erc-notify-allowed (nick target &optional delay)
  "Return true if a certain nick has waited long enough to notify"
  (unless delay (setq delay 30))
  (let ((cur-time (time-to-seconds (current-time)))
        (cur-assoc (assoc (format "%s|%s" nick target) erc-page-nick-alist))
        (last-time))
    (if cur-assoc
        (progn
          (setq last-time (cdr cur-assoc))
          (setcdr cur-assoc cur-time)
          (> (abs (- cur-time last-time)) delay))
      (push (cons (format "%s|%s" nick target) cur-time) erc-page-nick-alist)
      t)
    )
  )
(defun erc-notify-PRIVMSG (proc parsed)
  (let ((nick (car (erc-parse-user (erc-response.sender parsed))))
	(target (car (erc-response.command-args parsed)))
	(msg (erc-response.contents parsed)))
    ;;Handle true private/direct messages (non channel)
    (when (and (not (erc-is-message-ctcp-and-not-action-p msg))
               (erc-current-nick-p target)
	       (erc-notify-allowed nick target)
	       )
      ;Do actual notification
      (ding)
      (notify-desktop (format "%s - %s" nick
                              (format-time-string "%b %d %I:%M %p"))
                      msg 0 "gnome-emacs")
      )
    ;;Handle channel messages when my nick is mentioned
    (when (and (not (erc-is-message-ctcp-and-not-action-p msg))
               (string-match (erc-current-nick) msg)
               (erc-notify-allowed nick target)
	       )
      ;Do actual notification
      (ding)
      (notify-desktop (format "%s - %s" target
                              (format-time-string "%b %d %I:%M %p"))
                      (format "%s: %s" nick msg) 0 "gnome-emacs")
      )
    )
      
  )

(add-hook 'erc-server-PRIVMSG-functions 'erc-notify-PRIVMSG)

Emacs as a powerful Python IDE

Update 01/2009: this post is still valid, but see updated installation instructions here.

Last night at the Python user group I gave a short demo on using Emacs as a Python editor/IDE. My macbook pro refused to display on the projector so I thought my demo was going to be a 'no go'. Thankfully, sontek allowed me to use his Linux laptop. I hurriedly copied over my emacs environment, installed a few packages and was able to present after all. I think the demo went fairly well even though I think it was a bit hurried and I forgot to cover a few things, I think I was pretty nervous at the same time because of the fact that the mac didn't work and got me flustered. Oh well, I think people enjoyed it.

My Emacs Environment

Below are the Emacs features most applicable to Python development:

• Rope and Ropemacs Rope is a general (non-emacs specific) Python IDE library. It has awesome support for multiple refactoring methods and code introspection. Inside Emacs, this gives us:
  • Full (working!!) code-completion support of modules, classes, methods etc. (M-/ and M-?)
  • Instant documentation for element under the cursor (C-c d)
  • Jump to module/class/method definition of element under the cursor (C-c g). This works for any Python code it finds in your PYTHONPATH, including things from the stdlib.
  • Refactoring of code (like rename -- C-c r r)
  • List all occurences of of a name in your entire project
  • and More.
• YASnippet YASnippet is a snippet tool like TextMate. You can expand user defined keywords into whole blocks of predefined code. This is especially useful for the usual boilerplate that would go into a python file like

  #!/usr/bin/env python

  and

  if __name__ == '__main__':

  Granted, Python doesn't require much boilerplate, and therefore this package is much more suited to languages like Java, but I bring it up because I think its cool and if you get into the habit of using it, then a few keystrokes saved here and there will add up over time.

• Subversion support with psvn.el Psvn is a comprehensive subversion client for Emacs. It integrates well with ediff mode so you can use it to check changes between versions. It does all of the other boring subversion stuff well too.
• Ido-mode for buffer switching and file opening. Emacs, to the uninitiated, can be confusing because by default there is only one view into a single file at a time. How does one get to another file? Instead of cluttering the interface with GUIness and making the user click somewhere (and thereby forcing the user to waste their time by moving their hand off of the keyboard), Emacs gives powerful ways to switch between files. Ido-mode is one of these useful ways -- it makes a list of open files starting with the most frequently visted files and widdles this list down as you type part of the filename. You can have dozens of files open and only be a few keystrokes away from any one of them.

A lot of people, for whatever reason, don't feel that Emacs is an IDE at all. I don't personally care what you define it as -- the fact remains -- Emacs is a powerful Python environment and despite being over 32 years old has proven to be just as modern as any IDE today, and remains THE most configurable editor (operating system?) ever.

I've tarred up my Emacs environment for general consumption. Instructions:

• Install Pymacs
• Install Rope and Ropemacs
• BTW, those three packages should be the only packages other than Emacs you'll need. Everything else is self contained.
• Extract the tarball to your home directory. This creates a directory called ryan-emacs-env.
• Rename "ryan-emacs-env" to ".emacs.d"
• Symlink my dot-emacs file to your .emacs. Run "ln -s .emacs.d/dot-emacs .emacs".
• If you also want to do Java development run "tar xfvz jde-2.3.5.1.tar.gz". I leave it tarred because you don't need to pollute your environment if you're not going to use Java. (Also for whatever reason, jde doesn't like to be stuck in my subversion repository so I just leave it tarred up and untar on every machine I check it out on.)

Extra tips:

• Put your .emacs.d directory under version control. Never rely on your distros emacs packages, install all future elisp files yourself in your .emacs.d file and commit to your repository often. This way you've got an environment that is easily transportable and synchronizable across multiple machines. This is the major reason why my emacs environment was so fast to trasnfer from my macbook pro to sontek's laptop during the demo.
• Speaking of sontek, he brought up an excellent point in #utahpython the other day, he's not going to be using my emacs environment except for reference, instead he's starting with a clean slate. This is by far the best and most prudent thing to do. My emacs environment is a culmination of several years of plugging in and deleting various packages and writing various snippets of elisp. Your needs are always going to be different than mine and you are also going to be better off by educating yourself along the way by creating your own.

Some more fun Emacs evangelism:

• a fascinating screencast of Emacs in the role of Ruby editor (many tips here apply to Python too)
• Emacs blogs I read:
  • Emacs life
  • Life is too short for bad code
  • Minor Emacs Wizardry
  • M-x all-things-emacs
  • Platypope.org
  • Know of any other good ones? Let me know.