I know you asked for a simple IP hack, but failing that here's a not-so-simple one using mod_rewrite.
Code:
RewriteEngine On
RewriteCond %{REMOTE_ADDR} !^12\.34\.56\.78$
RewriteRule /$|^$ - [F,L]
With this, you leave Indexes turned on and the rewriter here looks to see if the requesting IP address is
not yours (assuming your IP is 12.34.56.78) and if that is the case, forbids access to urls ending in slash or empty (relative to the directory the .htaccess file is in), in other words where directory listings would be.