I got tired a long time ago of losing what I was currently working on in SSH sessions when they were lost due to disconnects from network connectivity issues. To combat this I have been using screen when running sessions that I can absolutely not lose, but the problem still persists in other sessions or when I forgot to run it. The easy solution to this was to add screen to one of my bash init scripts (~/.bashrc [or ~/.bash_profile]) as follows:

alias autoscreen="screen -x -RR && exit"
if [[ "$TERM" == cygwin* || "$TERM" == xterm* ]]; then autoscreen; fi

if [ $TERM == "xterm" ]; then screen -R pts-0.`hostname` && exit; fi

• The main screen command is now an “alias” to help out with some bash parsing problems.
• The resume parameters are now “-x -RR” which first attempts to multiplex a session, and if that fails, it creates a session. With multiplexing turned on, everyone uses the same screen session and can interact with each other, and you don’t have to worry about accidentally connecting to the wrong screen session or having new ones turned on. The only problem is sometimes you may accidentally step on other user’s toes :-)
• The special screen session name was removed so it always starts with the default name (easier to manually interact with)
• I added detection of multiple term names (cygwin and xterm), and added a wildcard at the end of each since there is often suffixes to these names. More term names can easily be added using this syntax.

Edit on 2010-12-30 @ 3:50am: I changed != "screen" to == "xterm" because otherwise scp and some other non-term programs were failing. You may have to use something other than “xterm” for your systems.

Edit on 2010-1-24 @ 2:00pm: I added the exit; so the terminal automatically exits when the screen session closes.

For a number of years now when on insecure network connections I have been routing my computer to the Internet through secure tunnels and VPNs, but I’ve been interested in trying out different types of VPN software lately so I can more easily help secure friends who ask of it. This would mainly include ease of installation and enabling, which partly requires no extra software for them to install.

Unfortunately, Windows 7 and Android (and probably most other software) only support PPTP and L2TP/IPSEC out of the box. While these protocols are good for what they do, everything I have read says OpenVPN is superior to them as a protocol. I was very frustrated to find out how little support OpenVPN actually has today as a standard in the industry, which is to say, you have to use third party clients and it is rarely, if ever, included by default in OSes. The OpenVPN client and server aren’t exactly the easiest to set up either for novices.

So on to the real point of this post. The sample client and server configurations for OpenVPN were set up just how I needed them except they did not include two important options for me: User authentication and full client Internet forwarding/tunneling/gateway routing. Here is how to enable both.

• Add the following options to the server configuration file:
  • push "redirect-gateway def1" #Tells the client to use the server as its default gateway
  • push "dhcp-option DNS 10.8.0.1" #Tells the client to use the server as its DNS Server (DNS Server's IP address dependent on configuration)
• Run the following commands in bash:
  • iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE #This command assumes that the VPN subnet is 10.8.0.0/24 (taken from the server directive in the OpenVPN server configuration) and that the local ethernet interface is eth0.
  • echo '1' > /proc/sys/net/ipv4/ip_forward #Enable IP Forwarding (This one is not mentioned in the OpenVPN howto)

To set up username/password authentication on the server, an authorization script is needed that receives the username/password and returns whether the login information was successful (0) or failed (1). The steps to set up this process are as follows:

• Add the following options to the server configuration file:
  • auth-user-pass-verify verify.php via-env #The third argument (method) specifies whether to send the username and password through either a temporary file (via-file) or environment variables (via-env)
  • script-security 3 system #Allows OpenVPN to run user scripts and executables and send password authentication information through environment variables. While "system" is deprecated, I had to use it or external commands like ifconfig and route were failing with "failed: could not execute external program"
• Add the following options to the client configuration file:
  • auth-user-pass #Request user credentials to log in
• The final step is to create the verify.php (see auth-user-pass-verify configuration above) script which returns whether it was successful, and also outputs its success to stdout, which is added to the OpenVPN log file.

  #!/usr/bin/php -q
  <?
  //Configuration
  $ValidUserFile='users.txt'; //This file must be in htpasswd SHA1 format (htpasswd -s)
  $Method='via-env'; //via-file or via-env (see auth-user-pass-verify configuration above for more information)
  
  //Get the login info
  if($Method=='via-file') //via-file method
  {
  	$LoginInfoFile=trim(file_get_contents('php://stdin')); //Get the file that contains the passed login info from stdin
  	$LoginInfo=file_get_contents($LoginInfoFile); //Get the passed login info
  	file_put_contents($LoginInfoFile, str_repeat('x', strlen($LoginInfo))); //Shred the login info file
  	$LoginInfo=explode("\n", $LoginInfo); //Split into [Username, Password]
  	$UserName=$LoginInfo[0];
  	$Password=$LoginInfo[1];
  }
  else //via-env method
  {
  	$UserName=$_ENV['username'];
  	$Password=$_ENV['password'];
  }
  
  //Test the login info against the valid user file
  $UserLine="$UserName:{SHA}".base64_encode(sha1($Password, TRUE)); //Compile what the user line should look like
  foreach(file($ValidUserFile, FILE_IGNORE_NEW_LINES) as $Line) //Attempt to match against each line in the file
  	if($UserLine==$Line) //If credentials match, return success
  	{
  		print "Logged in: $UserName\n";
  		exit(0);
  	}
  
  //Return failure
  print "NOT Logged in: $UserName\n";
  exit(1);
  ?>
  		

The following is a Python script that automatically pings a requested web address at a given interval. It was made as a quick favor for a friend. Here is the downloadable source code and Windows binary.

1. The URL to ping (The default pings the GetIP script)
2. The millisecond interval between pings (Default=600000=10 minutes)

#!python
from sys import stderr
from urllib import urlopen
from time import localtime, strftime, sleep

#Main function
def Main():
    #Open the settings file
    SettingFileName='PingURL.cfg';
    Settings=[] #Blank out settings file variable in case it can't be opened
    try:
        Settings=open(SettingFileName, 'r').readlines()
    except IOError as (errno, strerror):
        stderr.write('Cannot open {0} configuration file: I/O error({1}): {2}\n'.format(SettingFileName, errno, strerror))
        return

    #Confirm valid settings were passed
    if(len(Settings)<2):
        stderr.write('Not enough settings found in settings file\n')
        return
    try:
        IntervalTime=int(Settings[1])
    except:
        stderr.write('Invalid interval time\n')
        return
        
    #Ping the URL indefinitely
    while(True):
        try:
            URLText=urlopen(Settings[0]).read()
        except:
            URLText='READ FAILED'
        print 'URL Pinged At {0}: {1}'.format(strftime('%Y-%m-%d %H:%M:%S', localtime()), URLText)
        sleep(IntervalTime/1000)

#Run the program
Main()

I’ve recently been having problems using the Google Reader widget in iGoogle. Normally, when I clicked on an RSS Title, a “bubble” popped up with the post’s content. However recently when clicking on the titles, the original post’s source opened up in a new tab. I confirmed the settings for the widget were correct, so I tried to remember the last change I made in Firefox that could have triggered this problem, as it seems the problem was not widespread, and only occurred to a few other people with no solution found. I realized a little bit back that I had installed the HTTPS Everywhere Firefox plugin. As described on the EFF’s site “HTTPS Everywhere is a Firefox extension ... [that] encrypts your communications with a number of major websites”.

Once I disabled the plugin and found the problem went away, I started digging through Google’s JavaScript code with FireBug. It turns out the start of the problem was that the widgets in iGoogle are run in their own IFrames (which is a very secure way of doing a widget system like this). However, the Google Reader contents was being pulled in through HTTPS secure channels (as it should thanks to HTTPS Everywhere), while the iGoogle page itself was pulled in through a normal HTTP channel! Separate windows/frames/tabs cannot interact with each other through JavaScript if they are not part of the same domain and protocol (HTTP/HTTPS) to prevent Cross-site scripting hacks.

I was wondering why HTTPS Everywhere was not running iGoogle through an HTTPS channel, so I tried it myself and found out Google automatically redirects HTTPS iGoogle requests to non secure HTTP channels! So much for having a proper security model in place...

So I did a lot more digging and modifying of Google’s code to see if I couldn’t find out exactly where the problem was occurring and if it couldn’t be fixed with a hack. It seems the code to handle the RSS Title clicking is injected during the “onload” event of the widget’s IFrame. I believe this was the code that was hitting the security privilege error to make things not work. I attempted to hijack the Google Reader widget’s onload function and add special privileges using “netscape.security.PrivilegeManager.enablePrivilege”, but it didn’t seem to help the problem. I think with some more prodding I could have gotten it working, but I didn’t want to waste any more time than I already had on the problem.

The code that would normally be loaded into the widget’s IFrame window hooks the “onclick” event of all RSS Title links to both perform the bubble action and cancel the normal “click” action. Since the normal click action for the anchor links was not being canceled, the browser action of following the link occurred. In this case, the links also had a “target” set to open a new window/tab.

There is however a “fix” for this problem, though I don’t find it ideal. If you edit the “extensions\https-everywhere@eff.org\chrome\content\rules\GoogleServices.xml” file in your Firefox profile directory (most likely at “C:\Users\USERNAME\AppData\Roaming\Mozilla\Firefox\Profiles\PROFILENAME\” if running Windows 7), you can comment out or delete the following rule so Google Reader is no longer run through secure HTTPS channels:

<rule from="^http://(www\.)?google\.com/reader/" 
to="https://www.google.com/reader/"/>

That being said, I’ve been having a plethora of problems with Facebook and HTTPS Everywhere too :-\ (which it actually mentions might happen in its options dialog). You’d think the largest sites on the Internet could figure out how to get their security right, but either they don’t care (the more likely option), or they don’t want the encryption overhead. Alas.

In March of 2007 I was doing research on Mei and the Kittenbus (めいとこねこバス, Mei to Konekobasu), an animated short “sequel” to My Neighbor Totoro by Hayao Miyazaki of Studio Ghibli. It was “released” in 2003 for regular showing at the Ghibli Museum in Japan, and has unfortunately never seen a wide release. I believe the Ghibli museum shows different animated shorts on a random basis for their tours.

While researching, I was only able to find three sites on the whole internet at that time that had any real information on it. I decided to backup their contents for mirroring, which I am finally getting around to putting up :-). This information is directly copied from those sites and I do not claim to be the original source of, or in any way own, any of the below content in this post.

моцартинка (mozartianka) wrote in miyazaki_ru, @ 2007-01-28 21:44:00 Entry tags: Mei to Konekobasu/Мэй и Котобусёнок, Studio Ghibli Shorts/Короткометражки, Tonari no Totoro/Наш сосед Тоторо, картинки Mei to Konekobasu Album Полный альбом про Мей и Котобусёнка! обложка Громадное спасибо nazarovsky!

Mei and the Kittenbus     This is a short film about 20 minutes long, which was shown for a limited time only at the Ghibli Museum. It tells the story of Mei and her friend the kittenbus (the child of the original catbus from My Neighbor Totoro). Mei is just small enough to ride in the kittenbus, which is only big enough to stir up dust devils, rather than making whole fields of rice sway in its breeze. One night they have an adventure in which they fly into the forest with many other cat-based vehicles, including many buses and trains. There they meet Totoro and many similar spirits, all heading for a gigantic catliner. This liner cruises off into the sky and the kittenbus takes Mei home. Small Version Large Version Small Version | Large Version Small Version | Large Version Small Version | Large Version Small Version | Large Version Original story, screenplay and direction: Hayao Miyazaki Music: Joe Hisaishi Animation Directors: Maikiko Futaki, Sachiko Sugino, and Hiromasa Yonebayashi Length: 13 minutes 43 seconds Pictures and synopsis by Chris Kuan

This is definitely one of the things on my must-do list for when I visit Japan one day ^_^.

I often find SSH tunnels absolutely indispensable in my line of work for multiple reasons including secure proxies (tunneling) over insecure connections and connecting computers and programs together over difficult network setups involving NATs and firewalls.

One such example of this I ran into recently is that I have a server machine (hereby called “the client”) that I wanted to make sure I have access to no matter where it is. For this I created an auto initializing SSH remote port tunnel to a server with a static IP Address (hereby called “the proxy server”) which attempts to keep itself open when there is problems.

The first step of this was to create the following bash script on the client that utilizes the OpenSSH’s client to connect to an OpenSSH server on the proxy server for tunneling:

#!/bin/bash
for ((;;)) #Infinite loop to keep the tunnel open
do
	ssh USERNAME@PROXYSERVER -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes -o ServerAliveCountMax=2 -o ServerAliveInterval=10 -N -R PROXYPORT:localhost:22 &>> TUNNEL_LOG #Create the SSH tunnel
	echo "Restarting: " `date` >> TUNNEL_LOG #Write to the log file "TUNNEL_LOG" whenever a restart is attempted
	sleep 1 #Wait 1 second in between connection attempts
done

For this to work, you will need to set up public key authentication between the client and utilized user on the proxy server. To do this, you have to run “ssh-keygen” on the client. When it has finished, you must copy the contents of your public key file (most likely at “~/.ssh/id_rsa.pub”) to the “~/.ssh/authorized_keys” file on the user account you are logging into on the proxy server. You will also have to log into this account at least once from the client so the proxy server’s information is in the client’s “known_hosts” file.

For security reasons you may want the user on the proxy server to have a nologin shell set since it is only being used for tunneling, and the above mentioned public key will allow access without a password.

For network access reasons, it might also be worth it to have the proxy port and the ssh port on the proxy server set to commonly accessible ports on all network setups (that a firewall wouldn’t block). Or you may NOT want to have it on common ports for other security reasons :-).

If you want the proxy port on the proxy server accessible from other computers (not only the proxy server), you will have to turn on “GatewayPorts” (set to “yes”) in the proxy server’s sshd config, most likely located at “/etc/ssh/sshd_config”.

The next step is to create a daemon that calls this script. The normal method for this in Linux would be to use inittab. This can be a bit cumbersome though with Linux distributions that use upstart, like Ubuntu, so I just cheated in it and created the following script to initialize a daemon through rc.d:

#!/bin/bash
echo -e '#!/bin/bash\nfor ((;;))\ndo\n  ssh USERNAME@PROXYSERVER -o TCPKeepAlive=yes -o ExitOnForwardFailure=yes -o ServerAliveCountMax=2 -o ServerAliveInterval=10 -N -R PROXYPORT:localhost:22 &>> TUNNEL_LOG\n  echo "Restarting: " `date` >> TUNNEL_LOG\n  sleep 1\ndone' > TUNNEL_SCRIPT_PATH #This creates the above script
echo -e '#!/bin/bash\ncd TUNNEL_SCRIPT_DIRECTORY\n./TUNNEL_SCRIPT_EXECUTABLE &' > /etc/init.d/TUNNEL_SCRIPT_SERVICE_NAME #This creates the init.d daemon script. I have the script set the working path before running the executable so the log file stays in the same directory
chmod u+x TUNNEL_SCRIPT_PATH /etc/init.d/TUNNEL_SCRIPT_SERVICE_NAME #Set all scripts as executable
update-rc.d TUNNEL_SCRIPT_SERVICE_NAME defaults #Set the run levels at which the script runs

I have been having some problems regarding symlinks (symbolic links) for a project that I’ve been working on recently which is requiring work in at least 5 very different operating systems (and about a dozen programming languages). Not many programs support symlinks properly that I have the need to because support for it wasn’t added for NTFS until Windows Vista, and it still has some problems.

It is really great that Windows Vista and Windows 7 now support native symlinks so they can be utilized by programs out of the box. For example, one such instance where I need this a lot is directory relinking in Apache. While Apache’s mod_alias can duplicate the functionality of symlinks for many needs, creating special cases for this one piece of software when distributing a code repository is just not practical, and having proper symlinks natively followed without the program knowing they aren’t the actual file/directory is really the best solution so everything works without special cases.

The way to create NTFS symlinks in Windows Vista+ is through the “mklink” command, which is unfortunately implemented directly in the Window’s command shell, and not a separate executable, so it is not accessible to Cygwin. Further, Cygwin has made a stance to only support reading NTFS symlinks, and not creating them, because they can only be created by administrators, and require specification as to whether the link’s target is a directory or file. Cygwin itself in Windows has had support for symlinks for a long time, but these are not compatible with any program run outside of the Cygwin environment.

Now, my real problem started occurring when trying to use these NTFS symlinks with GIT. While GIT natively supports symlinks, TortoiseGIT doesn’t really support them at all, and throws errors when they are encountered. This is still a big problem that I am going to have to think about :-\. Fortunately, when working with GIT in Cygwin they still work, with caveats. As previously mentioned, only reading the NTFS symlinks in Cygwin work, so when you fetch/pull from a repository and it creates Cygwin style symlinks, Windows still does not read them properly. The following is a script I wrote to change the Cygwin style symlinks into NTFS style symlinks. It can be run from the root folder of the GIT project.

#!/bin/bash
IFS=$'\n' #Spaces do not count as new delimiters

function makewinlink
{
	LINK=$1
	OPTIONS=$2
	TARGET=`find $LINK -maxdepth 0 -printf %l`
	LASTMODTIME=`find $LINK -maxdepth 0 -printf "%t"`
	LINKDIR=`find $LINK -maxdepth 0 -printf %h`
	TARGET=`echo $LINKDIR/$TARGET`
	rm -f $LINK
	cmd /c mklink $OPTIONS "$(cygpath -wa $LINK)" "$(cygpath -wa $TARGET)"
	touch -h -d $LASTMODTIME $LINK
}

#Relink all directories
FILES=`find -type l -print0 | xargs -0 -i find -L {} -type d -maxdepth 0`
for f in $FILES
do
	makewinlink $f /D
done

#Relink all files
FILES=`find -type l -print0 | xargs -0 -i find -L {} -type f -maxdepth 0`
for f in $FILES
do
	makewinlink $f
done

Make sure when committing symlinks in a GIT repository in Windows to use Cygwin with Cygwin style symlinks instead of TortoiseGIT. Also, as previously mentioned, after running this script, TortoiseGIT will show these symlinks as modified :-\. If this is a problem, you can always reverse the process in Cygwin by changing the “cmd /c mklink $OPTIONS” line to a “ln -s” in the above script (note that “target” and “symlink’s name” need to be switched) along with a few other changes.

As previously mentioned, I had been trying to do an update every day on my site last month. Unfortunately, unforeseen circumstances revolving around work and Thanksgiving family obligations for almost an entire week made this impossible :-\. So I’ll try to make up the rest of the posts this month :-). At least I got in 23 in a row before I was overwhelmed.