I got tired a long time ago of losing what I was currently working on in SSH sessions when they were lost due to disconnects from network connectivity issues. To combat this I have been using screen when running sessions that I can absolutely not lose, but the problem still persists in other sessions or when I forgot to run it. The easy solution to this was to add screen to one of my bash init scripts (~/.bashrc [or ~/.bash_profile]) as follows:

alias autoscreen="screen -x -RR && exit"
if [[ "$TERM" == cygwin* || "$TERM" == xterm* ]]; then autoscreen; fi

if [ $TERM == "xterm" ]; then screen -R pts-0.`hostname` && exit; fi

• The main screen command is now an “alias” to help out with some bash parsing problems.
• The resume parameters are now “-x -RR” which first attempts to multiplex a session, and if that fails, it creates a session. With multiplexing turned on, everyone uses the same screen session and can interact with each other, and you don’t have to worry about accidentally connecting to the wrong screen session or having new ones turned on. The only problem is sometimes you may accidentally step on other user’s toes :-)
• The special screen session name was removed so it always starts with the default name (easier to manually interact with)
• I added detection of multiple term names (cygwin and xterm), and added a wildcard at the end of each since there is often suffixes to these names. More term names can easily be added using this syntax.

Edit on 2010-12-30 @ 3:50am: I changed != "screen" to == "xterm" because otherwise scp and some other non-term programs were failing. You may have to use something other than “xterm” for your systems.

Edit on 2010-1-24 @ 2:00pm: I added the exit; so the terminal automatically exits when the screen session closes.

For a number of years now when on insecure network connections I have been routing my computer to the Internet through secure tunnels and VPNs, but I’ve been interested in trying out different types of VPN software lately so I can more easily help secure friends who ask of it. This would mainly include ease of installation and enabling, which partly requires no extra software for them to install.

Unfortunately, Windows 7 and Android (and probably most other software) only support PPTP and L2TP/IPSEC out of the box. While these protocols are good for what they do, everything I have read says OpenVPN is superior to them as a protocol. I was very frustrated to find out how little support OpenVPN actually has today as a standard in the industry, which is to say, you have to use third party clients and it is rarely, if ever, included by default in OSes. The OpenVPN client and server aren’t exactly the easiest to set up either for novices.

So on to the real point of this post. The sample client and server configurations for OpenVPN were set up just how I needed them except they did not include two important options for me: User authentication and full client Internet forwarding/tunneling/gateway routing. Here is how to enable both.

• Add the following options to the server configuration file:
  • push "redirect-gateway def1" #Tells the client to use the server as its default gateway
  • push "dhcp-option DNS 10.8.0.1" #Tells the client to use the server as its DNS Server (DNS Server's IP address dependent on configuration)
• Run the following commands in bash:
  • iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE #This command assumes that the VPN subnet is 10.8.0.0/24 (taken from the server directive in the OpenVPN server configuration) and that the local ethernet interface is eth0.
  • echo '1' > /proc/sys/net/ipv4/ip_forward #Enable IP Forwarding (This one is not mentioned in the OpenVPN howto)

To set up username/password authentication on the server, an authorization script is needed that receives the username/password and returns whether the login information was successful (0) or failed (1). The steps to set up this process are as follows:

• Add the following options to the server configuration file:
  • auth-user-pass-verify verify.php via-env #The third argument (method) specifies whether to send the username and password through either a temporary file (via-file) or environment variables (via-env)
  • script-security 3 system #Allows OpenVPN to run user scripts and executables and send password authentication information through environment variables. While "system" is deprecated, I had to use it or external commands like ifconfig and route were failing with "failed: could not execute external program"
• Add the following options to the client configuration file:
  • auth-user-pass #Request user credentials to log in
• The final step is to create the verify.php (see auth-user-pass-verify configuration above) script which returns whether it was successful, and also outputs its success to stdout, which is added to the OpenVPN log file.

  #!/usr/bin/php -q
  <?
  //Configuration
  $ValidUserFile='users.txt'; //This file must be in htpasswd SHA1 format (htpasswd -s)
  $Method='via-env'; //via-file or via-env (see auth-user-pass-verify configuration above for more information)
  
  //Get the login info
  if($Method=='via-file') //via-file method
  {
  	$LoginInfoFile=trim(file_get_contents('php://stdin')); //Get the file that contains the passed login info from stdin
  	$LoginInfo=file_get_contents($LoginInfoFile); //Get the passed login info
  	file_put_contents($LoginInfoFile, str_repeat('x', strlen($LoginInfo))); //Shred the login info file
  	$LoginInfo=explode("\n", $LoginInfo); //Split into [Username, Password]
  	$UserName=$LoginInfo[0];
  	$Password=$LoginInfo[1];
  }
  else //via-env method
  {
  	$UserName=$_ENV['username'];
  	$Password=$_ENV['password'];
  }
  
  //Test the login info against the valid user file
  $UserLine="$UserName:{SHA}".base64_encode(sha1($Password, TRUE)); //Compile what the user line should look like
  foreach(file($ValidUserFile, FILE_IGNORE_NEW_LINES) as $Line) //Attempt to match against each line in the file
  	if($UserLine==$Line) //If credentials match, return success
  	{
  		print "Logged in: $UserName\n";
  		exit(0);
  	}
  
  //Return failure
  print "NOT Logged in: $UserName\n";
  exit(1);
  ?>
  		

The following is a Python script that automatically pings a requested web address at a given interval. It was made as a quick favor for a friend. Here is the downloadable source code and Windows binary.

1. The URL to ping (The default pings the GetIP script)
2. The millisecond interval between pings (Default=600000=10 minutes)

#!python
from sys import stderr
from urllib import urlopen
from time import localtime, strftime, sleep

#Main function
def Main():
    #Open the settings file
    SettingFileName='PingURL.cfg';
    Settings=[] #Blank out settings file variable in case it can't be opened
    try:
        Settings=open(SettingFileName, 'r').readlines()
    except IOError as (errno, strerror):
        stderr.write('Cannot open {0} configuration file: I/O error({1}): {2}\n'.format(SettingFileName, errno, strerror))
        return

    #Confirm valid settings were passed
    if(len(Settings)<2):
        stderr.write('Not enough settings found in settings file\n')
        return
    try:
        IntervalTime=int(Settings[1])
    except:
        stderr.write('Invalid interval time\n')
        return
        
    #Ping the URL indefinitely
    while(True):
        try:
            URLText=urlopen(Settings[0]).read()
        except:
            URLText='READ FAILED'
        print 'URL Pinged At {0}: {1}'.format(strftime('%Y-%m-%d %H:%M:%S', localtime()), URLText)
        sleep(IntervalTime/1000)

#Run the program
Main()

I’ve recently been having problems using the Google Reader widget in iGoogle. Normally, when I clicked on an RSS Title, a “bubble” popped up with the post’s content. However recently when clicking on the titles, the original post’s source opened up in a new tab. I confirmed the settings for the widget were correct, so I tried to remember the last change I made in Firefox that could have triggered this problem, as it seems the problem was not widespread, and only occurred to a few other people with no solution found. I realized a little bit back that I had installed the HTTPS Everywhere Firefox plugin. As described on the EFF’s site “HTTPS Everywhere is a Firefox extension ... [that] encrypts your communications with a number of major websites”.

Once I disabled the plugin and found the problem went away, I started digging through Google’s JavaScript code with FireBug. It turns out the start of the problem was that the widgets in iGoogle are run in their own IFrames (which is a very secure way of doing a widget system like this). However, the Google Reader contents was being pulled in through HTTPS secure channels (as it should thanks to HTTPS Everywhere), while the iGoogle page itself was pulled in through a normal HTTP channel! Separate windows/frames/tabs cannot interact with each other through JavaScript if they are not part of the same domain and protocol (HTTP/HTTPS) to prevent Cross-site scripting hacks.

I was wondering why HTTPS Everywhere was not running iGoogle through an HTTPS channel, so I tried it myself and found out Google automatically redirects HTTPS iGoogle requests to non secure HTTP channels! So much for having a proper security model in place...

So I did a lot more digging and modifying of Google’s code to see if I couldn’t find out exactly where the problem was occurring and if it couldn’t be fixed with a hack. It seems the code to handle the RSS Title clicking is injected during the “onload” event of the widget’s IFrame. I believe this was the code that was hitting the security privilege error to make things not work. I attempted to hijack the Google Reader widget’s onload function and add special privileges using “netscape.security.PrivilegeManager.enablePrivilege”, but it didn’t seem to help the problem. I think with some more prodding I could have gotten it working, but I didn’t want to waste any more time than I already had on the problem.

The code that would normally be loaded into the widget’s IFrame window hooks the “onclick” event of all RSS Title links to both perform the bubble action and cancel the normal “click” action. Since the normal click action for the anchor links was not being canceled, the browser action of following the link occurred. In this case, the links also had a “target” set to open a new window/tab.

There is however a “fix” for this problem, though I don’t find it ideal. If you edit the “extensions\https-everywhere@eff.org\chrome\content\rules\GoogleServices.xml” file in your Firefox profile directory (most likely at “C:\Users\USERNAME\AppData\Roaming\Mozilla\Firefox\Profiles\PROFILENAME\” if running Windows 7), you can comment out or delete the following rule so Google Reader is no longer run through secure HTTPS channels:

<rule from="^http://(www\.)?google\.com/reader/" 
to="https://www.google.com/reader/"/>

That being said, I’ve been having a plethora of problems with Facebook and HTTPS Everywhere too :-\ (which it actually mentions might happen in its options dialog). You’d think the largest sites on the Internet could figure out how to get their security right, but either they don’t care (the more likely option), or they don’t want the encryption overhead. Alas.

In March of 2007 I was doing research on Mei and the Kittenbus (めいとこねこバス, Mei to Konekobasu), an animated short “sequel” to My Neighbor Totoro by Hayao Miyazaki of Studio Ghibli. It was “released” in 2003 for regular showing at the Ghibli Museum in Japan, and has unfortunately never seen a wide release. I believe the Ghibli museum shows different animated shorts on a random basis for their tours.

While researching, I was only able to find three sites on the whole internet at that time that had any real information on it. I decided to backup their contents for mirroring, which I am finally getting around to putting up :-). This information is directly copied from those sites and I do not claim to be the original source of, or in any way own, any of the below content in this post.

моцартинка (mozartianka) wrote in miyazaki_ru, @ 2007-01-28 21:44:00 Entry tags: Mei to Konekobasu/Мэй и Котобусёнок, Studio Ghibli Shorts/Короткометражки, Tonari no Totoro/Наш сосед Тоторо, картинки Mei to Konekobasu Album Полный альбом про Мей и Котобусёнка! обложка Громадное спасибо nazarovsky!

Mei and the Kittenbus     This is a short film about 20 minutes long, which was shown for a limited time only at the Ghibli Museum. It tells the story of Mei and her friend the kittenbus (the child of the original catbus from My Neighbor Totoro). Mei is just small enough to ride in the kittenbus, which is only big enough to stir up dust devils, rather than making whole fields of rice sway in its breeze. One night they have an adventure in which they fly into the forest with many other cat-based vehicles, including many buses and trains. There they meet Totoro and many similar spirits, all heading for a gigantic catliner. This liner cruises off into the sky and the kittenbus takes Mei home. Small Version Large Version Small Version | Large Version Small Version | Large Version Small Version | Large Version Small Version | Large Version Original story, screenplay and direction: Hayao Miyazaki Music: Joe Hisaishi Animation Directors: Maikiko Futaki, Sachiko Sugino, and Hiromasa Yonebayashi Length: 13 minutes 43 seconds Pictures and synopsis by Chris Kuan

This is definitely one of the things on my must-do list for when I visit Japan one day ^_^.

I often find SSH tunnels absolutely indispensable in my line of work for multiple reasons including secure proxies (tunneling) over insecure connections and connecting computers and programs together over difficult network setups involving NATs and firewalls.

One such example of this I ran into recently is that I have a server machine (hereby called “the client”) that I wanted to make sure I have access to no matter where it is. For this I created an auto initializing SSH remote port tunnel to a server with a static IP Address (hereby called “the proxy server”) which attempts to keep itself open when there is problems.

The first step of this was to create the following bash script on the client that utilizes the OpenSSH’s client to connect to an OpenSSH server on the proxy server for tunneling:

#!/bin/bash
for ((;;)) #Infinite loop to keep the tunnel open
do
	ssh USERNAME@PROXYSERVER -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes -o ServerAliveCountMax=2 -o ServerAliveInterval=10 -N -R PROXYPORT:localhost:22 &>> TUNNEL_LOG #Create the SSH tunnel
	echo "Restarting: " `date` >> TUNNEL_LOG #Write to the log file "TUNNEL_LOG" whenever a restart is attempted
	sleep 1 #Wait 1 second in between connection attempts
done

For this to work, you will need to set up public key authentication between the client and utilized user on the proxy server. To do this, you have to run “ssh-keygen” on the client. When it has finished, you must copy the contents of your public key file (most likely at “~/.ssh/id_rsa.pub”) to the “~/.ssh/authorized_keys” file on the user account you are logging into on the proxy server. You will also have to log into this account at least once from the client so the proxy server’s information is in the client’s “known_hosts” file.

For security reasons you may want the user on the proxy server to have a nologin shell set since it is only being used for tunneling, and the above mentioned public key will allow access without a password.

For network access reasons, it might also be worth it to have the proxy port and the ssh port on the proxy server set to commonly accessible ports on all network setups (that a firewall wouldn’t block). Or you may NOT want to have it on common ports for other security reasons :-).

If you want the proxy port on the proxy server accessible from other computers (not only the proxy server), you will have to turn on “GatewayPorts” (set to “yes”) in the proxy server’s sshd config, most likely located at “/etc/ssh/sshd_config”.

The next step is to create a daemon that calls this script. The normal method for this in Linux would be to use inittab. This can be a bit cumbersome though with Linux distributions that use upstart, like Ubuntu, so I just cheated in it and created the following script to initialize a daemon through rc.d:

#!/bin/bash
echo -e '#!/bin/bash\nfor ((;;))\ndo\n  ssh USERNAME@PROXYSERVER -o TCPKeepAlive=yes -o ExitOnForwardFailure=yes -o ServerAliveCountMax=2 -o ServerAliveInterval=10 -N -R PROXYPORT:localhost:22 &>> TUNNEL_LOG\n  echo "Restarting: " `date` >> TUNNEL_LOG\n  sleep 1\ndone' > TUNNEL_SCRIPT_PATH #This creates the above script
echo -e '#!/bin/bash\ncd TUNNEL_SCRIPT_DIRECTORY\n./TUNNEL_SCRIPT_EXECUTABLE &' > /etc/init.d/TUNNEL_SCRIPT_SERVICE_NAME #This creates the init.d daemon script. I have the script set the working path before running the executable so the log file stays in the same directory
chmod u+x TUNNEL_SCRIPT_PATH /etc/init.d/TUNNEL_SCRIPT_SERVICE_NAME #Set all scripts as executable
update-rc.d TUNNEL_SCRIPT_SERVICE_NAME defaults #Set the run levels at which the script runs

I have been having some problems regarding symlinks (symbolic links) for a project that I’ve been working on recently which is requiring work in at least 5 very different operating systems (and about a dozen programming languages). Not many programs support symlinks properly that I have the need to because support for it wasn’t added for NTFS until Windows Vista, and it still has some problems.

It is really great that Windows Vista and Windows 7 now support native symlinks so they can be utilized by programs out of the box. For example, one such instance where I need this a lot is directory relinking in Apache. While Apache’s mod_alias can duplicate the functionality of symlinks for many needs, creating special cases for this one piece of software when distributing a code repository is just not practical, and having proper symlinks natively followed without the program knowing they aren’t the actual file/directory is really the best solution so everything works without special cases.

The way to create NTFS symlinks in Windows Vista+ is through the “mklink” command, which is unfortunately implemented directly in the Window’s command shell, and not a separate executable, so it is not accessible to Cygwin. Further, Cygwin has made a stance to only support reading NTFS symlinks, and not creating them, because they can only be created by administrators, and require specification as to whether the link’s target is a directory or file. Cygwin itself in Windows has had support for symlinks for a long time, but these are not compatible with any program run outside of the Cygwin environment.

Now, my real problem started occurring when trying to use these NTFS symlinks with GIT. While GIT natively supports symlinks, TortoiseGIT doesn’t really support them at all, and throws errors when they are encountered. This is still a big problem that I am going to have to think about :-\. Fortunately, when working with GIT in Cygwin they still work, with caveats. As previously mentioned, only reading the NTFS symlinks in Cygwin work, so when you fetch/pull from a repository and it creates Cygwin style symlinks, Windows still does not read them properly. The following is a script I wrote to change the Cygwin style symlinks into NTFS style symlinks. It can be run from the root folder of the GIT project.

#!/bin/bash
IFS=$'\n' #Spaces do not count as new delimiters

function makewinlink
{
	LINK=$1
	OPTIONS=$2
	TARGET=`find $LINK -maxdepth 0 -printf %l`
	LASTMODTIME=`find $LINK -maxdepth 0 -printf "%t"`
	LINKDIR=`find $LINK -maxdepth 0 -printf %h`
	TARGET=`echo $LINKDIR/$TARGET`
	rm -f $LINK
	cmd /c mklink $OPTIONS "$(cygpath -wa $LINK)" "$(cygpath -wa $TARGET)"
	touch -h -d $LASTMODTIME $LINK
}

#Relink all directories
FILES=`find -type l -print0 | xargs -0 -i find -L {} -type d -maxdepth 0`
for f in $FILES
do
	makewinlink $f /D
done

#Relink all files
FILES=`find -type l -print0 | xargs -0 -i find -L {} -type f -maxdepth 0`
for f in $FILES
do
	makewinlink $f
done

Make sure when committing symlinks in a GIT repository in Windows to use Cygwin with Cygwin style symlinks instead of TortoiseGIT. Also, as previously mentioned, after running this script, TortoiseGIT will show these symlinks as modified :-\. If this is a problem, you can always reverse the process in Cygwin by changing the “cmd /c mklink $OPTIONS” line to a “ln -s” in the above script (note that “target” and “symlink’s name” need to be switched) along with a few other changes.

As previously mentioned, I had been trying to do an update every day on my site last month. Unfortunately, unforeseen circumstances revolving around work and Thanksgiving family obligations for almost an entire week made this impossible :-\. So I’ll try to make up the rest of the posts this month :-). At least I got in 23 in a row before I was overwhelmed.

One thing I always really miss when working in other dynamic languages that aren’t JavaScript is the ability to access known (non dynamic) members of an associative array/object/hash (called a hash from here on out) through just a single dot. This matches C’s syntax of accessing struct members, as opposed to being forced into using array syntax which is harder to read IMO in languages like PHP and Perl. For example...

Zones[Info['Zone']]['DateInfo'][Info['Date']]['UniqueEnters']+=Info['Count'];

Zones[Info.Zone].DateInfo[Info.Date].UniqueEnters+=Info.Count;

Here is another compilation list from a few years ago for reasons previously stated. This one is a color coded list of Studio Ghibli films (Hayao Miyazaki being a favorite of mine). A more comprehensive list can probably now be found on Wikipedia, but oh well.

As previously noted, I’ve been doing a lot of work for a project in mostly C# and dynamic languages. I had a weird experience today when moving back to the C++ portion of the project. As I was going through and reading my code to pick back up on the rewrite I was needing to do, I could feel my mind making a sudden paradigm shift in thought and it gave me a half a second dizzy spell. This shift wasn’t about language syntax or structure, as I’m constantly switching between dozens of languages for different projects I’m working on. However, I have very different programming styles and ways to do things between dynamic versus structured languages, and I had been so deep rooted in those other sections of the project in the past few days, my mind just hadn’t caught up to the code I was trying to read yet and had to tell me to hold on for a second while it reorganized so it could properly read the current type of code :-).

As a matter of fact... I think there are at least 11 “languages” I am having to use for this project (for sake of the count I am including scripting languages 0:-) ). C, C++, C#, Perl, PHP, SQL, Flash, JavaScript, HTML, CSS, and Bash scripting.

I was running into a rather nasty .NET crash today in C# for a rather large project that I have been continuing development for on a handheld device that runs Windows CE6. When I was calling a callback function pointer (called a Delegate in .NET land) from a module, I was getting a TypeLoadException error with no further information. I started out making the incorrect assumption that I was doing something wrong with Delegates, as C# is not exactly my primary language ;-). The symptoms were pointing to the delegate call being the problem because the program was crashing during the delegate call itself, as the code reached the call, and did not make it into the callback function. After doing the normal debugging-thing, I found out the program crashed in the same manner every time the specific callback function was called and before it started executing, even if it was called in a normal fashion from the same class.

After further poking around, I realized that there was one line of code in the function that if included in any function, would cause the program to fail out on calling said function. Basically, resources were somehow missing from the compilation and there were no warnings anywhere telling me this. If I tried to access said resource normally, I was getting an easily traceable MissingManifestResourceException error. However, the weird situation was happening because I had the missing resource being accessed from a static member in another class. So here is some example code that was causing the problem:

public class ClassA
{
	public void PlaySuccess()
	{
		//Execution DOES NOT reach here
		Sound.Play(Sound.Success);
	}
}

public class Sound
{
	public static byte[] Success=MyResource.Success; //This resource is somehow missing from the executable
	public static byte[] Failure=MyResource.Failure;
	public static void Play(byte[] TheSound) { sndPlaySound(TheSound, SND_ASYNC|SND_MEMORY); }
}

ClassA Foo=new ClassA();
//Execution reaches here
Foo.PlaySuccess();

Oh well, at least it wasn’t an array overrun, those are fun to track down :-).

I’ve been really annoyed for a while by the unintuitive IDE tab ordering in Visual Studio 2005+. When you type [shift+]alt+tab, you don’t get the next/previous tab in the list as would be the OBVIOUS way to do it (which probably all other IDEs do this right). No, it switches between tabs in an arbitrary hidden order related to the last acces order of the tabs.

Searching the internet for a solution to this was pretty fruitless, so I tried to tackle the problem myself. I dug through all the possible structures I could find in the Visual Studio IDE macro explorer, and was unfortunately unable to find where the tab order was kept in a window pane (if it is even accessible to the user). I thought I had the solution at one point, but realized it also just switches tabs in the order they were originally opened :-(. This is the VB macro code I came up with to do at least that, which uses “DTE.ActiveWindow.Document.Collection” for the tab-open order.

	 Public Sub TabDirection(ByVal Direction As Integer)
		  'Find the index of the current tab
		  Dim i As Integer
		  Dim Index As Integer
		  Dim Count As Integer
		  Count = DTE.ActiveWindow.Document.Collection.Count
		  For i = 1 To Count
				If DTE.ActiveWindow.Document.Collection.Item(i).ActiveWindow.Equals(DTE.ActiveWindow) Then Index = i
		  Next i

		  'Determine the new index
		  Index = Index + Direction
		  If Index > Count Then Index = 1
		  If Index = 0 Then Index = Count

		  DTE.ActiveWindow.Document.Collection.Item(Index).Activate() 'Activate the next tab in the proper direction
	 End Sub

	 Public Sub TabForward()
		  TabDirection(1)
	 End Sub

	 Public Sub TabBackward()
		  TabDirection(-1)
	 End Sub

I got hit yesterday with a very time critical and large client project that will be taking up most all of my time over the next few weeks, so I may have to get in a lot of quick posts like this to keep up. :-\

A number of years ago I watched the new Teenage Mutant Ninja Turtle (2003) cartoon, and was shocked at how good it was. IMO there has been no other non-comedic American cartoon that has come out of its quality besides Avatar The Last Airbender (which is in its own league of quality for American cartoons). It was also saddening to me when watching the final (5th) season that a lot of those episodes hadn’t aired due to them being “too dark for kids” :-(. Also, for reference, I consider the show ended at 5 seasons. The “6th season” entitled “Fast Forward” is NOT the same show. I’d equate its quality to the Ninja Turtles cartoons of the 1980s, or perhaps worse.

Anywho, when I was giving the series’ DVDs to a friend, I felt the need to compile a list of the episodes, as they were made to be watched in a specific order, but were not aired or released (even remotely) in that proper order. There are some episodes repeated in the list as they were included on multiple DVDs. This information is probably much easier to find nowadays... but when I compiled it, it definitely wasn’t easy information to come by.

Back in May of 2007 one of my friends got me onto Second Life, the first and only MMORPG I’ve touch since my Ragnarok days. While Second Life had a strong pull for me due to its similarities to The MetaVerse in Snow Crash, my favorite book, I was of course more drawn to playing with the Engine and seeing what I could do with it.

I felt no real need to delve into the code or packet level of the client as it was open source, so I stayed mostly on the scripting level side of things in the world. IIRC I did find at least a dozen major security holes, but I unfortunately cannot seem to find logs of my research :-(.

• While an avatar could not pass through solid walls normally, if an object was visible that allowed “sitting” beyond the walls, the user could issue the sit command on that object which transported the avatar past the barriers.
• While there were optional restrictions on areas pertaining to if/where an object could be placed, once an object was placed somewhere, it could be “pushed” to almost any other location no matter the restrictions. When an object was pushed into another area beyond where it was placed, it was still inventoried as being in the originally placed location, but could interact with the world at the location it was actually at. Objects could even pass through solid barriers if the proper push velocities were given. The only way at the time to combat this was to have whole private islands as blocking anonymous objects. This security hole opened up multiple other security holes including:
  • If a user “sat” on the object, they could get to anywhere the object could.
  • These objects could be used to interact with the immediate world around them, including repeating private conversations in a private area.

I had also at the time planned on writing an application that allowed hijacking and reuploading any encountered texture or construct, which was trivial due to the open nature of the system. I never did get around to it for two reasons. First, I got distracted by other projects, and second, because it could have seriously destabilized the Second Life economy, which was built around selling said textures and constructs. I actually liked what Second Life was trying to accomplish and had no wish of making Linden Lab’s life harder or ruining the experiment in open economy.

I was however able to find a few pieces of my research and scripts that I figured I could post here. First, I do not recall what I did to find this, but the entire list of pre-defined “Last Names” was accessible, and IIRC the proprietary last names could be used for character creation if you knew how to access them (not 100% sure if this latter hack was available). Here was the list as of when I acquired it in 2007. I had the list separated into two columns, and I think they were “open” names and “proprietary” names. Each name is followed by its identifier.

The second piece I was able to find was a script I used to alert me via email whenever one of my friends signed on. I have unfortunately not tested this script before posting it as I no longer have Second Life installed or wish to waste the time testing it, but here it is none the less. ^_^;

//Users to watch
key DetectPersons=[ //List of UIDs of users to watch. (Real UIDs redacted)
    "fdf1fbff-f19f-ffff-ffff-ffffffffffff", //Person 1
    "f0fffaff-f61f-ffff-ffff-ffffffffffff" //Person 2
];

//Other Global Variables
integer NumUsers;
integer UsersParsed=0;
list UserNames;
list Status;

default
{
    state_entry()
    {
        NumUsers=llGetListLength(DetectPersons); //Number of users to watch

        //Get User Names
        integer i;
        for(i=0;i<NumUsers;i++)
        {
            llListInsertList(UserNames, [''], i);
            llListInsertList(Status, [0], i);
            llRequestAgentData(llList2Key(DetectPersons, i), DATA_NAME);
        }
    }

    dataserver(key requested, string data)
    {
        //Find User Position
        integer i;
        for(i=0;i<NumUsers;i++)
            if(llList2Key(DetectPersons, i)==requested)
                llListReplaceList(UserNames, [data], i, 1);

        if(++UsersParsed==NumUsers)
            state Running;
    }
}

state Running
{
    state_entry()
    {
        llOwnerSay((string)UserNames);
        llOwnerSay((string)Status);
        llSetTimerEvent(30);
    }

    timer()
    {
        llRequestAgentData(DetectPerson, DATA_ONLINE);
    }

    dataserver(key requested, string data)
    {
        if(data==IsOnline)
            return;
        IsOnline=data;
        if(data=="0")
            return;
        string Message="The user you are watching '"+UserName+"' signed on at "+llGetTimestamp();
        llEmail(EMAIL_ADDRESS, "User Signed on", Message);
        llOwnerSay(Message);
    }
}

Of course all this research was from 2007 and I have no idea what is capable now. I do really hope though that they at least updated the client’s interface because it was incredibly clunky. Also, Second Life has always been a neat experiment, and I hope it still is and continues to keep doing well :-).

There are two primary authentication methods for logging onto an SSH server as a user. The first is password based authentication, and the second is public key authentication. The public/private RSA key pair for public key authentication can be created using OpenSSH’s “ssh-keygen” application.

I’m not going to go into the exact method on accomplishing this because instructions can be found on countless other places on the internet. However, I was curious yesterday as to what exactly was in the public key (.pub) files created by ssh-keygen, as the data payload was larger than I expected (2232 bits for a 2048 bit key). I couldn’t find documentation on this ANYWHERE on the internet, so I downloaded the OpenSSH source code and looked at the generation code of the files. The format of the files is as follows:

• The public key files are ASCII based text files with each public key taking up exactly one line.
• Each line is formatted with 2 pieces of data as follows:

  KEY_TYPE DATA_PAYLOAD

• KEY_TYPE is the type of public key, which in our case (and most cases nowadays) is “ssh-rsa”.
• DATA_PAYLOAD contains the actual public key information encoded in base64 with the following format:

I also checked putty public key authentication files and they seemed to contain the exact same DATA_PAYLOAD.

When you send an email there may be multiple fields in the email header that specify the email address that it came from and how to reply back to that address. Some of these are:

• From: This is the field that the user sees in their email client as the "From" address. This field is the most easily (and most often) spoofable as you can put anything you want in this field and it doesn't change how the email is received or responded to. Most systems, in my experience, don't try to protect this field either.
• Envelope sender: This is used internally by email software to see who the email was really from. Different systems (i.e. spam blockers) can use this field for different purposes.
• Return path: This field specifies the email address to reply to when you click the "reply" button on your email client.

• Spam blockers may be more likely to identify the email as spam
• The email might be sent from the wrong IP address. Exim (which cPanel uses by default) might be configured to check /etc/mailips to determine what IP address to send from depending on the domain of the envelope sender.
• The recipient might reply to the wrong email address when replying to the email.

When sending an email from PHP via the mail function through Exim you can only manually set the "From" header field (of the three) through the "additional_headers" (4th) parameter. This might be possible to remedy on some systems however.

If your server is configured to allow it (it may require privileged user permission), you can pass to the "additional_parameters" (5th) parameter of the mail function the -f Exim option, which sets the envelope sender and return path. For example:

mail('example@gmail.com', 'This is an example', 'Example!', 'From: example@yourdomain.com', '-f example@yourdomain.com');

On a related security note, if you think an email may not be legitimate, don't forget to check the email headers by viewing the original email source. Our servers include many useful headers in emails to help combat fraud including (depending on circumstances) the account the email was sent from, the IP address it was sent from, if it was sent from PHP, and if so, the script it was sent from.

While UTF-8 has almost universally been accepted as the de-facto standard for Unicode character encoding in most non-Windows systems (mmmmmm Plan 9 ^_^), the BOM (Byte Order Marker) still has large adoption problems. While I have been allowing my text editors to add the UTF8 BOM to the beginning of all my text files for years, I have finally decided to rescind this practice for compatibility reasons.

While the UTF8 BOM is useful so that editors know for sure what the character encoding of a file is, and don’t have to guess, they are not really supported, for their reasons, in Unixland. Having to code solutions around this was becoming cumbersome. Programs like vi and pico/nano seem to ignore a file’s character encoding anyways and adopt the character encoding of the current terminal session.

The main culprit in which I was running into this problem a lot with is PHP. The funny thing about it too was that I had a solution for it working properly in Linux, but not Windows :-).

Web browsers do not expect to receive the BOM marker at the beginning of files, and if they encounter it, may have serious problems. For example, in a certain browser (*cough*IE*cough*) having a BOM on a file will cause the browser to not properly read the DOCTYPE, which can cause all sorts of nasty compatibility issues.

Something in my LAMP setup on my cPanel systems was removing the initial BOM at the beginning of outputted PHP contents, but through some preliminary research I could not find out why this was not occurring in Windows. However, both systems were receiving multiple BOMs at the beginning of the output due to PHP’s include/require functions not stripping the BOM from those included files. My solution to this was a simple overload of these include functions as follows (only required when called from any directly opened [non-included] PHP file):

<?
/*Safe include/require functions that make sure UTF8 BOM is not output
Use like: eval(safe_INCLUDETYPE($INCLUDE_FILE_NAME));
where INCLUDETYPE is one of the following: include, require, include_once, require_once
An eval statement is used to maintain current scope
*/

//The different include type functions
function safe_include($FileName)	{ return real_safe_include($FileName, 'include'); }
function safe_require($FileName)	{ return real_safe_include($FileName, 'require'); }
function safe_include_once($FileName)	{ return real_safe_include($FileName, 'include_once'); }
function safe_require_once($FileName)	{ return real_safe_include($FileName, 'require_once'); }

//Start the processing and return the eval statement
function real_safe_include($FileName, $IncludeType)
{
	ob_start();
	return "$IncludeType('".strtr($FileName, Array("\\"=>"\\\\", "'", "\\'"))."'); safe_output_handler();";
}

//Do the actual processing and return the include data
function safe_output_handler()
{
	$Output=ob_get_clean();
	while(substr($Output, 0, 3)=='?') //Remove all instances of UTF8 BOM at the beginning of the output
		$Output=substr($Output, 3);
	print $Output;
}
?>

I would have like to have used PHP’s output_handler ini setting to catch even the root file’s BOM and not require include function overloads, but, as php.net puts it “Only built-in functions can be used with this directive. For user defined functions, use ob_start().”.

As a bonus, the following bash command can be used to find all PHP files in the current directory tree with a UTF8 BOM:

grep -rlP "^\xef\xbb\xbf" . | grep -iP "\.php\$"

 find . -name '*.php' -print0 | xargs -0 -n1000 grep -l $'^\xef\xbb\xbf'

find . -name '*.php' -print0 | xargs -0 -n1000 grep -l $'^\xef\xbb\xbf' | xargs -i perl -i.bak -pe 'BEGIN{ @d=@ARGV } s/^\xef\xbb\xbf//; END{ unlink map "$_$^I", @d }' "{}"

find . -name '*.php' -print0 | xargs -0 -n1000 grep -l $'^\xef\xbb\xbf' | xargs -i perl -i -pe 's/^\xef\xbb\xbf//' "{}"

A number of my friends are participating in this year’s NaNoWriMo (National Novel Writing Month), which I was also personally debating participating in. While I wanted to, the deciding factor was I couldn’t come up with a plot I really wanted to write about, and forcing myself to write about something I wasn’t passionate about wouldn’t be any fun. In lieu of this, and to honor the spirit of the project, I decided I would instead get at least one piece of content updated on my site every day this month, which takes just as, if not more, effort. Besides, this course is slightly more productive. ^_^;

Due to another new project I’ve been working on, I was able to test out different methods on rendering sprites on the Android platform.

• Method #1: Use OpenGL ES to render to a buffer in C via JNI. This method turned out to be the fastest if polygons are batched via lists in a single (or a few) draw calls, and if using fixed instead of floating point numbers. It also opens up a lot of other effects natively not easily implemented through the other two methods (native 3D and matrices). However, it does not support antialiasing natively as is advertised and as I expected. The OpenGL ES 1.0 specs report “antialiased polygon rendering (with alpha border fragments, not multisample)”, but I could not find any information on this at all, and no antialiasing calls that I tried actually worked. There are methods that would allow antialiasing, but none efficient enough for a phone.
• Method #2: Drawing directly to a pixel buffer in C via JNI. This method came in a very close second place if using memory copying functions to blit images, but that also removes a lot of important effects like translucency. Results were less than optimal when not using direct memory copies (I didn’t even worry about including them below) :-\.
• Method #3: Rendering through the Android API in Java. While this method is the most robust in terms of feature set, it is also by far the slowest.

For my personal goals, I’ve come to the conclusion it would be best to use Method #1 for my heavy rendering tasks, and used baked translucent pixels around the borders in the images to emulate 2D antialiasing. The translucent borders could also be added at load time. I think I may also use Method #3 for some other special effects and overlays, though that will also make it harder to port, but I think worth it for now.

• The Test: 45*45 quads (2*45*45=4050 polygons) drawn to cover the whole screen each frame, none overlapping
• Emulator/Droid: The tests were run on 2 separate systems. An Android emulator running v2.2 on my Intel core i5 2.53ghz laptop, and my Motorola Droid also running Android v2.2.
• FLOAT/FIXED: This is only used on Method #1. OpenGL ES supports (in different functions) both floating point numbers and fixed point numbers. Since phones usually have no FPU, and the ones that do might have very slow ones, it’s much more safe efficiency wise to not use floating point numbers. The OpenGL ES solution was to have non-floating (fixed) point decimal arithmetic numbers that act virtually the exact same as normal integers when used with arithmetic functions, but do not have near the range as either floating point numbers or normal integers. The gain by going from floating to fixed was marginal in my tests. 16 bit integers (shorts), which I prefer, can also be used and showed near identical results to fixed point numbers.
• INDIVIDUAL/ALL: This is only used on Method #1. It means whether all the quads were drawn individually in a different call for each (which draws 2 triangles to make a quad), or in 1 call from a list of all the triangles. The gain by going from INDIVIDUAL to ALL was ~1fps on the emulator and ~12fps on the Droid in my tests.

So for all of June and half of July this year I was in Canada for a really big contract. It was a very intense and taxing (though rewarding!) project that basically tied me up 24/7 for the whole duration, minus the little sleep I could afford, and acquiring food. Unfortunately, during this time, during a very hectic and somewhat dangerous part of the job, my Dell XPS M1730 laptop took a grand fall and cracked open. To its credit, it lasted for 10 more days, which completely saved my butt. During that time it only had minor touchpad problems which required a reboot when they started happening, but then it completely bit the dust on the final day of the project (I coped) due to, I believe, an electrical short somewhere on the motherboard.

The previous laptop I had the pleasure of using for 3 years was a Dell XPS M1710, which I absolutely loved in every way, besides the constant hardware failures and having to get replacement parts sent out each time they occurred. It conveniently bit the dust just before its warranty was up, so I was sent the previously mentioned M1730 by Dell as a replacement, which was unfortunately a refurb[ished], and never worked very well. Because of this it had no warranty, and coupled with the sub-par performance, I decided it was time to consider it totaled when it stopped working, retire it, and get a new laptop.

The new laptop process however ended up taking about 6 weeks to complete due to horrible hardware failures and service. My requirements for a laptop were very specific and there were only about 5 laptops on the market I could find that even fit my specs, which was very disappointing. Within those 6 weeks, I have had the chance to use and review 3 separate laptops, each from different companies, and will be including my positive and negative points about them below (in regard to the many other laptops I’ve used over the years). It can be assumed that anything that is not mentioned is as expected.

• Supplier: Bought from Amazon for ~$1,600. A full refund was issued upon confirmed hardware failure by Amazon (otherwise a restocking fee is applied). It was sent out immediately and received within 1 day.
• Pros:
  • Huge 18.4” monitor
• Cons:
  • The reason I was forced to return it was the monitor went bad in less than a week. The monitor would sometimes turn on after boot or a resolution change, but would always turn back off within 5 seconds. During the short spurts it was on, the colors were way off on half the screen. I did a lot of tests using an external monitor to try and fix the problem, but determined it was an unfixable hardware issue.
  • Media/control buttons were located on the left side of the keyboard. They were touch sensitive buttons that were way too easy to trigger accidentally. Simply relaxing my left hand usually caused it to brush and trigger one of the pseudo-buttons. I had been planning on writing a little utility that required either a double tap, or a prolonged hold, to trigger the keys, but ended up not needing to due to having to return the laptop.
  • The power cord disconnected way too easily. It probably averaged coming out of the power slot about 3 times an hour with little movement of the laptop.
  • It had very bad overheating problems consistently, but especially when playing games. I believe this might have caused the failure of the monitor.
  • The speakers would cause the volume to fluctuate very randomly so music was always distorted as it increased and decreased in volume every few second or so.

• Supplier: Bought from Dell for ~$1,800 including extended 3 year mid-tier warranty and a few hardware upgrades. A full refund was issued upon return. Dell originally lied to me about the amount of time it would take to arrive and I almost canceled the order before it was sent out because of this.
• Pros:
  • The media buttons were in a very unobtrusive place (the best of any laptop I’ve ever had).
  • The laptop was probably half the weight of any other laptop I’ve had of its size, and the power supply was probably about a sixteenth the size of any power supply for same said previous laptops.
• Cons:
  • It’s Dell...
    • Dell has absolutely ABYSMAL phone support. It’s outsourced to India and the “representatives” are completely unknowledgeable and virtually unintelligible. The representatives and managers have absolutely no power to get anything done, and even the managers are now Indian so you can’t even escalate to a comprehensible conversation. The representatives do virtually nothing but read prompts from screens, and for knowledgeable computer users, it’s painful to explain to them you don’t need them to try and diagnose the problem as you already have, but they want to guide you through their script via the phone anyways. During the calls for this laptop I was even told at one point I would have a 2 hour wait time to talk to a manager, and I experienced so many dropped calls I stopped counting. Dell support was the worst in the industry 2 years ago. Since then, it’s gotten twice as bad. I will never use or recommend Dell again to anyone for this reason.
    • Before I gave Dell my credit card number and committed my order, I had been told by the website the laptop would ship immediately and I would have it within 2 days. Immediately after I committed to buying it, the website suddenly told me it would instead take OVER 3 WEEKS for me to receive it. I was flabbergasted, and this was the reason I spent hours with phone support over many days trying to get this fixed. I finally decided to cancel the order and get another laptop on the 6th day, but I guess due to my demands, they actually shipped it right before I was about to call, aborting my attempt. It arrived 8 days after I made the order, which still caused me major problems.
  • The hard drive had major freezing problems, which is what eventually made me return the laptop, as I did not want to have them send me a completely new Chinese assembled one, as it would take forever and most definitely be a refurb. The freezing even occurred during BIOS, and it often took up to 4 minutes to resume from hibernation.
  • The ATI video card was less than optimal compared to the nVidia cards on my other recent laptops. It just wasn’t performing in games.
  • The power cord was ridiculously short, was prone to falling out (not nearly as much as the Toshiba), and had a power led on it that was much too bright (it actually kept me up at night if left on).
  • The speakers were in a horrible spot on the palm rests. Having my wrists in the proper and comfortable position for the keyboard covered them up causing bad distortion and dampening.
  • The touchpad was far too big and had no dead zones in the touchpad driver properties. Because of this and the horrible over sensitivity of the pad, it was very hard to use as it often stopped working when it detected “multiple touches”. Even an apple charger cord barely touching it made it stop working.
  • There was no property key or pause break key (Even via a “Fn” key combo).
  • There was no indicator light for the caps and num locks.
  • Many of the keys started squeaking after a few days.

• Tired of horrible hardware from other companies, I decided to give in and get a MacBook Pro against my better judgment. It has turned out to be the keeper simply because I’m tired of dealing with finding a laptop and I hear they have spectacular technical support including (supposedly) often receiving your laptop back within 3 days of sending it in for hardware replacement!
• Supplier: Bought from Apple for ~$3,100 including extended warranty and a few hardware upgrades whole sale (RIDICULOUSLY expensive). However, I had a 15% friends and family discount through a friend who is an employee of Apple bringing the total down to ~$2,600. There is no way I would have gotten it without the discount, but even with, it was still hideously expensive for what you actually get. I received it within 7 days as I was told.
• Pros:
  • It actually has a 1920x1200 (WUXGA) resolution! Both of my previous Dell’s had this, but the only 2 computers I could find on the market currently with this that fit my specs were the MacBook Pro and an Alienware (which is Dell and also ludicrously expensive). The next step down I was forced to accept on other computers was 1920x1080 (Full HD/FHD/1080p).
  • The magnetic power connector is WONDERFUL. It never falls out!
  • The visible battery meter on the side of the computer is kind of nice, but I doubt I’ll ever use it.
  • I was able to get a matte screen for an extra $50. I HATE (but have always had to deal with) glossy screens because you often can’t see them if the sun is shining on your screen, and they are fingerprint magnets.
  • The time the computer can run off of battery seems pretty amazing. Windows is reporting almost double the amount of battery time as normal laptops, which seems to be accurate, though I have not fully tested this.
• Cons:
  (Most all regarding running in Windows on the MacBook Pro, which is what I pretty much only use it for)
  • The touchpad has virtually no settings and works absolutely horribly in Windows.
    • Some example settings most all other touchpads have, some of which are available for this touchpad in OSX include: sensitivity, dead zones, and scroll zones.
    • The available settings in Windows are: Tap to click, dragging, drag lock, which bottom corner is considered a secondary click, two fingers resemble a secondary tap.
    • The multi-touch nature mixed with the absolute farce that is the Windows drivers for the device is what causes the main problem. There are no separate mouse buttons, and it’s basically unusable to utilize the bottom left and right sides as buttons with all the glitches. I think I might end up trying to write my own drivers for it for Windows soon, and if that doesn’t work, I will attempt finding a mouse buttons peripheral I can plug into via USB.
    • The touchpad will not allow a right (secondary) click while another finger is also touching it, and the secondary click via 2 tapped fingers is very unstable. It also seems right clicking sends a left mouse down event (but not a left mouse up event), which often cancels context menus.
    • For the touchpad to be usable I have to make sure to keep only one finger on it at a time or it gets buggy.
    • I did research on the Apple multitouch touchpad a few nights ago and apparently Apple didn’t really support using the touchpad at all in windows until like 2009, and that was only a token gesture as they just don’t give a shit about the drivers, only making them barely usable to allow advertising Windows compatibility.
  • The keyboard key set is only a subset of a normal keyboard and missing a ton of keys:
    • No number pad (though many laptops do not have one)
    • A few of the missing keys are: Page up, page down, home, end, print screen, insert, delete (only has a backspace labeled as delete). Missing keys are mostly all replaced by “FN” key combos via Boot Camp, though not all of them are listed in the Boot Camp help file.
    • Due to the missing keys and the non standard layout of the Mac keyboard, I used KeyTweak to remap a good number of keys for my purposes. There is also a program available in the Windows Server 2003 Resource Kit Tools that accomplishes the same task, though with a worse GUI, called Remapkey.exe. Both of these programs just modify a registry value that has windows natively remap the keys. I also had to use a modification of my HalfKey Project for some other key remappings.
    • The “Fn” and “Eject CD” keys are hard wired and can’t be remapped through the above method. This has caused me a lot of annoyance so far as “Fn” and the left “Control” keys are swapped from standard layouts.
  • There is no way in Windows to disable/mute the startup sound when the laptop is turned on (which I find incredibly annoying and embarrassing in public venues). Fortunately, this can be fixed by running the 3rd party StartupSound.prefPane configuration dialog in OSX just once.
  • EFI adds another layer that can be used as a security weak point, invalidating my last security scheme. It didn’t work off the bat anyways as the EFI wouldn’t boot to the USB running GRUB, as I believe GRUB for EFI is required.
  • The keyboard backlight doesn’t work until the OS has loaded making the keyboard unviewable in dark situations. The monitor brightness is also unadjustable until Windows loads, and uses the last brightness set by OSX.
  • There are no drivers for the light sensor in Windows (though I personally don’t care about that).

Seeing as there are a number of applications on the market that have both a “Free” and “Full” version, you’d think this would be an easy thing to accomplish. Unfortunately, the marketplace uses an application’s package name as its unique identifier, so both versions have to have a different package name, which is again, a bit of a nuisance.

One method of remedying this is just having a recursive string replace through all the files [...]

I spent a bit of time coming up with a solution for this a few months ago for my latest project, [TEMPORARILY DOWN**]. This solution uses a shared package with a shared code base and resources that the different project versions pull from.

**The project that is a great example of how this process works should be uploaded very soon. At that time this message will disappear and appropriate links will be added. You’ll know this has happened when I upload my next project.

• First some definitions that will be used below*:
  • ProjectName: The base name of the project (e.g. “MyAndroidProject”)
  • VersionName: The name of separate versions (e.g. “Free” and “Full”)
  • SharedName: The name of the shared section of the code (e.g. “Shared”)
  • BasePackageName: The base name of the package group (e.g. “com.example.MyAndroidProject”)
  • BasePackageDirectory: The base path of the package group (e.g. “com/example/MyAndroidProject”)
  *Please note these definitions are used in code sections below.
• Create the directory structure:
  • A base directory of ProjectName (e.g. “~/MyAndroidProject”)
  • A subdirectory under the base directory named SharedName for the shared files (e.g. “~/MyAndroidProject/Shared”). It will hold any shared files in appropriate Android standard directories (e.g. “res”, “src”).
  • Subdirectories under the base directory named VersionName for each version’s project (e.g. “~/MyAndroidProject/Free”). Each of these will hold its own complete project including the AndroidManifest.
• Creating the shared resources: There’s nothing special about shared resources (probably in “SharedName/res”), except I suggest noting at the top of the files that they are shared, for reference sake.
• Creating the shared code:
  • Shared code goes in “SharedName/src/BasePackageDirectory/SharedName” (e.g. “~/MyAndroidProject/Shared/src/com/example/MyAndroidProject/Shared”).
  • As noted for shared resources, you might want to note at the top of the files that they are shared.
  • Set the package name for shared code to “BasePackageName.SharedName” (e.g. “package com.example.MyAndroidProject.Shared;”).
  • Shared code should never directly refer back to a version’s package (non shared) code except through reflection.
  • Resource IDs are still accessible in this shared package through the “R” class, but when accessed the function or class that does the accessing needs to be proceeded with “@SuppressWarnings({ "static-access" })”. The “R” variable also has a “Version” member that can be used to alter shared code flow depending on the version being used. This will be explained more in detail later.
  • *BONUS*
    If shared code needs to access information from a static member in a non-shared class, reflection can be used, for example:

    Class.forName("BasePackageName."+R.Version.name()+".NonSharedClassName").getDeclaredField("StaticMemberName").get(null)

    A static function can be called in a similar manner through reflection:

    Class.forName("BasePackageName."+R.Version.name()+".NonSharedClassName").getMethod("StaticMethodName", new Class[0]).invoke(null);

• *BONUS* Global Shared Class: I also suggest having a shared class that holds global variables that allows easy interaction from the shared to non shared classes, and holds static information that both use, with members including:
  • A reference to a Context (or Activity) from the main program
  • The BasePackageName (needed for reflection, and other stuff like preference storage)
  • Other useful interfaces like logging
• Creating the non-shared code:
  • Create a separate project for each version in their corresponding subdirectories listed in the third step of the Create the directory structure" section above.
  • Make the package name for a version as “BasePackageName.VersionName”.
  • When referencing shared classes form an android manifest, make sure to use their full package name, for example, a shared activity would look like “<activity android:name="BasePackageName.SharedName.ActivityName">”
  • Import the shared package into all non-shared class files “import BasePackageName.SharedName.*;”
• Linking the shared code into each project:
  • The shared code now needs to get integrated into each project. To do this, all the shared files need to be symbolically (or hard) linked back into their corresponding directories for each version.
  • First, make sure each project directory also contains the same subdirectories as those found in the shared directory.
  • The script I have written for this, which needs to be put in the base directory, is as follows: [EDIT ON 2011-01-03 @ 6:28am] See here for a better copy of the script. [/EDIT]

    #!/bin/bash
    
    #Run this file to install links to shared files into all branches
    
    LN="ln" #Use hard links, which work well in Windows and require less logic to calculate linking
    
    #if [ `uname || grep -vi 'cygwin'` ]; then #If not windows (NOT YET SUPPORTED)
    #	LN="ln -s" #Use symbolic links, which take some additional logic that is not yet programmed
    #fi
    
    LocalBranches=`find -maxdepth 1 -type d | grep -iPv "^\.(/SharedName|)$"` #Find version names, ignoring "." ".." and the shared directory
    
    #Propagate shared files into different versions
    cd Shared
    for i in $LocalBranches; do
    	find -type f -print0 | xargs -0 -i rm -f ../$i/{} #Clear out old links from version just in case the link has been undone
    	if [ "$1" != "clean" ]; then
    		find -type f -print0 | xargs -0 -i $LN {} ../$i/{} #Link shared files into directories
    	fi
    done
    				

• Tying the resources together:
  • The resources IDs in the “R” class might need to be accessible by the shared classes. To do this, an “R” class needs to be added into the shared namespace that duplicates the information. Unfortunately, the Android generated resources ID class “R” marks everything as “final” so class duplication requires a bit more work than would be preferable.
  • Each different version needs its own “R” class, put into the directory “~/MyAndroidProject/VersionName/src/BasePackageDirectory/SharedName” that the shared code reads from.
  • This class will also contain version information so the shared classes can know which version it is currently interacting with.
  • The code is as follows:

    package BasePackageName.SharedName;
    
    import BasePackageName.VersionName.R.*;
    
    public final class R //Mimic the resources from the (non-shared) parent project
    {
    	//There may be more resource ID groups than these that need to be shared
    	final static attr attr=null;
    	final static drawable drawable=null;
    	final static id id=null;
    	final static layout layout=null;
    	final static string string=null;
    	final static xml xml=null;
    	final static array array=null;
    	
    	public enum Versions { Version1, Version2 }; //List versions here
    	
    	public final static Versions Version=Versions.CurrentVersion;
    }
    			

  • Whenever this “shared” “R” class is accessed, the function or class that does the accessing needs to be proceeded with “@SuppressWarnings({ "static-access" })”. This is due to the hack required to reproduce the “final” class information from the original “R” class into a shared class.
• Working with shared projects and code in Eclipse:
  • When modifying shared code in Eclipse for any of the different versions, other version’s projects need to be refreshed to pick up the new code. I tried many different methods and processes to fix this different package name problem but this described method is still much quicker and easier than any of the others.

A fix for this issue has been recommended and I consider it resolved. There was always a fix for this but it had not been given yet, as there was confusion to the problem in the bug report. Other people are still reporting the problem but I cannot reproduce it after the suggested fix.

It has come to my attention that my Mini Acuity application suffers from a “known defect” in Android, namely, that it “requires” 2 security permissions I did not ask for or use.

The “Storage” and “Phone Calls” permissions are listed in the “Application Info” in the Android settings dialog even though the market does not specify that they are used during install.

This is a result of using an old SDK version as the base of my code. Google actually encourages authors to use the minimum possible SDK version, as Android is backwards compatible, so that as many users as possible are supported. This means my only solution to not having these security permissions listed would be to upgrade the SDK version, thereby disabling compatibility from some older phones. I wish there was a way I could see the distribution of Android Versions for my application’s downloads to help determine if this would be worth it.

I have updated the Mini Acuity project page accordingly with a “Permissions” section.

On a side note, it occurs to me how much of a security hole this [possibly] is. If an application is running on these old SDK versions, and the user sees an application has only network access permission, they might not worry about the application stealing their data while it could! Though, I have not yet done the research to confirm this, or plan on doing so. I feel more and more that Android’s security system leaves a lot to be desired.

So Microsoft sold us [company I am currently working for] a copy of Visual Studio 2010 Professional with lies about what it supported (Windows Mobile Legacy Versions [including CE]). When we complained, they spouted how VS2010 supports the newest version of Windows Mobile (Windows Phone 7), which doesn’t even exist yet (they promise a release in “Holiday of 2010”, I’ll believe it when I see it, as this is not the first missed expected deadline). Now they refuse to give us a refund on VS2010, or even let us buy VS2008 from them instead, as it’s “a legacy product”, even though we need it because it DOES support windows mobile legacy versions.

Microsoft has done this kind of thing to me, people I know, and pretty much everyone in the world too many times to count. They will never be receiving my business or money again. It feels great to see Google beating them hands down in every market Google decides to compete with them on.

Time to see if we can’t switch over to Linux or Android on these handheld systems as an alternative... (though unfortunately they seem to be locked in to running Windows CE *sigh*).

And Microsoft lied to me once again, though at least this time I was expecting it. I later found out I also had an MSDN subscription that came with VS2010, and called in to activate it, as online activation wasn’t working (don’t even want to mention all the mistakes they made during THAT process). I was told on the phone during this proceeding that the MSDN subscription license I had was compatible with the “VS Pro MSDN (Retail)” license, and was pointed to a list of products I could download from the MSDN Subscriptions page as soon as my subscription was activated (which took 3 days...). Low and behold, this was not true and I can not download many of the things I am needing and was planning on getting when the subscription came through (including VS 2005 or 2008), as the license is not compatible at all with what they told me.

Microsoft thrives on lying to their consumers and knowing they can get away with it. Microsoft specifically targets CEOs and tells them how important it is that they make their shop 100% Microsoft, giving completely falsified numbers and arguments to support this mockery. Microsoft jams their advertising so much into the heads of these non-tech-savvy individuals that when their IT staff tells them anything against the loud spoutings of Microsoft, the truth is lost in the wind, and even sometimes loses jobs. I have seen this happen at multiple companies, and have seen Microsoft’s lies and falsified reports more times than I can remember.

This somehow needs to be stopped.

One of the main selling points for me for the Android platform was that, as I understood it, the system was supposed to be very “open” in nature. It would allow a programmer to create virtually any application imaginable that they wanted to for users, as long as the user’s security is maintained. This is, of course, the antithesis of Apple’s philosophy with the iPhone/iPod Touch. However, I find this much spouted openness to not be the case at all. Security permissions are way too tight across the board, especially regarding interfacing with the hardware, making many things next to impossible to accomplish. This is especially true when interfacing with the phone functionality. While a programmer is free to do what they want within the scope of their own application(s) and their GUIs, working with the rest of the system can be a major PITA, or even impossible.

Some of this functionality can be gained back with rooted (jail broken) phones, but it is not always easy (or completely safe) to get one’s phone to such a state. It was simple with the Android 2.0 platform, which I originally had on my Motorola Droid, but not so much with the v2.1 software. Version 2.1 is (currently) a major PITA to root, as it entails having to restore the phone to its original state first, losing everything on it (which can, of course, be restored manually). I also, at this point, do not consider it worth it putting in the time to build things for rooted-only phones as the market is much smaller, and I myself haven’t even bothered rooting my phone on the current Android version.

Anyone can also compile their own version of the Android platform as it is open source. This would be worth it, for example, if an organization wanted to distribute their own compilation with modifications internally. However, it doesn’t much help application programmers like myself that want to reach a wide audience. I am also under the impression that putting your own flavor of the Android platform on your phone would lose certain functionalities/things included by the image provided by the provider of the phone (usually cell phone network carriers).

I really like how they did one section of the security system, which is, allowing an application to request special permissions from the operating system. A user is informed of the requested permissions before they install an application from the market place. The main problem is, though, that so many permissions are unavailable that should be possible. Another major downside is that way too many applications request permissions that they shouldn’t be requesting. This leaves users high and dry with applications they consider critical only available if they accept things they don’t want to. For example, many programs request full internet access with no need for it. It would be great to be able to selectively turn off these permissions, but I doubt the option for this is going to happen. I’m going to do more research myself on if an application can be written to do this, but I am not going to get even the slightest hope up on this possibility.

There are even examples of listed permissions that cannot be accessed by user submitted applications! For example, the INJECT_EVENTS permission can only be used by applications signed with the same signature as the system. I was unable to find this tidbit of information anywhere in the Android documentation and references (or the Internet). This all goes back to the problem of the documentation being less than optimal, as it leaves out a lot of important information.

• Call recording: I have written on this previously, but this functionality is unavailable, and Google is not commenting as to why. There are also countless other applications that could use the ability to access a call’s audio. This functionality was available on some older versions of the Android platform (and there are applications out there that take advantage of this), but it seems unavailable on newer versions for 1 of 3 reasons:
  • Legal reasons: It’s illegal to record calls in some areas (which would be dumb to revoke accessing call audio because of this because it’s legal in so many other places, including where I live in Texas).
  • Technological reasons: Some phone manufacturers might have it so the audio never even makes it to the operating system (it’s kept on the phone’s radio stack).
  • Google reasons: They decided it was a feature they no longer wanted to support. The fact of the matter is the interface is provided by the platform to do this, but bugs have been introduced into it and it no longer seems to work.
• Automated call menu: I would love to make an application that created an automated call menu on the phone, which could include leaving messages. I would personally use this so I could keep my phone on when sleeping, allowing the phone to direct the caller to either my [local or remote] voice mail or to wake me up if it’s an emergency. This is not possible due to the inability to access a call’s audio, as is explained in the above unimplementable application, but I am betting that there would be many more permissions that would make this not possible.
• Global Key interception: I have somewhat solved this problem, as I will be explaining in a post most likely coming tomorrow.

Having recently finished my first Android project (and hopefully not last), I decided to supply some notes I took about the process.

While I am going to try and keep impressions to a minimum on the rest of this post, and keep it to tangible notes, I must first comment that trying to find out things for the Android platform was often like pulling teeth. While its typical Java reference online documentation is all there with all the classes and cross-linking, that is about all it is, very dry and virtually useless beyond a base reference. The comments on variable parameters (and many other sections) in the reference are often coarse and not descriptive at all, for example, one parameter named mask has the basic description as “this is a mask”. Some functions don’t even have descriptions at all.

Perhaps I am getting too complacent as a programmer and getting used to excellent documentation like for Python or GTK (I’ve even grown to love Microsoft documentation after having used it for long enough!). After all, most required information is just a Google away, and being a programmer is often just about finding the proper magical incantations to hook into a certain library. Unfortunately, however, even web searches were often yielding less than fruitful results when dealing with Android, as the platform is relatively new.

• Some useful tasks and some problems:
  • Using the virtual (soft) keyboard without a TextView:
    • Showing the virtual keyboard:

      ((InputMethodManager)getSystemService(INPUT_METHOD_SERVICE)).toggleSoftInput(InputMethodManager.SHOW_FORCED, InputMethodManager.HIDE_IMPLICIT_ONLY);

    • Hiding the virtual keyboard:

      ((InputMethodManager)getSystemService(INPUT_METHOD_SERVICE)).hideSoftInputFromWindow(getWindow().getDecorView().getApplicationWindowToken(), 0);

      Note: “getWindow().getDecorView()” can also be replaced by a View on your screen
    • Getting the keyboard input: Add the following function to the Activity that opened the keyboard:

      @Override public boolean onKeyDown(int keyCode, KeyEvent msg)

      Note: This will not work if you’re not using default keyboard input (like if it’s set to enter Japanese or Chinese characters).
  • Determining the physical dimensions of the screen:

    This should be a trivial task using the DisplayMetrics (getWindowManager().getDefaultDisplay()) interface to get dpis and multiply by the screen dimensions getWindowManager().getDefaultDisplay().getWidth() (and .getHeight). However, it doesn’t always work as it should.

    The best method to get the DPI would be to use “DisplayMetrics.xdpi” and “DisplayMetrics.ydpi”, but unfortunately, these are misreported by at least the Motorola Droid. I’ve found “DisplayMetrics.density”*160 to be pretty accurate, but if true accuracy is needed, a calibration screen might be required.

  • Inform user of touch events: Many Android widgets (Views) change their visual state (highlight) when the user presses down on them to let the user know something is going to happen if the user lifts their finger while still on the widget. Unfortunately, there seems to be no text widget or layout view that does this automatic highlighting by itself (ListViews do in groups). The following is some example code to produce this effect.

    import android.view.View.OnTouchListener;
    
    public class CLASSNAME extends Activity
    {
    	@Override public void onCreate(Bundle savedInstanceState)
    	{
    		View HighlightView=findViewById(R.id.THE_VIEWS_ID);
    		HighlightView.setOnTouchListener(HighlightState);
    	}	
    	
    	private OnTouchListener HighlightState = new OnTouchListener() { public boolean onTouch(View v, MotionEvent event)
    	{
    		if(event.getAction()==MotionEvent.ACTION_DOWN)
    			v.setBackgroundColor(0xFF0000FF); //Set background color to blue
    		else if(event.getAction()==MotionEvent.ACTION_CANCEL || event.getAction()==MotionEvent.ACTION_UP)
    			v.setBackgroundResource(0); //No background color
    			
    		return false;
    	} };
    }

  • Retrieving the names and IDs of all resources in a resource group:

    import java.lang.reflect.Field;
    
    Field[] FieldList=R.drawable.class.getDeclaredFields();
    String[] Names=new String[FieldList.length];
    int[] IDs=new int[FieldList.length];
    for(int i=0;i<FieldList.length;i++)
    	IDs[i]=getResources().getIdentifier(Names[i]=FieldList[i].getName(), "drawable", getClass().getPackage().getName());

  • Setting a color matrix on an image: If you have 2 ImageViews that display the same resource image, and either has a color matrix set on it, the will both share one of the color matrices. If this occurs, copy the image the resource, or use a separate image resource. For kicks, here is an example of setting an inverse color matrix on an image.

    ((ImageView)findViewById(R.id.IMAGE_ID)).setColorFilter(new ColorMatrixColorFilter(new float[] {-1,0,0,0,255, 0,-1,0,0,255, 0,0,-1,0,255, 0,0,0,1,0}));

  • Setting to full screen:

    requestWindowFeature(Window.FEATURE_NO_TITLE); //This must be called before "setContentView", and hides the title bar
    getWindow().setFlags(FULLSCREEN ? WindowManager.LayoutParams.FLAG_FULLSCREEN : 0, WindowManager.LayoutParams.FLAG_FULLSCREEN); //Turns on/off the status bar

  • Starting another local activity: Instead of using Intent(String action) for Context.StartActivity, as the Reference explains, it is much easier to use Intent(Context packageContext, Class<?> cls) like the following: (called from inside an Activity)

    startActivity(new Intent(this, OTHER_ACTIVITY_NAME.class);

  • Creating a timed event that updates the UI: A function running through java.util.Timer cannot interact with the GUI. One solution to make a timer is with the android.os.Handler interface.

    import android.os.Handler;
    
    public class ExampleActivity extends Activity
    {
    	final int InitialDelay, RepeatDelay; 
    	Handler TimedHandler=new Handler();
    	
    	public void ExampleFunction()
    	{
    		TimedHandler.postDelayed(new Runnable() { public void run() {
    			//Do GUI stuff...
    			TimedHandler.postDelayed(this, RepeatDelay);
    		} }, InitialDelay);
    	}
    }

    
    Another solution is to post to a Handler from the Timer function.
• When dealing with putting on the market place:
  • Getting an account to put applications on the Android Market cost $25.
  • Screenshots shown on the Android Market description page are somewhat buggy, and seemingly randomly either stretch properly or crop. Viewing the full sized screenshots does seem to work properly.
  • Seeing as there are a number of applications on the market that have both a “Free” and “Full” version, you’d think this would be an easy thing to accomplish. Unfortunately, the marketplace uses an application’s package name as its unique identifier, so both versions have to have a different package name, which is again, a bit of a nuisance.

    One method of remedying this is just having a recursive string replace through all the files to change the package names. However, if using eclipse, so you don’t have to reopen it, it’s quicker to update the string first in the manifest, and then renaming the package under the “src” folder by pressing F2 (rename) on it when it is selected.

    Also, unfortunately, if you do this, when a person upgrades from the lite to the full version, preferences are not automatically transferred :-\.

  • The publisher’s market place page is very sparse and leaves a lot to be desired. It also seems to update only once every 24 hours or so (not sure of exact times).
  • If an application is put up, it WILL get downloads immediately. For example, I put up an application with a description of “This is a test, do not download this” for doing security tests that I took down within like 10 minutes. It already had 2 comments/ratings on it within that time ~.~; .
  • Google Checkout: Fees. When a copy of your application is purchased, the user has 24 hours to return it. The money is not deposited into your bank account until after this time (when it’s not a weekend). If you want to give your application to someone for free, they need to purchase it through the market, and then you can cancel the purchase transaction before the 24 hours are up. Unfortunately, this has to be done every time they want to update the application. It also seems you cannot buy your own applications, as the purchase server throws an error.
• Application Protection:

  You can download any Android application by default from your phone to your computer, modify them, and reinstall them back to any phone. An example use for this would be to crack a shareware application where just a single byte probably needs to be changed to make it a full version.

  The applications themselves are in an .apk file (which is just a .zip file), and the source code (classes) are encoded as a “Dalvik Executable” file within it (classes.dex), which as I understand it, is optimized Java bytecode. So, AFAIK right now, there is no way to decompile the .dex file back to the original source, like you can with normal Java. However, the Android emulator, part of the Android SDK, includes a tool called dexdump, which allows you to decompile it to bytecode.

  Once you have the bytecode, you can use that as reference to modify the compiled .dex file however you want, which is pretty much synonymous with assembly editing. Once that is done, the signature and checksum of the .dex file must be recalculated (Java source by Timothy Strazzere), and then the apk file must be resigned, and then it’s good to go!

  The marketplace also has an option to turn on Copy Protection. When this is turned on for an application, the user cannot backup or access the applications package file. I would assume however with a rooted phone you could still grab it from “/data/app-private”, and the rest of the process should be the same. I have not tested this as rooting Android 2.1 is much more of a pain in the butt, ATM, than I want to deal with.

I had the need to pass a program’s [standard] output to a web browser in real time. The best solution for this is to use a combination of programs made in different languages. The following are all of these individual components to accomplish this task.

Please note the C components are only compatible with gcc and bash (cygwin required for Windows), as MSVC and Windows command prompt are missing vital functionality for this to work.

The first component is a server made in C that receives stdin (as a pipe, or typed by the user after line breaks) and sends that data out to a connected client (buffering the output until the client connects).

PassThruServer source, PassThruServer compiled Windows executable.

• This compiles as C99 under gcc:

  gcc PassThruServer.c -o PassThruServer

• Define “WINDOWS” when compiling in Windows (pass “-DWINDOWS”)

#include <stdio.h>
#include <malloc.h>
#include <fcntl.h>
#include <sys/types.h> 
#include <sys/socket.h>
#include <netinet/in.h>
#include <signal.h>

//The server socket and options
int ServerSocket=0;
const int PortNumber=1234; //The port number to listen in on

//If an error occurs, exit cleanly
int error(char *msg)
{
	//Close the socket if it is still open
	if(ServerSocket)
		close(ServerSocket);
	ServerSocket=0;

	//Output the error message, and return the exit status
	fprintf(stderr, "%s\n", msg);
	return 1;
}

//Termination signals
void TerminationSignal(int sig)
{
	error("SIGNAL causing end of process");
	_exit(sig);
}

int main(int argc, char *argv[])
{
	//Listen for termination signals
	signal(SIGINT, TerminationSignal);
	signal(SIGTERM, TerminationSignal);
	signal(SIGHUP, SIG_IGN); //We want the server to continue running if the environment is closed, so SIGHUP is ignored -- This doesn't work in Windows
	
	//Create the server
	struct sockaddr_in ServerAddr={AF_INET, htons(PortNumber), INADDR_ANY, 0}; //Address/port to listen on
	if((ServerSocket=socket(AF_INET, SOCK_STREAM, 0))<0) //Attempt to create the socket
		return error("ERROR on 'socket' call");
	if(bind(ServerSocket, (struct sockaddr*)&ServerAddr, sizeof(ServerAddr))<0) //Bind the socket to the requested address/port
		return error("ERROR on 'bind' call");
	if(listen(ServerSocket,5)<0) //Attempt to listen on the requested address/port
		return error("ERROR on 'listen' call");

	//Accept a connection from a client
	struct sockaddr_in ClientAddr;
	int ClientAddrLen=sizeof(ClientAddr);
	int ClientSocket=accept(ServerSocket, (struct sockaddr*)&ClientAddr, &ClientAddrLen);
	if(ClientSocket<0) 
		return error("ERROR on 'accept' call");

	//Prepare to receive info from STDIN
		//Create the buffer
		const int BufferSize=1024*10;
		char *Buffer=malloc(BufferSize); //Allocate a 10k buffer
		//STDIN only needs to be set to binary mode in windows
		const int STDINno=fileno(stdin);
		#ifdef WINDOWS
			_setmode(STDINno, _O_BINARY);
		#endif
		//Prepare for blocked listening (select function)
		fcntl(STDINno, F_SETFL, fcntl(STDINno, F_GETFL, 0)|O_NONBLOCK); //Set STDIN as blocking
		fd_set WaitForSTDIN;
		FD_ZERO(&WaitForSTDIN);
		FD_SET(STDINno, &WaitForSTDIN);

	//Receive information from STDIN, and pass directly to the client
	int RetVal=0;
	while(1)
	{
		//Get the next block of data from STDIN
		select(STDINno+1, &WaitForSTDIN, NULL, NULL, NULL); //Wait for data
		size_t AmountRead=fread(Buffer, 1, BufferSize, stdin); //Read the data
		if(feof(stdin) || AmountRead==0) //If input is closed, process is complete
			break;
		
		//Send the data to the client
		if(write(ClientSocket,Buffer,AmountRead)<0) //If error in network connection occurred
		{
			RetVal=error("ERROR on 'write' call");
			break;
		}
	}
	
	//Cleanup
	if(ServerSocket)
		close(ServerSocket);
	free(Buffer);
	
	return RetVal;
}

The next component is a Flash applet as the client to receive data. Flash is needed as it can keep a socket open for realtime communication. The applet receives the data and then passes it through to JavaScript for final processing.

Compiled Flash Client Applet

import flash.external.ExternalInterface;
import flash.events.Event;
ExternalInterface.addCallback("OpenSocket", OpenSocket);

function OpenSocket(IP:String, Port:Number):void
{
	SendInfoToJS("Trying to connect");
	var TheSocket:Socket = new Socket();
	TheSocket.addEventListener(Event.CONNECT, function(Success) { SendInfoToJS(Success ? "Connected!" : "Could not connect"); });
	TheSocket.addEventListener(Event.CLOSE, function() { SendInfoToJS("Connection Closed"); });
	TheSocket.addEventListener(IOErrorEvent.IO_ERROR, function() {SendInfoToJS("Could not connect");});
	TheSocket.addEventListener(ProgressEvent.SOCKET_DATA, function(event:ProgressEvent):void { ExternalInterface.call("GetPacket", TheSocket.readUTFBytes(TheSocket.bytesAvailable)); });
	TheSocket.connect(IP, Port);
}
function SendInfoToJS(str:String) { ExternalInterface.call("GetInfoFromFlash", str); }
stop();

var NewSock=new XMLSocket();
NewSock.onData=function(msg) { GetPacket(msg); }
NewSock.onConnect=function(Success) { SendInfoToJS(Success ? "Connected!" : "Could not connect"); }
SendInfoToJS(NewSock.connect(IP, Port) ? "Trying to Connect" : "Could not start connecting");

JavaScript can then receive (and send) information from (and to) the Flash applet through the following functions.

• FLASH.OpenSocket(String IP, Number Port): Call this from JavaScript to open a connection to a server. Note the IP MIGHT have to be the domain the script is running on for security errors to not be thrown.
• JAVASCRIPT.GetInfoFromFlash(String): This is called from Flash whenever connection information is updated. I have it giving arbitrary strings ATM.
• JAVASCRIPT.GetPacket(String): This is called from Flash whenever data is received through the connection.

This example allows the user to input the IP to connect to that is streaming the output. Connection information is shown in the “ConnectionInfo” DOM object. Received data packets are appended to the document in separate DOM objects.

JavaScript+HTML Source

var isIE=navigator.appName.indexOf("Microsoft")!=-1;
function getFlashMovie(movieName) { return (isIE ? window[movieName] : document[movieName]);  }
function $(s) { return document.getElementById(s); }

function Connect()
{
	getFlashMovie("client").OpenSocket($('IP').value, 1234);
}

function GetInfoFromFlash(Str)
{
	$('ConnectionInfo').firstChild.data=Str;
}

function GetPacket(Str)
{
	var NewDiv=document.createElement('DIV');
	NewDiv.appendChild(document.createTextNode(Str));
	$('Info').appendChild(NewDiv);
}

Next is an example application that outputs to stdout. It is important that it flushes stdout after every output or the communication may not be real time.

inc source, inc compiled Windows executable.

inc counts from 0 to one less than a number (parameter #1 [default=50]) after a certain millisecond interval (parameter #2 [default=500]).

./inc 10 #Counts from 0-9 every half a second

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
	int NumLoops=(argc>1 ? atoi(argv[1]) : 50); //Number of loops to run from passed argument 1. Default is 50 if not specified.
	int LoopWait=(argc>2 ? atoi(argv[2]) : 500); //Number of milliseconds to wait in between each loop from passed argument 2. Default is 500ms if not specified.
	LoopWait*=1000; //Convert to microseconds for usleep

	//Output an incremented number every half a second
	int i=0;
	while(i<NumLoops)
	{
		printf("%u\n", i++);
		fflush(stdout); //Force stdout flush
		usleep(LoopWait); //Wait for half a second
	};
	
	return 0;
}

This final component is needed so the Flash applet can connect to a server. Unfortunately, new versions of Flash (at least version 10, might have been before that though) started requiring policies for socket connections >:-(. I don’t think this is a problem if you compile your applet to target an older version of Flash with the ActionScript v1.0 code.

This Perl script creates a server on port 843 to respond to Flash policy requests, telling any Flash applet from any domain to allow connections to go through to any port on the computer (IP). It requires Perl, and root privileges on Linux to bind to a port <1024 (su to root or run with sudo).

Flash Socket Policy Server (Rename extension to .pl)

#!/usr/bin/perl
use warnings;
use strict;

#Listen for kill signals
$SIG{'QUIT'}=$SIG{'INT'}=$SIG{__DIE__} = sub
{
	close Server;
	print "Socket Policy Server Ended: $_[0]\n";
	exit;
};

#Start the server:
use Socket;
use IO::Handle;
my $FlashPolicyPort=843;
socket(Server, PF_INET, SOCK_STREAM, getprotobyname('tcp')) or die "'socket' call: $!"; #Open the socket
setsockopt(Server, SOL_SOCKET, SO_REUSEADDR, 1) or die "'setsockopt' call: $!"; #Allow reusing of port/address if in TIME_WAIT state
bind(Server, sockaddr_in($FlashPolicyPort,INADDR_ANY)) or die "'bind' call: $!"; #Listen on port $FlashPolicyPort for connections from any INET adapter
listen(Server,SOMAXCONN) or die "'listen' call: $!"; #Start listening for connections
Server->autoflush(1); #Do not buffer output

#Infinite loop that accepts connections
$/ = "\0"; #Reset terminator from new line to null char
while(my $paddr=accept(Client,Server))
{
	Client->autoflush(1); #Do not buffer IO
	if(<Client> =~ /.*policy\-file.*/i) { #If client requests policy file...
		print Client '<cross-domain-policy><allow-access-from domain="*" to-ports="*" /></cross-domain-policy>'.$/; #Output policy info: Allow any flash applets from any domain to connect
	}
	close Client; #Close the client
}

1. Start the servers
   • In your [bash] command shell, execute the following

     Server/FlashSocketPolicy.pl & #Run the Flash Policy Server as a daemon. Don't forget sudo in Linux
     ./inc | ./PassThruServer #Pipe inc out to the PassThruServer

   • Note that this will immediately start the PassThruServer receiving information from “inc”, so if you don’t get the client up in time, it may already be done counting and send you all the info at once (25 seconds).
   • The PassThruServer will not end until one of the following conditions has been met:
     • The client has connected and the piped process is completed
     • The client has connected and disconnected and the disconnect has been detected (when a packet send failed)
     • It is manually killed through a signal
   • The Flash Policy Server daemon should probably just be left on indefinitely in the background (it only needs to be run once).
2. To run the client, open client.html through a web server [i.e. Apache’s httpd] in your web browser. Don’t open the local file straight through your file system, it needs to be run through a web server for Flash to work correctly.
3. Click “connect” (assuming you are running the PassThruServer already on localhost [the same computer]). You can click “connect” again every time a new PassThruServer is ran.

A client of mine wanted their website to have an applet that played streaming music from a SHOUTcast server. The easy solution would have been to just embed a Windows Media Player applet into the page, but that would only work for IE.

I thoroughly searched the web and was unable to find a Flash applet (or other solution) that already did this (and actually worked). Most of the information I was finding was people having problems getting this kind of thing working in Flash with no answer provided. After giving up on finding a resolution online, I decided to load up Flash and see what I could find from some tinkering.

Quite frankly, I’m shocked people were having so many problems with this. I started an ActionScript 2.0 project and put in the following code, and it worked right away in Flash CS3 (v9.0) with no problem:

var URL="http://example.shoutcast.castledragmire.com:1234/" //The URL to the SHOUTcast server
var MySound:Sound=new Sound(this);
MySound.loadSound(URL,true);

Unfortunately, once I exported the Flash applet and loaded it up in my browsers, it was no longer working. After a few minutes of poking around, I had a hunch that the SHOUTcast host might be sending different data depending on the [Browser’s] User Agent. I changed Firefox’s User Agent to “Flash” through a Firefox add-on (User Agent Switcher), and it worked :-D.

Once again, unfortunately, this was not a viable solution because I couldn’t have every user who visited the client’s web page change their browser User Agent string :-). The quickest solution at this point to the problem was to just create a passthrough script that grabbed the live stream on their server and passed it to the client. The following is the PHP script I used for this:

$streamname='example.shoutcast.castledragmire.com';
$port      ='1234';
$path      ='/';

header('Content-type: audio/mpeg');
$sock=fsockopen($streamname,$port);
fputs($sock, "GET $path HTTP/1.0\r\n");
fputs($sock, "Host: $streamname\r\n");
fputs($sock, "User-Agent: WinampMPEG/2.8\r\n");
fputs($sock, "Accept: */*\r\n");
fputs($sock, "Connection: close\r\n\r\n");
fpassthru($sock);
fclose($sock);

1. Setting the Flash Applet’s URL variable to the PHP file
2. Turning off PHP output buffering for the file. This can only be done through Apache or the php.ini depending on the server setup. This is very important, as if it’s on, the data will never get sent to the user.

The only problem with this method is that it taxes the server that is passing the data through, especially since it uses PHP... This kind of thing could very easily be done in C though (as a matter of fact, I will be writing a post on something very close to that very soon).

A friend of mine who mainly works in Linux is always frustrated when he has to deal with working in Windows and doesn’t have FFM (Focus Follows Mouse). FFM means focusing a window when the mouse cursor moves over it (rollover), preferably without raising the window’s z-order position. I told him I’d throw together a program that did this, but my original approach was problematic.

My original approach was to use LowLevelMouseProc (WinAPI:SetWindowsHookEx), which is the same basic approach as my HalfKey project. Whenever a non-focused window (WinAPI:GetActiveWindow) is moused over (WinAPI:WindowFromPoint), it would activate it (gain the focus) (WinAPI:SetWindowLong/WinAPI:SetForegroundWindow). Unfortunately, as I found out, Windows is specifically designed to make an activated window go to the top of the z-order, and there is no way around this. The only solution I could find was recording the original position of the activated window (WinAPI:GetParent) and restoring it after the operation. This was however less than optimal :-(.

After some additional research on the topic, I found out the FFM functionality is actually built into Windows through a little known “System Parameter” (WinAPI:SystemParametersInfo). Microsoft calls it “Active Window Tracking”. Below is the code to the FocusFollowsMouse.exe (command line executable) to activate this system feature (the executable can be run without the command line and it will use the default options).

• -h The help screen (cancels further actions)
• -i installs new settings permanently
• -d disables active window tracking
• -r raise the window when focusing

//Toggles active window tracking, with options
#include <windows.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
	BOOL SendUpdate=SPIF_SENDCHANGE; //How to send the update to the system. Default is to only send to the system, but not make the change permanent
	BOOL TurnOn=TRUE; //By default we are turning on the option
	BOOL RaiseWindow=FALSE; //Whether to raise the window that is gaining the focus
	int i;

	//Read in options
	for(i=1;i<argc;i++)
	{
		if(argv[i][0]!='-') //Parameters must start with a -
			continue;
		switch(argv[i][1]) //Execute a valid action on the first character. Characters after it are ignored
		{
			case 'h': //Help  screen
				printf("%s", "Turns on active window tracking in Windows\n"
					"-h this help screen (cancels further actions)\n"
					"-i installs new settings permanently\n"
					"-d disables active window tracking\n"
					"-r raise the window when focusing\n");
					return 0; //Cancel further actions
			case 'i': //Install permanently
				SendUpdate|=SPIF_UPDATEINIFILE; //Writes the new system-wide parameter setting to the user profile.
				break;
			case 'd': //Turn off76
				TurnOn=FALSE;
				break;
			case 'r': //Raise the window
				RaiseWindow=TRUE;
				break;
		}
	}

	//Execute the system parameters change
	SystemParametersInfo(SPI_SETACTIVEWINDOWTRACKING, 0, (PVOID)TurnOn, SendUpdate);
	SystemParametersInfo(SPI_SETACTIVEWNDTRKZORDER, 0, (PVOID)RaiseWindow, SendUpdate);

	return 0;
}

I finally gave in and got myself a Motorola Droid phone, as I’ve been meaning to do for quite some time, but was reluctant to do so due to some known problems with it. I have been incredibly happy with it, and find it leaps and bounds better than the iPhone platform (and many other smart phones).

• It’s on the Verizon network, which, from my experience, has by far the best coverage and quality
• The Android platform keeps things running in the background so you can go in and out of applications freely, as opposed to only being able to run 1 application at a time on the iPhone. This multitasking approach is much better because...
  • This means that applications can continue to run/process in the background for necessary tasks, like periodically talking and receiving data from servers (without push notifications, which aren’t as powerful)
  • Applications, since they remain open, take no extra time to start back up every time you want to go to them
  • Keeping applications open has (as I understand it) next to zero, if not zero, drain on system resources if they are not doing anything in the background
  • If the phone is running out of memory, it will kill an application that is not in use
• I find the touchscreen much more responsive and accurate than the iPhone
• The voice recognition is spectacular (especially useful when combined with GPS/mapping programs)
• It’s not Apple :-) (and it’s Google!)
• It has a physical qwerty keyboard (see Cons)
• The lock screen protection is pretty neat and quick (gesture-like unlocking)
• The platform is open source, which has many powerful advantages for programmers
• Music/books/etc are directly loaded from an SD Card with no need to go through something like iTunes. This can, however, be slow when indexing from the SD card after changes
• Music sorts first by folder and then by album. While this would have been great for me a year or so ago when I got my iTouch, it no longer is as useful as I have already gone through my whole music folder and properly re-tagged everything (and it has actually caused a few bugs in my music organization)
• Free tethering to a computer through PDANet (1 month trial that blocks HTTPS sites afterwards, but I SSL tunnel my connections anyways :-) ). [Edit on 5/25/2010]: Google has just announced Android 2.2 has a new added feature “a portable Wi-Fi hotspot that can be shared with up to 8 devices.”
• Great [5.0 megapixel] camera/video recorder
• Applications can use self signed certificates for distribution. Signed applications prevent tampering (which has both advantages and disadvantages :-) ), and self signing means you can develop for the platform for free

• It cannot be on the Internet and a phone call at the same time (Verizon [CDMA] problem). This is especially problematic when tethering
• Even though the system is Linux based, working with the terminal emulator leaves a lot to be desired
  • Most of my standard bash commands [coreutils] are not there
  • Permissions are so tight you can barely do anything without rooting
  • The shell itself is missing a lot of functionality (problems with finding files even though they are in the “path”, bash-type arrow shortcuts [autofill, history], etc)
  • Motorola has seemingly [possibly?] even removed some of the utilities that should have come with the Android platform (they are on the Android emulator at least) like the command line SQLite program
• The physical keyboard is a lot harder to use than my last phone (VX9800, I’ve been using one since 2004), and also leaves a lot to be desired. It’s better than having no keyboard though!
  • Since there is no spacing between the keys, it is very easy to make a mistake
  • The number of keys is very small and not always in typical qwerty layout (for numbers and symbols), so it’s hard to find some characters
  • Some characters (like the pipe | ) are not even listed, and you just have to know how to get to them (alt+shift+comma for the pipe)
• The phone does not let you record call conversations. This seems to be a hardware issue specific to the Droid (and possibly the Nexus One) and not necessarily because of the Android platform. I cannot find any official or confirmable information on the reason. [Edited this bullet on 3/1/2010]
• It’s made to interface with Java when programming, and you don’t have much of a choice with this :-( . However, C/C++ modules can be made that directly interface easily with the Java base.

Anywho, the original intent of this post was not to talk about the Droid itself, but I figured I might as well get that out of the way while I was on the topic :-). The real point was to talk about a problem I ran into and the solution.

I had a lot of trouble finding a way to quickly set a ringtone to many people. As I have many hundreds of contacts in my phone, and I need groups of people set to different ringtones (and sometimes just one individual), it would have taken many hours to accomplish the task of assigning ringtones to these contacts. I found the perfect solution though in an application, not on the market, called JUSTones (also called “Just Tones”). Not only does it allow you to set ringtones for many people at once, but it also lets you set SMS tones for contacts (Yay!), which is not native to the platform.

Unfortunately, it was made for a depreciated version of the Android platform, and did not work on my phone. I have been in communication with the author, and after sending him the solution to my main problem, he has sent me a new working version (with many more fixes for bugs I mentioned too) ^_^. He even refunded my original purchase and told me to not worry about paying for it, which was pretty cool of him. It’s nice to find other software authors as nice and willing to work with you as this :-).

The problem was that the ringtones being set through the program were not actually being set to the contacts. The reason was that ringtone values are now set in 2 separate places in the OS, and only 1 of them was actually being used. I presume the old location of the value was still being set so legacy applications could still read the ringtone for the contact, but not necessarily set it.

The following is the code I sent to him that copies over the values from the old value location to the new one.

//Prepare to read from the old contacts database
Cursor cur=managedQuery(People.CONTENT_URI, new String[] { People.NAME, People.CUSTOM_RINGTONE }, null, null, People.NAME + " ASC"); //Query the old contacts database
cur.moveToFirst(); //Move to the first record of the old contacts database

//Prepare queries to run on new contacts database
ArrayList<ContentProviderOperation> ops = new ArrayList(); //This holds the queries to run
do //Loop over each [old] record of the database
{
	String TheName=String.valueOf(cur.getString(cur.getColumnIndex(People.NAME))); //Get the name from the old record
	String TheRingtone=cur.getString(cur.getColumnIndex(People.CUSTOM_RINGTONE)); //Get the ringtone from the old record
	ops.add(ContentProviderOperation.newUpdate(ContactsContract.Contacts.CONTENT_URI) //Prepare a query on the new contacts database
		.withSelection(ContactsContract.Contacts.DISPLAY_NAME + "=?", new String[]{TheName}) //On any record where the Name = OLD_NAME
		.withValue(ContactsContract.Contacts.CUSTOM_RINGTONE, TheRingtone) //Set Ringtone = OLD_RINGTONE
		.build());
} while (cur.moveToNext());

//Execute the queries on the new contacts database
try
{
	getContentResolver().applyBatch(ContactsContract.AUTHORITY, ops);
} catch(Exception e) { }

My domain has undergone many changes over the years, a few of which I have cataloged below. I found these old home pages as I’ve been going back through all my old websites and their scripts, and consolidating and organizing the information. Hopefully there will be more stuff to follow from this effort, including more web scripts.

As mentioned in other posts, my online handle before Dakusan was DarkSide. I have also been using Hyrulean Productions as my company name for, I believe, about 12 years, but I often misspelled it, oops.

FREE SPACE IS NO LONGER OFFERED. SORRY.
All sites below are now closed and remain viewable for archiving and history purposes.
.
This page designed for 1024x768 resolution at max colors.

RABiD BUNNY FEVER

This one was done by Adam Shen, who is the designer for most other websites I have worked on.

As you may have noticed, I took his original layered-design and turned it into an animated flash, viewable on my current home page (no www).

To continue the subject in my last post, these next cross-window bugs also derive from objects not being recognized properly when being passed between windows in JavaScript.

I needed the ability to dynamically run functions in the secondary window form the primary window where the parameters are taken from an array. Since a “function” from a secondary window is not seen as a function object from the primary window in IE, the apply member was not working.

I have included a fix for this below in the “RunFunctionInRemoteWindow” function, which is just a wrapper function in the second window that calls the apply function. This function manually copies the array through a for loop, instead of using slice, because in IE7 (but not IE8), the passed arrays were not seen as valid JSObjects, so the slice method (which is a standard method used for copying arrays by value) was not working.

<html><body>
<input type=button onclick="RunTest();" value='Click me when the second window has opened to run the test'>
<script type="text/javascript">

//Spawn the second window
var NewWindow=window.open('RemoteWindow.html');

//Run the test
function RunTest()
{
	LocalAlertString('This is an alert generated from the local window');
	NewWindow.RemoteAlertString('This is an alert generated from the remote window');
	alert('The local window alert function is of type function: '+(LocalAlertString instanceof Function));
	alert('The remote window alert function is of type function: '+(NewWindow.RemoteAlertString instanceof Function));
	LocalAlertString.apply(window, ['This is an alert generated from the local window through the APPLY member']);

	try {
	NewWindow.RemoteAlertString.apply(NewWindow.window, ['This is an alert generated from the remote window through the APPLY member. This will not work in IE because the remote window\'s function is not actually a function.']);
	} catch(e) { alert('The REMOTE APPLY failed: '+e.message); }

	NewWindow.RunFunctionInRemoteWindow('RemoteAlertString', ['This is an alert generated from the remote window through the FIXED APPLY function.']);
}

//Generate an alert in the local window
function LocalAlertString(TheString)
{
	alert('Local String: '+TheString);
}

</script></body></html>

<html><body><script type="text/javascript">
//Generate an alert in the remote window
function RemoteAlertString(TheString)
{
	alert('Remote String: '+TheString);
}

//Call functions in this window remotely through the "apply" member
function RunFunctionInRemoteWindow(FunctionName, Parameters)
{
	//Manually copy the passed Parameters since "Parameters" may not be a valid JSObject anymore (this could be detected and array.slice used if it is still valid)
	var ParametersCopy=[];
	for(var i=0;i<Parameters.length;i++)
		ParametersCopy[i]=Parameters[i];
	
	window[FunctionName].apply(window, ParametersCopy);
}
</script></body></html>

I was doing some research around April of 2009 on JavaScript interaction between web browser windows. I was doing this because web browsers are starting to split off each tab/window into separate processes/threads (Firefox is lagging in this), which can lead to some useful new implementations in the browser world, including multithreading. I wanted to explore the interaction between these windows to make sure there were no caveats that might creep up if I decided to take advantage of this.

The first one I found was that each browser window has its own instance of all of the base object classes, so prototypes do not carry over, and instanceof will not work as expected.

For example, if in WindowOne, you add a prototype to the Array class called IsArray, it is only accessible by arrays created in WindowOne. If you pass an array created in WindowOne into a second window, the prototype is still available on that one array (IIRC this was not true of some of the browsers at the time, but I tested again today, and it worked for IE8, Firefox3, and Google Chrome). Also, since the base object class in Window1 and other windows are not the same, an object created in Window1 and passed to another window will return false in a instanceof Object operation in that other window.

<html><body>
<input type=button onclick="RunTest();" value='Click me when the second window has opened to run the test'>
<script type="text/javascript">
Array.prototype.IsArray=true;
var NewWindow=window.open('RemoteWindow.html'); //Spawn the second window
function RunTest() { NewWindow.RunTest({}, [], new ExampleObject()); }; //Send the test data to remote window
function ExampleObject() { } //An example class
</script></body></html>

<html><body><script type="text/javascript">
function RunTest(AnObject, AnArray, AnExampleObject)
{
   var MyTests=[
      'AnObject instanceof Object',
      'AnObject.IsArray',                               //Object.prototype does not have this (Array.prototype does)
      'AnArray instanceof Object',
      'AnArray instanceof Array',
      'AnArray.IsArray',                                //This was added to the Array.prototype in the parent window
      'AnArray instanceof opener.Array',                //This example does not work in IE7 because opener.x cannot be properly accessed
      'AnExampleObject instanceof opener.ExampleObject',//This example does not work in IE7 because opener.x cannot be properly accessed
      'AnExampleObject instanceof ExampleObject'        //This test should error because "ExampleObject" does not exist in this window
   ];
   
   for(var i=0;i<MyTests.length;i++) //This runs each test like the following: alert("TEST: "+(TEST));
      try {
         eval('alert("'+MyTests[i]+': "+('+MyTests[i]+'));');
      } catch(e) {
         alert('Error on test "'+MyTests[i]+'": '+(e.hasOwnProperty('message') ? e.message : e.toString()));
      }
}
</script></body></html>

I’ve had these 2 sitting around my hard drive for a day when I didn’t have time to get something else up to post :-).

HTTP Status 500 -

---

type Exception report

message

description The server encountered an internal error () that prevented it from fulfilling this request.

exception

org.apache.jasper.JasperException

at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:207)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:240)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:187)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:200)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:146)
at com.papajohns.online.common.servlet.NoPersistantConnectionFilter.doFilter(NoPersistantConnectionFilter.java:18)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:146)
at com.papajohns.online.common.servlet.NoCacheFilter.doFilter(NoCacheFilter.java:20)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:146)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:209)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:948)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:144)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.valves.CertificatesValve.invoke(CertificatesValve.java:199)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:948)
at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2358)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:133)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:118)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:116)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:534)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:948)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:127)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:948)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:152)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:595)

root cause

java.lang.NullPointerException

at org.apache.jsp.checkout_jsp._jspService(checkout_jsp.java:166)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:92)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:162)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:240)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:187)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:809)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:200)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:146)
at com.papajohns.online.common.servlet.NoPersistantConnectionFilter.doFilter(NoPersistantConnectionFilter.java:18)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:146)
at com.papajohns.online.common.servlet.NoCacheFilter.doFilter(NoCacheFilter.java:20)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:146)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:209)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:948)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:144)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.valves.CertificatesValve.invoke(CertificatesValve.java:199)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:948)
at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2358)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:133)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:118)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:116)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:534)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:948)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:127)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:948)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:152)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:595)

---

Apache Tomcat/4.1.31

I recently (like a week ago) decided to finally upgrade my ancient Trillian install because it had many problems that needed addressing (OTR problems, no plugins for some networks, etc).

I decided to test out the new Trillian (Astra), which I had been waiting to be released for over 3 years. While I was very impressed with it as it is much cleaner, better designed, has a better assortment of options, and seems much more stable than either the previous Trillian or the current version of Pidgin, I still decided not to go with it. I was very unhappy with the fact that it forced you to create an account with them and sign in to it on startup, connecting to their servers for the full time you are using their product. This makes me not want to trust their product, as doing stuff like this can show a trend that generally leads to worse things. They even openly admit to bouncing your IM messages off their servers for different [legitimate] reasons.

So I went with Pidgin instead, and am generally pleased with it. It does most everything I want, and well enough. I was able to configure it and find plugins to do most everything I needed, but there was one major thing missing. Trillian had a script host plugin that allowed executing actions triggered from events. I was very surprised that Pidgin did not have a similar plugin. I contemplated making my own fully fledged one, as it would be a great and useful project to many, but I just don’t think I have the time right now :-\.

This is why I went ahead and made my Status Hot Keys plugin; as it isn’t quite as full fledged and useful as a full scripting host for the community, but it does what I want well enough to fill in my remaining requirements from Pidgin.

This is a modification of the utility I made in yesterday’s post, chunk, and most of the information mentioned in that post applies to this one too. This utility instead writes bytes from STDIN to a certain byte offset of a preexisting file. Below is the source code for the result, which I call chunkwrite (Windows Executable).

Chunkwrite writes bytes to a file at a given offset from STDIN. The parameters are:
1) The file to write to
2) The byte offset to write at (hex is supported like 0xA)

//Copyright 2009 by Dakusan (http://www.castledragmire.com/Copyright). Licensed under Dakusan License v2.0 (http://www.castledragmire.com/Misc/Software_Licenses/Dakusan_License_v2.0.php).
//See http://www.castledragmire.com/Posts/chunkwrite for more information

#define __LARGE64_FILES
#include <stdio.h>
#include <stdlib.h> //strtoull

#ifdef WIN32 //STDIN only needs to be set to binary mode in windows
	#include <io.h> //_setmode
	#include <fcntl.h> //_O_BINARY
#endif

typedef unsigned long long UINT64;
const UINT64 MaxSizeToRead=1024*1024*10; //The maximum number of bytes to read at a time to our buffer (Must be < 2^31)

UINT64 GetNumberFromString(const char* S) //Extract both hexidecimal and decimal numbers from a string
{
	bool IsHex=S[0]=='0' && (S[1]|32=='x'); //If string starts as 0x, then is a hex number
	return strtoull(S+(IsHex ? 2 : 0), NULL, IsHex ? 16 : 10); //Hex number starts after 2 characters and uses base 16
}

int main(int argc, char *argv[], char *envp[])
{
	//Determine if proper number of parameters are passed, and if not, output help info
	if(argc!=3)
		return fprintf(stderr, "Chunkwrite writes bytes to a file at a given offset from STDIN. The parameters are:\n1) The file to write to\n2) The byte offset to write at (hex is supported like 0xA)\n") & 0;

	//Open the file to output to
	FILE *TheFile=fopen64(argv[1], "r+b");
	if(TheFile==NULL)
		return fprintf(stderr, "File not found or cannot open file\n") & 0;

	//Determine the requested start offset
	UINT64 Offset=GetNumberFromString(argv[2]);

	//Write the data 10MB at a time from STDIN to the file
	char *Buffer=new char[MaxSizeToRead];
	fseeko64(TheFile, Offset, SEEK_SET); //Seek to the beginning write offset of our file
	#ifdef WIN32 //STDIN only needs to be set to binary mode in windows
		_setmode(_fileno(stdin), _O_BINARY);
	#endif
	do
	{
		size_t AmountRead=fread(Buffer, 1, MaxSizeToRead, stdin); //Read the data from STDIN
		fwrite(Buffer, AmountRead, 1, TheFile); //Write the data to the file
	}
	while(!feof(stdin)); //Keep reading and writing until STDIN is complete

	//Cleanup
	delete[] Buffer;
	fclose(TheFile);
	return 1;
}