iGoogle Security Problems

For a company that stresses security...

I’ve recently been having problems using the Google Reader widget in iGoogle. Normally, when I clicked on an RSS Title, a “bubble” popped up with the post’s content. However recently when clicking on the titles, the original post’s source opened up in a new tab. I confirmed the settings for the widget were correct, so I tried to remember the last change I made in Firefox that could have triggered this problem, as it seems the problem was not widespread, and only occurred to a few other people with no solution found. I realized a little bit back that I had installed the HTTPS Everywhere Firefox plugin. As described on the EFF’s site “HTTPS Everywhere is a Firefox extension ... [that] encrypts your communications with a number of major websites”.

Once I disabled the plugin and found the problem went away, I started digging through Google’s JavaScript code with FireBug. It turns out the start of the problem was that the widgets in iGoogle are run in their own IFrames (which is a very secure way of doing a widget system like this). However, the Google Reader contents was being pulled in through HTTPS secure channels (as it should thanks to HTTPS Everywhere), while the iGoogle page itself was pulled in through a normal HTTP channel! Separate windows/frames/tabs cannot interact with each other through JavaScript if they are not part of the same domain and protocol (HTTP/HTTPS) to prevent Cross-site scripting hacks.

I was wondering why HTTPS Everywhere was not running iGoogle through an HTTPS channel, so I tried it myself and found out Google automatically redirects HTTPS iGoogle requests to non secure HTTP channels! So much for having a proper security model in place...

So I did a lot more digging and modifying of Google’s code to see if I couldn’t find out exactly where the problem was occurring and if it couldn’t be fixed with a hack. It seems the code to handle the RSS Title clicking is injected during the “onload” event of the widget’s IFrame. I believe this was the code that was hitting the security privilege error to make things not work. I attempted to hijack the Google Reader widget’s onload function and add special privileges using “netscape.security.PrivilegeManager.enablePrivilege”, but it didn’t seem to help the problem. I think with some more prodding I could have gotten it working, but I didn’t want to waste any more time than I already had on the problem.

The code that would normally be loaded into the widget’s IFrame window hooks the “onclick” event of all RSS Title links to both perform the bubble action and cancel the normal “click” action. Since the normal click action for the anchor links was not being canceled, the browser action of following the link occurred. In this case, the links also had a “target” set to open a new window/tab.

There is however a “fix” for this problem, though I don’t find it ideal. If you edit the “extensions\https-everywhere@eff.org\chrome\content\rules\GoogleServices.xml” file in your Firefox profile directory (most likely at “C:\Users\USERNAME\AppData\Roaming\Mozilla\Firefox\Profiles\PROFILENAME\” if running Windows 7), you can comment out or delete the following rule so Google Reader is no longer run through secure HTTPS channels:

<rule from="^http://(www\.)?google\.com/reader/" 
to="https://www.google.com/reader/"/>

That being said, I’ve been having a plethora of problems with Facebook and HTTPS Everywhere too :-\ (which it actually mentions might happen in its options dialog). You’d think the largest sites on the Internet could figure out how to get their security right, but either they don’t care (the more likely option), or they don’t want the encryption overhead. Alas.

Android is not as open as is advertised

Giving the impression of complete freedom

One of the main selling points for me for the Android platform was that, as I understood it, the system was supposed to be very “open” in nature. It would allow a programmer to create virtually any application imaginable that they wanted to for users, as long as the user’s security is maintained. This is, of course, the antithesis of Apple’s philosophy with the iPhone/iPod Touch. However, I find this much spouted openness to not be the case at all. Security permissions are way too tight across the board, especially regarding interfacing with the hardware, making many things next to impossible to accomplish. This is especially true when interfacing with the phone functionality. While a programmer is free to do what they want within the scope of their own application(s) and their GUIs, working with the rest of the system can be a major PITA, or even impossible.

Some of this functionality can be gained back with rooted (jail broken) phones, but it is not always easy (or completely safe) to get one’s phone to such a state. It was simple with the Android 2.0 platform, which I originally had on my Motorola Droid, but not so much with the v2.1 software. Version 2.1 is (currently) a major PITA to root, as it entails having to restore the phone to its original state first, losing everything on it (which can, of course, be restored manually). I also, at this point, do not consider it worth it putting in the time to build things for rooted-only phones as the market is much smaller, and I myself haven’t even bothered rooting my phone on the current Android version.

Anyone can also compile their own version of the Android platform as it is open source. This would be worth it, for example, if an organization wanted to distribute their own compilation with modifications internally. However, it doesn’t much help application programmers like myself that want to reach a wide audience. I am also under the impression that putting your own flavor of the Android platform on your phone would lose certain functionalities/things included by the image provided by the provider of the phone (usually cell phone network carriers).

I really like how they did one section of the security system, which is, allowing an application to request special permissions from the operating system. A user is informed of the requested permissions before they install an application from the market place. The main problem is, though, that so many permissions are unavailable that should be possible. Another major downside is that way too many applications request permissions that they shouldn’t be requesting. This leaves users high and dry with applications they consider critical only available if they accept things they don’t want to. For example, many programs request full internet access with no need for it. It would be great to be able to selectively turn off these permissions, but I doubt the option for this is going to happen. I’m going to do more research myself on if an application can be written to do this, but I am not going to get even the slightest hope up on this possibility.

There are even examples of listed permissions that cannot be accessed by user submitted applications! For example, the INJECT_EVENTS permission can only be used by applications signed with the same signature as the system. I was unable to find this tidbit of information anywhere in the Android documentation and references (or the Internet). This all goes back to the problem of the documentation being less than optimal, as it leaves out a lot of important information.

There are at least 3 applications ATM I wanted to create but could not due to permissions:

• Call recording: I have written on this previously, but this functionality is unavailable, and Google is not commenting as to why. There are also countless other applications that could use the ability to access a call’s audio. This functionality was available on some older versions of the Android platform (and there are applications out there that take advantage of this), but it seems unavailable on newer versions for 1 of 3 reasons:
  • Legal reasons: It’s illegal to record calls in some areas (which would be dumb to revoke accessing call audio because of this because it’s legal in so many other places, including where I live in Texas).
  • Technological reasons: Some phone manufacturers might have it so the audio never even makes it to the operating system (it’s kept on the phone’s radio stack).
  • Google reasons: They decided it was a feature they no longer wanted to support. The fact of the matter is the interface is provided by the platform to do this, but bugs have been introduced into it and it no longer seems to work.
• Automated call menu: I would love to make an application that created an automated call menu on the phone, which could include leaving messages. I would personally use this so I could keep my phone on when sleeping, allowing the phone to direct the caller to either my [local or remote] voice mail or to wake me up if it’s an emergency. This is not possible due to the inability to access a call’s audio, as is explained in the above unimplementable application, but I am betting that there would be many more permissions that would make this not possible.
• Global Key interception: I have somewhat solved this problem, as I will be explaining in a post most likely coming tomorrow.

Droid Impressions and Custom Ringtones

Yay, new Google phone

I finally gave in and got myself a Motorola Droid phone, as I’ve been meaning to do for quite some time, but was reluctant to do so due to some known problems with it. I have been incredibly happy with it, and find it leaps and bounds better than the iPhone platform (and many other smart phones).

Pros (when compared to other phones/platforms on the market):

• It’s on the Verizon network, which, from my experience, has by far the best coverage and quality
• The Android platform keeps things running in the background so you can go in and out of applications freely, as opposed to only being able to run 1 application at a time on the iPhone. This multitasking approach is much better because...
  • This means that applications can continue to run/process in the background for necessary tasks, like periodically talking and receiving data from servers (without push notifications, which aren’t as powerful)
  • Applications, since they remain open, take no extra time to start back up every time you want to go to them
  • Keeping applications open has (as I understand it) next to zero, if not zero, drain on system resources if they are not doing anything in the background
  • If the phone is running out of memory, it will kill an application that is not in use
• I find the touchscreen much more responsive and accurate than the iPhone
• The voice recognition is spectacular (especially useful when combined with GPS/mapping programs)
• It’s not Apple :-) (and it’s Google!)
• It has a physical qwerty keyboard (see Cons)
• The lock screen protection is pretty neat and quick (gesture-like unlocking)
• The platform is open source, which has many powerful advantages for programmers
• Music/books/etc are directly loaded from an SD Card with no need to go through something like iTunes. This can, however, be slow when indexing from the SD card after changes
• Music sorts first by folder and then by album. While this would have been great for me a year or so ago when I got my iTouch, it no longer is as useful as I have already gone through my whole music folder and properly re-tagged everything (and it has actually caused a few bugs in my music organization)
• Free tethering to a computer through PDANet (1 month trial that blocks HTTPS sites afterwards, but I SSL tunnel my connections anyways :-) ). [Edit on 5/25/2010]: Google has just announced Android 2.2 has a new added feature “a portable Wi-Fi hotspot that can be shared with up to 8 devices.”
• Great [5.0 megapixel] camera/video recorder
• Applications can use self signed certificates for distribution. Signed applications prevent tampering (which has both advantages and disadvantages :-) ), and self signing means you can develop for the platform for free

Cons:

• It cannot be on the Internet and a phone call at the same time (Verizon [CDMA] problem). This is especially problematic when tethering
• Even though the system is Linux based, working with the terminal emulator leaves a lot to be desired
  • Most of my standard bash commands [coreutils] are not there
  • Permissions are so tight you can barely do anything without rooting
  • The shell itself is missing a lot of functionality (problems with finding files even though they are in the “path”, bash-type arrow shortcuts [autofill, history], etc)
  • Motorola has seemingly [possibly?] even removed some of the utilities that should have come with the Android platform (they are on the Android emulator at least) like the command line SQLite program
• The physical keyboard is a lot harder to use than my last phone (VX9800, I’ve been using one since 2004), and also leaves a lot to be desired. It’s better than having no keyboard though!
  • Since there is no spacing between the keys, it is very easy to make a mistake
  • The number of keys is very small and not always in typical qwerty layout (for numbers and symbols), so it’s hard to find some characters
  • Some characters (like the pipe | ) are not even listed, and you just have to know how to get to them (alt+shift+comma for the pipe)
• The phone does not let you record call conversations. This seems to be a hardware issue specific to the Droid (and possibly the Nexus One) and not necessarily because of the Android platform. I cannot find any official or confirmable information on the reason. [Edited this bullet on 3/1/2010]
• It’s made to interface with Java when programming, and you don’t have much of a choice with this :-( . However, C/C++ modules can be made that directly interface easily with the Java base.

Anywho, the original intent of this post was not to talk about the Droid itself, but I figured I might as well get that out of the way while I was on the topic :-). The real point was to talk about a problem I ran into and the solution.

I had a lot of trouble finding a way to quickly set a ringtone to many people. As I have many hundreds of contacts in my phone, and I need groups of people set to different ringtones (and sometimes just one individual), it would have taken many hours to accomplish the task of assigning ringtones to these contacts. I found the perfect solution though in an application, not on the market, called JUSTones (also called “Just Tones”). Not only does it allow you to set ringtones for many people at once, but it also lets you set SMS tones for contacts (Yay!), which is not native to the platform.

Unfortunately, it was made for a depreciated version of the Android platform, and did not work on my phone. I have been in communication with the author, and after sending him the solution to my main problem, he has sent me a new working version (with many more fixes for bugs I mentioned too) ^_^. He even refunded my original purchase and told me to not worry about paying for it, which was pretty cool of him. It’s nice to find other software authors as nice and willing to work with you as this :-).

The problem was that the ringtones being set through the program were not actually being set to the contacts. The reason was that ringtone values are now set in 2 separate places in the OS, and only 1 of them was actually being used. I presume the old location of the value was still being set so legacy applications could still read the ringtone for the contact, but not necessarily set it.

The following is the code I sent to him that copies over the values from the old value location to the new one.

//Prepare to read from the old contacts database
Cursor cur=managedQuery(People.CONTENT_URI, new String[] { People.NAME, People.CUSTOM_RINGTONE }, null, null, People.NAME + " ASC"); //Query the old contacts database
cur.moveToFirst(); //Move to the first record of the old contacts database

//Prepare queries to run on new contacts database
ArrayList<ContentProviderOperation> ops = new ArrayList(); //This holds the queries to run
do //Loop over each [old] record of the database
{
	String TheName=String.valueOf(cur.getString(cur.getColumnIndex(People.NAME))); //Get the name from the old record
	String TheRingtone=cur.getString(cur.getColumnIndex(People.CUSTOM_RINGTONE)); //Get the ringtone from the old record
	ops.add(ContentProviderOperation.newUpdate(ContactsContract.Contacts.CONTENT_URI) //Prepare a query on the new contacts database
		.withSelection(ContactsContract.Contacts.DISPLAY_NAME + "=?", new String[]{TheName}) //On any record where the Name = OLD_NAME
		.withValue(ContactsContract.Contacts.CUSTOM_RINGTONE, TheRingtone) //Set Ringtone = OLD_RINGTONE
		.build());
} while (cur.moveToNext());

//Execute the queries on the new contacts database
try
{
	getContentResolver().applyBatch(ContactsContract.AUTHORITY, ops);
} catch(Exception e) { }

Google Search API Failure

Google Search failed me again

I am once again disappointed by Google Search functionality, which has hindered my implementation of proper searching on this site :-(. (See relevant update).

The Google Search API returns a vastly stripped result set compared to using actual Google Search. I have checked and done a bit of research and have not found a good reason for this. And, no, it has nothing to do with local or personalized searches, which has been confirmed by using searches without any kind of cookies or localizations.

My guess is that the Google Search API and normal Google Search itself are just tapping into different result sets from the start :-(.

An example of this problem is as follows: Searching for “Fractal” in the Projects section returns the following results:

• Google API Search [actual background search]
  • Project: Fractal Landscape
  • Project: Eyes
• Google Search
  • Project: Fractal Landscape
  • Project: Eyes
  • Project List: Table View
  • Project List: Games

At some point I’ll probably have to find another site search solution for this reason, blegh. :-\

Google Search Removed Functionality

Has Google been removing search options to make things run faster?

I’ve been meaning to get searching working on my site for what seems like forever, and I decided to finally get around to getting some manner of search working via the temporary “use Google” solution. Unfortunately, it seems Google no longer does boolean searches completely properly as advertised. I am sure Google Search still supports boolean logic (as opposed to the assumed “and” between each word) because the Advanced Search, linked to from their front page, still has it, and it returns a few of the results it should.

As an example:
If I wanted to search the Projects and Updates sections of my sites for either the keywords fractal or font I would use the following search:

(site:www.castledragmire.com/Projects OR site:www.castledragmire.com/Updates) AND (Fractal OR Font)

This currently only returns 3 results, when it should return 11 different results, enumerated by using the 4 separate searches (with return results):

1. site:www.castledragmire.com/Projects Fractal
   • Projects
   • Projects/Fractal_Landscape
   • Projects/Julia_Fractals
   • Projects/Eyes
   • Projects/Games
2. site:www.castledragmire.com/Projects Font
   • Projects
   • Projects/ViewFonts
3. site:www.castledragmire.com/Updates Fractal
   • Updates/Fractal_Landscape_v0.5_and_v1.0
   • Updates/Fractal_Landscape_Technical_Paper_v1.0
   • Updates/Fractal_Landscape_v1.05
   • Updates/Julia_Fractals_Initial_Release
4. site:www.castledragmire.com/Updates Font
   • Updates/Site_Color_Scheme

Actual Google returned results:

• Projects
• Projects/Fractal_Landscape
• Projects/ViewFonts
  
  

A simple example of this through the Google Advanced Search Page is as follows:

Fractal OR Font site:www.castledragmire.com/Projects [Advanced Search]

Which only returns 3 results (following) instead of the 6 (see above) that it should:

• Projects
• Projects/Fractal_Landscape
• Projects/ViewFonts

Because of this, I need to go ahead and get real searching up via MySQL (or possibly another solution), as originally planned, sooner than later, since Google will not work as a temporary solution for what I want.

I wrote up a paper on what could be done through Google Search over 5 years ago as a job request [to be posted soon], which I believe is very informative. I’m sure it’s a little outdated, but it shows how much can Google can [could] do for you.

Chrome no longer doing separate processes

Google broke Chrome :-(

There were at least 3 really neat things about Google Chrome when it made its spectacular entrance onto the web browser market a few months ago that made it a really viable option compared to its competitors. These features were [“are”, going to write it in present tense as they are still true]:

1. It is fast, especially with JavaScript.
   • I have done speed tests on the JavaScript engines between browsers (which unfortunately I can’t post), and function calls, especially recursion, in the JavaScript engine in Chrome are incredibly faster when compared to the other Web Browsers.
   • However, SpiderMonkey, the new JavaScript engine being used in Firefox, seriously kicks all the other browsers in the butt in speed optimizations when it comes to loop iterations and some other areas. SpiderMonkey is available in the newest non-stable builds of Firefox (v3.1), but is not turned on by default.
2. Different tabs run in different processes; which was very heavily advertised during Chrome’s launch. This carries with it two great advantages.
   1. A locked or crashed tab/window (usually through JavaScript) won’t affect the other tabs/windows.
   2. Since each tab is in a separate OS process, meaning they are also being run on separate OS threads, they can be run on separate logical operating cores (CPUs). This means that browser tabs can be run in parallel and not slow each other down (depending on the number of logical CPUs you have).

   Unfortunately, this is not as completely true as is widely advertised. New processes are only opened when the user manually opens a new window or tab. If a new window or tab is opened by JavaScript or by clicking a link, it still runs in the same process!

   Google has a FAQ Entry on this as follows:

   16. How can my web page open a new tab in a separate process?

   Google Chrome has a multi-process architecture, meaning tabs can run in separate processes from each other, and from the main browser process. New tabs spawned from a web page, however, are usually opened in the same process, so that the original page can access the new tab using JavaScript.

   If you’d like a new tab to open in a separate process:

   • Open the new tab with about:blank as its target.
   • Set the newly opened tab’s opener variable to null, so that it can’t access the original page.
   • Redirect from about:blank to any URL on a different domain, port, or protocol than that of the page spawning the pop-up. For example, if the page spawning the pop-up is on http://www.example.com/:
     • a different domain would be http://www.example.org
     • a different port would be http://www.example.com:8080
     • a different protocol would be https://www.example.com

   Google Chrome will recognize these actions as a hint that the new and old pages should be isolated from each other, and will attempt to load the new page in a separate process.

   The following code snippet can be used to accomplish all of these steps:

   var w = window.open();
   w.opener = null;
   w.document.location = "http://different.example.com/index.html";
   			

   
   

   The only problem is... THIS NO LONGER WORKS! Google recently (within the last 7 days) broke this FAQ recommendation with an automatic update to Chrome, so new tabs that are not manually opened by the user cannot be forced to new processes even with their little code snippet. Personally, I think this behavior is really lame and every tab should be able to open in separate processes every time no matter what, and still be able to talk to each other through process message passing. It may slow things down a little, but it’s a much more powerful model, IMO. An option for this in the window.open’s options parameter would be really nice...

3. And of course, it’s Google, who, in general, “does no evil”. :-)
   • I can’t find the original article I was looking for on this “don’t do evil” topic :’( ... it basically said something to the extent that the “don’t be evil” motto only applies to business inside the USA, or something like that.
   • I have been a long time fan of Google though, and I still think that pretty much everything they’ve done, in general, has been for the good of everyone. There are always going to be blemishes on a company that size, and for how big they are and all they do, they’ve done a pretty damn good job, IMO. Just my two cents.