Home Page
Archive > Posts > 2008 > May
Search:

Online credit card misinformation
Check your gut suspicions before acting

I was just doing my accounting and I noticed I had 3 double-charges on my Capital One credit card that all happened within a 2 day period. I found this to be very odd since I have never been double-charged on any of my credit cards since I started using them 10 years ago when I was 14.

So I went ahead and submitted 2 charge disputes with Capital One, and a third with the other company I saw double-charged. I then finished my accounting, and noticed that the balance showing up on my Capital One did not include those 3 charges. I validated my suspicions by calling up their customer relations department (getting a lady in India) and confirming that the charges only show up once in my account.

I then did my emails to rescind my previous queries into having the double-charges refunded, and also included in the email to Capital One that their web system (or possibly statement system) has an error and needs to be fixed. The double-charges actually weren’t showing up on the same statements. They showed up once (for May 16th and 17th) on my last month’s statement, and then again (May 17th and 19th) on my current month’s statement. Go Figure.


[Edit on 6/13/08] A few days ago, after an annoying downtime on the Capitol One credit card site, I noticed they added a new feature that now shows your latest charges within a certain period of days (15, 30, etc) instead of just the current billing cycle. So I’m pretty sure the above problem was due to them implementing this new system without warning the user or having any indication of the system change in the interface. I do know how annoying change control is, and the problems that come along with implementing new features on websites which may temporarily confuse users, but I’d expect better from a multinational corporation like this. Then again, this isn’t the first time this kind of thing has happened on their website, so I shouldn’t be surprised.
Project About Pages
Big things come in small packages
About Window Concept

I’ve been thinking for a while that I need to add “about windows” to the executables of all my applications with GUIs. So I first made a test design [left, psd file attached]

Unfortunately, this requires at least 25KB for the background alone, and this is larger than many of my project executables themselves. This is a problem for me, as I like keeping executables small and simple.

PNG Signature I therefore decided to scratch the background and just go with normal “about windows” and my signature in a spiffy font [BlizzardD]: (white background added by web browser for visibility)
The above PNG signature is only 1.66KB, so “yay”, right? Wrong :-(, it quickly occurred to me that XP does not natively support PNG.

GIF SignatureMy next though is “what about a GIF?” (GIF is the predecessor to PNG, also lossless): (1.82KB)
I remembered that GIF files worked for VB, so I thought that native Windows’ API might support it too without adding in extra DLLs, but alas, I was wrong. This at least partially solves the problem for me in Visual Basic, but not fully, as GIF does not support translucency, but only 1 color of transparency, so the picture would look horribly aliased (pixilated).

The final solution I decided on is having a small translucency-mask and alpha-blending it and the primary signature color (RGB(6,121,6)) to the “about windows’ ” background.
GIF Signature MaskSince alpha-blending/translucency is an 8 bit value, a gray-scale (also 8 bits per pixel) image is perfect for a translucency mask format for VB: (1.82KB, GIF)
You may note that this GIF is the exact same size as the previous GIF, which makes sense as it is essentially the exact same picture, just with swapped color palettes.

The final hurdle is how to import the picture into C with as little space wasted as possible. The solution to this is to create an easily decompressable alpha-mask (alpha means translucency).
BMP Signature Mask I started with the bitmap mask: (25.6KB, BMP)
From there, I figured there would be 2 easy formats for compression that would take very little code to decompress:
  • Number of Transparent Pixels, Number of Foreground Pixels in a Row, List of Foreground Pixel Masks, REPEAT... (This is a form of “Run-length encoding”)
  • Start the whole image off as transparent, and then list each group of foreground pixels with: X Start Coordinate, Y Start Coordinate, Number of Pixels in a Row, List of Foreground Pixel Masks
It also helped that there were only 16 different alpha-masks, not including the fully transparent mask, so each alpha-mask could be fit within half a byte (4 bits). I only did the first option because I’m pretty sure the second one would be larger because it would take more bits for an x/y location than for a transparent run length number.

Other variants could be used too, like counting the background as a normal mask index and just do straight run length encoding with indexes, but I knew this would make the file much larger for 2 reasons: this would add a 17th alpha-mask which would push index sizes up to 5 bits, and background run lengths are much longer (in this case 6 bits), so all runs would need to be longer (non-background runs are only 3 bits in this case). Anyways, it ended up creating a 1,652 byte file :-).


This could also very easily be edited to input/output 8-bit indexed bitmaps, or full color bitmaps even (with a max of 256 colors, or as many as you wanted with a few more code modifications). If one wanted to use this for normal pictures with a solid background instead of an alpha-mask, just know the words “Transparent” means “Background” and “Translucent” means “Non-Background” in the code.

GIF and PNG file formats actually use similar techniques, but including the code for their decoders would cause a lot more code bloat than I wanted, especially since they [theoretically] include many more compression techniques than just run-length encoding. Programming for specific cases will [theoretically] always be smaller and faster than programming for general cases. On a side note, from past research I’ve done on the JPEG format, along with programming my NES Emulator, Hynes, they [JPEG & NES] share the same main graphical compression technique [grouping colors into blocks and only recording color variations].


The following is the code to create the compressed alpha-mask stream: [Direct link to C file with all of the following code blocks]
//** Double stars denotes changes for custom circumstance [The About Window Mask]
#include <windows.h>
#include <stdio.h>
#include <conio.h>

//Our encoding functions
int ErrorOut(char* Error, FILE* HandleToClose); //If an error occurs, output
UINT Encode(UCHAR* Input, UCHAR* Output, UINT Width, UINT Height); //Encoding process
UCHAR NumBitsRequired(UINT Num); //Tests how many bits are required to represent a number
void WriteToBits(UCHAR* StartPointer, UINT BitStart, UINT Value); //Write Value to Bit# BitStart after StartPointer - Assumes more than 8 bits are never written

//Program constants
const UCHAR BytesPerPixel=3, TranspMask=255; //24 bits per pixel, and white = transparent background color

//Encoding file header
typedef struct
{
	USHORT DataSize; //Data size in bits - **Should be UINT
	UCHAR Width, Height; //**Should be USHORTs
	UCHAR TranspSize, TranslSize; //Largest number of bits required for a run length for Transp[arent] and Transl[ucent]
	UCHAR NumIndexes, Indexes[0]; //Number and list of indexes
} EncodedFileHeader;

int main()
{
	UCHAR *InputBuffer, *OutputBuffer; //Where we will hold our input and output data
	FILE *File; //Handle to current input or output file
	UINT FileSize; //Holds input and output file sizes

	//The bitmap headers tell us about its contents
	BITMAPFILEHEADER BitmapFileHead;
	BITMAPINFOHEADER BitmapHead;

	//Read in bitmap header and confirm file type
	File=fopen("AboutWindow-Mask.bmp", "rb"); //Normally you'd read in the filename from passed arguments (argv)
	if(!File) //Confirm file open
		return ErrorOut("Cannot open file for reading", NULL);
	fread(&BitmapFileHead, sizeof(BITMAPFILEHEADER), 1, File);
	if(BitmapFileHead.bfType!=*(WORD*)"BM" || BitmapFileHead.bfReserved1 || BitmapFileHead.bfReserved2) //Confirm we are opening a bitmap
		return ErrorOut("Not a bitmap", File);

	//Read in the rest of the data
	fread(&BitmapHead, sizeof(BITMAPINFOHEADER), 1, File);
	if(BitmapHead.biPlanes!=1 || BitmapHead.biBitCount!=24 || BitmapHead.biCompression!=BI_RGB) //Confirm bitmap type - this code would probably have been simpler if I did an 8 bit indexed file instead... oh well, NBD.  **It has also been programmed for easy transition to 8 bit indexed files via the "BytesPerPixel" constant.
		return ErrorOut("Bitmap must be in 24 bit RGB format", File);
	FileSize=BitmapFileHead.bfSize-sizeof(BITMAPINFOHEADER)-sizeof(BITMAPFILEHEADER); //Size of the data portion
	InputBuffer=malloc(FileSize);
	fread(InputBuffer, FileSize, 1, File);
	fclose(File);

	//Run Encode
	OutputBuffer=malloc(FileSize); //We should only ever need at most FileSize space for output (output should always be smaller)
	memset(OutputBuffer, 0, FileSize); //Needs to be zeroed out due to how writing of data file is non sequential
	FileSize=Encode(InputBuffer, OutputBuffer, BitmapHead.biWidth, BitmapHead.biHeight); //Encode the file and get the output size

	//Write encoded data out
	File=fopen("Output.msk", "wb");
	fwrite(OutputBuffer, FileSize, 1, File);
	fclose(File);
	printf("File %d written with %d bytes\n", 1, FileSize);

	//Free up memory and wait for user input
	free(InputBuffer);
	free(OutputBuffer);
	getch(); //Pause for user input
	return 0;
}

int ErrorOut(char* Error, FILE* HandleToClose) //If an error occurs, output
{
	if(HandleToClose)
		fclose(HandleToClose);
	printf("%s\n", Error);
	getch(); //Pause for user input
	return 1;
}

UINT Encode(UCHAR* Input, UCHAR* Output, UINT Width, UINT Height) //Encoding process
{
	UCHAR Indexes[256], NumIndexes, IndexSize, RowPad; //The index re-mappings, number of indexes, number of bits an index takes in output data, padding at input row ends for windows bitmaps
	USHORT TranspSize, TranslSize; //Largest number of bits required for a run length for Transp[arent] (zero) and Transl[ucent] (non zero) - should be UCHAR's, but these are used as explained under "CurTranspLen" below
	UINT BitSize, x, y, ByteOn, NumPixels; //Current output size in bits, x/y coordinate counters, current byte location in Input, number of pixels in mask

	//Calculate some stuff
	NumPixels=Width*Height; //Number of pixels in mask
	RowPad=4-(Width*BytesPerPixel%4); //Account for windows DWORD row padding - see declaration comment
	RowPad=(RowPad==4 ? 0 : RowPad);

	{ //Do a first pass to find number of different mask values, run lengths, and their encoded values
		const UCHAR UnusedIndex=255; //In our index list, unused indexes are marked with this constant
		USHORT CurTranspLen, CurTranslLen; //Keep track of the lengths of the current transparent & translucent runs - TranspSize and TranslSize are temporarily used to hold the maximum run lengths
		//Zero out all index references and counters
		memset(Indexes, UnusedIndex, 256);
		NumIndexes=0;
		TranspSize=TranslSize=CurTranspLen=CurTranslLen=0;
		//Start gathering data
		for(y=ByteOn=0;y<Height;y++) //Column
		{
			for(x=0;x<Width;x++,ByteOn+=BytesPerPixel) //Row
			{
				UCHAR CurMask=Input[ByteOn]; //Curent alpha mask
				if(CurMask!=TranspMask) //Translucent value?
				{
					//Determine if index has been used yet
					if(Indexes[CurMask]==UnusedIndex) //We only need to check 1 byte per pixel as they are all the same for gray-scale **This would need to change if using non 24-bit or non gray-scale
					{
						((EncodedFileHeader*)Output)->Indexes[NumIndexes]=CurMask; //Save mask number in the index header
						Indexes[CurMask]=NumIndexes++; //Save index number to the mask
					}

					//Length of current transparent run
					TranspSize=(CurTranspLen>TranspSize ? CurTranspLen : TranspSize); //Max(CurTranspLen, TranspSize)
					CurTranspLen=0;

					//Length of current translucent run
					CurTranslLen++;
				}
				else //Transparent value?
				{
					//Length of current translucent run
					TranslSize=(CurTranslLen>TranslSize ? CurTranslLen : TranslSize);  //Max(CurTranslLen, TranslSize)
					CurTranslLen=0;

					//Length of current transparent run
					CurTranspLen++;
				}
			}

			ByteOn+=RowPad; //Account for windows DWORD row padding
		}
		//Determine number of required bits per value
		printf("Number of Indexes: %d\nLongest Transparent Run: %d\nLongest Translucent Run: %d\n", NumIndexes,
			TranspSize=CurTranspLen>TranspSize ? CurTranspLen : TranspSize, //Max(CurTranspLen, TranspSize)
			TranslSize=CurTranslLen>TranslSize ? CurTranslLen : TranslSize  //Max(CurTranslLen, TranslSize)
			);
		IndexSize=NumBitsRequired(NumIndexes);
		TranspSize=NumBitsRequired(TranspSize); //**This is currently overwritten a few lines down
		TranslSize=NumBitsRequired(TranslSize); //**This is currently overwritten a few lines down
		printf("Bit Lengths of - Indexes, Trasparent Run Length, Translucent Run Length: %d, %d, %d\n", IndexSize, TranspSize, TranslSize);
	}

	//**Modify run sizes (custom) - this function could be run multiple times with different TranspSize and TranslSize until the best values are found - the best values would always be a weighted average
	TranspSize=6;
	TranslSize=3;

	//Start processing data
	BitSize=(sizeof(EncodedFileHeader)+NumIndexes)*8; //Skip the file+bitmap headers and measure in bits
	x=ByteOn=0;
	do
	{
		//Transparent run
		UINT CurRun=0;
		while(Input[ByteOn]==TranspMask && x<NumPixels && CurRun<(UINT)(1<<TranspSize)-1) //Final 2 checks are for EOF and capping run size to max bit length
		{
			x++;
			CurRun++;
			ByteOn+=BytesPerPixel;
			if(x%Width==0) //Account for windows DWORD row padding
				ByteOn+=RowPad;
		}
		WriteToBits(Output, BitSize, CurRun);
		BitSize+=TranspSize;

		//Translucent run
		CurRun=0;
		BitSize+=TranslSize; //Prepare to start writing masks first
		while(x<NumPixels && Input[ByteOn]!=TranspMask && CurRun<(UINT)(1<<TranslSize)-1) //Final 2 checks are for EOF and and capping run size to max bit length
		{
			WriteToBits(Output, BitSize+CurRun*IndexSize, Indexes[Input[ByteOn]]);
			x++;
			CurRun++;
			ByteOn+=BytesPerPixel;
			if(x%Width==0) //Account for windows DWORD row padding
				ByteOn+=RowPad;
		}
		WriteToBits(Output, BitSize-TranslSize, CurRun); //Write the mask before the indexes
		BitSize+=CurRun*IndexSize;
	} while(x<NumPixels);

	{ //Output header
		EncodedFileHeader *OutputHead;
		OutputHead=(EncodedFileHeader*)Output;
		OutputHead->DataSize=BitSize-(sizeof(EncodedFileHeader)+NumIndexes)*8; //Length of file in bits not including header
		OutputHead->Width=Width;
		OutputHead->Height=Height;
		OutputHead->TranspSize=(UCHAR)TranspSize;
		OutputHead->TranslSize=(UCHAR)TranslSize;
		OutputHead->NumIndexes=NumIndexes;
	}
	return BitSize/8+(BitSize%8 ? 1 : 0); //Return entire length of file in bytes
}

UCHAR NumBitsRequired(UINT Num) //Tests how many bits are required to represent a number
{
	UCHAR RetNum;
	_asm //Find the most significant bit
	{
		xor eax, eax //eax=0
		bsr eax, Num //Find most significant bit in eax
		mov RetNum, al
	}
	return RetNum+((UCHAR)(1<<RetNum)==Num ? 0 : 1); //Test if the most significant bit is the only one set, if not, at least 1 more bit is required
}

void WriteToBits(UCHAR* StartPointer, UINT BitStart, UINT Value) //Write Value to Bit# BitStart after StartPointer - Assumes more than 8 bits are never written
{
	*(WORD*)(&StartPointer[BitStart/8])|=Value<<(BitStart%8);
}

The code to decompress the alpha mask in C is as follows: (Shares some header information with above code)
//Decode
void Decode(UCHAR* Input, UCHAR* Output); //Decoding process
UCHAR ReadBits(UCHAR* StartPointer, UINT BitStart, UCHAR BitSize); //Read value from Bit# BitStart after StartPointer - Assumes more than 8 bits are never read
UCHAR NumBitsRequired(UINT Num); //Tests how many bits are required to represent a number --In Encoding Code--

int main()
{
	//--Encoding Code--
		UCHAR *InputBuffer, *OutputBuffer; //Where we will hold our input and output data
		FILE *File; //Handle to current input or output file
		UINT FileSize; //Holds input and output file sizes
	
		//The bitmap headers tell us about its contents
		//Read in bitmap header and confirm file type
		//Read in the rest of the data
		//Run Encode
		//Write encoded data out
	//--END Encoding Code--

	//Run Decode
	UCHAR* O2=(BYTE*)malloc(BitmapFileHead.bfSize);
	Decode(OutputBuffer, O2);

/*	//If writing back out to a 24 bit windows bitmap, this adds the row padding back in
	File=fopen("output.bmp", "wb");
	fwrite(&BitmapFileHead, sizeof(BITMAPFILEHEADER), 1, File);
	fwrite(&BitmapHead, sizeof(BITMAPINFOHEADER), 1, File);
	fwrite(O2, BitmapFileHead.bfSize-sizeof(BITMAPINFOHEADER)-sizeof(BITMAPFILEHEADER), 1, File);*/

	//Free up memory and wait for user input --In Encoding Code--
	return 0;
}

//Decoding
void Decode(UCHAR* Input, UCHAR* Output) //Decoding process
{
	EncodedFileHeader H=*(EncodedFileHeader*)Input; //Save header locally so we have quick memory lookups
	UCHAR Indexes[256], IndexSize=NumBitsRequired(H.NumIndexes); //Save indexes locally so we have quick lookups, use 256 index array so we don't have to allocate memory
	UINT BitOn=0; //Bit we are currently on in reading
	memcpy(Indexes, ((EncodedFileHeader*)Input)->Indexes, 256); //Save the indexes
	Input+=(sizeof(EncodedFileHeader)+H.NumIndexes); //Start reading input past the header

	//Unroll/unencode all the pixels
	do
	{
		UINT i, l; //index counter, length (transparent and then index)
		//Transparent pixels
		memset(Output, TranspMask, l=ReadBits(Input, BitOn, H.TranspSize)*BytesPerPixel);
		Output+=l;

		//Translucent pixels
		l=ReadBits(Input, BitOn+=H.TranspSize, H.TranslSize);
		BitOn+=H.TranslSize;
		for(i=0;i<l;i++) //Write the gray scale out to the 3 pixels, this should technically be done in a for loop, which would unroll itself anyways, but this way ReadBits+index lookup is only done once - ** Would need to be in a for loop if not using gray-scale or 24 bit output
			Output[i*BytesPerPixel]=Output[i*BytesPerPixel+1]=Output[i*BytesPerPixel+2]=Indexes[ReadBits(Input, BitOn+i*IndexSize, IndexSize)];
		Output+=l*BytesPerPixel;
		BitOn+=l*IndexSize;
	} while(BitOn<H.DataSize);

/*	{ //If writing back out to a 24 bit windows bitmap, this adds the row padding back in
		UINT i;
		UCHAR RowPad=4-(H.Width*BytesPerPixel%4); //Account for windows DWORD row padding
		RowPad=(RowPad==4 ? 0 : RowPad);
		Output-=H.Width*H.Height*BytesPerPixel; //Restore original output pointer
		for(i=H.Height;i>0;i--) //Go backwards so data doesn't overwrite itself
			memcpy(Output+(H.Width*BytesPerPixel+RowPad)*i, Output+(H.Width*BytesPerPixel)*i, H.Width*BytesPerPixel);
	}*/
}

UCHAR ReadBits(UCHAR* StartPointer, UINT BitStart, UCHAR BitSize) //Read value from Bit# BitStart after StartPointer - Assumes more than 8 bits are never read
{
	return (*(WORD*)&StartPointer[BitStart/8]>>BitStart%8)&((1<<BitSize)-1);
}

Of course, I added some minor assembly and optimized the decoder code to get it from 335 to 266 bytes, which is only 69 bytes less :-\, but it’s something (measured using my Small project). There is no real reason to include it here, as it’s in many of my projects and the included C file for this post.

And then some test code just for kicks...
//Confirm Decoding
BOOL CheckDecode(UCHAR* Input1, UCHAR* Input2, UINT Width, UINT Height); //Confirm Decoding

//---- Put in main function above "//Free up memory and wait for user input" ----
printf(CheckDecode(InputBuffer, O2, BitmapHead.biWidth, BitmapHead.biHeight) ? "good" : "bad");

BOOL CheckDecode(UCHAR* Input1, UCHAR* Input2, UINT Width, UINT Height) //Confirm Decoding
{
	UINT x,y,i;
	UCHAR RowPad=4-(Width*BytesPerPixel%4); //Account for windows DWORD row padding
	RowPad=(RowPad==4 ? 0 : RowPad);

	for(y=0;y<Height;y++)
		for(x=0;x<Width;x++)
			for(i=0;i<BytesPerPixel;i++)
				if(Input1[y*(Width*BytesPerPixel+RowPad)+x*BytesPerPixel+i]!=Input2[y*(Width*BytesPerPixel)+x*BytesPerPixel+i])
					return FALSE;
	return TRUE;
}

From there, it just has to be loaded into a bit array for manipulation and set back a bitmap device context, and it’s done!
VB Code: (Add the signature GIF as a picture box where it is to show up and set its “Visible” property to “false” and “Appearance” to “flat”)
'Swap in and out bits
Private Declare Function GetDIBits Lib "gdi32" (ByVal aHDC As Long, ByVal hBitmap As Long, ByVal nStartScan As Long, ByVal nNumScans As Long, lpBits As Any, lpBI As BITMAPINFOHEADER, ByVal wUsage As Long) As Long
Private Declare Function SetDIBitsToDevice Lib "gdi32" (ByVal hdc As Long, ByVal x As Long, ByVal y As Long, ByVal dx As Long, ByVal dy As Long, ByVal SrcX As Long, ByVal SrcY As Long, ByVal Scan As Long, ByVal NumScans As Long, Bits As Any, BitsInfo As BITMAPINFOHEADER, ByVal wUsage As Long) As Long
lpBits As Any, lpBitsInfo As BITMAPINFOHEADER, ByVal wUsage As Long, ByVal dwRop As Long) As Long
Private Type RGBQUAD
		b As Byte
		g As Byte
		r As Byte
		Reserved As Byte
End Type
Private Type BITMAPINFOHEADER '40 bytes
		biSize As Long
		biWidth As Long
		biHeight As Long
		biPlanes As Integer
		biBitCount As Integer
		biCompression As Long
		biSizeImage As Long
		biXPelsPerMeter As Long
		biYPelsPerMeter As Long
		biClrUsed As Long
		biClrImportant As Long
End Type
Private Const DIB_RGB_COLORS = 0 '  color table in RGBs

'Prepare colors
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)
Private Declare Function GetBkColor Lib "gdi32" (ByVal hdc As Long) As Long

Public Sub DisplaySignature(ByRef TheForm As Form)
    'Read in Signature
    Dim BitmapLength As Long, OutBitmap() As RGBQUAD, BitInfo As BITMAPINFOHEADER, Signature As PictureBox
    Set Signature = TheForm.Signature
    BitmapLength = Signature.Width * Signature.Height
    ReDim OutBitmap(0 To BitmapLength - 1) As RGBQUAD
    With BitInfo
            .biSize = 40
            .biWidth = Signature.Width
            .biHeight = -Signature.Height
            .biPlanes = 1
            .biBitCount = 32
            .biCompression = 0 'BI_RGB
            .biSizeImage = .biWidth * 4 * -.biHeight
    End With
    GetDIBits Signature.hdc, Signature.Image, 0, Signature.Height, OutBitmap(0), BitInfo, DIB_RGB_COLORS
    
    'Alpha blend signature
    Dim i As Long, Alpha As Double, BackColor As RGBQUAD, ForeColor As RGBQUAD, OBC As Long, OFC As Long
    OFC = &H67906
    OBC = GetBkColor(TheForm.hdc)
    CopyMemory BackColor, OBC, 4
    CopyMemory ForeColor, OFC, 4
    For i = 0 To BitmapLength - 1
        Alpha = 1 - (CDbl(OutBitmap(i).r) / 255)
        OutBitmap(i).r = ForeColor.r * Alpha + BackColor.r * (1 - Alpha)
        OutBitmap(i).g = ForeColor.g * Alpha + BackColor.g * (1 - Alpha)
        OutBitmap(i).b = ForeColor.b * Alpha + BackColor.b * (1 - Alpha)
    Next i
    
    SetDIBitsToDevice TheForm.hdc, Signature.Left, Signature.Top, Signature.Width, Signature.Height, 0, 0, 0, Signature.Height, OutBitmap(0), BitInfo, DIB_RGB_COLORS
    TheForm.Refresh
End Sub

C Code
//Prepare to decode signature
	//const UCHAR BytesPerPixel=4, TranspMask=255; //32 bits per pixel (for quicker copies and such - variable not used due to changing BYTE*s to DWORD*s), and white=transparent background color - also not used anymore since we directly write in the background color
	//Load data from executable
	HGLOBAL GetData=LoadResource(NULL, FindResource(NULL, "DakSig", "Sig")); //Load the resource from the executable
	BYTE *Input=(BYTE*)LockResource(GetData); //Get the resource data

	//Prepare header and decoding data
	UINT BitOn=0; //Bit we are currently on in reading
	EncodedFileHeader H=*(EncodedFileHeader*)Input; //Save header locally so we have quick memory lookups
	DWORD *Output=Signature=new DWORD[H.Width*H.Height]; //Allocate signature memory

	//Prepare the index colors
	DWORD Indexes[17], IndexSize=NumBitsRequired(H.NumIndexes); //Save full color indexes locally so we have quick lookups, use 17 index array so we don't have to allocate memory (since we already know how many there will be), #16=transparent color
	DWORD BackgroundColor=GetSysColor(COLOR_BTNFACE), FontColor=0x067906;
	BYTE *BGC=(BYTE*)&BackgroundColor, *FC=(BYTE*)&FontColor;
	for(UINT i=0;i<16;i++) //Alpha blend the indexes
	{
		float Alpha=((EncodedFileHeader*)Input)->Indexes[i] / 255.0f;
		BYTE IndexColor[4];
		for(int n=0;n<3;n++)
			IndexColor[n]=(BYTE)(BGC[n]*Alpha + FC[n]*(1-Alpha));
		//IndexColor[3]=0; //Don't really need to worry about the last byte as it is unused
		Indexes[i]=*(DWORD*)IndexColor;
	}
	Indexes[16]=BackgroundColor; //Translucent background = window background color

//Unroll/unencode all the pixels
Input+=(sizeof(EncodedFileHeader)+H.NumIndexes); //Start reading input past the header
do
{
	UINT l; //Length (transparent and then index)
	//Transparent pixels
	memsetd(Output, Indexes[16], l=ReadBits(Input, BitOn, H.TranspSize));
	Output+=l;

	//Translucent pixels
	l=ReadBits(Input, BitOn+=H.TranspSize, H.TranslSize);
	BitOn+=H.TranslSize;
	for(i=0;i<l;i++) //Write the gray scale out to the 3 pixels, this should technically be done in a for loop, which would unroll itself anyways, but this way ReadBits+index lookup is only done once - ** Would need to be in a for loop if not using gray-scale or 24 bit output
		Output[i]=Indexes[ReadBits(Input, BitOn+i*IndexSize, IndexSize)];
	Output+=l;
	BitOn+=l*IndexSize;
} while(BitOn<H.DataSize);

//Output the signature
const BITMAPINFOHEADER MyBitmapInfo={sizeof(BITMAPINFOHEADER), 207, 42, 1, 32, BI_RGB, 0, 0, 0, 0, 0};
SetDIBitsToDevice(MyDC, x, y, MyBitmapInfo.biWidth, MyBitmapInfo.biHeight, 0, 0, 0, MyBitmapInfo.biHeight, Signature, (BITMAPINFO*)&MyBitmapInfo, DIB_RGB_COLORS);

This all adds ~3.5KB to each VB project, and ~2KB to each C/CPP project. Some other recent additions to all project executables include the Hyrulean Productions icon (~1KB) and file version information (1-2KB). I know that a few KB doesn’t seem like much, but when executables are often around 10KB, it can almost double their size.

While I’m on the topic of project sizes, I should note that I always compress their executables with UPX, a very nifty executable compressor. It would often be more prudent to use my Small project, but I don’t want to complicate my open-source code.


One other possible solution I did not pursue would be to take the original font and create a subset font of it with only the letters (and font size?) I need, and see if that file is smaller. I doubt it would have worked well though.
Useful Bash commands and scripts
Unix is so great
First, to find out more about any bash command, use
man COMMAND

Now, a primer on the three most useful bash commands: (IMO)
find:
Find will search through a directory and its subdirectories for objects (files, directories, links, etc) satisfying its parameters.
Parameters are written like a math query, with parenthesis for order of operations (make sure to escape them with a “\”!), -a for boolean “and”, -o for boolean “or”, and ! for “not”. If neither -a or -o is specified, -a is assumed.
For example, to find all files that contain “conf” but do not contain “.bak” as the extension, OR are greater than 5MB:
find -type f \( \( -name "*conf*" ! -name "*.bak" \) -o -size +5120k \)
Some useful parameters include:
  • -maxdepth & -mindepth: only look through certain levels of subdirectories
  • -name: name of the object (-iname for case insensitive)
  • -regex: name of object matches regular expression
  • -size: size of object
  • -type: type of object (block special, character special, directory, named pipe, regular file, symbolic link, socket, etc)
  • -user & -group: object is owned by user/group
  • -exec: exec a command on found objects
  • -print0: output each object separated by a null terminator (great so other programs don’t get confused from white space characters)
  • -printf: output specified information on each found object (see man file)

For any number operations, use:
+nfor greater than n
-nfor less than n
nfor exactly than n

For a complete reference, see your find’s man page.

xargs:
xargs passes piped arguments to another command as trailing arguments.
For example, to list information on all files in a directory greater than 1MB: (Note this will not work with paths with spaces in them, use “find -print0” and “xargs -0” to fix this)
find -size +1024k | xargs ls -l
Some useful parameters include:
  • -0: piped arguments are separated by null terminators
  • -n: max arguments passed to each command
  • -i: replaces “{}” with the piped argument(s)

So, for example, if you had 2 mirrored directories, and wanted to sync their modification timestamps:
cd /ORIGINAL_DIRECTORY
find -print0 | xargs -0 -i touch -m -r="{}" "/MIRROR_DIRECTORY/{}"

For a complete reference, see your xargs’s man page.

grep:
GREP is used to search through data for plain text, regular expression, or other pattern matches. You can use it to search through both pipes and files.
For example, to get your number of CPUs and their speeds:
cat /proc/cpuinfo | grep MHz
Some useful parameters include:
  • -E: use extended regular expressions
  • -P: use perl regular expression
  • -l: output files with at least one match (-L for no matches)
  • -o: show only the matching part of the line
  • -r: recursively search through directories
  • -v: invert to only output non-matching lines
  • -Z: separates matches with null terminator

So, for example, to list all files under your current directory that contain “foo1”, “foo2”, or “bar”, you would use:
grep -rlE "foo(1|2)|bar"

For a complete reference, see your grep’s man page.

And now some useful commands and scripts:
List size of subdirectories:
du --max-depth=1
The --max-depth parameter specifies how many sub levels to list.
-h can be added for more human readable sizes.

List number of files in each subdirectory*:
#!/bin/bash
export IFS=$'\n' #Forces only newlines to be considered argument separators
for dir in `find -type d -maxdepth 1`
do
	a=`find $dir -type f | wc -l`;
	if [ $a != "0" ]
	then
		echo $dir $a
	fi
done
and to sort those results
SCRIPTNAME | sort -n -k2

List number of different file extensions in current directory and subdirectories:
find -type f | grep -Eo "\.[^\.]+$" | sort | uniq -c | sort -nr

Replace text in file(s):
perl -i -pe 's/search1/replace1/g; s/search2/replace2/g' FILENAMES
If you want to make pre-edit backups, include an extension after “-i” like “-i.orig”

Perform operations in directories with too many files to pass as arguments: (in this example, remove all files from a directory 100 at a time instead of using “rm -f *”)
find -type f | xargs -n100 rm -f

Force kill all processes containing a string:
killall -9 STRING

Transfer MySQL databases between servers: (Works in Windows too)
mysqldump -u LOCAL_USER_NAME -p LOCAL_DATABASE | mysql -u REMOTE_USER_NAME -p -D REMOTE_DATABASE -h REMOTE_SERVER_ADDRESS
“-p” specifies a password is needed

Some lesser known commands that are useful:
screen: This opens up a virtual console session that can be disconnected and reconnected from without stopping the session. This is great when connecting to console through SSH so you don’t lose your progress if disconnected.
htop: An updated version of top, which is a process information viewer.
iotop: A process I/O (input/output - hard drive access) information viewer. Requires Python ? 2.5 and I/O accounting support compiled into the Linux kernel.
dig: Domain information retrieval. See “Diagnosing DNS Problems” Post for more information.

More to come later...

*Anything staring with “#!/bin/bash” is intended to be put into a script.
Zelda Treasure Flaws
The only time when having too much money is a problem

I had meant to write this post back when I beat “Zelda: Twilight Princess” a few days after it and the Nintendo Wii came out in 2006, but never got around to it, and the idea of writing about a game that came out long ago seemed rather antiquated. The initiative to write this post popped up again though as I just finished replaying “Zelda: Ocarina of Time” (N64).

I have been a really big Zelda fan for a very long time, and have played most of the series. I got to a GameStop ~8 hours, IIRC, before they started preordering the Wii to make sure I could play Twilight Princess as soon as it came out, as I was very anxious to play it. It was a good thing I did too, because when the Wii actually came out, they were next to impossible to acquire. I knew of many people having to wait in lines well over 15 hours to get one soon after the release, and they were still rarities to attain well over a year later.

While I really enjoyed Twilight Princess, I was very frustrated by a rupee and treasure problem. “Zelda” (NES) and “Link to the Past” (SNES) had it right. Whenever you found a secret in those games it was something really worth it, namely, a heart piece (increased your life meter), or often even a new item. Rupees (in game money) were hard earned through slaying enemies, only rarely given in bulk as prizes, and you almost always needed more. As I played through Twilight Princess, I was very frustrated in that almost every secret I found, while hoping for something worth it like a heart pieces, was almost always a mass of rupees. There were at least 50 chests I knew of by the end of the game filled with rupees that I couldn’t acquire because I was almost always maxed out on the amount I could carry. What’s even worse is that the game provided you a means to pretty much directly pinpoint where all heart pieces were. These problems pretty much ruined the enjoyment of the search for secret treasures in the game. You could easily be pointed directly to where all hearts were, new game items were only acquirable as primary dungeon treasures, and the plethora of rupees was next to worthless.

So, as I was replaying Ocarina of Time, I realized how unnecessary rupees were in that game too. There are really only 2 places in the whole game you need rupees to buy important items; one of which is during your very first task within the first few minutes of the game. The only other use for rupees is for a side quest to buy magic beans which takes up a small chunk of your pocket change through the game, but besides that, there is no point to the money system in the game as you never really need it for anything. What’s even more a slap in the face is that one of the primary side quests in the game just rewards you with larger coin purses to carry more rupees, which again, you will never even need to use.

While these games are extremely fun, this game design flaw just irks me. Things like this will never stop me from playing new Zelda games however, or even replaying the old ones from time to time, especially my by far favorite, Link to the Past, as they are all excellent works. I would even call them pieces of art. Miyamoto forever :-).