Once uploaded to our hosting server, however, the site no longer worked correctly. More specifically, the page that was supposed to ‘list all properties’ was reporting that there were no properties in the database. This was definitely not the case! The connection string and code were identical on our development server and the hosting server.

A little debugging showed that the ODBC driver was actually returning an empty result set (0 rows) on a particular query. Some digging through Google showed that this is a known bug in old version of the MySQL ODBC driver.

It turns out, however, that older version of this driver have an annoying bug that causes some (valid) queries to fail silently and return an empty result set.

Dan.

Paul has a computer science degree from Staffordshire University and has a wealth of commercial experience developing C#.NET systems.

Ryan has a computer science degree from Edinburgh University and has a wealth of commercial experience developing dynamic database-driven websites.

Acid Computer Services have designed and developed their new website.

For recruiters, Cast UK can help with the following executive recruitment:

• Procurement Recruitment
• Supply Chain Recruitment
• Logistics Recruitment
• Engineering Recruitment
• Manufacturing Recruitment

For job seekers, Cast UK can help you find a job in the following sectors:

• Procurement Jobs
• Supply Chain Jobs
• Logistics Jobs
• Engineering Jobs
• Manufacturing Jobs

Passion For Perfume currently have 35 perfume stores across the UK and now have a full ecommerce store to help continue their fast expansion plans. The new online store sells a large range of men’s designer aftershaves and women’s designer perfumes from top brands such as Bvlgari, Hugo Boss, Ralph Lauren, Dolce and Gabbana, DKNY, Gucci, Calvin Klein and Yves Saint Laurent.

Firstly, you will need this free barcode font and to have the GD or GD2 extension installed on your server.

The page takes a value “string” which it converts into an barcode, for example I have been using “string=*00001234*” for my barcode testing. It can also take “width”, “height” and “fontsize” optional parameters.

//set the content type this page returns
Header (”Content-type: image/png”);

//set the default values
$width = 200;
$height = 35;
$fontsize = 35;
$text = ”;

//Get the parameters passed to the page
if(isset($_REQUEST[’width’]))
   $width = $_REQUEST[’width’];
if(isset($_REQUEST[’height’]))
   $height = $_REQUEST[’height’];
if(isset($_REQUEST[’fontsize’]))
   $fontsize = $_REQUEST[’fontsize’];
if(isset($_REQUEST[’string’]))
   $text = $_REQUEST[’string’];

//align the barcode at the bottom and one pixel from the left
$startlocation_y = $height;
$startlocation_x = 1;

//create and image of the correct size
$img_handle = imagecreate($width,$height);
//add the black and white colours to the image
$white = imagecolorallocate($img_handle,255,255,255);
$black = imagecolorallocate($img_handle,0,0,0);

//write the barcode text to the image using the “Free 3 of 9 Extended”
//barcode font which must be in the same directory.
//The $black colour is negated to remove anti-aliasing.
//The 0 indicates the angle of the text.
imagettftext($img_handle, $fontsize , 0, $startlocation_x, $startlocation_y, -$black, ‘FRE3OF9X’, $text);

//return the image and release resources
imagepng ($img_handle);
imagedestroy ($img_handle);
?>

Information retrieval (IR) is the science of searching for documents / within documents. Information retrieval techniques form some of the most fundamental elements of web search engine technology. This article will discuss information retrieval in the context of search engines.

Indexes

It is unrealistic to remotely access documents in real-time when performing a search, as it would be exceptionally slow and unreliable. Therefore a local index is created, which for search engines is done by a crawler (aka spider). Thus, when you perform a search you are not actually searching the web, but are searching a version of the web as seen and stored by the crawler at some point in the past.

The index would not usually contain the whole document (this may, however, be stored in a separate document cache), but stores a representation of the terms relevant to the document that is quickly and easily searchable. There are various stages to this process (not all systems will include all of these stages):

1. Document
   This is the document in its raw format with all text, structure and formatting.
2. Structure Analysis
   Recognising headings, paragraphs, titles, bold text, lists, …, etc.
3. Lexical Analysis
   Converting the characters in the document into a list of words. This process may include analysing digits, hyphens, punctuation and the case of letters. Proper Noun Analysis can use the case and format of words/phrases to identify important information such as names, places, dates and organisations.
4. Stopwords Removal
   The removal of words which occur very often and provide no ability to discriminate between documents. For example: “the”, “it”, “is”. However, it can be seen that some search engines leave these words in the index and remove them at the user query level. This allows “+word” queries to be performed.
5. Stemming
   This is a conflation procedure which reduces variations of a word into a single root. For example, both “worked” and “working” may be reduced to “work”. The Porter Stemming Algorithm can be used to perform stemming.

After these processes have been performed we have a list of index terms for this particular document.

Index Term Weighting

We now need to calculate to what degree a term is relevant to a particular document. The following is an example of a weighting scheme:

• Index Term Frequency
  This is the frequency of a term inside a document. The frequency is usually normalised within the particular document:
  TermFrequency(term, document) = (no. occurrences of term in document) / (no. occurrence of term with max occurrences in document)
• Inverse Document Frequency
  The inverse of the frequency of a term between all the documents in the set. Terms that appear in many documents are not very useful as they do not allow us to discriminate between documents.
  IDF(term) = log([no. documents in collection] / [no. documents in collection containing term])
• Weight
  This is the actual index term weight for a particular term in a particular document:
  Weight(term, document) = TermFrequency(term, document) * IDF(term)

Other items may be a factor in deciding weight, such as: the terms position in the document, whether it was in the title, whether it was bold, whether it was in a list, …, etc.

Reverse Index

We now have a list of terms (with their weights) for a given document. However, a list of documents that contain a particular word would be much more useful, rather than a list of words for a particular document. This is called a reverse index.

For example, if we had the following three documents:

1. This is a file about website search engine optimisation
2. A website design tutorial file
3. A file about bespoke software design and development

Then the index terms for each document may be as follows (weights would be in parentheses):

1. file(?), website(?), search(?), engine(?), optimisation(?)
2. website(?), design(?), tutorial(?), file(?)
3. file(?), bespoke(?), software(?), design(?), development(?)

However, the reverse index would be:

The reverse index then allows us to easily find the relevant documents for a particular word

Similarity Matching

This is the process for computing the relevance of a document to a particular query. It can comprise:

• Query Term Weighting
  Applies weights to each term in a query. For example, terms at the beginning of a query may be weighted more heavily.
• Similarity Coefficient
  Uses the query term weights and document term weights to compute the similarity between a query and a document. The similarity could be calculated using the vector space model and calculating the cosine coefficient (this will not be discussed here).

Refreshing the Index

Documents can continually change, therefore the index needs to be continually refreshed. The crawler needs to decide how often to reindex particular documents, based on how often they are updated. If a document is not updated very often, then reindexing it very often would be a waste of resources. However, documents that are always changing need to be continually reindexed as they may no longer be relevant to terms they are currently indexed for.

Measuring Accuracy of IR Systems

Two of the simplest ways to assess the accuracy of a basic information retrieval system are Precision and Recall. These are calculated using the number of relevant documents and the number of retrieved documents (the documents perceived to be relevant by the system), the documents actually returned to the user are where these two sets of document overlap.

• Precision
  Ratio of no. relevant documents returned to the total number of documents retrieved - i.e. the number of documents returned that are relevant.
• Recall
  Ratio of no. relevant documents returned to the total number of relevant documents - i.e. the number of relevant documents that are returned.

The documents actually returned from the retrieved documents set will be decided using some form of ranking mechanism (discussion of this is beyond the scope of this article).

Generally, there is a compromise between precision and recall, as increasing the number of documents retrieved is likely to also increase the number of irrelevant documents in the set of retrieved documents.

Web Search Engines

Web search engines (such as Google, Yahoo! and MSN) usually combine information retrieval techniques with link structure analysis, as well as many other unknown techniques. Obviously, the above techniques are very easily spammed, so any useful search engine would need to try to filter out spamming where possible.

About the author

Michael Hawthornthwaite works at Acid Computer Services who specialize in graphic design (including web design), web development and bespoke software development, based in Manchester, UK. Michael is a project manager and specializes in advanced algorithms and web technologies.

Any reproduction of this article must include the above “About the author” information and preserve any links within it.

The sub-headings on each of the services pages (design, web development and software development) are now links to individual pages detailing the services we offer.

We have also added a website map and a free consultation request form.

It was quickly realised that we would also need a spam filter to avoid spam messages clogging up the system. Firstly we created a black list and a white list to allow us to pre-approve/disapprove emails from certain addresses.

Next, we needed some way of retrieving a spam score (or probability) for the remaining messages. We looked immpediately to SpamAssassin and easily got it working on our Windows server using the excellent how to guide: Using SpamAssassin with Win32.

However, we did not want the mail to be filtered at the mail server level, we wanted to spam check the messages in our system as they were retrieved and store their spam score.

To do this we created a SpamAssassinWrapper class library in C# (as our system is in ASP.NET). This class simply called “spamassassin.bat” with the email message file as the argument (in its raw format), by starting a new Process (System.Diagnostics.Process).

The Standard Output can be redirected to a StreamReader (see this MSDN article), allowing us to get the results.

We then retrieved each line from the output until we found the “X-Spam-Status” header. The value of the “score” attribute of the header was then retrieved and returned as a nullable float. It is null if for some reason there was no score.

The only problem with this method is that calling “spamassassin” on each individual email is slow. This could be solved by setting up “spamc” and “spamd” on the server and calling this instead as they are C-compiled and much faster, but this looks tricky (on Windows) and for now time is not an important factor.