Not every site needs a site map, they can certainly be a good idea. Site maps provide a dual purpose: They provide search engine spiders easy access to all of your site pages and they provide site visitors easy access to all of your site pages. The difference is that search engines and visitors access your site map differently and therefore there are different methods that need to be applied to creating site map(s) that are friendly for both engines and search spiders.

Small sites typically don't need a site map so long as all pages are linked in the main navigation. Once you get into main and sub-navigation menu's then site maps are helpful in allowing search engines and visitors to quickly find anything they are looking for within just a couple of clicks. A single site map can be used for both purposes or multiple site maps can be created. Here we'll address creating site maps for spiders and humans separately.

Site Map For Spiders

.xml file

An .xml document should be added to your site's root directory containing links to all site pages. This .xml file should then be referenced should be compiled and placed into a proper .xml document which should be uploaded into the root directory.

Robots.txt file

With your .xml site map file in place you must then make it accessible to search spiders. Reference the site map in your robots.txt file by adding a line for sitemap: URL (example: sitemap: http://www.polepositionmarketing.com/sitemap.xml)

Keep current

Be sure that your .xml file is updated and uploaded frequently, or at least as often as pages are added or removed from your site. Large sites should implement an automated site map update monthly or even weekly or daily.

Site Map For Visitors

Navigation links

A site map designed for human visitors is just like any other website page. Link to the site map page should be included in the primary navigation or the site's global footer. Visitors should be able to find this link without too much searching.

Additional page links

Site map should also be linked from various pages within the site such as Help pages and your custom 404-redirect page. This helps point visitors to the site map as a quick and easy means to find what they need.

Overview

It's helpful to provide a short overview paragraph at the top of your site map page. This can be a nice introduction should someone land on this page directly from a search engine or elsewhere.

Heading and layout

The layout of the site map should present a clear visual hierarchal structure or your website. Both headings and sub-headings should be used above properly grouped links.

Text links & descriptions

Site map should primarily use textual links and page should not be cluttered with images or other distractions. It is also a good idea to provide an additional short description (provided there is room) for each link that allows the visitor to better understand where each link will take them.

Keep current

Be sure that your site map is updated and uploaded frequently, or at least as often as pages are added or removed from your site.

As I said above, not every site needs a site map. But those that do should be sure that the site map(s) they create are actually benefiting them. And the best way to do that is to make sure your site maps provide maximum usability for visitors and search engine spiders.

Don't be fooled by seeming simplicity. A lot of developers get comfortable with a certain way of designing a database for their web applications that they miss out on techniques they should rather employ to make things run faster and more efficiently. A lot of developers don't bear in mnd that the small site they are creating now might grow into something incredibly large and complex, and the database they designed has become bloated and doesn't scale well to meet the demands of the increased traffic.

This article hopes to provide web developers with a few techniques to help make their database and queries faster and more efficient.

1. Avoid Character Types

When you are designing a database, it is so easy to set all data types to the VARCHAR type as it can then contain any data you want; numbers or text. But character data is amongst the most inefficient data type you can get. If a field is only going to contain numbers, then make it one of the appropriate types (INT, DOUBLE, etc).

Also, wherever possible in your web development code, try to use numeric data types as opposed to characters. One of the most common things a script has to store are flags like whether someone answered yes or no to a question, etc. You could of course store it as 'Y' or 'N' but why not store it as 0 and 1?

The reason this makes a difference is when you have a database, for example, with over 500 000 entries, and are running a SELECT on that field, comparisons are processed a lot faster for numeric data types than character types. Also, if you need to return data to the calling script, numeric data is less memory intensive than character data. In addition, your web development language (PHP, ASP, etc) would also be able to process and perform functions on numeric data better than character data.

I am not trying to convince you never to use character data types. Sometimes it is a necessity, but if you can find ways to reduce the amount of character data processed by your SQL database, the better your server will cope.

2. Normalization

Normalizing a database is really quite a complex process. It is a process that describes a way to design a database structure to avoid repetition of data in your database and can lead to significant performance benefits if employed correctly. However, the entire process of normalisation is a bit beyond the scope of this article as it can fill books on its own, but any developer designing a database should seriously consider becoming knowledgable about normalisation and employing it in their own designs.

For a good tutorial on this process: http://www.keithjbrown.co.uk/vworks/mysql/mysql_p7.php

3. DateTime vs Timestamp fields

This actually relates to 1. a bit. The big difference to bear in mind here is that a field of type DATETIME is actually stored as a series of characters. A field of type TIMESTAMP is actually stored as an integer. So therefore, a more efficient way of storing dates is using the timestamp method. The timestamp has its drawbacks however. For one, you cannot store a date early than 1 January, 1970. Also, timestamps in your script will need recalculating to get to the character format. Because of this recalculation, it may not be better to store as timestamp. It really is a case of testing which format works better for your needs.

4. Use LIMIT where possible

In your queries, if you are doing a SELECT to a database and you only expect a certain number of results, using the LIMIT statement can speed your query up incredibly.

For example, if you have a table of users and you need to run a query to search for one users record, you can use a query like:

SELECT user_name FROM users WHERE user_id = 453;

This query is perfectly valid and will return the right result. But you also know there will only be ONE result. The query above will search the database, find what you want, but then still continue searching after that. It would run a lot faster if you could tell the query that once it has found what you are looking for to stop searching. LIMIT can do this, as this query shows:

SELECT user_name FROM users WHERE user_id = 453 LIMIT 1;

Imagine this scenario. You have a table called logins, that records every login from a user. It currently contains over 2 000 000 records, and you want to find the first time a user logged in. Now bear in mind that because this table inserts data over time, it is already sorted for by date. You could do the following query:

SELECT MIN(login_date) FROM logins WHERE user_id = 4876;

This will return the record you want, but SQL will now have to get all dates for that user, sort them and then return the lowest value to you. Our table is already date sorted simply because of the way it records data for us. So using LIMIT can be more effective:

SELECT login_date FROM logins WHERE user_id = 4876 LIMIT 1;

Because it is sorted, the first one will always be a users first login.

5. Avoid using LIKE

If you have tried to employ 1. above, then hopefully you will be in a scenario where you do not need to use LIKE all that much. LIKE is one of the most inefficient ways of searching a table. LIKE performs a text comparison search in a field and with no wildcards is as efficient as a direct comparison; i.e. WHERE name = 'Jane' is equivalent to WHERE name LIKE 'Jane'. It is when you start introducing the wildcard characters like '%' that things get really hairy.

If you do have to use LIKE, then at least try and make efficient use of the wildcards. These are '_' (underscore) and '%'. Let me explain all this with a real world example.

In a project I was involved in, we had a SQL database storing logs generated automatically from a mail server. Unfortunately, the mail server pretty much just dumped a very long string of text data into a field that contained the data we wanted. A script had to be written to find all logs that referred to a login by a user into the POP server. The only way we could do this was to search every record for a string in the msg field that had the text "User logged in" in it. The first query developed was something like this:

SELECT msg FROM logs WHERE msg LIKE '%User logged in%';

This query took on average of about 35 minutes to process. Obviously not an ideal situation. The way the LIKE worked here was that it had to parse through every single portion of each and every record in the msg field looking for text that matched "User logged in" anywhere in the text. We were able to determine eventually that the text "User logged in" occured at the end of that text in the msg field and so we altered the query:

SELECT msg FROM logs WHERE msg LIKE '%User logged in';

The '%' at the end was removed as we do not want to worry about text after because there is none. The query now only compares text to our string in the msg field at the end of the field and no longer parses through the entire piece of text stored in msg. The query now ran in under 2 minutes. (This was actually still too long, but how we optimised from there is a little beyond the scope of this article.)

Hopefully with all these elements put into practice on your next web development project, you can have a database that runs quickly, efficiently, uses as little resources as possible and wont grind to a halt when the load suddenly increases.

Importance of a navigation scheme

One of the most important tasks when developing your website is creating a navigation system that is effective and easy to use. People have a tendency to try to include everything in the navigation tree all at once. Instead they should follow the same process as setting up the structure for the site and break it down in levels. This would be information relevant to the homepage that directs visitors to the main sections of the site. From the main sections of the site, the user gains access to the sub sections containing content pertinent to that section only. Remember to always include links back to the main sections and your homepage on every page in your site. While you are creating your navigation setup, try to look through your user's eyes. Remember, you are the one creating the structure, so you are biased in your opinions. If you think about what someone else might say or do, then you are one step ahead of the game.

As you create you website navigation, make sure that links can be added in the future with no difficulty. You must always keep in mind that your site must be flexible and open to change because it is never completed. All links should be clear and to the point. It makes no sense to have your visitor try to figure out where they are going. They should be able to quickly look for what they want and then access it in a timely and efficient method. There is a better chance of a visitor going somewhere else if it takes too long for them to find the information they are looking for. Your navigation scheme should stick out and be located in a common spot on every page of your site. Consistency allows your visitors to focus on the content instead of trying to figure another navigation system for the same site.

Linear navigation

Linear navigation provides the same capabilities as your forward and back button on your browser. With this navigation style, you cannot jump around and skip pages. Reading a book or viewing a PowerPoint presentations are also examples of linear navigation. As you read through the pages, it makes no sense to jump around or you might miss an important piece of information. People who create sites with this style often are directing the visitor from a starting point to a predetermined end in a step by step fashion. Your links will not allow the user to anywhere other than where you want.

Hierarchical navigation

Hierarchical navigation allows the visitor to go from a homepage to the main sections of your site and then to the subsections. Visitors can travel through your site without any restrictions. This type of navigation ties all the areas of your site together so any page can be accessed from another in as few clicks as possible. Hierarchical navigation is best used on sites that are filled with information and to be utilized like a library.

Sitemaps for navigating through your website

Sitemaps provide a list of organized links to the content of your website. The same way a table of contents tells you what is inside a book, a sitemap does the same for your website. Your visitors are given a one page view of the information structure that your site is based on. Sitemaps are not meant to be your websites primary navigation but more as a compliment to it.

Navigation bars

A navigation bar gives your visitors the ability to move between the different sections of your website. It should be placed on every page of your site and should be consistent as well. Placement of the navigation bar is entirely up to you. The navigation bar can go on the left side of your page because most people read from left to right. It can also be placed on the top of your page so it is the first thing your visitors see. Some people place it on the right hand side of the page intentionally making a visitor browse through content to then be able to navigate the rest of the site.

Theme and navigation links

The theme for a website is created with the way you use color, fonts and images. During the theme creation process, remember to not clutter your pages with unnecessary content that will draw your visitor's attention from the main content. There are many ways to create a theme. One example might be a newspaper look consisting of black and white colors with a plain font. Themes give a website character and often leave a strong visual impression on the visitor. Your website theme needs to be used on all your pages which helps tie your site together.

Navigation

Navigation of a website should be an easy process and not require the visitor to take much time in figuring out how to move through your site. Visitors have come to your site looking for information and should be able to find it easily. If a visitor has to waste time figuring out how to get what they want, they will go to another site. Navigation is not just for people, but search engines as well. Set up the navigation to allow search engines to follow the links to index your entire website. Your navigation scheme should appear in the same place on all pages. When a visitor navigates through your site, make sure they can flow through it. Do not set it up so a visitor has to use the back button of the browser to access another page. Through navigation, your visitor will have access to pages in your site, to other sites and to different sections of the pages on your site. You will need to place navigation menus in more than one area on your pages. There are many good ways to help the visitor move through your site. The most simple is a text link. The next is a navigation bar placed on the top or sides. Another is the use of graphic buttons created to help compliment the overall look of your site. It is always good idea is to include links at the bottom of your page in case someone does not feel like scrolling back to the top. Through navigation, your visitor should be able to get where they want quickly, know where they currently are on your site and be able to access other pages on your site for additional information.