How i m Increasing Google Position - Page 2 - Webmaster Forums - Webmaster forum for HTML, PHP, ASP, CSS and more

JourneyDeep · 06-19-2006, 05:28 PM

what is this robot.txt, how do I create it and where on the server should it be placed? If I understand correctly, this is for giving search engines keywords to crawl, correct?

**etono** · 06-19-2006, 05:57 PM

The robots.txt file does not have anything to do with keywords. When a robot, spider, etc. crawls your website, it is supposed to first look for the robots.txt file before it crawls anyfiles. You can specify which bots you do or do not want searching your site. You can also disallow them from accessing spefic pages or directories. For example, maybe you have a private area where members must login to access, you can specify that you do not want any robots to access this directory. All you do to create it is create a new text doc if your on windows use like notepad, so there is no formatting. Here is an example of a robots.txt file that allows all robots and disallows them from the cgi-bin.

Code:

User-agent:	*
Disallow:	/cgi-bin

Upload this file to the top level publicly available directory, so that it can be accessed by typing in http://yourdomain.com/robots.txt

For an example check out the White House's robots.txt file.

JourneyDeep · 06-19-2006, 06:04 PM

wow, thank you.

Weblinksonline · 09-03-2006, 12:46 AM

Quote:

Originally Posted by WebLady

Google likes sites which have been around for a while, so don't get too disheartened if you don't get a good position at first.

#1 as a new site and still there

See message #4

swoopster · 09-15-2006, 04:44 PM

The answer is just a few clicks away. Search Engine Optimization techniques will certainly get you index faster and ranked higher.

Click the links in my signature block to download a FREE SEO eBook that will explain what to do and what common mistakes webmaster usually make. I'll give you one tip. Never submit your site to a search engines "submit your site" page, Especially Google. You will be almost certainly be put at the back of the bus.

Peace

zobor · 09-16-2006, 02:09 PM

I think your website must also contain common keywords which are widely being searched for on google, which will make your site appear in many search results and people clicking on it would probably make your site go up in the results.

Weblinksonline · 10-14-2006, 11:02 PM

Most of the sites that i have optimised come @ #1 in Google, Yahoo and about 20 or 30 other search engines. When they are not @ #1 they hover around the top 5.

There is one exception to this, our home site

weblinksonline.co.uk . I am still trying to figure out why. Our success with the other sites has varied from #1 in 4 months to #1 in only a few weeks.

visio · 11-05-2006, 03:29 PM

at the moment Google as 65 Million web sites on its database for "Custom Web Design" the keyword you are going for, if I where you I would check wordtracker to find a less competitive keywords.

The articles at the bottom are terrible, change with the name of the article.

appraisal57 · 11-07-2006, 11:43 PM

Great SEO article!

The Ultimate SEO Checklist [Search Marketing]

brentp · 12-09-2006, 06:28 AM

there are many ways to promote your site and many long hours are required the best solution is to wait and all will be panned out in time but still you need to work at it through articles,forum,promoting the best places to start from in my opinion are higher pr sites and not waste time with other sites

digg
reddit
fark
google (dir)
yahoo (dir)
myspace
youtube
blogs

in the next post i will add more in depth details and through more light on this subject