Robots.txt is a plain text file that is implemented in the root directory of a URI as a configuration file used by some search engine spiders and internet robots/bot programs to help direct them to what you want to be indexed and what you don’t. Although many robots will read and follow your instructions in the “/robots.txt” file, many ‘less compliant’ programs may actually ignore this file completely.

Here are a few examples of robots.txt file (plain text):

Ask all search engines to NOT index or follow links on the entire website:

#asks all search engines to NOT index and NOT follow any pages or links on the entire website
User-agent: *
Disallow: /

Allows all search engines to index and follow links on the entire website by Disallowing nothing:

#allows search engines to index and follow all pages and links on the entire website by Disallowing nothing
User-agent: *
Disallow:

Disallows specific folders and files from indexing and following:

User-agent: *
Disallow: /uploads/ # since this folder may contain secure, private, cached or temporary files, we should disallow this entire folder from being indexed.
Disallow: /tmp/ # since this folder may contain cached or temporary files, we should disallow this entire folder from being indexed
Disallow: /page.php

Also see:

1. Robots Meta Tag
2. .htacess and mod_rewrite

Bookmark to:

Server-side Programs

• Enarion phpSitemapsNG ([PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.])
• Google Sitemap Generator ([Linux|Linux is a generic term referring to Unix-like computer operating systems based on the Linux kernel. Their development is one of the most prominent examples of free and open source software collaboration; typically all the underlying source code can be used, freely modified, and redistributed by anyone under the terms of the GNU GPL and other free software licences.] or [Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.], 32/64bit, [open-source|Open source is an approach to the design, development, and distribution of software, offering practical accessibility to software source code. Some consider open source as one of various possible design approaches, while others consider it a critical strategic element of their operations. Before open source became widely adopted, developers and producers used a variety of phrases to describe the concept; the term open source gained popularity with the rise of the Internet, which provided access to diverse production models, communication paths, and interactive communities.])
• GS Master (Polish, [PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.]/[Perl|Perl is a high-level, general-purpose, interpreted, dynamic programming language. Perl was originally developed by Larry Wall, a linguist working as a systems administrator for NASA, in 1987, as a general-purpose Unix scripting language to make report processing easier. Since then, it has undergone many changes and revisions and become widely popular amongst programmers. Larry Wall continues to oversee development of the core language, and its upcoming version, Perl 6.])
• Outil en PHP (French, [PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.])
• Perl Sitemap Generator ([Perl|Perl is a high-level, general-purpose, interpreted, dynamic programming language. Perl was originally developed by Larry Wall, a linguist working as a systems administrator for NASA, in 1987, as a general-purpose Unix scripting language to make report processing easier. Since then, it has undergone many changes and revisions and become widely popular amongst programmers. Larry Wall continues to oversee development of the core language, and its upcoming version, Perl 6.])
• Python Sitemap Generator ([Python|Python is a general-purpose high-level programming language. Its design philosophy emphasizes code readability. Python claims to \'combine remarkable power with very clear syntax\', and its standard library is large and comprehensive. Its use of indentation as block delimiters is unusual among popular programming languages.])
• Simple Sitemaps ([PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.])
• SiteMap XML Dynamic Sitemap Generator ([PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.]) $
• XML Sitemap Generator ([PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.]) $

CMS and Other Plugins

• ASP.NET – Sitemaps.Net
• ClickCartPro $
• Coppermine
• DotClear (Spanish)
• DotClear (2)
• Drupal
• ECommerce Templates ([PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.]) $
• Ecommerce Templates ([PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.] or [ASP|Active Server Pages (ASP), also known as Classic ASP or ASP Classic, was Microsoft\'s first server-side script engine for dynamically-generated web pages. Initially released as an add-on to Internet Information Services (IIS) via the Windows NT 4.0 Option Pack, it was subsequently included as a free component of Windows Server (since the initial release of Windows 2000 Server). It has now been superseded by ASP.NET.]) $
• LifeType
• mnoGoSearch
• OS Commerce
• PHPNuke (German)
• PHPNuke – NukeSEO Sitemaps Generator
• phpWebSite
• Plone
• RapidWeaver
• Textpattern
• vBulletin
• Wikka Wiki ([PHP|PHP, or PHP: Hypertext Preprocessor, is a widely used, general-purpose scripting language that was originally designed for web development, to produce dynamic web pages. It can be embedded into HTML and generally runs on a web server, which needs to be configured to process PHP code and create web page content from it. It can be deployed on most web servers and on almost every operating system and platform free of charge. PHP is installed on over 20 million websites and 1 million web servers.])
• WordPress

Downloadable Tools

• CoffeeCup Sitemapper ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.]) $
• Devintelligence SitemapGenerator ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.])
• GSiteCrawler ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.])
• GWebCrawler & Sitemap Creator ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.])
• G-Mapper ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.])
• Inspyder Sitemap Creator ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.]) $
• IntelliMapper ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.]) $
• KeyLimeTie Sitemap Generator ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.]) $
• Map-IT Sitemapper ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.], [Mac|Mac OS is the trademarked name for a series of graphical user interface-based operating systems developed by Apple Inc. for their Macintosh line of computer systems. The Macintosh user experience is credited with popularizing the graphical user interface. The original form of what Apple would later name the Mac OS was the integral and unnamed system software first introduced in 1984 with the original Macintosh, usually referred to simply as the System software. It was a trimmed down version of the operating system underpinning Apple\'s earlier Lisa product.]) $
• MediaCET Diadematus ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.])
• Microsys A1 Sitemap Generator ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.]) $
• Rage Google Sitemap Automator $ ([OS-X|Mac OS X is a line of computer operating systems developed, marketed, and sold by Apple Inc., and since 2002 has been included with all new Macintosh computer systems. It is the successor to Mac OS 9, the final release of the classic Mac OS, which had been Apple\'s primary operating system since 1984.])
• Site Magellen ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.])
• Site Map Pro ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.]) $
• Sorrowmans Sitemap Tools ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.])
• Vigos Gsitemap ([Windows|Microsoft Windows is a series of software operating systems and graphical user interfaces produced by Microsoft. Microsoft first introduced an operating environment named Windows in November 1985 as an add on to MS DOS in response to the growing interest in graphical user interfaces. Microsoft Windows came to dominate the personal computer market, overtaking Mac OS, which had been introduced previously. As of July 2009, Windows had approximately 93 percent of the market share of the client operating systems for usage on the internet.])
• Web-design-pros.ca Sitemap.xml.gz Generator ([Java Webstart Application|In computing, Java Web Start (also known as JavaWS or as javaws), a framework developed by Sun Microsystems, allows users to start application software for the Java Platform directly from the Internet using a web browser. Unlike Java applets, Web Start applications do not run inside the browser, and the sandbox in which they run need not have as many restrictions, although this can be configured. Web Start has an advantage over applets in that it overcomes many compatibility problems with browsers\' Java plugins and different JVM versions. On the other hand, Web Start programs cannot communicate with the browser as easily as applets. To assist migration, users can also invoke a Java Applet as a Java Web Start application.])

Online Generators/Services

• AuditMyPc.com Sitemap Generator
• AutoMapIt
• Autositemap $
• Enarion phpSitemapsNG
• Free Sitemap Generator
• Neuroticweb.com Sitemap Generator
• Safe Area SitemapGenerator
• ScriptSocket Sitemap Generator
• ResourceSEO Online Sitemap Generator
• SitemapDoc
• Sitemapspal
• Smart-IT-Consulting Google Sitemaps XML Validator
• XML-Sitemaps Generator

Code Snippets / Libraries

• Java library
• Perl script
• ASP script
• PHP class

Bookmark to:

Google Talk Chat Hyperlinks

It’s occurred to me that [Hyperlinks|A hyperlink is a reference in a document to an external or internal piece of information. The most common usage is in the Internet to browse through web pages: some text in the current document is highlighted so that when clicked, the browser automatically displays another page or changes the current page to show the referenced content.] are a part of every day life for pretty much everyone these days. Whether you use them in basic internet browsing, copying and pasting YouTube and Facebook links to your friends, or making websites, knowing how to create [hyperlinks|A hyperlink is a reference in a document to an external or internal piece of information. The most common usage is in the Internet to browse through web pages: some text in the current document is highlighted so that when clicked, the browser automatically displays another page or changes the current page to show the referenced content.] is an important thing to know!

Some people don’t realize there are a lot more kinds of [hyperlinks|A hyperlink is a reference in a document to an external or internal piece of information. The most common usage is in the Internet to browse through web pages: some text in the current document is highlighted so that when clicked, the browser automatically displays another page or changes the current page to show the referenced content.] than just ones that use the [HTTP|Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia information systems. Its use for retrieving inter-linked resources led to the establishment of the World Wide Web.] or [HTTPS|Hypertext Transfer Protocol Secure (HTTPS) is a combination of the Hypertext Transfer Protocol and a cryptographic protocol. HTTPS connections are often used for payment transactions on the World Wide Web and for sensitive transactions in corporate information systems.] (HTTP with SSL encryption) [protocols|In computing, a protocol is a set of rules which is used by computers to communicate with each other across a network. A protocol is a convention or standard that controls or enables the connection, communication, and data transfer between computing endpoints. In its simplest form, a protocol can be defined as the rules governing the syntax, semantics, and synchronization of communication. Protocols may be implemented by hardware, software, or a combination of the two. At the lowest level, a protocol defines the behavior of a hardware connection.]. Many of which are very useful to today’s [socially-networked|A social network is a social structure made of individuals (or organizations) called nodes, which are tied (connected) by one or more specific types of interdependency, such as friendship, kinship, financial exchange, dislike, sexual relationships, or relationships of beliefs, knowledge or prestige.] [webmasters|A webmaster (portmanteau of web and postmaster), also called a web architect, web developer, site author, website administrator, or (informally) webmeister, is a person responsible for maintaining a website(s). The duties of the webmaster may include ensuring that the web servers, hardware and software are operating accurately, designing the website, generating and revising web pages, replying to user comment, and examining traffic through the site.] and internet gurus.

Here are some useful examples:

Email HyperLink:
Example Email Link

<a href="mailto:youremail@yourdomain.com?subject=Email+Subject">Example Email Link</a>

Phone Number HyperLink:
Example Phone Link

<a title="Call 503-555-1212" href="callto:5035551212">Example Phone Link</a>

Skype Call HyperLink:
Example Skype Call Hyperlink

<a title="Call YourScreenName on Skype" href="skype:yourscreenname?call">Example Skype Call Hyperlink</a>

Skype Chat HyperLink:
Example Skype Chat Hyperlink

<a title="Chat with YourScreenName on Skype" href="skype:yourscreenname?chat">Example Skype Chat Hyperlink</a>

Google Talk Instant Message HyperLink:
Example Google Talk IM Hyperlink

<a title="Instant Message YourScreenName" href="gtalk:chat?jid=yourscreenname">Example Google Talk IM Hyperlink</a>

Google Talk Call Hyperlink:
Example Google Talk Call Hyperlink

<a title="Call YourScreenName on Google Talk" href="gtalk:call?jid=yourscreenname">Example Google Talk Call Hyperlink</a>

MSN/Windows Live HyperLink:
Example MSN/Windows Live IM Hyperlink

<a title="Instant Message YourScreenName on MSN/Windows Live Messenger" href="msnim:chat?contact=yourscreenname">Example MSN/Windows Live IM Hyperlink</a>

Yahoo! Instant Messanger HyperLink:
Example Yahoo! Instant Messenger IM Hyperlink

<a title="Chat with YourScreenName on Yahoo! Instant Messenger" href="ymsgr:sendim?yourscreenname">Example Yahoo! Instant Messenger IM Hyperlink</a>

AOL Instant Messenger HyperLink:
Example AOL Instant Messenger IM Hyperlink

<a title="Chat with YourScreenName on AOL Instant Messenger" href="aim:goim?screenname=yourscreenname">Example AOL Instant Messenger IM Hyperlink</a>

I’ll update this post and add more useful examples as I think of them.

Bookmark to:

AddPro.com

AddPro.com provides a free service that performs a basic submission of your site to 18 of the top search engine providers:

Visit AddPro.com

AddMe.com

AddMe.com also provides a free search engine submission service that auto-submits your website to the following Search Engines:

Visit AddMe.com

FreeWebSubmission.com

FreeWebSubmission.com provides free manual and auto submission to the highest-rated, Free Internet Search Engines and Directories.

Visit FreeWebSubmission.com

Bookmark to:

Frequently Asked Questions About Firefox

Top 5 Reasons to Switch from Internet Explorer to Mozilla Firefox

Speed:

going from IE to Firefox is like trading in your minivan for a sports car.

Safety:

our open-source security process means we address threats faster to keep you safe online.

Easy to Use:

Firefox’s cutting-edge features are so intuitive you won’t know how you lived without them.

Customization:

we offer literally thousands of ways for you to adapt Firefox to meet your unique browsing needs.

Free:

they say the best things in life are free, and Firefox is totally, 100% free of charge.

Fun Fact:

Four out of five Firefox users say they’d recommend it to a friend.

View the Firefox FAQs for more information

Bookmark to: