fishScript.com d
Home| Progetto| Web| Faq| Acronimi

Argomenti







PLSQL... Script: Errori
Architetture web... Faq: Quali sono gli standard web?

Shell scripting... Script: Cercare e visualizzare file per dimensione

A User Agent, like a web browser, uses HTTP to request a HTML document



In programming the danger of exploring the small details of coding is that you get obsessed with every line of code you write, thereby greatly reducing productivity.
In general, is not worth obsessing over code.
On the other hand you should code with deliberation and intent. Steven Feurstein Oracle Magazine July /August 2004

A User Agent, like a web browser, uses HTTP to request a HTML document

Architetture web

Home >Architetture web > Utilizza i file robots.txt

Stampa  Stampa


Quando un robot visita un sito web il primo file che cerca robots.txt nella cartella principale:

www.w3c.org/robots.txt

Se il file contenuto in un'altra cartella del sito web i robots lo ignoreranno.

Attrraverso il file robots.txt file possibile restringere l'azione di ricerca dei robots, disabilitando l'accesso a particolari cartelle o file.

Un esempio del file robot.txt per disabilitare qualsiasi robots a visitare l'intero sito
Here is a sample robots.txt file that prevents all robots from visiting the entire site

User-agent: * # istruzione per tutti i robots
Disallow: / # disabilita l'indicizzazzione di tutte le pagine


http://www.w3.org/robots.txt

Ci devessere un

# exclude some access-controlled areas
User-agent: *
Disallow: /Team
Disallow: /Project
Disallow: /Systems
Disallow: /Web
Disallow: /History
Disallow: /Out-Of-Date
Disallow: /2002/02/mid
Disallow: /mid/
Disallow: /People/all/
Disallow: /2003/03/Translations/byLanguage
Disallow: /2003/03/Translations/byTechnology

*************************************************************
Esempio da www.google.com/robot.txt

User-agent: *
Disallow: /search
Disallow: /groups
Disallow: /images
Disallow: /catalogs
Disallow: /catalog_list
Disallow: /news
Disallow: /pagead/
Disallow: /relpage/
Disallow: /imgres
Disallow: /keyword/
Disallow: /u/
Disallow: /univ/
Disallow: /cobrand
Disallow: /custom
Disallow: /advanced_group_search
Disallow: /advanced_search
Disallow: /googlesite
Disallow: /preferences
Disallow: /setprefs
Disallow: /swr
Disallow: /url
Disallow: /wml
Disallow: /hws
Disallow: /bsd?
Disallow: /linux?
Disallow: /mac?
Disallow: /microsoft?
Disallow: /unclesam?
Disallow: /answers/search?q=
Disallow: /local
Disallow: /froogle?
Disallow: /froogle_


*************************************************************
http://www.altavista.com/robots.txt
User-agent: *

Disallow: /search
Disallow: /sidebar
Disallow: /advanced
Disallow: /alchemist
Disallow: /customize
Disallow: /go
Disallow: /go2
Disallow: /cgi-bin
Disallow: /g/
Disallow: /web
Disallow: /r
Disallow: /babelfish
Disallow: /urltrurl
Disallow: /translate
Disallow: /image/results
Disallow: /image/samepage
Disallow: /image/res_detail
Disallow: /audio/results
Disallow: /audio/samepage
Disallow: /audio/res_detail
Disallow: /video/results
Disallow: /video/samepage
Disallow: /video/res_detail
Disallow: /news/more


Il valore asterisco (*) indica che le politiche di accesso sono valide per ogni robot o user-agent

Il campo "Disallow" specifica quali url non devono essere visitate.

Per esempio, la direttiva:

Disallow: /help

disabilita sia /help.html che /help/index.html

mentre
Disallow: /help/
Disabilita l'accesso a /help/index.html ma non a /help.html.

Se il valore per "Disallow" non indicato viene indicato al robot che pu cercare ovunque nel sito
Disallow:





signal Marco Magnani marcomagnani@fishscript.com



Cerca





Anger is creative, depression is useless.
Dyson, Freeman J.


Anger is creative, depression is useless.
Dyson, Freeman J.


Architetture web... Citazioni: The Web is a young medium

Architetture web... Definizioni: Session

Oracle... Definizioni: Transazione





fishScript.Com is accessible by Mobile access technology as mobile phones, Palm and Pocket PC .

Nicoleta e Marco Magnani tutorial, examples, courses, esempi, corsi, esercizi, appunti vari Dottoressa Nicoleta Dragu Formatrice Docente Insegnante Mediatrice Culturale Dott. Marco Magnani Universita La Sapienza Roma Master Computer Science Hunter College New York , Data Base Administrator DBA oracle System architect

Last modified: 2017-11-30 amministratore@fishscript.comNico and Marco Magnani Software Production
Home|About this Site © 2003-2008 www.fishScript.com ®