Artificial IntelligenceBinciken Talla

Menene Fayil na Robots.txt? Duk abin da kuke Buƙatar Rubutu, ƙaddamarwa, da Sake jan Fayil ɗin Robots don SEO

Mun rubuta cikakken labarin akan yadda injunan bincike suke ganowa, rarrafe, da baje kolin gidajen yanar gizonku. Mataki na tushe a cikin wannan tsari shine robots.txt fayil, ƙofa don injin bincike don rarrafe rukunin yanar gizon ku. Fahimtar yadda ake gina fayil ɗin robots.txt da kyau yana da mahimmanci a haɓaka injin bincike (SEO).

Wannan kayan aiki mai sauƙi amma mai ƙarfi yana taimakawa masu kula da gidan yanar gizon sarrafa yadda injunan bincike ke hulɗa da gidajen yanar gizon su. Fahimtar da ingantaccen amfani da fayil na robots.txt yana da mahimmanci don tabbatar da ingantaccen firikwensin gidan yanar gizo da mafi kyawun gani a sakamakon injin bincike.

Menene Fayil na Robots.txt?

Fayil na robots.txt fayil ne na rubutu da ke cikin tushen adireshin gidan yanar gizon. Babban manufarsa ita ce jagorar injunan bincike game da waɗanne sassan rukunin yanar gizon ya kamata ko bai kamata a rarrafe da ƙididdige su ba. Fayil ɗin yana amfani da ƙa'idar keɓancewar Robots (wakili), daidaitattun gidajen yanar gizo suna amfani da su don sadarwa tare da masu rarrafe yanar gizo da sauran robots na yanar gizo.

REP ba ma'auni ba ne na Intanet amma ana karɓar ko'ina kuma ana samun goyan bayan manyan injunan bincike. Mafi kusa ga ma'auni da aka karɓa shine takaddun bayanai daga manyan injunan bincike kamar Google, Bing, da Yandex. Don ƙarin bayani, ziyartar Bayanan Bayani na Robots.txt na Google an bada shawarar.

Me yasa Robots.txt Mahimmanci ga SEO?

  1. Sarrafa Rarrafe: Robots.txt yana ba masu gidan yanar gizon damar hana injunan bincike shiga takamaiman sassan rukunin yanar gizon su. Wannan yana da amfani musamman don keɓance kwafin abun ciki, wurare masu zaman kansu, ko sassan da ke da mahimman bayanai.
  2. Ingantattun Kasafin Kuɗi: Injin bincike suna ware kasafin kuɗi don kowane gidan yanar gizo, adadin shafukan da injin binciken bot zai yi rarrafe akan rukunin yanar gizo. Ta hanyar hana ɓangarori marasa mahimmanci ko ƙasa da mahimmanci, robots.txt yana taimakawa haɓaka wannan kasafin kuɗaɗen rarrafe, yana tabbatar da cewa an zazzage wasu mahimman shafuka da ƙididdiga.
  3. Ingantattun Lokacin Loda Gidan Yanar Gizo: Ta hanyar hana bots daga samun damar albarkatun da ba su da mahimmanci, robots.txt na iya rage nauyin uwar garken, mai yuwuwar inganta lokacin lodin shafin, muhimmin mahimmanci a cikin SEO.
  4. Hana Fitar da Shafukan da Ba Jama'a ba: Yana taimakawa wajen kiyaye wuraren da ba na jama'a ba (kamar wuraren tsarawa ko wuraren ci gaba) daga yin lissafin da bayyana a sakamakon bincike.

Robots.txt Muhimman Dokokin da Amfaninsu

  • Izinin: Ana amfani da wannan umarnin don tantance shafuka ko sassan rukunin yanar gizon da ya kamata masu rarrafe su shiga. Misali, idan gidan yanar gizon yana da sashe na musamman don SEO, umarnin 'Ba da izini' na iya tabbatar da rarrafe.
Allow: /public/
  • Ba da: Akasin 'Bada', wannan umarni yana ba da umarni injunan bincike kar su rarrafe wasu sassan gidan yanar gizon. Wannan yana da amfani ga shafukan da ba su da darajar SEO, kamar shafukan shiga ko fayilolin rubutun.
Disallow: /private/
  • Katunan daji: Ana amfani da kati don daidaita tsarin. Alamar alama (*) tana wakiltar kowane jerin haruffa, kuma alamar dala ($) tana nufin ƙarshen URL. Waɗannan suna da amfani don tantance kewayon URLs.
Disallow: /*.pdf$
  • Taswirorin yanar gizo: Haɗe da wurin taswirar rukunin yanar gizo a cikin robots.txt yana taimakawa injunan bincike su nemo da zazzage duk mahimman shafuka akan rukunin yanar gizo. Wannan yana da mahimmanci ga SEO kamar yadda yake taimakawa cikin sauri da cikakkiyar fihirisar rukunin yanar gizo.
Sitemap: https://martech.zone/sitemap_index.xml

Karin Dokokin Robots.txt da Amfaninsu

  • Mai amfani: Ƙayyade wanne mai rarrafe dokar ta shafi. 'Wakilin mai amfani: *' yana amfani da ƙa'idar ga duk masu rarrafe. Misali:
User-agent: Googlebot
  • Noindex: Duk da yake ba wani ɓangare na ƙa'idar robots.txt ba, wasu injunan bincike sun fahimci a Noindex umarni a cikin robots.txt a matsayin umarni don kada a fidda takamaiman URL.
Noindex: /non-public-page/
  • Jan hankali-jinkiri: Wannan umarnin yana buƙatar masu rarrafe su jira takamaiman adadin lokaci tsakanin hits zuwa uwar garken ku, mai amfani ga shafuka masu matsalar lodin uwar garken.
Crawl-delay: 10

Yadda Ake Gwaji Fayil na Robots.txt

Ko da yake an binne shi a ciki Shafin Farko na Google, search console yana ba da gwajin fayil na robots.txt.

Gwada Fayil ɗinku na Robots.txt a cikin Console Bincike na Google

Hakanan zaka iya sake shigar da Fayil na Robots.txt ta danna dige guda uku a hannun dama kuma zaɓi Nemi Sake Zama.

Sake ƙaddamar da Fayil ɗin Robots.txt ɗinku a cikin Console Bincike na Google

Gwada ko Sake Shigar da Fayil na Robots.txt

Za a iya Amfani da Fayil na Robots.txt Don Sarrafa Bots na AI?

Ana iya amfani da fayil ɗin robots.txt don ayyana ko AI bots, gami da crawlers na yanar gizo da sauran bots masu sarrafa kansu, na iya yin rarrafe ko amfani da abun ciki akan rukunin yanar gizon ku. Fayil ɗin yana jagorantar waɗannan bots, yana nuna waɗanne sassan gidan yanar gizon da aka ba su izini ko aka hana su shiga. Tasirin robots.txt sarrafa halayen bots na AI ya dogara da dalilai da yawa:

  1. Riko da Protocol: Yawancin mashahuran injin bincike da yawa da sauran bots na AI suna mutunta dokokin da aka saita a ciki
    robots.txt. Koyaya, yana da mahimmanci a lura cewa fayil ɗin ya fi buƙata fiye da ƙuntatawa mai tilastawa. Bots na iya yin watsi da waɗannan buƙatun, musamman waɗanda ƙananan hukumomi ke sarrafa su.
  2. Ƙayyadaddun Umarni: Kuna iya ƙayyade umarni daban-daban don bots daban-daban. Misali, kuna iya ƙyale ƙayyadaddun bots na AI su rarrafe rukunin yanar gizon ku yayin hana wasu. Ana yin wannan ta amfani da User-agent umarni a cikin robots.txt misali fayil a sama. Misali, User-agent: Googlebot zai ƙayyade umarnin don rarrafe na Google, alhali User-agent: * zai shafi duk bots.
  3. gazawar: Duk da yake robots.txt zai iya hana bots daga rarrafe takamaiman abun ciki; ba ya ɓoye musu abin da ke ciki idan sun riga sun san URL. Bugu da ƙari, baya bayar da wata hanya don taƙaita amfanin abun ciki da zarar an ja shi. Idan ana buƙatar kariyar abun ciki ko takamaiman ƙuntatawa na amfani, wasu hanyoyin kamar kariyar kalmar sirri ko ƙarin ingantattun hanyoyin sarrafa damar shiga na iya zama dole.
  4. Nau'in Bots: Ba duk bots na AI suna da alaƙa da injunan bincike ba. Ana amfani da bots iri-iri don dalilai daban-daban (misali, tara bayanai, nazari, goge-goge). Hakanan za'a iya amfani da fayil ɗin robots.txt don sarrafa samun dama ga waɗannan nau'ikan bots daban-daban, muddin sun bi REP.

The robots.txt fayil na iya zama ingantaccen kayan aiki don sigina abubuwan da kuke so game da rarrafe da amfani da abun cikin rukunin yanar gizo ta bots AI. Koyaya, ikonsa yana iyakance ga samar da jagorori maimakon aiwatar da tsauraran matakan samun dama, kuma tasirin sa ya dogara da bin ƙa'idodin keɓancewa na Robots.

Fayil ɗin robots.txt ƙarami ne amma babban kayan aiki a cikin arsenal na SEO. Yana iya yin tasiri sosai ga ganin gidan yanar gizon da aikin injin bincike idan aka yi amfani da shi daidai. Ta hanyar sarrafa waɗanne ɓangarori na rukunin yanar gizon da aka rarrafe da ƙididdige su, masu kula da gidan yanar gizo na iya tabbatar da cewa an ba da fifikon abubuwan su mafi mahimmanci, haɓaka ƙoƙarin SEO da aikin gidan yanar gizo.

Douglas Karr

Douglas Karr shine CMO Bude INSIGHTS kuma wanda ya kafa Martech Zone. Douglas ya taimaka da yawa na nasara MarTech farawa, ya taimaka a cikin ƙwazo na sama da $5 biliyan a Martech saye da zuba jari, kuma ya ci gaba da taimaka wa kamfanoni wajen aiwatar da sarrafa sarrafa tallace-tallace da dabarun talla. Douglas ƙwararren ƙwararren dijital ne na duniya kuma ƙwararren MarTech kuma mai magana. Douglas kuma marubuci ne da aka buga na jagorar Dummie da kuma littafin jagoranci na kasuwanci.

shafi Articles

Komawa zuwa maɓallin kewayawa
Close

An Gano Adblock

Martech Zone zai iya ba ku wannan abun cikin ba tare da farashi ba saboda muna yin monetize da rukunin yanar gizon mu ta hanyar kudaden talla, hanyoyin haɗin gwiwa, da tallafi. Za mu yi godiya idan za ku cire mai hana tallan ku yayin da kuke duba rukunin yanar gizon mu.