Robots.txt kya hai? Ise apni website me kahan place karein?

Robots.txt ek text html file hoti hai jise hum apni website me isliye put krte hai taki hum search engine ko bta ske ki kon se pages ya path ko visit nhi krna hai. Usi command ke according search engine hamari sites ke links ko visit krega. Robots.txt Search engines ke liye anivarya nhi hai but search engine unhi links ya path ko visit nhi krta ya hum keh skte hain ki search engine usi ka palan krta hai jo chiz use nhi krne ke liye khi jati hai.

Sath hi kuch mahatvpurna tathya batana chahunga ki robots.txt ka aim apni site ko search engine se crwaling se bacane ke liye nhi balki sirf search engine ko yeh batane ke liye kiya jata hai ki yeh hamare dwara note kiye huve kuch pages ya path hai jo hamare liye bahut important hai please aap is path ko read mt kijiye.

For example
Simple se sabdo m aap soch skte hai humne sadak pe ek notice board lagaya hua hai jisme kha hua hai aage koi danger hai. Toh log us notice board ke according samajh jayege ki aage sayad kuch danger hai or unhe us raste pr nhi jana chahiye. Usi prakar robots.txt command ko read krke search engine decide krta hai ki kon se url ya path ko use follow nhi krna hai. Isliye agar hamare paas bahut hi important data hai jise hum index aur search results mein display hone se bachana chahte hai uske liye hum robots.txt pr depend ho skte hain.

Robots.txt file ko apni site me kha place karein?

Robots.txt ko kha place krna hai yeh bhut hi important Question hai. Aur iski jankari hona ek Seo Beginner ke liye bahut hi Mahatvapurna hai. Toh chlo aapko batate hain ki robots.txt file ko kha place kiya jata hai. Is file ko hame main directory ke andar place krna hota hai agar yeh file main root directory ke andar nhi hogi toh search engine ise find nhi kr payega or jo v command humne de rkhi hai uske according kaam nhi krega, or uske according yhi hoga ki hamari site mein robots.txt file nhi hai. Or voh hamari site ke sbhi pages or paths ko follow krega and search results mein v display krega. isiliye robots.txt ki location main root directory ke andar honi chahiye.

Syntax of a Robots.txt File-

This is below are the syntax of a robots.txt file-

User-agent: *
Disallow: /temp/page2

Yahan * means jo v humne user agent ke baad command di hai woh sbhi search engine ke liye hogi. Yadi humne kewal google search engine ke liye command deni hai uske liye syntax kuch is prakar hoga-

User-agent: googlebot

Disallow: /temp/page2

Ab uprokt syntax ke according iska matlab yeh hua ki google search engine ka robot temp/page2 ko jo v path h use visit na kre.

How to Generate and Validate a Robots.txt File

Agar aap robots.txt file create karne ke liye kisi tool ki madad lena chahte hain toh internet pr bhut sare tools available hain. Jiski sahayta se aap easily robots.txt file generate kr sakte hain
Uske baad kewal aapka kaam yeh hoga ki aap us generated file ko apne c-panel m jakar root directory ke andar uplaod kr dein. Kuch online robots.txt generator tool niche diye gye hain-

tools.seobook.com/robots-txt/generator

https://lxrmarketplace.com/robots-txt-generator-tool.html

https://smallseotools.com/robots-txt-generator/

http://tools.seochat.com/tools/online-robots-txt-generator/