What is Robots.txt ? And its importance.

Crawling and indexing are two different concepts which we really need to understand. We know how to see robots.txt but there is misconception about robots.txt. Most of the peoples consider robots.txt is for "noidex" or "doindex". But please understand robots.txt is not for used for indexing purpose.