CREATE A ROBOTS TXT FILE

Quite simply, your site’s robots.txt file tells search engine crawlers the pages and files that web crawlers can or can’t request from your site. most commonly, it is used to prevent certain sections of your site from being crawled and is not intended to be used as a way to de-index a webpage and stop it from showing on Google.

You can find your site’s robots.txt file at https://www.domain.com/robots.txt.