For websites in production, it is a good idea to block search engines from accessing your content and caching your website for search. Wordpress makes it easy to block Google and other search engines.
Click on the "Settings" item in your left hand menu in Wordpress. Then go to "Reading" which provide some top-level settings for how your content is viewed. This settings page also includes a function for blocking search engines. Keep in mind, this function does not block visitors from going to your website and viewing your content. The function simply is a request for Google and other search engines to not include your website in search results.
When an admin clicks on "Discourage search engines from indexing this site," it causes Wordpress to add a nofollow and index tag to appear in the header.
<meta name='robots' content='noindex,nofollow' />
The code can only be generated in the <head> </head> section if the wp_head inclusion is used by your Wordpress theme. This will cause search engine spiders to ignore your site (if they follow the rules).
By requesting search engines to not index your website, a robots tag is added to the head space of your theme. This does not block visitor access to your website. You can also block your website from being viewed by the general public by creating an .htaccess password easily.
Here are the instructions for using .htaccess and .htpasswd to protect a website