Switch on search engine indexing directive based on environments

[rija]

User story

As a third party system
I want to find GigaDB content through search engines
So that I can conveniently find GigaDB content relevant to my needs

Acceptance criteria

Given I am navigating live website
When I go to "/robots.txt"
Then I can see that search engines are allowed to crawl our website

Given I am navigating non-live website
When I go to "/robots.txt"
Then I can see that search engines are not allowed to crawl our website

Additional Info

Product Backlog Item Ready Checklist

• Business value is clearly articulated
• Item is understood enough by the IT team so it can make an informed decision as to whether it can complete this item
• Dependencies are identified and no external dependencies would block this item from being completed
• At the time of the scheduled sprint, the IT team has the appropriate composition to complete this item
• This item is estimated and small enough to comfortably be completed in one sprint
• Acceptance criteria are clear and testable
• Performance criteria, if any, are defined and testable
• The Scrum team understands how to demonstrate this item at the sprint review

Product Backlog Item Done Checklist

• Item(s) in increment pass all Acceptance Criteria
• Code is refactored to best practices and coding standards
• Documentation is updated as needed
• Data security has not been compromised (with particular reference to the personal information we hold in GigaDB)
• No deviation from the team technology stack and software architecture has been introduced
• The product is in a releasable state (i.e. the increment has not broken anything)