Wiki is by nature semi-private. However as we look to explore and extend wiki, we can look to add a fully public Published Wiki and a fully Private Wiki.
# Don't crawl
A robots.txt file added to the root of wiki, can prevent Google and other Search Engines from indexing wiki. We can respect an authors request as well in our own graph database of wiki, while providing a greater level of control that enables authors to define explicit domains of cooperation.
At the moment however wiki does not serve:
http://wiki.example.com/robots.txt
This should be added as a route to the current reference server implementation.
# See also - Graph database - Paragraph Level Atomicity - Data analysis of wiki - Wiki Publishing Process