Why stop being in search engines?
Last year, I noticed thatmy git server was starting to slow down, and use a lit more memory. Not only that, but web traffic from it was never ending. This isn't normal for that type of server, especially when I'm the most active user by far, and I hardly touch it. I went to see what was happening, and it turned out to be web crawlers. Nothing but bots loading every page, just so places like google and duckduckgo searches can find the page. Problem is that on a git server, files are generated on the fly, and large projects like QMK, which I used to keep a personal copy of, could generate literally billions of pages. Not only is it liking at every file, but every version of every file that's ever existed.