GitHub's Copilot lies about its own documentation. So why would I trust it with my code?
https://shkspr.mobi/blog/2024/10/githubs-copilot-lies-about-its-own-documentation-so-why-would-i-trust-it-with-my-code/
In the early part of the 20th Century, there was a fad for "Radium". The magical, radioactive substance that glowed in the dark. The market had decided that Radium was The Next Big Thing and tried to shove it into every product. There were radioactive toys, radioactive medicines, radioactive chocolate bars, and a hundred other products.
The results weren't pretty.
In the early part of the 21st Century, there was a fad for "AI". The magical, Artificial Intelligence which provided all the answers. The market had decided that AI was The Next Big Thing and tried to shove it into every product. You can probably see where this is going, right?
I don't particularly mind companies experimenting with AI. It's good to explore a problem and see if it fits a user's needs. But the current crop are just so shit it makes me wonder whether anyone tested them.
GitHub has forced its new Copilot button on to every page. The first thing I asked it was whether it could be turned off.

It pointed me to this page: https://docs.github.com/en/copilot/getting-started-with-github-copilot/disabling-github-copilot
Except - and I hate to be a pedant - that link 404s. There's nothing there. It doesn't exist. It is made up.
This AI, which I am supposed to trust with my code, doesn't even understand itself.
This isn't an AI Mirror Test. This isn't me trying to find out if the large-language model is conscious, aware, or has a soul. I'm not asking complex reasoning, or asking it to make an æsthetic judgement.
This is a basic functionality test.
Is the computer able to accurately provide information about itself?
That's it. That's all I want. My first interaction with Copilot it lied to me about itself. Why would I trust it again?
You can leave feedback for GitHub about this problem. I'm sure a human will answer you.
https://shkspr.mobi/blog/2024/10/githubs-copilot-lies-about-its-own-documentation-so-why-would-i-trust-it-with-my-code/
#AI #github #LLM