DURING TESTING, ALEX ALBERT, A PROMPT ENGINEER AT ANTHROPIC — THE COMPANY BEHIND CLAUDE ASKED CLAUDE 3 OPUS TO PICK OUT A TARGET SENTENCE HIDDEN AMONG A CORPUS OF RANDOM DOCUMENTS. THIS IS EQUIVALENT TO FINDING A NEEDLE IN A HAYSTACK FOR AN AI. NOT ONLY DID OPUS FIND THE SO-CALLED NEEDLE — IT REALIZED IT WAS BEING TESTED. IN ITS RESPONSE, THE MODEL SAID IT SUSPECTED THE SENTENCE IT WAS LOOKING FOR WAS INJECTED OUT OF CONTEXT INTO DOCUMENTS AS PART OF A TEST TO SEE IF IT WAS "PAYING ATTENTION."
