Mind Reading 2.0 from Meta: The Company Released Brain2Qwerty v2

The company has developed a model that can accurately reconstruct text that a person types silently on a keyboard. It seems they have proven for the first time that non-invasive methods can approach the quality previously thought achievable only through brain surgery, as seen with Neuralink.

So, Brain2Qwerty v2:

– The system is based on Brain2Qwerty v1. However, unlike the first version, v2 reconstructs text without knowing the timing of each key press. It receives a continuous stream of brain activity and immediately generates complete sentences.

– The internal task is divided into three levels: there is an encoder that converts raw MEG signals into individual characters. Next, this sequence is processed by an Aligner, which attempts to understand where words begin and end, constructing their embeddings. These embeddings, along with the output signals, are then sent to an LLM, which forms the final output.

– It all looks like a massive setup costing millions of dollars.

The accuracy compared to previous methods is, one might say, off the charts. An average word accuracy of 61% is claimed (around 70% for the best participants). At the same time, the system can allow for more symbolic errors than the first version, but the meaning turns out to be much closer to the original.

facebookresearch.github.io/brain2qwerty/