Credit: Shinji Nishimoto, An T. Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu & Jack L. Gallant
Movie Clips Reconstructed From Brain Waves
Welcome toِ theِ future: Scientists canِ now peer insideِ the brain andِ reconstruct videos ofِ what a person hasِ seen, based onlyِ onِ their brain activity.
The reconstructed videos couldِ beِ seen asِ a primitive — andِ somewhat blurry — form ofِ mind reading, thoughِ researchers areِ decades fromِ beingِ able toِ decode anythingِ as personal asِ memories orِ thoughts, ifِ such a thingِ isِ evenِ possible.
But inِ the long term, similar methods couldِ beِ used toِ communicate withِ stroke patients orِ coma patients living inِ a locked-in state, saidِ study researcher Jack Gallant, a neuroscientist atِ the University ofِ California, Berkeley.
Decoding the brain
Gallant’s team hasِ decoded theِ brain before.
In 2008, theِ researchers reported thatِ they’d developed a computer model thatِ takes inِ brain activity data fromِ functional magnetic resonance imaging (fMRI), compares itِ to a library ofِ photos, andِ spits outِ the photo thatِ the person wasِ mostِ likelyِ looking atِ when theِ brain activity measurements wereِ taken.
That’s becauseِ fMRI doesn’tِ measure theِ activity ofِ brain cells directly; itِ measures blood flow toِ active areas ofِ the brain.
Movie night … for science
Next cameِ the fun part: Three volunteers, allِ neuroscientists onِ the project, watched hours ofِ video clips whileِ insideِ anِ fMRI machine.
Outside volunteers weren’tِ usedِ becauseِ of theِ amount ofِ time andِ effort involved, andِ because theِ neuroscientists wereِ highly motivated toِ focus onِ the videos, ensuring betterِ brain images.
Using theِ brain-imaging data, Gallant andِ his colleagues built a dictionary thatِ linked brain activity patterns toِ individual video clips — muchِ likeِ their 2008 study didِ with pictures.
Watching a mind movie
The blurriness isِ largely becauseِ the YouTube library ofِ clips isِ so limited, making exact matches tough, Gallant said.
Eighteen million seconds isِ really a vanishingly small fraction ofِ the thingsِ you couldِ see inِ your life, heِ said.
The mind-reading method isِ limited onlyِ to theِ basic visual areas ofِ the brain, notِ the higher-functioning centers ofِ thought andِ reason suchِ asِ the frontal cortex.