The Doubt of the Benefit: Fake Progress and Lumosity’s “Brain Games”

,"And then they applied electrodes here, and now I'm happy and not crazy aren't you happy for me!"

Matt Carey

When ShuffleBrain CEO Amy Jo Kim took the stage at the 2011 CasualConnect game design conference, she started her speech “Smart Gamification” with a strangely defensive proviso:

“I’m going to give you a quick run-through of how a game designer thinks about gamification,” she said, “and rather than arguing about what’s good and bad about it, I’m just going to give you an approach and some smart tips for doing a great job applying gaming principles to whatever it is that you’re working on.”

"Consider our hit game, Dinner Time, where we ask players to produce saliva at the chime of a bell..."

I was immediately encouraged by this introduction because I personally think the loosely connected bag of game mechanics sold under the label “gamification” is the worst thing since moldy bread, so I naturally expect any honest account of anybody’s experiences following that design philosophy to be preceded by at least a vague manifestation of guilt.

ShuffleBrain is essentially a gamification consulting agency. Companies can hire them to adorn their products with features like badges and experience levels. In the words of ShuffleBrain’s website, they can “embed game design expertise into your product team” to “develop games, services, and apps that embody the compelling pull of ‘smart games.’” Of course “gamification” is a term that makes more sense in marketing than game design, because how do you gamify something that’s already a game? I’d define it as the process of making a company’s marketing campaign more like a really awful casino game so the company’s cheap-ass customers will keep poking at the marketing materials for some interminable about of time in the hope free prizes will fall out, and then finally give up and buy something so they’ll have something to show for all their sunk time. But the message of Dr. Kim’s speech was largely about applying gamification to games themselves, which I took to mean making ordinary acceptable-looking video games function more like awful casino games.

ShuffleBrain claims to have done gamification work on the regrettable match-3 game Bejeweled 2 as well as Moshi Monsters, a vastly successful UK-based virtual pet game that now claims 60 million users. Moshi Monsters was a typical success story of the gamification gold rush, building up hapless seven-year-olds’ emotional investment in virtual pets that can only be kept happy and healthy with daily free-to-play grinding sessions, then leveraging the kids’ desperate need to feel loved into purchases of paid subscriptions and a bunch of real-world rubbish with codes on it for virtual items. These are typical guilty-pleasure types of games, the kind that don’t claim to do anything more than kill time and aren’t looking for any higher praise than to be called “addictive.” But there’s also a surprising entry on ShuffleBrain’s client list: ShuffleBrain claims to have “re-designed the onboarding process for Lumosity.com, which offers games that improve your cognitive functioning.” I had heard of Lumosity before, and to me it seemed like an unlikely partnership. Why would a company like Lumosity, which goes to so much trouble to cultivate a positive and healthy image, hire consultants who were so open about their goal to manipulate their users?

Lumosity is part of the “brain-training” genre of games, much like Brain Age on the DS or Big Brain Academy on the Wii, but unlike its competitors which prudently position themselves mainly as entertainment products, Lumosity isn’t shy about making attention-grabbing claims about its health benefits. In a 30-second TV ad now found on YouTube, a bunch of happy young people claim to have experienced various cognitive benefits from playing Lumosity, and an announcer makes the bold claim that “No matter why you want a better brain, Lumosity.com can help. It’s like a personal trainer for your brain, using the science of neuroplasticity, but in a way that just feels like games.”

There’s also a stark difference between the way Lumosity describes its effects on players and the way Dr. Kim, drawing on her PhD in behavioral neuroscience, describes the effect of ShuffleBrain’s grind-heavy design. In “Smart Gamification” Kim talks about game design as a way of “getting people to do things.” She illustrates her point with a Powerpoint slide with a picture of a slot machine and a chart graphing “cumulative number of responses” over time with various reward schedule patterns, showing that users respond more eagerly to rewards that come at randomized intervals than they do when they can anticipate and control the outcomes of their actions.

True story: I once saw a woman sitting at a slot machine in small town Nevada, with a kid leashed to her chair. This isn't a joke.

“What happens over time is what’s known as a reinforcement schedule in behavioral psychology,” Dr. Kim says. “Patterns of reinforcement over time that have variability in them are much more addictive than very straightforward ‘do this, get that’ in a deterministic sense. Game designers know this. That red pattern which is the most addictive is exactly what’s programmed into a slot machine. And that’s what gets you addiction, surprise, and it’s actually very good at building habits. It’s that blend of something you can count on with a certain element of surprise. Those are the sorts of things that no matter what you’re doing you can build into your experience.” She also shows herself to be an avid proponent of all kinds of typical grind-design cruft, specifically endorsing points, levels, leaderboards, badges, virtual goods, appointments, and progressive unlocks.

I found Dr. Kim’s description of her design philosophy very suspicious, especially as applied to Lumosity. She seemed to be talking about finding ways to make essentially tedious tasks feel beneficial and productive—what Jonathan Blow, in a very preposterously-named speech, called “engineering around boredom.” This approach concerns me, as I tend to think of boredom as a precious evolutionary gift, a natural defense system that prevents us from whiling away our days collecting rubber bands or counting the blades of grass in our lawns. It’s an important engine for creativity, and many people willingly choose to do hard, valuable work to alleviate boredom. The way I see it, if we’re truly doing something boring, instead of allowing our natural boredom to go to waste, we should let ourselves experience it and let it direct us in more valuable directions. If gamification was the only thing keeping Lumosity from being boring, then I was very suspicious about whether Lumosity would be worth playing at all.

See also: tricks for avoiding marriage.

If people who are concerned about their cognitive health are going to expose themselves to psychological devices like the “Engagement Loops” Kim showed in her presentation to make boring video games more tolerable, they ought to have a really compelling justification for assuming that these seemingly mind-numbing activities are actually good for their brains. So I decided to actually try some Lumosity games to figure out whether their players were motivating themselves to do something healthy, like setting a schedule to remind themselves to jog or eat their vegetables, or something useless but addictive like (to take Kim’s example) playing a slot machine.

Before I begin, allow me to note that I explained my concerns and asked for comments from Lumosity, ShuffleBrain, and all five members of Lumosity’s Scientific Board whose bios indicated research interests in cognition, memory, or learning. They all either declined to comment or didn’t respond.

Feel The Burn: Training My Brain

As soon as I logged in with a Lumosity starter account in I saw the numbers “00010.” I’d been given 10 “Lumosity Points” on a scoreboard that apparently could go into the tens of thousands. Lumosity Points are one of the three scales used to measure progress for Lumosity players, the others being game scores and Brain Performance Index. Lumosity Points come from things like posting personal best game scores and playing all the games Lumosity assigns you for a play session, and as far as I can tell, no matter how badly you’re performing on the games, your Lumosity Points can never go down. In fact, because of the needlessly complex way your various scores interact in Lumosity, you can be pretty confident that no matter what you’re doing on the website at any given time, some score of yours somewhere is going up.

Never brag about your BPI in public.

Much more attention goes to Brain Performance Index. Lumosity claims that “BPI is Lumosity’s version of IQ” and it promises in a money-back guarantee that if its two-year subscribers play Lumosity at least three times a week they’ll experience a BPI increase of at least 400. In fact, after my first session Lumosity gave me a chart showing that my BPI was projected to improve from around 450 to 850 in just the first three months of training. 450 to 850 of what, I’ll probably never find out, because beyond explaining that it’s based on a statistical distribution of game scores, Lumosity keeps the BPI formula secret as its own “proprietary algorithm.” Nonetheless, Lumosity did helpfully announce that the increase shown on the chart would mean an 87 percent improvement in my brain (obviously using the highly scientific process of “divide the higher number by the lower number”). The message was clear: whatever the heck this BPI stuff is, it’s worth having.

But any frail illusions I had about Lumosity’s trustworthiness in reporting their game’s cognitive benefits were shattered by their blog post “Picture Your Improvement With This BPI Chart.” They made this claim:

On average, users who played at least 1000 games saw their BPIs more than double — the equivalent of moving from the 25th percentile to the 75th percentile. Furthermore, this holds true for people of every age and starting ability. Everyone can improve with sufficient training!

 

Just like an investment fund manager who says “on average my investors double their money every year” is really saying “I am running a Ponzi scheme,” a game designer who says “on average I move my customers from the 25th to the 75th percentile on an intelligence test” is really saying “I measure intelligence based on how much money people give me.” What seems obvious from these claims, which seem to indicate a heretofore unheard of increase in mental capacities over a short period of time, is that what is happening is less an increase in mental prowess and more a shifting of the goalposts, a false measure designed to suggest progress that is, in fact, fake.

I only really understand these charts when they are eggs and frying pans.

Nevertheless, my first session with Lumosity was pretty pleasant. The site casts itself in the role of a personal trainer, choosing three or four simple browser games for you to play each day and chiming in with lists of mental skills you’re supposedly improving and optimistic projections of your future performance. The first game, Speed Match, involved being shown a series of shapes and pushing a button to indicate whether the current shape was the same as, or different from, the previous one. This was essentially a 1-back task, a much easier variation on the N-back task that some researchers believe can improve working memory in children. An encouraging sound played for every correct shape, as well as a Mario-mushroom power-up sound indicating a score multiplier when I got several right in a row. Lumosity said Speed Match exercised three suspiciously similar-sounding skills: “Thinking faster,” “Faster reaction time,” and “Speeding up cognitive processes.” The second game involved reproducing patterns of squares after they flashed on the screen, and the third one involved focusing on a formation of five birds where one was facing a different direction from the rest. Each right answer uncovered part of a picture of a bird. Uncover the whole bird, and I’d get to add it to my permanent Bird Journal. Fun.

I love to tout the mental health benefits of my shitty flash games. "It's so boring, that playing it for more than five minutes will exercise your patience and focus!"

I had a sense that the scores in these games might be engineered more to keep players motivated than to measure their performance. Lumosity claimed its records of player performance gave it the “largest database of human cognition ever assembled,” and that researchers were actively exploring that database “to understand the determinants of cognitive performance and cognitive enhancement,” but it seemed unlikely that Lumosity could treat game scores as scientifically valid measurements even after introducing elements like multiplier bonuses that effectively meant the same score could mean two different things for two different people.

Later I played a full game of Playing Koi, a trivially easy task involving clicking on fish, and gained a perfect score of 1050. The only problem was that 1050 was not a high enough score to get a good Brain Performance Index. So how was I supposed to increase my “performance” to meet Lumosity’s standards? The game explained that by playing once, I had unlocked a new, higher-scoring fish pond. So I played another trivially easy round, with my brain feeling closer to shutting down from boredom than it was to being “trained,” and this time received a perfect score of 1600. And then I unlocked a new pond, the per-round bonuses arbitrarily increased from 100 to 200, and so on.

I played a koi once. My mother always told me that there are no small parts - only small actors. Ironically, I received the part because I'm so short.

The game mechanics were nothing out of the ordinary for a free Flash game, and there’s nothing more commonplace than a video game that makes you play through a few easy levels before moving on to higher-scoring ones. So why was I offended? Mainly because Lumosity claims the authority to certify that I’m now more intelligent purely because I participated in its level unlocks and point grinding. Because Lumosity calculates Brain Performance Index based on some kind of weighted average of your scores on different playthroughs, it was clear that my scores from my first few playthroughs would continue to drag down my average for as long as I play. That means that each additional time I play on the highest-scoring pond my BPI will continue to rise, even if I never improve at fish-clicking in any way, simply because I’m atoning for my unenlightened and doltish past as a person who hadn’t yet unlocked the higher-scoring levels. Familiar Faces was another game with level unlocks, requiring players to work their way up through ten levels compared to Playing Koi’s four. A look at YouTube showed that even the first Lumosity game I played, Memory Match, would have unlocked a higher-scoring “2-back” version if I stuck with the program long enough. So if Lumosity’s plan for generating fake progress against its BPI standard wasn’t obvious from the fact that they hired ShuffleBrain for a gamification makeover, it seemed clear from a close look at Lumosity’s games. Apparently a big part of Lumosity players’ BPI improvement was simply doled out for repetitive point grinding regardless of whether they got any better at the games.

There’s a serious question about whether misleading progress reports would undermine the benefits of brain training by causing players to focus on the wrong exercises. But that question assumes the mental exercises have any cognitive benefit in the first place, which is exactly what Lumosity has been unable to prove for most of its users. In a study published in Nature in 2010, a team of neuroscientists and psychiatrists performed a brain-training study on over 11,000 subjects, including 4,000 who were trained on tests of memory, attention, visuospatial processing and math similar to those found in commercial brain training games like Lumosity. The scientists concluded unequivocally that their results “provide no evidence for any generalized improvements in cognitive function following ‘brain training’ in a large sample of healthy adults,” and “that training-related improvements may not even generalize to other tasks that tap similar cognitive functions.” The basic problem was that Lumosity teaches to the test, much like a grade school teacher who shortchanges his students’ actual education by wasting weeks drilling them on the trivial quirks of a standardized exam. Some of Lumosity’s games might be a valid measurement of cognitive abilities for people who have never seen them before, but if you actually train on them you develop skills that are good for playing Lumosity games and nothing else.

You might wonder whether Lumosity’s fake progress could still be valuable because it motivates you to stick to your training plan and eventually reach some high-level content that’s incredibly healthy for your brain. I didn’t personally test that hypothesis, but I did find a good example of high-level Lumosity play in this video, where a player reaches a high level in a game about solving incredibly simple arithmetic problems. Instead of making the math problems on the higher levels actually, you know, harder, Lumosity just spams the easy problems until they fill the screen. The design problem here is that players can score points and advance by giving the right answer to any of the dozens of problems on the screen, so the YouTube player maxes out his score just by mashing every number on the number pad. This player is not exploiting the game in any way—he’s playing it exactly the way it was designed. He reveals at the end of the video that Lumosity certifies him with a perfect BPI of 1700, meaning as far as Lumosity is concerned he literally has the highest-functioning brain of anybody on the planet, but it’s obvious that any notion that he’s seriously exercising his intelligence has gone out the window.

The only studies that claimed healthy adults could benefit from Lumosity were tiny and scientifically dubious—the flagship study Lumosity relies on to validate its health claims involves only 14 training subjects, and uses research methods so bogus the company couldn’t get it published anywhere other than its own website. The study trained participants on only two mental skills: visual attention and working memory. In the visual attention training, Lumosity committed the basic error of testing the subjects’ intelligence using the same test they’d just been training on (minus the cartoon graphics). In the working memory training, Lumosity’s test results simply showed no significant difference in improvement in people who played the Lumosity games versus those who didn’t

Lumosity also showcases a study it published in the Mensa Research Journal which involved only 23 participants with an average age of 54. Again Lumosity was only able to able to make its trained group barely edge out the untrained control group, and once again the test subjects only showed gains on tests that were very similar to the Lumosity games they’d been trained on. But Mensa (which has decades of experience in the business of taking people’s money to praise their supposed intelligence) nonetheless allowed Lumosity to publish their bizarre conclusion that “the benefits of cognitive training transferred to untrained measures of core cognitive abilities.” Not only that, but Mensa has now moved to cash in on the brain game business on their own. Mensa Academy has now been published by Square Enix and was released in late July on Xbox Live, PSN, WiiWare, and Steam.

To be fair, there were studies suggesting that Lumosity games had their uses for at-risk groups. Children recovering from brain damage caused by cancer and chemo seemed to experience some benefit. There was evidence that seniors at risk of dementia might benefit from brain training in general, but a study showed that Lumosity itself improved their “visual sustained attention” and not much else. As the study results trickled in after its 2007 founding, Lumosity was starting to look like, at best, a minor medical product that doctors would recommend for a limited number of patients—not at all like the nationwide phenomenon every Bay Area startup wants to become. Lumosity needed a way to appeal to a much broader audience far beyond any populations proven to experience any health benefit from their product.

It succeeded. Lumosity’s Alexa traffic rank has improved from about 6,000 to less than 2,000 in just the last two years. Lumosity now claims 20 million users, and according to Alexa, the most common age ranges for Lumosity users are over 45 and under 25 (they’re also disproportionately less affluent, more educated, female, and childless). Why would hordes of women under 25 willingly spend hours every month and pay a recurring subscription fee for a training regimen invented to stave off senile dementia? One reason might be the marketing, which presents young women as Lumosity’s star “success stories” and relies on their anecdotal experiences to support sweeping claims about Lumosity’s health benefits. But another likely reason is Lumosity’s barrage of misleading and outrageously optimistic feedback.

As a venture capitalist said in a 2011 interview with Fast Company, “When we first invested, we were concerned this was just a niche area for people with Alzheimer’s or other cognitive problems. But Lumosity has proved there’s universal demand for this among all demographics.”

I'm sorry Mr. Shortz, Crosswords are NOT brain training. Have you tried "Playing Koi"?

Lumosity even partnered with the New York Times to provide the “Brain Games” feature on the Times’s website. The Times claimed that Lumosity can “improve the health and function of your brain with the right mental workouts” and that it’s “scientifically designed” by “some of the leading experts in neuroscience and cognitive psychology.” (The Times neglected to mention that one neuroscience PhD who worked on Lumosity was Amy Jo Kim, and her objective apparently was to keep players coming back to Lumosity, not to improve their cognitive health.)

The irony behind the Times making these claims was that the New York Times Magazine published an article earlier this year called “Can You Make Yourself Smarter?” The article quoted experts who explained that reputable studies had failed to show any transferrable intelligence benefits from “brain training.” Although the article held out hope that the newer N-back test could grant its users transferrable mental skills, most of the games the New York Times offered on its “Brain Games” page had nothing to do with the N-back test. When I emailed the Times’s PR department about how they justified the health claims packaged with their games, they refused to disclose even how long the “Brain Games” page had existed on the grounds that the Times was “no longer doing business” with Lumosity and the page should have been taken down. Indeed, within hours of my inquiry, braingames.nytimes.com began redirecting to this adorably perplexed cartoon boffin.

I actually picture this guy as responsible for every website on the internet (he makes them out of chemicals).

For the time being, the New York Times’s Brain Games page can still be seen on the “partners” section of Lumosity, and Lumosity still has many outposts at health insurance companies’ sites as well major publications like the Guardian, the Telegraph, Popular Science, and Woman’s Day. At many of those sites you can play games for free that would cost money if you went to Lumosity’s own site. Although the Times’s PR representative insisted that the Times’s relationship with Lumosity “was not an advertising relationship” and the Times’s Brain Games page was not “advertorial” content, Lumosity does market an affiliate program that allows popular websites to earn commissions on sales and leads they refer to Lumosity.

Maybe these outside sites can also help explain how Lumosity guarantees its players will progress in BPI. Nonsubscribers who encounter Lumosity games will probably sensibly quit playing after one or two repetitions and never unlock the higher-scoring versions of the same games, and even if they do unlock them they’re more likely to lose their progress because they don’t have account credentials. It would be interesting to know whether Lumosity adds nonsubscribers’ games to what it calls its “database of human cognition,” because Lumosity assigns BPI based on where a player falls on the spectrum of games that have already been played. If Lumosity is like most online games where only a small percentage of people who play for free get converted into paying customers, then it’s logical to suspect that Lumosity’s statistical curve of BPI scores is heavily weighted with players who were limited to the introductory score caps on many of the individual games. That would mean that when Lumosity subscribers picked up the score bonuses for level grinding, they wouldn’t just be told they were exceeding their old performance—they’d be told that they were outthinking the vast majority of humanity and rapidly ascending into genius territory. That could be a pretty convincing incentive to keep paying the monthly fee, at least for a certain audience.

My experience left me believing Lumosity still had questions it needed to answer for its users. When a game markets itself as a health product, doesn’t the developer have a responsibility to market the game only to users who can benefit and only to the extent that they can benefit? When players use a game for health reasons and not for entertainment, shouldn’t the developers be expected to give positive feedback only for behaviors proven to be healthy and not to engineer around the players’ natural aversion to exercises that don’t challenge them? Or should Lumosity’s health claims be considered so outlandish that no one would seriously rely on them anyway?

This entry was posted in Critical Conversation and tagged , , , , , , , , , , , , , , , , , , , , , , , , . Bookmark the permalink.
  • brighter future

    holy smokes your article dragged on.. the games do get difficult, and it doesn’t take long, you can switch the settings to difficult and within a short period of time you will really testing your brain.

    You say that the BPI is a meaningless number, however the people at my University with noticeably higher IQ’s climbed to the top percentiles within two weeks. Where as someone with a lower IQ it may take years.

    wow that article was a waste of time. thanks

    • http://www.mediumdifficulty.com/ Karl Parakenings

      To be fair, so are IQs.

      • Brian Higdon

        If IQs were truly meaningless, we would’ve stopped using them a long time ago. We still use IQ testing in certain circumstances, while acknowledging its limitations.

        • Phil Lichen

          EQ is as important if not more important than IQ to be successful.

          • Jeff Little

            It depends. Do you want to be a successful real estate investor or a successful physicist?

          • John R. Miller

            It depends on the career.

            There have been studies conducted over about the past 50 years and to make a long story short, the three things that are most important to success are IQ, practical intelligence, and creative intelligence. (EQ is a factor of practical intelligence.)

          • Ubwart

            No even close, Bill Gates has an EQ of -1 and look at him.

    • Phil Lichen

      I agree BPI is totally meaningless… My BPI has gone down 100 over two months playing on a regular basis. Should I be worried?

      • John R. Miller

        Here’s a metaphore to encourage you to keep at it. Think of it like the economy. Occasionally there is a recession, but in the long term there’s mostly inflation.

        There are a lot of other factors that could be hurtingyour BPI as well. For example, I play worse when I’m about to go to bed and play better in the afternoon. Do you play at a different time than you did 2 months ago? Other factors could be stress, drowsiness, the amount that you’re working a week, the distractions around you, and literally hundreds of others that I can’t think of right now.

        - John

      • Lex

        Make sure you are in a quiet setting, or one that allows you to focus every time you play. If you don’t, you might see declines like you have. Also, give your best effort. If you give a half effort after working hard to get to a certain level, your BPI will naturally go down.

  • Brian Higdon

    Yeah, your article was lengthy and perhaps a little overblown, but you did pretty well at debunking Lumosity’s BPI as actually measuring real improvements.

  • jason cowan

    I’ve noticed significant memory decline playing lumosity games, although they do appear to help in making decisions more rapidly and in problem solving. In my experience, they may even be harmful to memory! For me, improvement in my memory capacity and functionality was the main reason I joined Lumosity, and I have to say, I’ve been sorely disappointed in that regard.

    • Dude

      I know this is 2 months late but I have experienced similar results. I realized that I was playing everyday and this may have tired out my brain to the point that I was sluggish in many cognitive areas. I will try to rest from the games and see if things improve.

      • Ruchira Datta

        Did you experience this memory decline in real life, or in your memory BPI? In either case I doubt it has anything to do with Lumosity. I’ve been playing Lumosity games for four years, haven’t noticed any memory decline in real life, and my memory BPI is at its highest ever (1592).

    • Lex

      You can also change the areas of focus in your training to memory if you haven’t already. But I would recommend, as “Dude” already has, don’t play everyday. In order to have a good memory, you need three things 1) multiple exposure to the material (you’ll need less and less for new material/subjects/etc as your memory improves), 2) training over a longer period of time, i.e. cramming for exams will get you a worse grade than if you start studying two weeks prior, and 3) time off and plenty of sleep. Sleep especially is an important part of memory.

  • http://www.facebook.com/april.padula April Padula

    Thank you for taking the time to do all of this research! I was poking around on Lumosity, and before I was going to pay for any kind of subscription I wanted to see if the “training” had any validation in actual research. Of course not, just another slot machine to take your money!

  • http://twitter.com/cphowe Christopher Howe

    All such websites are after your money. But really, its $1/week! Does it really matter if the games don’t do everything they claim to? The point is, with any training, it’s specific. Want to run faster? Practise running faster. Want to know more words? Read half a page of dictionary entries every day and try finding ways to use them. I have a Lumosity BPI of 1300 (puts me above 95% for my age group) and I find at this level the memory grid rotation, koi pond, and brain shift overdrive pretty challenging. Familiar faces isn’t the best game, for sure. But eagle eye – those birds and dummies flash up pretty fast at the highest level. That’s good peripheral vision and attention training, whatever the score. And – you get a reminder everyday. For most people, whether its a real gym or a brain gym, its the little push that gets you going that makes all the difference. At less than 10 cents a day, that’s good value.

    • Collier Smith

      $1/week is over 14 cents a day, not “less than 10 cents a day”. Hope your brain improvement includes a little basic math…..

    • Dan

      I also use Lumosity (more because I enjoy the games than because I’m at all convinced they’re doing me any good), and wouldn’t you know, my “BPI” is also above the 95th percentile for my age group. So, reportedly, are the BPI’s of two other people I know who use it. What’s more is that all of us seem to have started out somewhere around the 30th percentile and then made clear gains for a couple of months before leveling off where we are now.

      That everybody who uses it seems to have the same scores is suspicious, though not half as much as the fact that I have on multiple occasions achieved a new high score for a particular game, only to be told that I had just driven my overall BPI down. What led to me to this article, actually, was the observation that the site has stopped telling you what effect any individual score has on the overall BPI. Apparently too many other people noticed how little sense their system seems to make.

  • Adam

    With all due respect, Mr. Carey, you are an idiot. God damnit, everyone thinks if they have one modicum intelligence that they need to write some fucking blog and claim to be a scientist. My BPI went down for a period of time and I actually had to work to do something about it. By the way, I don’t pay for the site (although I would like to), but I still learn despite the fact that I’m using it for free.

    For those of you who are optimistic that Lumosity can help your brain performance: At the heart of everything, life is one big game. I find that Lumosity has significantly helped me to think on my toes, concentrate, and connect ideas in the straightest line possible. I am taking the GRE (Graduate Record Exam) in roughly two weeks. I would be FUCKING SHOCKED if all of the practice I have done is not helping me to get a better score on that motherfucker. Lumosity practice = high GRE score = good graduate school = high income after graduation. I may fuck up on a lot of things I’ve done in my life, but if there’s one thing I have faith in, this is it…

    • Amy

      Hopefully after graduate school you will find a better vocabulary.

  • Lex

    I have a master’s degree in medical science and I’ve been using lumosity for about 7 months. First off, if you would take a neuroscience or neurobiology course, you would understand why lumosity works. Second, I have noticed benefits from using the system. I started it in the last semester of my master’s degree and noticed a remarkable difference in my test taking abilities, memory, and speed of thought. I am also prepping for the MCAT and have noticed a huge difference in my test taking abilities (part of that no doubt is from studying the material, but you have an average of 1.5 minutes to answer each question on the exam and they are not easy). I also have become better at remembering people’s names upon meeting them. Third, regarding the New York Times article, certain kinds of intelligences aren’t transferable, but lumosity games work on brain skills that are transferable (more general skill rather than specific abilities/skills). A neurobiology course would help you understand this. Lastly, lumosity is sponsored and endorsed by major research universities including Stanford and Harvard. I’m pretty sure that those universities wouldn’t be affiliated with something that is illegitimate.

    • John R. Miller

      I knew this writer was full of it when she said right off the bat that gamification simply turns programs into “crappy” slot machines. What bullshit! This is the logical fallacy of generalization at it’s finest.

      Thank you for this post. It’s good to see some sense among nonsense.
      I guess the moral of the story is to not believe everything you hear off the internet.

    • Al

      So to recap your attack on the article: 1) an understanding of neurobiology would mean the author would understand not only that Lumosity does work by why; 2) Lumosity is working for you; 3) some intelligences are transferable and Lumosity works on those; and 4) Stanford and Harvard would not be affiliate with something illegitimate. The article, however, is fundamentally about the gamification of the Lumosity games and how this suggests that the BPI and the game play are suspect. Your four points don’t really go to this at all, let’s examine: 1) this attack (if you knew neurobiology you would understand Lumosity rocks!) is so vague as to be meaningless, I think that if you knew the theories behind gamification and the social economics of incentives you would understand Lumosity sucks — see how neither statement is actually meaningful? 2) this is an anecdote — other commentators say Lumosity is hurting their mental cognition — this is not evidence. 3) I am not sure what you are saying here but Lumosity claims to focus on specific skills (remembering peoples names, aptitude in quickly solving math problems) that are not clearly transferable. The YouTube video should confirm that. 4) seriously? This argument from authority is sad. Because Stanford’s names is/was somehow connected to Lumosity it must be scientifically valid? Stanford has a football team that it sponsors does that mean that football is a legitimate sport? (To give you the answer it is “no” Stanford sponsors football for a variety of reasons arising from history, culture, and certain beliefs (the value of organized sport for example) that do not say anything about the inherent value of football qua football. A specific critique of the article that did not rely on vague pseudo-science would seem more genuine and less like you are a Lumosity shill who is trying to prevent people from reading the article and doubting the value of spending money on Lumosity.

      • Lex

        The only reason I wrote my first response to this article was because
        a) the article itself had shoddy scientific/clinical evidence, b) was
        playing on several social theories, but offered no actual legitimate
        information stating that Lumosity arbitrarily increases your progress,
        c) because the author had no concept of medical or neurobiological
        science in the article anywhere, which I felt was relevant, and d)
        because I wanted to avoid writing a massive response, such as the one I
        am about to write–thank you for forcing my hand. I will respond to
        each of the points rebutted from my previous post and then I’m going to
        be done defending this because there is too much information to dispense
        in a post on a forum in order to validate Lumosity’s benefits. So here
        we go–

        1) I gave the “vague” “take a neurobiology course and
        you’ll understand” argument because in order to actually understand what
        your brain does and how this program works, you would have to take an
        entire course to begin to understand. But because you insist that I am
        “so vague” and this argument is therefore invalid, I’ll give you a few
        keywords that anyone is welcome to look up and they will better
        understand neuroscience: neuroplasticity, long-term potentiation, and
        the homonculus and studies on its neuroplasticity. Those should keep
        you busy for a while and most people still won’t comprehend everything
        because it requires somewhat of a foundation in science to understand
        the material, but that is not to say you can’t build that foundation by
        looking up terms and concepts you don’t understand as you go. Also, I
        am an avid gamer, and while I would never say I am an expert in
        gambling, I do understand the theories and how reward plays into
        behavior. It’s called Classical or Pavlovian Conditioning and that was
        actually a psychological/neuroscience finding. I also majored in
        economics for a time, while taking several psychology and sociology
        courses, so I am familiar with the theories being presented in this
        article.

        2) Yes, this is an anecdote. If someone’s testimonial is
        ever considered evidence (aside from a courtroom, oddly enough), the
        people considering that will have a very hard life as they have no
        critical thinking skills of their own. You also said some other
        commentors say Lumosity is hurting their mental cognition, which I doubt
        to be true simply because playing a game cannot hurt your mental
        faculties–there must be another factor at play or it is personal
        perception, which can often times be subjective and inaccurate as
        evidenced by eyewitness accounts of the same event. While I find that
        for the majority of people that would probably be untrue, I will not say
        that it is impossible. Every human is unique in its make, and while
        some medications work great for most people with a particular condition,
        it can seriously harm or injure others. Hence, not everything works
        for everyone, which I’m sure most people with any life experience know.

        3)
        I have already asked for you and others to look up information on
        transferable intelligences, but I’ll reiterate simply because of you
        obviously don’t have a background in neuroscience and were too lazy to
        look up the details of transferable intelligences yourself. You think
        that Lumosity works on specific skills. While to the layman (person not
        familiar with neuroscience) it might seem specific, but in the world of
        neuroscience studies, they are not considered specific skills. A
        specific skill would be more like being able to balance on one foot
        while pressing a button with your finger or the Tower Of Hanoi. The
        intelligences Lumosity works on are more “general brain skills” as
        deemed in a neuroscience context, but you wouldn’t know that unless
        you’ve actually had a course in it.

        4) The reason I even brought
        this up is because Stanford and Harvard are two of the biggest and
        well-funded private research institutions in the world. Many scientific
        breakthroughs have been made at these places. Also, these institutions
        are using the data from users of Lumosity to better understand
        neuroplasticity and long term potentiation. With your asinine
        comparison to football/sport, you are saying (I’m inferring) that the
        legitimacy of sport is under equivalent scrutiny as the legitimacy of
        scientific study. If that is the case, I’m sorry to be so blunt, but
        you would be the biggest ass with which I’ve ever had the pleasure of
        rebutting an argument. While the legitimacy of certain sports is a
        conversation that goes on in the sports world (I would know, I was a
        Division I athlete), it is not nearly on the same level of discussion or
        importance when it comes to the legitimacy of academic study, because
        this actually involves the health and well-being of people, whereas the
        discussion of whether or not football is a legitimate sport is not going
        to affect, good or bad, the health and well-being of people.

        Lastly,
        I want to say a few things. I have no regrets spending money on
        Lumosity. While I am fortunate enough to say that it did not cause me
        huge financial difficulties from purchasing it, I still believe it is a
        helpful tool and can be for others if they can afford it. If you
        cannot, there are other methods that produce similar results in such
        areas, such as Sudoku, Crossword Puzzles, and arithmetic. Elderly who
        keep their minds sharp with these activities tend to have a lower
        incidence of Alzheimer’s and other neurological deterioration, i.e.
        dementia. My original response was not to encourage every single person
        to buy Lumosity, as I have no financial ties or interests in it other
        than my own purchase for access to the program. My purpose was to bring
        light to this article that was void of any medical/ neuroscience.
        While I do not know Lumosity’s algorithm for BPI, how it works, or if it
        is truly rigged like a slot machine (while I don’t believe this is true
        because I have increase very little in BPI in the past 4 months), I can
        say that I do know this program works on sound neurobiological
        principles and is helpful despite the progress and percentile charts.

        To Al:

        I
        don’t know what your angle is here–maybe to appear smart and
        condescending, maybe to thwart Lumosity from gaining any speed, maybe
        because you’re bored and you want to pick an argument with someone.
        Regardless, I would ask that next time you attempt to respond to
        something of which you know very little about, that you actually educate
        yourself as to not elicit the impression of a complete ass while
        responding.

        Again, I hope what I have said can encourage people
        to learn about neuroscience and the possible benefits of activities
        like, but not always necessarily, Lumosity.

        • disqus_ZKffLzRBXN

          Your response is well articulated, butnot true. The article provides sound arguments for why Lumosity artificially increases your BPI and therefore exihibits a “programmed” progress. Your four points do not provide any sound argument. Point 1 and 3 applies to anything: training in any activity makes you better in that activity – at least initially – and in similar activities. I see no evidence why Lumosity develops mental skills better than other activities. Point 2, you admit it yoursel that it has limited validity, and point 4 I have no idea of what the situation is (a simple endorsment by someone there or an agreement to use data does not mean Lumosity works). I actually play lumosity for a while and do not feel my mental skills have improved, but hey it’s just a game and I play it for fun just like I play other games that develops many of the same skills. Just not pretend it’s more.

    • Keshet59

      Gee whiz, anything that would cause a “huge difference” in test-taking abilities viz-a-viz the MCAT would be valuable snake oil indeed. Perhaps it’s changed in the 20 years since I took it. The only thing that helped me was studying. I don’t have a Master’s in “Medical Science” (I’m not familiar with MS degrees that are not in a specific scientific discipline, so I might need to be educated). I do have an MS in biochemistry, which didn’t help much with the MCAT, since its science is more basic. It did mean that I could blow off biochem classes and put more effort into anatomy in my preclinical years in medical school. But, I digress. The claim that Lumosity is “sponsored and endorsed” by major research universities… please. Not true. It’s a pleasant and diverting entertainment, but any claims that it improves one’s…neurobiology… are sheer hucksterism.

  • readysetawesome

    This article makes a lot of general assertions without really concretely debunking anything about Lumosity, as far as I can tell. I’ve been using it for 4 months, have seen unmistakable differences in my abilities. My wife specifically noticed recently that I’m better at remembering where things are, and doing my chores around the house.

    I do agree 100% that this program, like any other similar commercial product you’d pay for, will show you “improvement” – according to their standards – no matter what. They have to retain users to stay viable and profitable: so do most web software companies. But that does not mean that you should dismiss the whole thing as quackery.

    Try it for yourself people – my brain moves faster, my memory is improved, I remember faces+names now. Its unmistakable.

    • Al

      This comment, again, follows a classic con approach. Because the article is persuasive the comment initially makes you doubt whether you should have been persuaded (“[t]his article makes a lot of general assertions without really concretely debunking anything about Lumosity”), but the attack on the article is so general as to be meaningless. What “general assertions?”

      The comment then segues into a personal story “I love it, my wife loves me more now because I am smarter and do my chores.”

      The comment then confirms that the reader is maybe right to doubt Lumosity because it has a profit motive and does show “improvement,” but then immediately undercuts that by saying “everyone on the web has a profit motive that does not mean they are all quacks.” Thus, assuaging the fears of the doubtful reader by dealing only with the profit motive part of the concern and ignoring the real science concern that if everyone improves on the games what is really being tested in the first place.

      Finally, there is another pitch — just go play it and pay for it, I got smarter maybe you will too — meant to close the deal. I really don’t think that these comments are for real given their uniform structure and their complete lack of engagement with the actual thrust of the article.

  • http://www.mediumdifficulty.com/ Kyle Carpenter

    An American scientist once visited the offices of the great Nobel
    Prize-winning physicist, Neils Bohr, in Copenhagen, and was amazed to
    find a horseshoe was nailed to the wall over his desk.

    The American said with a nervous laugh, “Surely you don’t believe
    that horseshoe will bring you good luck, do you, Professor Bohr?”
    Bohr chuckled. “I believe no such thing, my good friend. Not at
    all. I am scarcely likely to believe in such foolish nonsense.
    However, I am told that a horseshoe will bring you good luck whether you
    believe in it or not!”

    (This particular phrasing is from http://www.laughbreak.com/stories/horseshoe_logic.html, though the story is verifiably true)

  • Abby Dachman

    At times you need to take the “facts”, science (especially when you are simply regurgitating someone else’s findings, not your own), and papers, studies, journals, etc. out of the equation, and simply ask yourself, “Has (insert here) made life more worth living either for today or for the future?” I have used Lumosity for a month or so, almost everyday for about an hour … Am I smarter? No. Do I find myself having capabilities now that I did not have before? No. I can say however, that I have more confidence in my thought processes; they feel cleaner as long as I am combining the exercises with adequate sleep. I can also say that I have more energy for said cognitive processes. Where before I may have perceived a task at school, work, etc. as tedious, my completion of the task feels more organized. I think the end result of any similar “brain training” may not be to raise your IQ or awaken any dormant areas of your brain, but simply to tune up whatever it is your already working with. I don’t really even care if Lumosity markets itself as a new engine for the car that is your mind, anyone should know by adulthood that such advertised revelations aren’t real … any product that promises miraculous improvement in anything is a ploy. I however am happy to pay a small amount a month for a mental “oil change and tire rotation”. The world is by majority a lie; always has been, always will be. Find what makes that easier to accept and don’t spend so much time bitching and moaning about it.

    • John R. Miller

      Trying to notice whether or not lumosity is working by means of introspection is like a person trying to notice if their hair’s gotten any longer the past month by staring into a mirror. Try it out for one or two years, then you might be able to think back and notice a difference.

  • Jorge

    I was lucky enough to have been offered a limited access account for free from Lumosity, it came as a pop-up on my Words With Friends app and I’ve been using it for since like Nov. 2012. I’ll try keep it short for anyone reading this (Vs the author of this article). These games wake me up and it feels like I’m doing something good for my brain by paying more attention to things before I click on my keyboard or mouse. I typically start my day at work off by getting a cup of coffee and playing Lumosity, so I personally believe it helps. Yes, after playing a few games a few times, some have a few things that can be figured out to help me improve my BPI scores (kind of like cheating I guess), but they still require me to use my brain and I might not have gotten a chance to use those cognitive skills on a certain day otherwise. I’ve personally experienced a better flow of speech and thought that a friend of mine (that I know since childhood), also noticed. I’m now looking at my options to purchase the service.

  • Charles Gnarlson

    you must be an athiest too. Horrible article. This is why people like you shouldn’t have a voice

  • DILLON OLIVIERO

    I have a 902 bpi and i’m 11.

  • DILLON OLIVIERO

    and a iq of 150

    • Garrett Facer

      You are also very proficient in the English language……………..

  • Ubwart

    I have an IQ of 143 and have been an off again, on again Mensa member for years. I wonder how I would do on Lumosity? Would I be in the 25% percentile until I spend money playing their games. It sounds like Scientology to me.

    • Dude

      Let me start by saying Mensa members rarely show much more than pride in their capability of thought. As for your money = rating ideas, sure, you may be right, but take for example Netflix. Would a company like netflix allow trial users access to all of its movies/shows without limitations? Now it’s true that netflix does not rate your cognitive abilities. However, wouldn’t you think that you might need to be using the full website to get a accurate rating anyway? So maybe paying for it would make your bpi rating on the website go up, but I think your point was more to make notice of your IQ rather than to seriously comment on the article.

    • Becky Adler

      At the moment

  • Eucalyptus Wolfe

    My BPI hovers around 1550, with 97-9th percentile across all measures, and I can say for sure that if you played the levels at this difficulty, you would know what a challenge it is. Far harder than any video game at the hardest levels (which I also play). Video games have established research into their cognitive benefits. Lumosity provides an additional challenge.

    Isn’t there a scientific fallacy of confirmation bias where a scientist skims the information supporting the position they oppose, whilst maximising the rhetorical depth of their own position? It is so easy to knock down a man of straw…

  • Dude

    You need to verify a lot of your research before you post and article like this. Right off the bat I noticed more than a couple “facts” about the website which just are not true. I don’t have to argue with your conclusion because just or not you loose credibility when your supporting topics are fraudulent. I’m not going to go into what was wrong about your article because honestly I don’t care about the answer, but I will say that if you are going to make statements like this you’d better make sure that what your saying is true.

    • Al

      This comment, like many in response to the article, ignores the main thesis of the article — that the gamification of the Lumosity games suggests that the data derived from the games are suspect — and instead attacks the article for failing to “verify a lot of your research.” Interestingly though, the comments never identify the specific facts/research/premises that the commentator believes were not properly supported. For example, is the section about the small sample size of the Lumosity study wrong? Indeed, these comments, with their mix of conversion stories/anecdotes, vague pseudo-scientific attacks, and “hey, this may or may not work but it really is fun and not too expensive” pitches, read like the work of a whole group of Lumosity employees who are attempting to discredit the article for marketing reasons.

      • Dude

        i’m not attacking it. Either way, false progress or real, it doesn’t matter to me. And it’s not up to me to do the research for them, I’m not writing a rebuttal article. But when I read articles I’d like to read something that is unbiased and based on actual research. This article is a rant and nothing more. It becomes clear early on that he doesn’t know much about the website or the topic. All the research can be found in the research done by MIT, Harvard, and Stanford. If your interested you can actually read that and not some article by someone who probably went to school for journalism. I’m just letting people know that this is not a fact based article, but the facts are available.

  • Felis

    I caught Lyme disease last summer and noticed quite a decrease in my mental abilities during and afterward that was not repairing itself- plus my memory has always been rather poor- within 6 weeks of playing (nearly every day), *I* noticed a drastic increase in these abilities. I was thinking faster and remembering much more than I ever had. I can’t comment on the whole BPI/percentile thing.. I don’t really follow it to be honest. What counts for me is how I function.
    I got a one-year membership originally and while I still have 8 months left, I am seriously considering going for the lifetime. I don’t want to be cotton-brained again.

  • Andrew

    What’s the problem of having a “proprietary algorithm”? The quote shows that she doesn’t understand business, since she mocks it the way she wrote.

    I think she just grabbed a lot of questions and unanswered propositons to prove her point that everyone have the same IQ ever (since it seems for her that there’s no need for anyone to train the brain – like neural networks are trained for any tasks and get better at it!) because she might think she’s overly smart, but had a low BPI in the first weeks… lol

    This text appears to be from an old person that believes children will be smarter if they don’t play any video games ever (because they’re bad and will make poor child dumb).

    The last phrase is hilarious, it sounds like she’s trying to convice you that everyone is doubting the benefits of neural training and this is another conspiracy.

    I agree with Lex’s comment: someone needs to study neurobiology and neural networks.

  • Carlos

    Yeah, and one more thing about BPI.

    I guarantee that stuff is rigged. Everyone who talks about their score pretty much says they have a BPI around 1300-1400, so I would say the average is about 1350. I mean, you might say, “But hey, they also give a percentage here! They say I’m blahblah% better than others! They totally can’t be lying there!”

    Well, truth be told, maybe they ARE lying. I mean, do they ever give you any raw data lists? No. They don’t. And you can’t see the names of every single person’s score every single day, and they are in complete control of the system. For all you know, they could be including tons of beginner accounts which have zero BPI, therefore lifting everyone’s % higher.

    Look, everyone wants to feel better about themselves and brag. Lumosity doesn’t want anyone to feel less than average, or else they *lose money* because people feel upset. So as long as all of their users can hop up above 50% at some point, they can all be satisfied with their self-esteem and glad they joined Lumosity, even if, little do they know, it could be somewhat fabricated.

  • Todd Brackman

    You know what I like about this article? Even though it is based partially on anecdotal evidence, it raises thoughtful questions unlike some of the biased comments below from Lex and Miller.

  • http://modigmovie.weebly.com/ Modig_liani

    I’d like to point out that you only ascend to the higher levels of a game if you score successfully. The Koi pond is a perfect example. The author had an easy time with the first few levels. So the author ascends. Eventually, the author would reach a stage where he couldn’t get a perfect score. At that point, you stop forging ahead. If you can’t get a perfect score on the pond with 14 fish, you won’t move on to the higher level where you get more points. It’s not a “no child left behind” policy where you ascend to higher ranks based on how many times you play.

    That makes sense, right? So someone who struggles with six Koi won’t progress to the higher level until he/she can conquer that pond.

    The memory puzzle is a great example as well. For the first couple days, I kept getting to 7 blocks and not moving ahead. As soon as the 8th one was thrown in: I failed. The game wasn’t just ascending me to the next level. I had hit a point ceiling. So I played that game over and over again. Now 8, 9, 10 blocks are no problem. My high score is 42,000.

    It’s no different than weight lifting. Hand me a 5 lb dumbbell, I curl it. Hand me 10, I curl it. Hand me 15, I curl it. Hand me 20, I curl it. Hand me 25, I get there but I actually have to work at it. Hand me 30…okay, got it. 35…Ah!, got it. 40. No. Someone else might get to curling 65 lbs. You repeat that weight until you’re stronger. And then you up the weight.

    It’s a good point about many users reaching point caps so not maximizing BPI, which means the paying users seem smarter than what they really are because they see their percentile index and are like, “Yeah, look at me go!” Can’t argue that.

    At the end of the day: I like it. My brain feels more unified.

  • Listen to this guy

    Disregard this rant, the writer of the above article is just angry because the tests said he/she’s stupid and this is article is a reactive, retaliation. This writer is probably below average by lumosity’s standards.

    Neuroscientists > angry, unscientific, uneducated, opinionated and ignorant writers any day.

  • Tommy G

    It is proven unequivocally that learning new skills, such as a musical instrument or a new language, increase mental acuity and can decrease the onset and/or progress of dementia. Also, the physical therapy of brain-injured patients, Alzheimers/demential patients involves mental acuity exercises that have been shown to have quantifiable positive results. Learning to think fast under pressure is an acquired skill. Being able to make estimates and do mental math are not skills you are born with. Is the science behind “BPI” solid? Probably not. But the age-comparative performance in each category represents a pretty decent measuring stick of improvement for the vast majority of people who are using the system to improve themselves. Anyone who would waste their time cheating at Lumosity is truly pathetic. The games are not enjoyable enough to play solely for entertainment in my opinion, and you don’t get to have your own screen name on standings. There are plenty of games out there you can cheat on and get your Avatar up at the top with the other cheats, and they are probably a good deal more entertaining. Lumosity is a positive mental acuity training program with somewhat artificial improvement measurement.

  • Tezurak

    I think what is the most interesting thing about this Luminosity site is that as an avid gamer from a young age to current day, the first thing I thought after my free session was, “This doesn’t do anything for my intelligence, I am getting high scores because I am just really good at video games.” After that, I just play the three free games each morning to help me wake up more than actual “brain training.”

  • harryplayer1

    I have to say that many aspects of my daily function have improved since starting lumosity training. Also there is nothing false about the rankings or points system. It is always a fact that as soon as I have a bad effort that I lose my overall score. These marginal accuracies are best noticed when you have 100 per cent in your age group for certain disciplines, because it easy to lose your ranking with a sloppy performance or two. I reached 100 per cent for ny age group after a few months graft, then one night I played drunk and lost my position. I can also remember my wifi pin which is 16 numbers and letters long, which I can assure you would have been impossible beforehand.
    Stop knocking the product, and learn to be a decent human being. People who make a habit of criticism really are lacking, maybe they need to train a bit more..

  • Red Haired Rock Head

    I’m a curmudgeon and a sceptic so I don’t pay attention to the gamification of Lumosity. I’ve no idea what my BPI is and I don’t look at my scores. I think probably that seems strange to some, but I don’t play video games and I already have a fine concept of my intelligence from living life. I only play the free games. After a few weeks I feel like my memory is better. That’s good enough for me. I will say I do other things to keep my brain stimulated and ‘plastic’ – I read, play cards, write. I would say Lumosity is worth my time but certainly not worth paying for! You would have to be a real dope to think BPI is a real thing, or that playing so e computer games is going to make a really big difference in your life.

  • AriD2385

    It’s interesting to me that this article directly attacks the “gamification” aspect of Lumosity, as that is pretty much why I started playing in the first place. As I was wasting time on one of those addictive Facebook games (Candy Crush in particular), I thought that there had to be something that combined the addictive reward mechanism of video games with developing a skill that’s actually helpful. So I went Googling and came across Lumosity. So, to start, I don’t agree with the premise that even intentionally manipulating users’ reward circuitry itself indicates that the program is without actual value. The gamification aspects keeps you engaged, but it’s the exercises themselves that provide the benefit.

    Being of the video game generation, I did initially think that it didn’t seem much different than playing increasingly more difficult levels of Donkey Kong. Sure, you get better, but aren’t you just getting more familiar with the tasks required? Well, research on video games in general shows that gaming definitely impacts certain cognitive skill and develops them in a certain way.

    It also seems true simply from life observation that if you practice certain cognitive tasks, that you will in fact, get better at executing them. That is not to say that your overall intellectual capability will increase, but rather that you will get better at certain mental tasks the more often you do them. For instance, people teach themselves how to memorize large amounts of information and get better at doing so through practice. If I knew more about neuroplasticity I’m sure I could explain why that is. But just the fact that we know from experience that we can develop various cognitive skills means that Lumosity’s claims don’t seem like so much of a stretch.

  • Nerd Alert

    I have used the subscription version of Lumosity on a daily basis for several months now. While I cannot claim to know Lumosity inside and out, I have had a chance to evaluate some of the strengths and weaknesses of the site.

    I think articles like this one are important. Many people have been bombarded by Lumosity ads on YouTube and elsewhere. They need a dose of healthy scepticism to balance things out at this point.

    In my experience, the “gamification” of Lumosity scoring becomes obvious to anyone who plays for more than a few days. The BPI is clearly intended to increase for any active user, as this author suggests, simply by way of level unlocks and “points grinding”. I don’t think anyone who is familiar with the site would argue otherwise. In other words, the BPI score and the graphs that go with it are a bit of fun, and can make the site more interesting, but they shouldn’t be taken too seriously.

    As far as noticeable changes in cognitive functioning in daily tasks – I would say I haven’t noticed a great deal of change, even with the dramatic increases in Lumosity BPI. I still forget where I put my keys. Reading research articles is still tedious. I still use a calculator when I run payroll at the end of the month.

    I would say there has been one exception. I have started to remember names when meeting people, which has been a lifelong struggle for me personally. I can’t credit that change to anything else, I’ve been making efforts to improve in that area for many years, so that piece has been valuable to me personally.

    I think the true value of Lumosity is that it gets people doing SOMETHING with their brains on a daily basis. It is far to easy to exist in a vegetative state in our society – driving the same route to the same job, watching the same television shows and having the same conversations at the water cooler with the same people. Many of us could sleepwalk through our day-to-day lives. At least Lumosity requires a person to focus and think for 15 consecutive minutes. Use it or lose it, no?

    The personal trainer comparison is actually not bad, in my opinion. The program is laid out for you, and you just have to show up and do it. People argue that you could save your money and complete a crossword or do some Sudoku to get the same benefit. I would fully agree with that point. In that same spirit, everyone has the freedom to exercise at the gym independently, but only a portion of the population actually follows through. You could ask why people are wasting money on personal trainers when they could simply exercise on their own for free. Well, it seems some people enjoy planning on their own, and other people enjoy being assigned challenging tasks. That’s the funny thing about people, everyone’s different.

    So my conclusions after a few months of using Lumosity? 1) It does challenge the brain to solve puzzles 2) It seems to have helped me remember names, but no other dramatic changes are noticeable at this point 3) if you like solving puzzles on your own anyway, save your money and keep doing your own thing 4) if you are the type of person who would hire a personal trainer so your daily program is decided for you, then Lumosity will probably suit your personality

    To people who have never used Lumosity, I would say don’t knock it until you’ve tried it for a few months. To people who do use Lumosity, I would say don’t expect miracles, and don’t get excited about the BPI scoring system, it is designed to increase as you play, whether you become smarter or not.

  • John James

    If you want to really learn something new, then study it. Buy a book and read it.

    If you want to learn music, then get an instrument and learn to play it with a teacher and practice with it regularly.

    I do not see how playing games makes one learn anything else than playing the game. I see activity like Lumosity as the lazy mans’ way to ‘get smart’ without actually doing anything intellectual. Lumosity reminds me of the words in the Dire Straits song “Money for Nothing”…

    Money for nothin’ and chicks for free
    Now that ain’t workin’ that’s the way you do it

    • Nerd Alert

      Well, no matter what concept is presented, there always seems to be a naysayer who knocks it out of the park with a quote from Dire Straits. There’s just no way to counter an argument this solid. Game, set, and match. Pack it up, Lumosity. You had a good run but this gentleman clearly had your number.

    • Jennifer Bell

      I am a relatively new player – have made steady increases in all areas. Don’t know whether it is making me smarter but I am enjoying the challenges. I have questions about the statistics; for instance, when comparing with others, does that mean others online in a particular time-frame? Most of the posts I have skimmed through are people showing up in the top end. So what about the low achievers – what’s in it for them? If a subscriber drops out because they are making disappointing or no progress, if their low scores stay in the system, does this skew the figures to make the high achievers achieve even higher (comparative) scores? If a subscriber leaves, I’d be interested to know what happens with their results. J

  • Martin Payne

    I have been playing Lumosity games for 4 months now with no payed subscription at all and can speak to some of the assertions of this article. While Lumosity does start every game at ridiculously low levels and uses very common “gamification” methods to encourage players to continue playing the games will increase in difficulty to a point where they are truly challenging regardless of whether you pay for a subscription. The issue is since Lumosity only allows 3 games per day that most people are too impatient to wait to play the next level so they most likely either quit or buy a subscription. This likely leads to a flooding of low scores in statistics which is likely what causes to most people to report being in very high percentiles of BPI. All this said, past the initial almost guaranteed spike in progress that is designed to encourage players, once the games become challenging progress for me slowed down to a gradual pace yet continued regardless. My BPI actually drops when I perform worse than usual on the games so that alone tells you that it is actually measuring performance and not just a silly number that pointlessly increases over time through grinding the games. If you don’t consistently score better in games your BPI will not improve, furthermore if you were to consistently perform worse your BPI would drop significantly. Lumosity just starts all of the games incredibly easy to give users an impression of incredibly fast improvement when the actual improvement isn’t quite as dramatic, this does not mean there is no improvement in the skills the games are training it just means that Lumosity is designed to encourage users by starting really easy before baffling users with incredibly difficult games. Also its necessary to require every person to pass easy levels to ensure that players are capable of doing the next difficulty. The only pure gamification factor on Lumosity is the Lumosity points which are awarded for just completing games regardless of how well you do but that doesn’t factor into BPI at all. Just because Lumosity uses basic marketing techniques to encourage people to use its product doesn’t mean the product doesn’t work at all, the marketing just exaggerates the effectiveness of Lumosity to some degree. All this said over time my scores on game categories and overall BPI have yet to stop their very gradual upward trend which shows practicing the games over time will improve your skills at them. The main question is how directly applicable are skills acquired in Lumosity to life as the only game that I noticed specifically helping my was the spatial short term memory game helping me do math in my head along with helping me at chess.

  • AndyK

    This is a terrific piece. As a psychologist who does evaluations, I am particularly keen to know of any programs with evidence that they can improve cognitive functioning, especially attention and working memory. Lumosity’s ubiquity and clever marketing had me on the brink of believing (sucker for that NYTimes connection!). This patient take-down is just the tonic needed. Even as they acquire millions of users and tons of data, Lumosity’s unwillingness to submit itself to anything approaching scientific rigor is a sure sign that the emperor has no clothes, yet again.

  • Jack Cornish

    Though I agree with your claims, that Lumosity doesn’t improve cognitive function, I do believe that it can give you a decent summation of your overall skills and intelligence. No, I don’t feel smarter after having played the games, but I do notice that people less intelligent than me have lower scores and those more intelligent than me have higher ones. For example, when sharing my scores with an acquaintance, I noticed that mine were three times as high as hers (Hers averaged ranged from 500-800, mine 1300-1800). My IQ is most likely not triple hers, but I am significantly more intelligent than her, so that part is at least accurate. Similarly, while allowing a more intelligent friend of mine play a couple games on my account, he was able to attain scores with ease that I essentially had to grind for. The conclusion I have come to is that, though they don’t improve your cognitive function, they do give you a good idea of your relative intelligence. If you play the games fairly and don’t cheat, when you reach your “cap” you’ve really reached your mental capacity. Of course the games seem easy at first, but when you unlock the more difficult levels, you will be challenged.

  • Aral

    I joined Lumosity yesterday and these are my BPIs after I’ve played it for ONE HOUR.

    Speed – 1566
    Memory – 1888
    Attention – 1501
    Flexibility – 1654
    Problem Solving – 1693

    No training was needed to get this score and I think I would get 1800s in a few hours if I keep playing it.