Bitcoin Forum

It seems that poker has finally fallen the next victim to insatiable and unstoppable AI, after chess and go world champions were defeated in the past. Poker seems to have been a more difficult problem for AI to solve than the games mentioned, primarily due to hidden information as well as bluffing, but now it's over for human beings. An AI developed by researchers was successful in defeating top professionals at No-limit Texas Hold’em, which is thought to be one of the most challenging types of poker. If anyone is interested who played for the mankind, the players were Jason Les, Dong Kim, Daniel McCauley and Jimmy Chou, though I'm not sure if these names are telling you anything.

Here're the links to the popular-science (https://techcrunch.com/2017/12/18/carnegie-mellons-superhuman-ai-bests-leading-texas-holdem-poker-pros/) article and actual (http://science.sciencemag.org/content/early/2017/12/15/science.aao1733.full) paper in Science magazine.

I've never heard of any of these professionals, I want to see it beat Phil Ivey and Tom Dwan.

I think it was only a matter of time. The development in AI in the past years was overwhelming but this is only the beginning. Imagine where we will be in 10-20 years?! There will possibly an AI for almost every task we can imagine... these are exciting times!  :o

I am not surprised with this, as humans even professionals will feel emotion.

Also poker players study their opponents mannerism and movement but with if their rival is an  AI they will only rely on their risk management.

Doug Polk and his friends (Jason Les, Dong Kim) are amoung the best HU poker players of the world. They beat Ivey and Dwan.
Now AI beat the best poker player on HU (2 players). It was just a question of time.

We are waiting for AI to move to 6-max poker and to beat some of the best players (like OTB Red Baron). But it'll be more difficult to reach the Nash equilibrium (game theory) with 6 players than it was for 2.

Yup, me too.  I think those guys are some of the pros that play the 50/100 NLHE tables on stars when they run.  And I agree on Ivey and Dwan, but the guy I'd like to see play is Victor Vlom.  I wanna see how the bot reacts vs him.


Even better than Jungleman or Vlom?  Sorry...  I haven't been in touch with the poker world for years now.  :)

I should also add that Libratus, the AI in question, wasn't using any "bluff analysis" or whatever it is called. It played exclusively by the preset rules of the game, just like it played chess or go, the so-called "perfect-information" games. And it still was able to defeat the reigning poker champions hands down. As the article says, Libratus was able to "master a game where bluffing is a core, necessary component".

AI will take over lots of things in future and it has  already started although can't understand whole bunch of technical words and description on article.

Looks like this AI is not only built for poker or card games.

If this is really true and if this gets implemented on a larger scale and things get available to everyone easily, then the future of online gambling will surely be doomed.

However, poker is a form of gambling and how can anyone, be it how intelligent are they can beat someone in game of chance, maybe its just purely luck that the AI has beaten them, but in the longer run, AI cannot always beat the human brain neither can this be true vice-versa.

Well I certainly believe that some artificial intelligence could win some hands of poker. But  turn poker you can make absolutely 100 percent the correct and proper move but with the variance in poker you could still lose the bet. What I’m saying is I could see this but being Have a decent playing caliber but there’s no way it’s unbeatable. There’s just certain things  I believe AI Wouldn’t be able to pick up on.

Lol.  Not sure if serious.  Negeaunu turned out to be one of the weaker poker players after a generation of online players started taking the scene during the mid 2000's.  Guys like Cole South, Brian Townsend and I guess Patrik Antonious and a whole lot of young guns.

I understand what you mean tokeweed But you have to give Daniel a ton of respect for being able to continue to compete, for what like 20 years.Even with all the new Internet wizards and Young Guns that popped up in the 2010’s, I would definitely still consider him a Top caliber poker pro. That being said he does make some really stupid moves sometimes, he just flat out plays himself alot it seems.

Boy, Technology are scary things to be honest..
I wont be surprise if they make a real robot that can live like human someday..
I wonder how the AI can catch the techniques in poker, like baiting, and stuff..
the programmer must have done a hell of a great job programming it..
like those chess AI that beat those grandmasters all around the world...

Your comment is just wrong on so many levels.

How will this change the future on online gambling? Maybe online poker, but to say that this has any impact on online gambling in general is just plain stupid.
Slots aren't impacted by this, roulette isn't, etc.

From the first article:

I doubt this is just luck if the bot played for 20 days against these pros.

Obviously, this is not luck since all moves played by all parties have been properly analyzed. I read across the second article and although I couldn't really understand half of it or even more, the authors specifically point it out that it wasn't luck, absolutely. Long story short, the AI managed to find weaknesses in playing patterns of its human opponents and used them to the full. When the players changed their approach, the AI changed its approach too, and it was always able to find an advantage and take it.

This is good - it's showing that everybody could improve and even the poker pro's are making mistakes and can be exploited.

I ever read that now it is possible for some bot to win against pro poker gambler. Last time winners already lose by them easily, they calculate everything like the deck, gameplay and even recognize your face when make a move here. There is nonway to cheat on them because it is like a perfect solution for them. If I not mistaken the name is libratus which is really good on defeated pro player. But I do not know if the bot against each other who is the winner

Interesting, but I guess it is the next frontier for it. Probably will next beat all casinos, since in online gambling there is really no "true random" because all numbers are actually just generated using a code that tries to simulate random numbers. I guess it is possible that an AI could break this code and be able to predict the next number. Well, that would give gambling sites a run for their money.

From an AI point of view, with billion lots of possible combinations. It can clearly predict probable patterns on what the next card to get. It is still an unfair in level of playing field if we use AI against human to play that kind of game. Maybe someday we the humans could just be an spectator of the poker games where machine vs machine are the real players.

Very interesting. It was only a matter of time for this to happen and I was anticipating it. I myself am a backgammon player and the reason why no one plays backgammon online for money anymore is because bots will crush even the best human players in the long run. Poker, of course, will encounter a similar fate.

What a surprise by me, Before Chess grand Champion getting defeated at AI now, even poker without any more notification this is not just a bluff I'm really serious about this what if one of real casino made an AI that have program that way to play poker for real holy no...!! :-X :-X :-X

That makes sense, I don’t know a lot about the inns and outs of backgammon. I would assume it comes down to analyzing and executing the perfect move for any given situation. Kind of in the way that a chess robot would work. I can see how parts could be programmed to be unstoppable But poker is a whole Other game.  The pot may be able to do the most optimal play, but you can’t Teach human emotion.

I'm afraid this is not quite so. If you run an online casino, you can buy a hardware random number generator plugged into your USB port for as low as 50 dollars. It is a device that is capable of generating truly random numbers from physical stochastic processes such as photoelectric effect, thermal noise, whatever. I don't think you will be able to crack it because the numbers generated are not simulated, they are as random as the world around us.

Just like in the e-sport dota 2, a year ago they introduce "Open Ai" which was really good and beaten the top players of dota 2, but now that AI has been getting some smack in the ass because top player know the Ai`s potential, eventually the poker players will defeat that AI just like "black" "sumail" and other pro players who beat the AI.

Well, I don't really know much about Dota games. Are they shooters or what? But I don't think that it is possible to put the genie back into the bottle when it has already been released. You can now easily beat chess champions using common desktop computers, so how is poker different? Yeah, I know that it is in fact different because poker is an "imperfect-information" game as they call it, but the AI that beat poker champions was treating poker pretty much like chess. I mean no bluff, no hidden info.

This is indeed interesting however it won't do much in either the real physical casino or in any online casinos.

In online casinos you can't see bluffing or be able to bluff.

In a physical casino you can't use an AI to bet. You are allowed to card count only without the help of electronics. However using an AI is impossible to do without getting caught.

Many people had spotters before which had a small laptop or tablet or phone that they used to determine a strategy and they all got caught by casinos cameras.

So Las Vegas has nothing to worry about.

I tend to disagree. Along with the AI technologies the technologies which make use of them are also being developed. As I see it, in the future, and not so distant future I suspect, it will be possible to extend your capabilities and expand your intelligence directly by using brain implants.  Yeah, I know it sounds a bit scary, but there is no other way we are going to beat the machines apart from implementing the same technology on ourselves. Evidently, people who will be the first will try to monetize their competitive advantage, and casinos will likely be the places where these new "improved" people are going to show up.

Yeah, those players are sort of good, but they are not the top players, so maybe not everything's lost.
I think it is not very nice, that people keep creating AI which will beat people in games. AI development is a good project, but I think it should be concentrated more on other things, such as adaptation for people's social needs, exploration of human consciousness and stuff like that. People need games as a part of their life. Game presupposes that one can't always win, for thus the game makes no sense and doesn't really make people feel better. As a person can't always win, AI doesn't become cleverer when it does, it remains being a cold machine that doesn't understand human experience.

wow, this is amazingly scary to see, where many human things are involved in real life poker games.
Ai is getting stronger and stronger every day and working in a company where the AI is being developed and used for human needs like medical field, cancer researches, rare sickness and so on, it's good to see the versatility and usefulness of machines either way.

Dota 2 is a MOBA (Multiplayer Online Battle Arena Video Games) , anyways well i think that those poker player were just amazed on having am AI to compete with, AI`s are able to beaten on large numbers of game even on gambling and poker is no excuse, let them play for over a year so that poker players (champions) can fell more on what its like playing with an AI.

AI in poker cant beat pros on the long run just because poker is a range game and ranges can be fooled therefore AI gets caught, a godo bot should learn from the villain mistakes to make them lose more money with each mistake

If so, why the pros couldn't beat the AI in the games they played?

It was a 20 day competition with a prize pool of $200,000, surely not something to sneeze at. The players tried to change their strategy but to no avail. Indeed, over a longer course, some weaknesses will be discovered in the playing patterns of Libratus which could ultimately lead to its defeat. But there is nothing that prevents it from evolving. Right now, for example, top chess players don't stand a chance against the best chess programs out there, so why poker, and especially online poker, should be different?

You know in chess all cards are open so it is difficult than poker. AI already solved issue in chess so pokes is easier and funny one for AI

But who are these guys? Let that machine meet STEPHEN CHIDWICK or Daniel Negreanu with all the skills these guys have!

I do think that one day AIs will be extremely smart to even outwit any human player. The thing is, AIs are mechanical and they can do almost anything that would be better than a human since they commit less error. Maybe one day the world will be ruled by robots and it could lead to those scifi movies about robots in the future we all know and love (Matrix, iRobot, Terminator, etc).

I'm pretty sure those guys are much better than your Stephen Chidwick and the overrated Daniel Negreanu.  The latter I know for sure made bank before the internet generation of poker players.  Games were passive back then and almost devoid of 3 betting and for betting in both NLHE and PLO.  I doubt he can build a bankroll on Stars if he started at the micros.

So in artificial intelligence being so good at chess and now poker.
Does this mean that the house can always win?

If that is so then what is the point in playing against them online in a poker match.

Cause you know the outcome will always be the same
They will win 100% of the time and you are just wasting your money in playing an AI opponent. :-[

That is the future we are signed up for, where robots rule everything because they are faster and smarter. Maybe it will help make people begin to get smarter too.

This will obviously hurt online poker.  If people are playing in person, you know they aren't an AI.  Online someone could have an AI open on their computer and beat the human players over time.

That is the case even know ? Don't you think ? Why do you think the real poker players stay away from online communities or only join rooms which support live poker ? Is is not now,even the chess playing bots are better at stuff than humans which are in the games before the AI.

I think this will give birth to a chain based system where you will have programmable AI bots which play against each other.Machines vs Machines,you are both even know. :)

That's how matters stand these days. Ultimately, in my view, there is nothing wrong with that because we're not competing with, for example, a tower crane or freight train in strength as we don't compete with a calculator in calculations. But this doesn't mean that humans cannot compete with other humans in whatever it might be. Whether some people are going to cheat is another question though. But people have always been cheating anyway.

There comes a time when artificial intelligence will dominates human lives. The AI that used in chess and poker might be the first step in modernizing the gambling world. The gambling machines inside the house will also equipped with AI in the future.

Well it's not really surprising is it?
I know that poker has even been solved in texas holdem limit. Which means that an IA has managed to find the perfect way to play the game, no matter what the human does. But it's only on head up (two players) and with limit to the bet amount.

So next one to fall might be limitless poker :p

Interesting article and unbeatable proof that texas holdem isn't gambling.
I wonder how this will affect whole online poker industry in future.
They are good because they can read opponents. They can't read AI.
On side note he took third place as top 10 poker players in 2017.

This should basically not be a surprise to anyone in this day in age. I remember when they had the computer go up against the chess world champion player. I remember how mad i was when i lost $500 because i told myself there was no way a computer would win. Granted this was a few years ago so computers was not as good as they are nowadays.

Yes, it is very reckless not to believe in new technologies. AI does a better job than man. In computer logic there are no mistakes. Cold calculation wins the human intellect.

Yes, but AI is not about cold calculations. In fact, it is not about calculations at all because it is about what is called fuzzy logic and machine learning. The former is about dealing with the concept of partial truth, which defies "cold calculations", while the latter is about adapting to a changing environment. These two factors are what makes AI so formidable up to a point of being frightening.

Bots have been known in online poker for long, but it was easy to exploit their weaknesses using stats programs as Holdem Manager or Poker Traker (legal programs). Anyway, bots are banned, and it is difficult to see one of them in the major sites.

This is different, though, but I’m not surprised. AI will change the world as we know it, and this is just a small part of it. It’s just been tested for HU but as those machines keep learning, I think they will soon beat 6-max or 9-max tables. However, I’m not so sure about other formats, like MTTs, where there is high variance.

In short human themselves produce an AI to defeat something that is undefeatable but in other words it's human itself who defeat the opponents since they are the creator of AI. Nothing change about that game who always win is the poker devs and admins at all. But all Ina all money has a big role to human beings that is why it is circulated into humanity. Great, Cheers!

The big difference here is that people get tired and make mistakes, while computers will always do exactly what they are told. Someone who has a lot of poker skill can program it to do better than they could because of going on tilt, being tired, being rushed, distracted, and other things.

Perhaps I missed it in the article, but these hands were played online, not live, right?

Also, I think it's pretty interesting that it wasn't until the 30k hand mark that the AI started to become consistently profitable, and it wasn't until 80k hands played that this profitably was statistically greater than 0. I think this suggests that the AI's ability to win isn't from some sort fundamentally better strategy against all opppnents, rather, its improved ability to learn from opponents and exploit them over a large sample size. This bot can't just sit down at a random online table and start crushing, which is a good thing.

Yeah, it may well be so for the first time, but after a few months of continuous playing the AI should be able to squash all competition at once, at the first game. It is basically the same with chess, though AI doesn't seem to be used there so much. Chess bots use, for example, opening and endgame databases so they don't need to play every game from a cold start, so to speak. This roughly corresponds to the level of AI "expertise" in playing poker acquired after a number of games.

This is what we humans call experience.

It can do well as a tight player setup but people bluff and it has to learn how each individual plays before doing well against them. Chess is all open and public, while poker is mostly hidden and is about tells, so they're two different things.

True, it seems from the article that the AI also improved in performance against other AI over time. One thing I will say is that I'm sure the players' knowledge that they were playing against an AI probably had some influence on their playstyle. Whether that means playing tighter or more aggressively, I'm not sure, but I know I'd play differently against an AI than a human opponent (probably more aggressively, since you have much less fold equity against bad players who will call with marginal hands, and I doubt an AI would do this). Regardless, I wonder if the results would be the same if players were unaware their opponent wasn't human.

As long as you can predict move on AI, I do not think that it will be difficult thing to do. But the problem here is how are we going to predict it if many inputs for the AI including face detector, how to react if A using this kind of face, what move should the AI takes. I do not think that it is really easy to know that it is AI or not if the bot is pro like libratus

It may be so in a face-to-face game, where physical presence is required, but poker is played mostly online these days, as many other games, for that matter. So how are you going to bluff in such conditions? Further, it actually cuts both ways. Just as you can say that players can get advantage by reading their opponents' facial expressions, there is no "face" to read in AI. While the AI can still do that.

Are you sure you will be able to deceive it?

Yeah , AI slowly beats people in everything . Will they conquer the world too ?

The online world, yeah. Maybe then, we'll go back to a simple life haha!

I think tells are overrated on poker. IA don't care about tells. IA care about position, stack, ranges, odds, blockers, sizings, etc.

AI will undoubtedly follow you no matter where you go or what you are going to do until it gets you. Yeah, it may take some time until it reads looks and uses its own visual tricks and facial expressions (don't ask me what it would look like), but we will get there eventually, or rather, it will get there. In fact, it may require a lot of time and major break-throughs in the field but we will be there even if we have to build a copycat of entire human brain. Or more than that.

I don't want to play against someone who doesn't care about paying his bills while playing poker. AI seems to be that kind of emotionless opponent. It is so hard to bluff AI. Ya, you can win one or two hands but in the long run, you will run out of diverse patterns while AI will be learning and improving with your every move. Not only that AI will have the access to millions of hands played by other people in that very specific scenario which is enough to beat f normal players like me.
But again there are chess engines who can beat even top-class grandmasters but still in the real world you have to play against only live human being.
So, yes in the world of online poker it is very hard to determine if you are playing against AI or real player but I don't think in live poker matches it will be a big deal.

For sure this is just for an online kind of poker, so poker faces and bluffing would be different if it is online. It is quite a lot easier to determine the play style of a player that is online since the AI just needs a few games before it would be able to know the play style of its opponent. What that means is that if it knows how you play then it would be able to determine what are your tendencies. The thing is, it can analysis indeed the millions of hands it has in its data base and base its decision from there. there is really no new player style so I guess it would just depend on how quick an AI will do a read and decide what kind of player it is playing against.

Not a nice news.

I looked through the whole thread and everyone seems to be whining about AI defeating mankind in the next field of human activity, poker this time. But what about one AI playing against another? As I got it from the thread, poker is different from chess because the latter is called a perfect-information game while the former not so perfect. My idea is that it should be a lot more fun to watch one machine playing against another, especially if all tricks, deceits and bluff that human beings are so proud of are implemented and allowed!

So could one machine actually deceive another?

AI's will rule the world someday.

I think we are still very far from that. Really, how do you imagine one computer deceiving another? If we take chess as an example, you can sacrifice a piece with the objective of gaining advantage and winning the game later. In my view, there's no deceit, this is only our interpretation of some moves.

Nevertheless, if we speak about an advanced AI, or superintelligent AI, it may ultimately develop some features or qualities that we could consider as trickery or cunning. Whether it will be able to consider them as such itself remains to be seen though.


We may not live up to that. And it may in fact turn out to be good.

There was that professional poker player that became famous with his betting bitcoins I believe.
He is on youtube and twitter all the time.
He is the one with a beard and has alot of near naked women all over him in these videos so he must be still winning. ;)

Like the Matrix movie? I am not discounting the fact that there is a possibility of this happening but with our current AI and robotics technology today this will be in the very far future.

For the OP, I am not surprised that AI was able to beat this Pro poker players as most poker players usually read body movements and of course AI will not show that. AI will always be logical and compute the probability of them winning the hand.

Frankly, I'm not very familiar with poker and especially with its "show-off" part, but what about online poker then? As far as I know, there is no way you can reveal your movements and facial expressions to your opponent unless you are playing via skype and making faces at them. So in online poker there is a level playing field for both AI and human players. What prevents human beings to be as logical as the machine playing against them?

I have recently read a news when two AI robots started talking on their own, then the company (Facebook) had to shut it down. This is kind of alarming.

I do not think that they is playing that way, by decieving any player, I do not think they even have the abilities to do that. You know that AI is a robot, which is the designer can always put every information on each AI to have more information about each pro players here. So as long as they can make an AI to have big memory so they will keep every data on it and can beat everyone in just a second, because they memorize every pattern that you have. May be different pattern will do but the chance of winning is still too small

I don't have a lot of experience and profound knowledge into the operation of AI systems but they definitely don't work the way you describe. AI is not a robot, it doesn't have a specific algo or program that was coded into it. AI works pretty much like human brain does, and it should be taught first before it can do anything intelligible. But I understand what you mean and this is exactly what I'm asking. Could it have some moral value or judgment after it invents a trick or deceit? Or will it be yet another sequence of moves leading to a state which we humans proudly call victory?

The way i see it, every ai will be trained with different sets of data or learn for different purposes, so it probably won't have any moral values if it's trained/learn to be perfect at certain job, it will try it's best to be good at it.

Sorry am not quoting everyone, already a pretty long quote string! But I was drawn to the last few discussions on AI designed for deception - I wonder if there would be any conflict with some of the Ethically-Aligned Design (EAD) treaties already drawn up by organisations like IEEE. I know there are already steps towards advocating compliance within medical fields for example, though of course none of this has resulted in regulations.

Could AI be designed to still be ethical while being good in a game like poker (or even chess?). If one threw the ethics rulebook out, could not one deploy a similar AI to online poker rooms and just defeat reliably all human opponents?

@kryptorian: AI does work a little like you mention, in that it is like a human brain. It can only learn what it is taught, but there are becoming more and more capable of self-learning (or self teaching). But no, AI cannot yet determine moral or value judgment without predefined codes of conduct, at least not in practice - but EAD AI tries to create such that form values based on learning rather than a set universal code. The challenge is to get them to process context.

Yeah... No...
I mean don't worry, the AI can only beat players in head to head games with limited poker.
At least last time I heard about their progress they managed to finalize the poker in those circumtances. To finish a game or to solve it is to manage to have all the possibilities of the game and to always chose the best option. Some games are easy to solve (like small games) but games like chesses are impossible to solve because of the far too many possibilities.

So happily enough the game isn't solved here. It's not because the AI managed to fight one pro poker player that it means it's better than them.
Remember that poker is a chance based game. Even without any AI if the only move you make is to go all-in to each hand, you might have something like 25% chance to win even against a pro player...

Until we have more information for me the AI isn't a problem for the poker world ;)

Huh, when I said about teaching I meant that AI needs some data source, and if we want to teach it something useful, we need to be particular about what we teach it. Otherwise, it can be taught whatever is being fed to it, so of course it is capable of self-learning as long as there is a source to learn from. Neural networks are intentionally built this way. I also don't think that AI can be made ethical unless we impose some strict rules on it like instinct of self-preservation. It may make AI unethical or even anti-ethical (a perfect killing machine) but it is not the same as being completely non-ethical, void of the very idea of ethics.

the way to beat AI is do the opposite on how you bet

It looks more like you don't quite believe in what you say yourself here. Rather, you are clutching at the idea that the game, poker in this case, is basically a chance based game, and AI can't overcome that. You are right, of course, if that comforts you, but as the real tournament has shown, there is not enough chance in poker to keep AI constrained and in the long term it still invariably crashes all human competition. Note that chances may equally work in its favor, while unlike humans it won't miss them.

How are going to do this? Can give more explanation here? Because what I know, AI is only things that people made so if the programmer do not input really hard thing into AI, then there is nothing to be scared of. The only matter if pro programmer which can input many things for that bot and let him learn everything that pro players have, from their gesture and everything then it can't be beaten and poker games will be ended in really bad way

I'm sorry to say it but AI is not a bot, it is not a program either, conceptually. By the way, AI means artificial intelligence, and while it is definitely artificial, it is intelligence nonetheless. It does things which are not coded in it because you don't code it like you code a program. Essentially, it is a neural network where links between individual neurons are not set beforehand. The are set and removed when AI learns and it can learn pretty much constantly if it needs to adjust and adapt as in the poker tournament discussed here.

Am sure the AI model has been feed with all sorts of scenarios and the best possible moves to play but i hope this wasn't based on one game
lol what is the probability of winning if this strategy is used

But people can code it anyway with so many conditions and with this so many conditions and probability things that make the AI keep learning and get a lot of experience so they can win against pro players. Since when player can beat on AI? And I believe there is no way people want to compete with some AI in a big tournament, this will be meaningless because in the end they will lose it. I think it is not easy to win an AI with really complicated mode on it and I am sure pro players will be proud if they win it

If you think that AI is a sort of complicated program with millions or even billions if-then operators, then you are probably right, in a sense. It can in fact be represented in this manner at any given point in time when it operates. But what you are missing in the whole picture is that the code structure is not a result of some effort done by a big team of developers but what is called machine learning. The most important thing here is that "code" representing AI is dynamic and can be changed on the fly depending on the data received by it. You don't code AI like you code a regular program. Formally, AI is not even a computer, though it can simulated on a typical desktop computer but it will be devastatingly slow and "unintelligent".

At present computers lack originality and random variation.  Famously computers cannot be trusted to be used in lottery schemes because its too easy to engineer or scheme the results they produce, many multi million dollar schemes result to natural variation in results.  Ultimately computers only do what they are told, AI is very much behind in what it should be doing so Im not convinced they would do well in poker especially.

https://thewest.com.au/news/offbeat/how-computer-programmer-eddie-tipton-rigged-lotteries-around-the-us-for-a-2-million-score-ng-b88573886z

Computers could alter their choices but it would always end up predictable and not a safe way to earn an advantage.   A good program could beat a poor player at least as a standing advantage as the person will learn, so its possible 1 program could be superior over another in some way I guess.