[TylerE]

Something has to be done about GIB. It's just horrible!

PLaying on medium speed setting, the following deal was played:

Scoring: Total Points

---

Auction:

1♦ - (2♥) - P - P
2♠ - P - 3NT - AP

Opening lead is the J , ducked to dummys king. Notice the hand contains 11 top tricks GIB managed to go down 2 here. At trick 2 it lead the 9♦ off dummy and PLAYED THE TEN from hand. Beyond belief.

[jchiu]

The description isn't really that accurate, unless you mean to reverse dummy and hand. While I agree that the play was horrible, and I'm surprised the robot didn't act directly over 2♥, these indiscrestions are a result of an unlucky random simulation. I certainly would just dismiss this because the $16.00 that it cost here(I doubt you have right to complain if it were less) is less than the edge that a good player has on an average player playing here.

Edited: Took OP's eleven tops for granted, only nine tricks.

[TylerE]

This was just the last straw. I've just seen it do SO much dumb stuff today. Like doubling a slam on a balanced hand with 2 Q's... just on and on.

[Gerben42]

GIB ran a simulation and had to compare the chance of 0 - 4 ♦ with ♦J offside on the extra information that W bid 2♥. As this is a process of randomly generating hands, MOST of the time playing from the top still wins, but this time it didn't.

[mike777]

TylerE, on Apr 24 2007, 01:20 PM, said:

Something has to be done about GIB. It's just horrible!

PLaying on medium speed setting, the following deal was played:

<!-- FULLHAND begin --><table border=1> <tr> <td> <table> <tr> <td>Dealer:</td> <td> South </td> </tr> <tr> <td>Vul:</td> <td> Both </td> </tr> <tr> <td>Scoring:</td> <td> Total Points </td> </tr> </table> </td> <td> <table> <tr> <th> </th> <th> <table> <tr> <th class='spades'>♠</th> <td> J9 </td> </tr> <tr> <th class='hearts'>♥</th> <td> KQ </td> </tr> <tr> <th class='diamonds'>♦</th> <td> 9643 </td> </tr> <tr> <th class='clubs'>♣</th> <td> KQ832 </td> </tr> </table> </th> <th> </th> </tr> <tr> <th> <table> <tr> <th class='spades'>♠</th> <td> T42 </td> </tr> <tr> <th class='hearts'>♥</th> <td> AT8763 </td> </tr> <tr> <th class='diamonds'>♦</th> <td> J7 </td> </tr> <tr> <th class='clubs'>♣</th> <td> T7 </td> </tr> </table> </th> <th> </th> <th> <table> <tr> <th class='spades'>♠</th> <td> Q873 </td> </tr> <tr> <th class='hearts'>♥</th> <td> J9 </td> </tr> <tr> <th class='diamonds'>♦</th> <td> 82 </td> </tr> <tr> <th class='clubs'>♣</th> <td> J9654 </td> </tr> </table> </th> </tr> <tr> <th> </th> <th> <table> <tr> <th class='spades'>♠</th> <td> AK65 </td> </tr> <tr> <th class='hearts'>♥</th> <td> 542 </td> </tr> <tr> <th class='diamonds'>♦</th> <td> AKQT5 </td> </tr> <tr> <th class='clubs'>♣</th> <td> A </td> </tr> </table> </th> <th> </th> </tr> </table> </td> <td>  </td> </tr> </table><!-- FULLHAND end -->

Auction:

1♦ - (2♥) - P - P
2♠ - P - 3NT - AP

Opening lead is the J , ducked to dummys king. Notice the hand contains 11 top tricks GIB managed to go down 2 here. At trick 2 it lead the 9♦ off dummy and PLAYED THE TEN from hand. Beyond belief.

What are you talking about you do not have 11 top tricks you have 7 top tricks on the J of hearts lead.
2 spades
1 heart
3 Diamonds
1 club

Do you even know what "top tricks" means?

Yes you can "develop" 11 tricks when the Diamonds are 2-2 or 3-1 , but those are not "top tricks". Those are tricks that are dependent, not "top".

[jdonn]

mike777, on Apr 24 2007, 04:30 PM, said:

What are you talking about you do not have 11 top tricks you have 7 top tricks on the J of hearts lead.
2 spades
1 heart
3 Diamonds
1 club

Do you even know what "top tricks" means?

Yes you can "develop" 11 tricks when the Diamonds are 2-2 or 3-1 , but those are not "top tricks". Those are tricks that are dependent, not "top".

Do you even know what "top tricks" means?

You have 9 top tricks, not 7
2 spades
1 heart
3 diamonds
3 clubs

Even if you can't take them easily, they are still top tricks.

I have no idea what you thought is the point of getting so nitpicky when it's obvious what he meant.

[mike777]

Disagree only one club top trick not three. You need to develop a D entry for the clubs. Give West 4 diamonds and 7 hearts those tricks may disappear. Yes this is rare but more than zero%.

Actually I do not know what he means. We all know GIB does not play the best line or double dummy. That means we Know GIB will often repeat often not play the best line. What exactly does he mean, I do not know..I can guess he may mean several different things..not sure which one. In any case we Know Gib does not play the best line often, more than 5%.

[jdonn]

mike777, on Apr 24 2007, 05:06 PM, said:

Disagree only one club top trick not three.

You are wrong. That is a fact not an opinion.

[mike777]

jdonn, on Apr 24 2007, 05:34 PM, said:

mike777, on Apr 24 2007, 05:06 PM, said:

Disagree only one club top trick not three.

You are wrong. That is a fact not an opinion.

Ok willing to learn, How do I take 3 top club tricks and nine top tricks against any distribution?

[jdonn]

No one said you can take them, they are simply there.

[helene_t]

Overall, GIB plays quite well. But the kind of mistakes it makes are different from those done by a human. It's also different from the kind of mistakes made by a more rule-based approach such as Jack.

It's easier to notice GIB's strange mistakes than GIBs failure to make more common mistakes. There was a very good review of the Jack in the Dutch expert magazine "IMP". The author wrote that it takes time to get used playing with a robot. "What the h... is the idiot doing .... oh no, it's breaking the communication with the dummy ..... (relieved) oh ♣4 is an entry since the finesse over the 5 is marked due to the 1-3-5 lead.... 3NT+1 = 50%, despite completely crazy declarer play it managed to get a normal result." GIB is a similar experience. Although the approach is different from that of Jack, the discrepancy relative to humans is similar, I think.

[Flame]

Hi Tyler
As i was the one who took your money here i should tell you that on bidding gib makes lots of mistakes much worse then this one. However since you are a good player you will win on the long run sometimes due to those stupid mistakes.

[hotShot]

GIB makes a little less than one playing mistake in 3 boards, this is about that what human Experts/ World Class player can do.
But there is a difference in the kind of errors they make. Human player use some sort of reasoning, while GIB uses statistics. So while expert errors still look reasonable, GIB errors seem "mindless".
In this case there is a big chance that the 2♥ bidder is void in ♦ and since we know he is weak and did not have ♥KQJ there is a big chance he does not have ♦J.
Obviously GIB estimated the chance that the ♦ 4-0 distributed higher than the chance that the 2♥ bidder has the ♦J.

[vang]

jdonn, on Apr 25 2007, 12:59 AM, said:

No one said you can take them, they are simply there.

so, there ARE 9 top tricks (they are there) but you HAVE 7 top tricks (since 9 tricks are there but you cannot take them always).

both mike777 and jdonn are right ;-)

[fred]

Here is a my (probably futile) attempt at a simple explanation of why GIB does things like this on occasion:

In order to solve the problem of "what card should I play?", GIB generates a sample of random deals that are consistent with the bidding (and previous play if there has been any).

Then GIB looks at each of the deals in this sample in order to evaluate which play will work out best for that sample. It is not difficult to see that, if the sample that GIB generates is flawed in some way, then the card that GIB decides to play may not be the card that would be best for the set of all possible deals.

If GIB was able to look at "all possible deals" then it would always make the percentage play. In practice this is not possible because:

1) There number of possible deals is massive and it takes GIB time to evaluate each and every deal. Given the speed of today's computers, the speed of GIB's algorithms for hand generation and evaluation, and the level of patience that most humans have, the practical limit for sample size is no more than a couple of dozen deals (and it is less than that if you want GIB to play "quickly").

2) Sometimes the sample is biased because GIB does not understand the humans' bidding the way the humans understand it. What GIB sees as "possible deals" are often "not possible" from a human's point of view (and vice versa).

So the size of GIB's sample must be smallish (due to time constraints) and this increases the chances that GIB will be "unlucky" in the random deals it generates. Furthermore, sometimes GIB will make incorrect assumptions about the bidding and include deals in the sample that are not possible in the real world.

The bidding in the hand is question is pretty simple and probably GIB understands that hearts are 6-2 and that the random deals it constructs should have that property. The odds of the Jack of diamonds being onside are about 11 to 7 (because there are 11 empty spaces in the hand with the short hearts but only 7 empty spaces in the hand with the long hearts).

If there were only 10 deals in GIB's random sample, the odds would be less than 100 to 1 (11/7 to the 10th power) that the Jack of diamonds would be onside in all 10 of the deals that GIB generated. Suppose in one of the 10 deals diamonds were actually 4-0 making the diamond finesse a necessity.

GIB would (wrongly) conclude that the diamond finesse was the percentage play because it would see that the finesse would sometimes gain, but that it would never lose.

By increasing the sample size to 100 deals, the odds of GIB coming to the same conclusion would be astronomically small, but that would also slow down the speed of GIB's play by a factor of 10 (which would make it too slow for the taste of almost all human players).

Hopefully this will give some of you a better appreciation for the incredible complexity of the problem of programming a computer to play strong bridge (especially fast strong bridge) and what a remarkable piece of software GIB is despite the fact that it is far from rare that the program makes a really bad bid or play.

These really bad bids or plays can be frustrating, especially when there is money involved, but one of the nice things about our money bridge games is that these things happen with equal frequency to all of the players. The inherant fairness of these games makes it easier for some people to deal with some bad bridge on an emotional level.

Making GIB play better bridge is one of our main development goals. Unfortunately this is much easier said than done...

Fred Gitelman
Bridge Base Inc.
www.bridgebase.com

[mike777]

I would be willing to bet that GIB is already in the top 10% of all bridge playing players in the world now. That would place it in the top 2.5 million. That sounds like a good start.

I would be shocked if it does not play as well as the top 1% (250,000) in less than two years if not today. That sounds like a great rate of improvement.

Does anyone doubt Gib or whatever replaces Gib could be in the top 25,000 (.1%) players in less than 4 years from today?

[Flame]

fred, on Apr 25 2007, 01:00 PM, said:

These really bad bids or plays can be frustrating, especially when there is money involved, but one of the nice things about our money bridge games is that these things happen with equal frequency to all of the players. The inherant fairness of these games makes it easier for some people to deal with some bad bridge on an emotional level.

Fred all you said is true about play but not about bidding.
Yes it effects the bidding too but the main problem about bidding is the programmers misdefining bidding situations, those usually leads to funny -1100 or more .
Personally i no longer have a problem with it because i played lots of money already and i know more then most of my opponenents how to handle gib (not that the crazy bids never hit me) and on the long run i win more from gib mistakes (or bugs) then i lose from it.

[helene_t]

mike777, on Apr 25 2007, 10:24 PM, said:

I would be willing to bet that GIB is already in the top 10% of all bridge playing players in the world now. That would place it in the top 2.5 million.  That sounds like a good start.

I would be shocked if it does not play as well as the top 1% (250,000) in less than two years if not today. That sounds like a great rate of improvement.

Does anyone doubt Gib or whatever replaces Gib could be in the top 25,000 (.1%) players in less than 4 years from today?

I dono about GIB, but Jack was tested against a selection of the strongest established Dutch partnerships last year. The sample was too small to conclude whether Jack is stronger or weaker than those pairs, but if you accept the conditions of the contest it's safe to conclude that Jack is among the top 100 Dutch players which might place it among top 5000 or such players worldwide.

I don't know if there's any data on the strength of GIB relative to Jack. Jack is World champion computer bridge while GIB did not participate in the World championship. In the IMP article about the championships it was mentioned (and regreted) that some very strong bridge programs, most notably GIB, did not participate because the costs are so high that even for a program that expects to win metal it may not make comercial sense. My personal impresion from playing with Jack and GIB is that there strength is simiiar. I haven't tested it statistically, though, and also the conditions under which I play against GIB and Jack are different.

Another issue is the fairness of the Jack-vs-Dutch-top contest. The operator noticed that the unusual enviroment had an adverse impact on some of the human player's attention. Also, when playing in an indy, a robot playing with an identical robot has in theory perfect partnership harmony. It could be argued that a mixed human/robot indy where robots where not allowed to partner another robot would be a better test of the strength of the robot relative to that of humans.

[junyi_zhu]

TylerE, on Apr 24 2007, 06:20 PM, said:

Something has to be done about GIB. It's just horrible!

PLaying on medium speed setting, the following deal was played:

Scoring: Total Points

---

Auction:

1♦ - (2♥) - P - P
2♠ - P - 3NT - AP

Opening lead is the J , ducked to dummys king. Notice the hand contains 11 top tricks GIB managed to go down 2 here. At trick 2 it lead the 9♦ off dummy and PLAYED THE TEN from hand. Beyond belief.

seems the basic design for gib's bidding is wrong. it's based on around 10 randomly generated hands, and the constraints for these hands can be very wrong. Also that simply leads to random bids, I have tested gib, in the same situation, it gives some completely different auctions, including a choice between pass 6M and raising it to 7M or raising 2nt to 3nt and bidding 6D with Jxxxx over 2nt opening. I somehow think this is the major barrier to attract more players to play money bridge. Some of my friends quit money bridge basically due to this problem. If computer player plays a decent game, I am sure many really don't mind partnering with them even if they lose some money. It's just like playing in ACBL, you pay a table fee, partnering with gib, you also pay a table fee and sometimes you may win good amount of money. I heard Jack's bidding is based on rules which is definetely a superior design comparing with gib's, because there is little randomness and jack's behavior in bidding can be predictable if it's truly based on rules.

[junyi_zhu]