Copyright © 2005– by CCLA. All rights reserved. |
[top] |
Advanced Chess | |
Hints to Get Started in | |
Players new to the world of engine-assisted play are often puzzled to find no published "how to" articles. Unlike traditional chess, there are no teaching sites, advice columns, instruction books or recommended reading lists, not even suggestions on what equipment to purchase or how to use it in "advanced chess". A Google search just isn't productive, as many authors have used "advanced chess" to mean any article about chess beyond the beginner level. | |
It was Garry Kasparov who first used the term Advanced Chess, describing his 1998 match with Veselin Topalov in which both players used chess computers. Today we differentiate between freestyle chess, computer-assisted chess played over-the-board, and advanced chess, computer-assisted chess played by correspondence. (Freestyle chess is actually what Kasparov and Topalov were playing, and today it has all but died out, as it soon became obvious human players have nothing to contribute in computer vs computer matches with time clocks running.) Many question the "human contribution" in advanced chess, too. | |
Hardware and software specs are closely guarded secrets by successful advanced chess players, as are their methods for using engines. The advanced chess newbie, in search of the "right" equipment, must do internet research, slogging through techno-jargon only a computer geek could hope to understand. As a general rule, when it comes to computers "bigger is better," or perhaps we should say "more is better". | |
Our friends at ICCF assure us "chess engines run on smart phones.
You do not need to invest any money on hardware to enjoy ICCF." If you already own a smart phone, that's great, but my LG Lucid3
costs about $ 300. This Android phone features 4.7-inch (540 x 960) display, quad-core 1.2 GHz Qualcom Snapdragon 400 processor and
1GB of RAM. | |
| |
Hint: for upgrades, and especially if purchasing a new computer, order the largest power supply, speediest motherboard / multi-core processor, most memory, largest hard drive and best cooling system you can fit into the case. Think high-end gaming machine, not a generic or "internet" computer. Although their capabilities continue to improve, you are not likely to find high-end specs in a laptop computer and cooling can still be problematic. Buy a desktop model computer. The operating system should be Microsoft Windows®. Currently, leading chess engines and GUI's are mostly developed for Windows®. Stockfish is an exception, also available for Mac, Linux, iOS and Android. It's one of the strongest engines and, best news of all, open source which means a free download (but GUI is not included.) | |
So what equipment should an aspiring advanced chess player be using? Today, advanced chess players regularly analyze positions to depths of 35-40 ply. We know this by analyzing published games using a chess engine, waiting for the moves actually played to "bubble up" to number one in the analysis window, or else suddenly appear there and remain on top (note: some gui's display the reverse, with best line at the bottom of the stack). What hardware do you need to achieve such performance? A minimum would be a quad-core processor, but it will require hours to accomplish such deep analysis. A dual-quad core machine will cut the time down, though not in half. You will need a "deep" or multi-core version of a chess engine and 64-bit motherboard/version of Windows®. And, last but not least, enough memory (8, better 16 gigabytes) to hold the millions of positions generated by an engine. Some of the top engines/GUI's display your computer's resource usage while the engine is running; as ply depths increase, you will quickly max out your processor (95-100%) and also slowly but surely use up to 100% of available memory. When memory utilization reaches 90-100%, the speed of analysis slows to a crawl. With an under-powered computer, leaving the engine running all night while you sleep gets you ... nowhere. Eight or ten hours later, maybe you will have added 2 or 3 plies to yesterday's analysis. Running processors at 100% for hours on end generates a humongous amount of heat (the reason adequate cooling is necessary.) You are strongly advised to know the maximum operating temperature (mfg. spec.) of your processor and motherboard, and to monitor these carefully, visually or (better) electronically by alarm and/or auto shut-down. Let's be clear about this; the fan system (size, speed, number of fans in the case) that came from the factory is adequate for normal workloads. If you run processor-intensive software such as a chess engine, the core temperature will overwhelm stock cooling. For example, my Intel quad core 2.4 Ghz processor shoots up to 212 degrees almost immediately when running Stockfish 6 64. I have to run a 10" desk fan aimed directly at the intake vent to keep the core temperature down in the recommended 150-160 degree range. Most motherboards have a shut-off feature if the core temperature reaches some pre-set limit, but I wouldn't rely on that. Regardless, running your processor under excessive heat conditions will reduce the useful life of this very expensive component. | |
How much does a computer system like this cost? Visit any computer manufacturer's web site, pick a model, plug in all the options
you need and the site will compute the price. You will not be able to purchase such a computer for $ 500 at Best Buy or Walmart! A desktop x64 with i7
processor (4 actual cores, not 2 cores and hokey hyper-threading) and 16 gigabytes of memory will be more like $ 2,000. Add in a temp monitor/alarm,
extra cooling fans or a water-cooling unit, another 16 gigabytes of memory - the price goes up accordingly. | |
Robert Houdart advertises his state-of-the-art chess engine, Houdini 4 Pro, as capable of utilizing 32 threads (16 cores) and 256 gigabytes of hash memory. There's not a pc in the world can meet those standards (Intel and AMD just began offering 8, 10 and 12-cores in a single processor configuration.) One must move-up to the workstation class of computer to run that many cores and that much memory. Workstation prices start around $ 2,500 for a "bare bones" system; add in 16 processor cores ( just two 8-core xeon processors = $ 6,000) and 256+ gigabytes of fast memory (one 16GB DDR4 ram "stick" is approximately $250) and you now own a $ 12,000 computer. The top advanced chess players (competitors in ICCF's world championship cycles, for example) are using xeon workstations or equivalent, and can analyze deeper than 40 ply. Equally important, they reach those ply depths in comparatively little time, enabling them to conduct many top-level games (several 17-player sections) simultaneously. | |
But I don't compete at the world championship level. Can't I just enter a class event and play advanced chess
with my existing computer? | |
After you invest in high-end equipment and your skill level propels you out of the "class" level (your rating now exceeds Elo 2000) be prepared for a new kind of frustration - most if not all your games will end in draws! See "Chess On the Server" for discussion of the coming draw death of advanced chess. History indicates computers will get faster, software will improve and market pressure will drive prices lower. More and more players will be able to upgrade their hardware and software. The “trickle-down” effect will spread the draw death to rank and file players as well. | |
As shown in the chart above, excessive draws have become a huge problem for the ICCF and "advanced chess." See ICCF World Championships (Table II) for statistics on all the world corres. chess championships. Results in the recently completed (30th) world correspondence chess championship are 93.4% draws. The ICCF mantra says "if you blindly follow an engine's recommendations you will lose every game." ICCF officials apparently believe the "draw problem" won't proliferate - only world champion-level players possess the "human genius" to guide their chess engines. Last year's preliminary round of the German Championship, only a national event, must have produced some shockwaves: | |
100% DRAWS in GER/C47/sf02 | |
Can I analyze with Houdini 4 Pro, or some other high-end engine, on my home pc? | |
There are more than a hundred chess engines available on the internet, many offered as free downloads. An inquisitive player will visit distributors' web sites and independent "testing" sites which publish performance statistics comparing engines, different processor / memory configurations, speed tests involving a standardized set of positions, and so forth. And, of course, there are engine vs. engine tournaments going on all the time, giving a programmer bragging rights (and a potential bump in sales) should his program defeat the other engines in head-to-head competition. | |
| |
Which is more important, hardware or software? The "best" chess software in the world can be had for $100 but it won't accomplish much coupled with a subpar computer. With equal hardware, the top chess engines come up with the same cluster of top-ranked moves; the exact ordering (ranking) of these moves may vary from engine to engine, and from move to move. I can hear the howls of protest, but I stand by the statement. The big differences in engines are 32-bit vs. 64-bit, whether they are designed for single or multi-core processors, and features/ease of use. Some users put a lot of faith in computer vs. computer tournament results and engine ratings, but these should not be the sole reason for purchasing, as explained above. Unless chess engines/computers are of special interest to you, I recommend using one of the established, highly rated engines to get started in advanced chess. Otherwise, you may waste a lot of time experimenting with little-known engines that are mediocre performers and/or lack necessary features. | |
To keep requests for assistance to a minimum, not to mention reducing complaints, major engine sites such as Fritz/Chessbase, Houdini and Rybka do a decent job in the "help" area, offering user forums, technical articles, features and benefits, and detailed download / installation instructions. They also give potential users the minimum computer specs required to run their products. A bundle includes one or more versions of an engine, an interface (GUI) and a manual. Clearly these are more expedient, avoiding the time and effort needed to research, purchase and install separate components from several sources. | |
At a minimum, you need to know if the engine matches your system (32-bit or 64-bit, single or multi-core processor) and whether the GUI and engine are compatible (x-board or uci.) Most GUI's can be configured for either type of engine. A 64-bit computer will run 32-bit apps, but not vice-versa; running 32-bit engines in a 64-bit machine slows the computer, about the last thing you want to do. Last but not least, the memory in your computer must meet, better exceed the engine's minimum requirement. | |
| |
You will also need to download and install a chess GUI (graphical user interface) compatible with the engine you wish to use. Some GUI's are very fussy about where an engine's executable file is located, so follow the directions carefully. Installing an engine in a GUI is often frustrating because Windows® users must type in a DOS-type path statement, telling the software exactly where the engine's executable file is located on the hard drive. You may "name" your engine anything you like; this will show up in a drop-down menu of engine selections. But the full path, and exact executable file name, must be precise or the engine will not start. Hint: since most GUI's come with one or more chess engines pre-installed, select one and click edit (don't change anything!) and you will see it's path statement. From that you will be able to figure out how to create one for your new engine. Force the download manager to place these files in a folder of your choice, not the Windows® default folder, or be prepared to move them later. Many engine downloads are one large but compressed (zipped) file; extract all the files to the folder of your choice, probably something like \(your engine)\bin\engines\. Do not just copy the engine's executable file over to your GUI's folder, as other supporting files and dll's are also likely required. Some free GUI's like Tarrasch or CB Reader, have limited and/or disabled features, while others, such as Arena and SCID, offer a wide range of features. The new Aquarium GUI featuring IDEA is all the rage, but only available for purchase, either alone or bundled with an engine. There is a trial version of Aquarium but many features (including IDEA) are disabled. The fully- featured Chessbase is only available for purchase. Chessbase Reader is just that, a no-frills .pgn file reader, but it is a free app. | |
Commercial chess engines / GUI's are designed to accommodate all players, from beginner to master. Many features are of little use to advanced chess players; they're designed for the benefit of novice players or "engine vs engine" enthusiasts. There are two configuration menus to deal with, settings for engine and for the GUI. Manuals, if available usually leave much to be desired, and so learning a host of technical concepts (hash size, threads, split depth, multi-pv etc.) and how to apply them becomes a process of research (users forums, web sites, wiki's and so forth) and trial-and-error. | |
What about opening and tablebase (endgame) databases? For the latter, the most widely used are for 6 (Nalimov) or 7 (Lomonosov) total pieces (includes kings and pawns, white and black.) They are of | |
Continued top right-hand column ... | |
[top] |
limited use beyond novice level, as players tend to resign lost games rather than play out simple endings, and complex endgames can't be analyzed with them. If a tablebase is loaded, the engine will make use of it when projecting lines that may simplify into such end games. Some engine / GUI packages will include pre-configured but limited tablebases (Gaviota, Syzygy etc.) Be aware, a completely installed tablebase consumes a huge amount (as in terabytes) of hard drive space! Nalimov (6 piece) tablebase can be used for free on the ChessOK website by entering the position in the GUI and clicking "analyze"; access to the Lomonosov (7 piece) tablebase requires purchase of a "VIP membership". | |||||||
Many engine packages include a digital opening "book"; configuration menus enable the chess engine to access its "book". More opening books may be purchased, either as downloads or dvd's. Players can also create, and configure their chess engines to access databases of their own .pgn game collections. Some of the "deep" chess engines are also "smart;" they can be configured to learn from previous opening analysis and game results and to record these for future reference. | |||||||
Purchase of some software, e.g., Chessbase, includes access to its huge on line database of games. At one time, Chessbase insisted on a proprietary (.cbh) format but this has been resolved. There are free programs to convert .cbh to .pgn format. Many download sites offer a choice of formats. Most engines are configured for universal .pgn format. Most database sites, including ICCF's game archives, offer free .pgn downloads. Note: you must be registered and log in with your ICCF ID# in order to access the download area. ICCF's .pgn files are bloated with 0- and 1-move forfeits and do not have ECO codes. Pgn files are easily cleaned-up, with SCID or similar software. Some free sites offer pgn downloads pre-sorted by opening variation, player or event, such as pgnmentor.com, while others such as chesslab.com allow further refinement, additional sorting by player name, year, rating range and/or result. | |||||||
Houdini 1.5a engine and SCID gui, both free downloads, analyzing Black's 14th move in single-line ( infinite analysis ) mode. | |||||||
Analyzing with a chess engine is straightforward: simply load a pgn game (or select one from a loaded file of such games), highlight the move immediately preceding the point where you wish to begin analyzing and start the engine. The engine/GUI will analyze/display as many lines as you have selected (called multi-pv mode) and rank the output. Evaluations in hundredths of a pawn are displayed for each move - these cause ranking of moves to change as the engine progresses through successive plies. Typically, move rankings change frequently early on but eventually stabilize and only change by minute amounts. When an initially lower-ranked move rises toward the top of the heap, we say it "bubbles up" (note: some GUI's display best line/most recent ply at the bottom of the stack - see screenshot above.) | |||||||
A high-ranked move may drop in ranking, sometimes even disappearing altogether. And, sometimes, a "disappeared" move will re-appear in the rankings a few plies deeper. In infinite mode, an engine will analyze until you shut it down or the computer's resources are exhausted and the machine locks-up. Advanced chess isn't speed chess. Never get in a hurry to send a reply to your opponent. Let the chess engine do its job! | |||||||
In correspondence chess, a non-engine player will lose to an engine-assisted player every time. Unlike conventional chess, in advanced chess it's extremely difficult to recover from even one weak move. Once your opponent's engine obtains a certain numeric advantage (the range of critical evaluations), it will steadily outplay your engine every time. Do not "second guess" the engine or start thinking you can play better moves. The only time you may need to think for yourself is where the top two (or more) ranked moves have identical, or nearly identical evaluations and the engine cannot break the tie, no matter how many plies have been examined. This occurs in "even" positions where the engine can find no tactical or maneuvering lines that force an edge. In these situations, you have no choice but to use your own "chess brain." | |||||||
Sometimes the top two (rarely, three or more) lines have identical evaluations because they are merely transpositions of moves reaching an identical position. Look for such a possibility by using the slider at the bottom of the analysis window, or perhaps by enlarging the GUI to full screen. If you observe identical moves in all lines beyond the end of the transposition sequence, further ply depths are pointless. An engine in multi-pv mode will not break such ties no matter how long it analyzes. | |||||||
Note: where actual engine analysis is presented below, moves are preceded with the vital statistics. For example, "{[Stockfish 5 64] 31:-0.48}" reveals the engine version is "Stockfish 5", "64" means running on a 64-bit operating system, "31:" is ply depth reached and "-0.48" means almost a half-pawn advantage to Black. | |||||||
Example 1. Position after 9. Ne4 ... | |||||||
| |||||||
The two transposition sequences (9... Nxe4 10.Bxe4 Nc6 11.O-O and 9...Nc6 10.O-O Nxe4 11.Bxe4 result in the same position. Does this mean the two moves truly are equal, that it makes no difference which one is played? As explained in subsequent paragraphs, switching to single-line or infinite analysis mode, and analyzing each move separately will often break the tie: | |||||||
| |||||||
Stockfish 5 discards the original variation after 9 ... Nc6, switching from 10. 0-0 to 10. Nxf6+ etc. In fact, the moves have changed in both lines, fairly typical of chess engines. The above lines are projections, subject to change at any time. As these lines are each analyzed one ply at a time, an engine will usually abandon its previous projection in favor of a different move sequence. The exceptions are sequences involving forced moves. After 27 ply, Stockfish 5 has evaluated 9... Nxe4 slightly better for Black. Four hundredths of a pawn difference is more like a photo-finish than a tie-breaker; further clarification is needed. Either analyze deeper or start over, returning to multi-pv mode and expanding the search with additional candidate moves. Finally, analyze these in single-line mode, one ply at a time. | |||||||
Example 2. Position after 33. Re1 ... | |||||||
In the diagram above, although only one pawn ahead, Stockfish 5 has taken positional considerations into account and evaluates Black as having a full piece advantage (-3.06). But after 34 ply, the engine cannot project a "best" move from among the top four candidates: | |||||||
| |||||||
The first two variations are transpositions, the last two lead to different positions. This situation is also typical, i.e., chess engines are notoriously weak in evaluating simplified positions (as in chess end games) where strategic concepts outweigh tactics. Here, as in the previous example, Black will have to switch to infinite analysis and examine each move in depth, to determine the "best" move. | |||||||
Some authors have recommended playing for such positions as part of an anti-computer strategy, i.e., to force the opponent to think for himself, especially in the opening phase. While this sounds like good advice, in practice not many truly even positions occur. Experienced players are aware of such "drawing lines" and can easily avoid them. To break evaluation ties, exit multi-pv mode and go to single line or infinite analysis, letting the engine sort out the best variation. Do this for all "tie" moves until you have your answer. By the way, this is also good practice for every move before you send it in a live game. If the evaluation drops significantly in single-line mode, compared to multi-pv mode, then re-analyze the position. | |||||||
Expanding the number of multi-pv lines often produces a different set of candidate moves to be further explored. Some players advocate a sort of trial-and-error method, letting the engine evaluate moves based on the player's intuition. The engine will quickly reveal how well such moves stack up against the cluster of top moves obtained in multi-pv mode. Some players also analyze with two or more engines, hoping to gain additional insight into a position, a "second opinion" as it were. Although the leading engines tend to produce the same cluster of "best" moves, due to the idiosyncracies of various programs, different engines may come up with additional candidate moves or slightly different evaluations. | |||||||
Occasionally multiple lines all evaluate to "0.00" - this is a drawn position. Positive numbers mean advantage to white, negative numbers are advantage black; the larger the number, the greater the advantage. There are also "fortress" positions and end games that are "book" draws, even though one side has material advantage or a higher engine evaluation. This was true in conventional chess and remains so in advanced chess. Chess engines have resolved (as in sequences of hundreds of moves) a few end games formerly considered drawn, but such "solutions" greatly exceed the limitation imposed by the 50-move rule. | |||||||
You will notice the number one-ranked move at 15 ply often does not change after 20 ply, 25 ply, even 30 ply and beyond. Naive players are thus tempted to save time and cut-off the engine at 20 ply or so, especially if the evaluation hasn't changed for many plies. In fact, one study of four popular engines found the top move at 15 ply does not change 85% of the time, even when 30 ply are reached. But that other 15% can ruin your day, not to mention what may happen when you quit at 30 ply but your opponent analyzes to 40 or more ply. Let the engine run to target depth, or as far as your equipment is capable, before sending that number one move in a live game. Keep in mind 30 ply is no longer "state of the art." Today (2014) it's 40 ply, with greater depth possible using the right hardware. Who knows what future technology may bring? | |||||||
A "dark horse" or "surprise" move may suddenly appear in the upper rankings, sometimes capturing the number one spot immediately. This only occurs when ply depth is deep and the engine has gathered enough new information to re-evaluate moves previously dismissed. Former World Champion Garry Kasparov once revealed he used a Hiarcs engine set on infinite analysis to check out TN's before playing them in otb events. Veteran advanced chess players spend a lot of time looking for just such anomalies. When they occur in "book" openings, they're called "theoretical novelties" or TN's. In the middle game, they constitute "traps" and advanced chess players are clever at misdirecting the unwary with "if" moves. In multi-pv mode, you may pre-set (also change on-the-fly, but evaluations begin anew) the number of different lines the engine will analyze/display. If the position contains 33 legal moves, and you set multi-pv to 6 moves, you will not be able to see the remaining 27 moves analyzed and you may have arbitrarily excluded a strong move. However, the more moves you tell the engine to analyze and display, the slower it runs (one more reason to buy fast processors and many gigabytes of memory!) If there are 33 legal moves in the position, then set multi-pv on 33 if the software allows it, or set it on the largest number the program will accept. Run this to a reasonable ply depth, long enough to observe the natural breaks in the evaluations; you should be able to pause the engine, dial multi-pv back to 3 or 4 moves and re-start the analysis. When you terminate multi-pv mode a second time, you'll have your top-ranked candidate move(s) which you should then final check in single-line (infinite analysis) mode, one at a time. | |||||||
Steve Lopez publishes an excellent series "Steve's Chessbase Tips" on the USCF sales website. Although explanations and visuals concentrate on products sold by USCF, such as Chessbase, Fritz and Rybka, much of his material applies to chess engines in general, their features, settings and how to use them. I found his posts about Fritz and Rybka in 2011 most informative: Steve's Chessbase Tips. He has continued this series with product reviews and articles on into 2014. | |||||||
Read as much as possible, ask questions when/where you can, and experiment. Unlike traditional chess, however, losing a bunch of advanced chess games is not cured by reading more chess books or paying for lessons from a chess coach. You will hit a rating/performance plateau that can only be improved by (1) purchasing better hardware and software and (2) becoming proficient in using an engine. Sorry, but that's just the way it is with advanced chess. | |||||||
Download pgn games | |||||||
Opening Databases with search capabilities | |||||||
| |||||||
GUI's | |||||||
Chess Engines (general information.) | |||||||
A selection of Chess Engines (in alphabetical order) | |||||||
Access Nalimov 6-piece endgame tablebase online | |||||||
Discussion forums on advanced chess technique | |||||||
[top] | |||||||