4. Game-Playing Subsystem : Performance Evaluation

 

4.1 Experimental Design

  This system's performance was very difficult to measure experimentally. It would be difficult to design and execute a test that accurately measured the system's overall ability.

Instead, system effectiveness was informally measured by a series of matches against opponents of varying strength. Of the human opponents against which the system played complete games, it recorded 4 victories and 2 defeats. Again, these players very from a variety of skill levels. The system was consistently defeated by its only computer opponent: the Free Software Foundation's GNUChess.

The most highly-publicized game was a match against Taylor University philosophy professor Jim Spiegel. In a well-fought game, Machbanai was defeated in a rook-rook-bishop-pawn combination. In his commentary on the game, Dr. Spiegel, recognized on campus as a very strong player, rated the robot's performance as "about average" among the opponents he's faced.

4.2 System Statistics

  4.2.1 Run Time
  Run times for a single move, averaged over 93 moves:
  • Min: 0.17 seconds
  • Average: 5.69 seconds
  • Max: 46.23 seconds
4.2.2 Memory Requirements
  Memory allocated in choosing a single move, avaraged over 93 moves:
  • Min: 0.59 MB
  • Average: 18.27 MB
  • Max: 143.99 MB
Note: This figures represent total memory allocations, not the maximum concurrent allocation. In the course of searching, much of this memory is returned to the heap and reclaimed by gc.

4.3 Test Results

  (See Section 4.1 above)
[Previous Section] [Table of Contents] [Next Section]