Name: Exploring Expect: A Tcl-based Toolkit for Automating Interactive Programs (Nutshell Handbooks)
Rating: 3.94 (4 reviews)
ISBN: 9781565920903

Jerry

Author 10 books27 followers

April 15, 2020

It is almost always easier to fix problems if you made some attempt at detecting them in the first place.

I ran into Expect as the solution to a problem many years ago, probably in the mid-2000s, and I thought at the time that it seemed like a good idea to learn it more. But I put it on the back burner until I happened to see this book in a used bookstore’s discount rack in Raleigh, North Carolina.

I think it remains worth knowing, and is likely to be a useful part of a serious scripting toolbox.

Expect is mainly for scripting poorly-designed programs that (a) don’t timeout well, or (b) require user input instead of command-line switches or (in the case of passwords) some form of automating the acquisition of them without sticking them on the command line. These problems are much rarer now than when this book was written.

The age of the book—it was published in January 1995, which means it was probably written over 1994—means that many of the examples are outdated. They tend to involve telnet, login, rsh, ftp, and even finger. The first three don’t exist on my Macintosh, and I’m slightly surprised that the last one does. The examples provided for these, especially ftp and rsh, are unnecessary in modern sftp and ssh.

And of course programs change over time, so that while the script that attempts to re-run rogue until it generates a strength of 18 works, it never ends, because rogue no longer generates a strength of 18, or if it does, it is much less often than the one-in-twenty chance the author was expecting in 1995.

Even more outdated is the mindset behind some of the examples, such as the “passmass” script that automates setting your password to the same thing on every host you have a login to. This script is not provided in the book, it is merely described, because the script was part of the standard Expect installation. It doesn’t appear to be now, unless I don’t have a complete installation.

The examples use, rather than example.com addresses, existing sites at the time, such as next.com or uunet.uu.net. (While the publisher lists the release date as August 2010 on their ebook site, this appears to be simply the date the ebook was released—the snippets available and the page count match the 1995 print version exactly.)

A few of the examples would still be useful for system administrators in special cases, such as scripting ping, but even ping now includes functionality that comes close to what the examples do.

One intriguing example involves bc, the command-line calculator. It was very difficult for me to write the Perl script padmath (that I included in 42 Astounding Scripts) to make bc accessible as a service on the Mac, because bc remains unsuitable for interaction with scripts. It looks like padmath would have been easier to write and more reliable in Expect.

Another interesting example is using Expect to make the command-line chess script play against itself. Expect can be used like putting two telephones together to connect different programs that each are expecting user input.

A lot of the need for expect has gone. Command-line programs are a lot smarter now, and the options are greater. The successors to ftp and telnet are very scriptable, and in a more secure fashion than storing the password in a script-accessible fashion; sudo probably didn’t even exist at the time Libes wrote this book—while sudo existed before 1994, it’s unclear to me how widespread it was—and the spread of web-based REST APIs that are easily accessed via scripts was still well in the future. The web was only a few years old and is barely mentioned. He mentions toward the beginning that some of the documentation for Expect is on the WWW, and he briefly mentions Mosaic toward the end.

Some of the examples even involve dialup, and there’s a special variable that controls how fast Expect outputs text, because dialup often lost data when data was sent too quickly. Even more interesting is that this functionality is augmented by the ability to act like a human, with a variable rate of typing.

There’s a technique he uses in writing this book that I find annoying, which is that he’ll produce Expect code for performing some common task, describe how the code works and why, and then says something like, this task is so common there’s a built-in way to do it in Expect, so you don’t need all that code.

What this means is that there are several code chunks in the book that you do not want to use. Reading the book through, that’s fine, because you’ll read the whole text. But if you use it as a reference, you might see the code chunk and not the explanation that there’s a better way of doing it.

He also provides code snippets that simply don’t work. For example, in the chapter on “interact”, there is code that, if attempted, provides the error “cannot interact with self”. The solution is to add “spawn $env(SHELL)” before the “interact” line.

This really does make me want to find reasons to write Expect scripts. It seems especially useful for its ability to mediate between the operator and the shell. One script I took from the book and have put into my toolbox is one that asks for user input, but with a timeout, so that the script can ask for input but not get locked up if there is no answer. This seems like it ought to be useful.


#!/usr/bin/expect --
# wait x seconds for a response and return it
# timedRead 
# Don Libes, Exploring Expect, p. 77
# Modifications Jerry Stratton
# astoundingscripts.com

#help text
if {$argc != 1 || [lsearch $argv "--help"] > -1} {
	puts "Usage: $argv0 "
	puts "\twait  seconds for a typed response, and return that response"
	exit 1
}

set timeout $argv
expect "\n" {
	send [string trimright "$expect_out(buffer)" "\n"]
}

Another idea is a script that itself starts up the default shell, continuing to allow normal shell commands, but that which logs the time spent on each command to a file. I have a handful of commands, mainly backup commands, that I run weekly and am always forgetting to time them, either by hand or by using the built-in command-line timer.

Interact seems like a powerful tool for customizing both the shell and any command-line app.

Expect uses Tcl as its language, and this book provides a decent amount of info on how to use Tcl, how to debug Tcl, how to use Tk, and how to use Tcl’s globs and regular expressions.

I need to reiterate that this is a scripting book from 1995. The fact that a quarter-century-old scripting tutorial is useful at all is amazing, and many of its flaws were not widely considered flaws at the time. Many of my complaints boil down to the book not being designed for using the examples while following along. In 1995, in order to follow along on the command line while reading this book I would either have had to go to work early, stay at work late, read it during work hours, or keep my telephone tied up while reading it at home, just for the occasional desire to type something or play around with the concepts.

The fact that my current off-the-shelf user-friendly home computer has Unix built into it is amazing. That my home computer sports as much of a command-line environment as a server from 1995 is also amazing. Both combine to obscure that this book was written for an entirely different era and a nearly entirely different computing environment. My own 42 Astounding Scripts will not be half as useful in 2044 as this book is in 2020.

All sorts of sins can be covered up in this way.

Exploring Expect: A Tcl-based Toolkit for Automating Interactive Programs

Don Libes

About the author

Don Libes

Ratings & Reviews

Friends & Following

Community Reviews

Join the discussion

Can't find what you're looking for?