Working with Unix Tools

Line-oriented textual data streams are the lowest useful common denominator for a lot of data that passes through our hands. We can use such streams to represent program source code, Web server log data, version control history, file lists, symbol tables, archive contents, error messages, profiling data, and so on. For many routine, everyday tasks, we might be tempted to process the data using a "Swiss army knife" scripting language such as Perl, Python, or Ruby. However, doing that often requires writing a small, self-contained program and saving it into a file. By that point, we\´ve lost interest in the task and end up doing the work manually, if at all. Often, it\´s more effective to combine Unix tool chest programs into a short and sweet pipeline that we can run from our shell\´s command prompt. With modern shell command-line editing facilities, we can build our command bit by bit, until it molds into exactly the form that suits us. Nowadays, many different systems--including GNU/Linux, Mac OS X, and Microsoft Windows--offer the original Unix tools, so there\´s no reason to not add this approach to your arsenal. Many one-liners that you\´ll build around the Unix tools follow a pattern that goes roughly like this: fetching, selection, processing, and summarization. You\´ll also need to install some plumbing to unify these parts. This column will give you a quick overview.