A Tcl/Tk Front End for the Filter Processor

Mark Wroth

Revision History
Revision 030 June 2001
Initial implementation.

Table of Contents
1. Background
2. Installation
3. Implementation
A. Command Line Use of Filter

1. Background

This Tcl/Tk script provides a simple front end for Filter. The purpose is to provide a somewhat configurable graphical front end for repetitive invocation of the program.

Filter is a search program written by Joel Polowin, then of the Department of Chemistry, Queen's University, Kingston, Ontario (), with some minor modifications by the author of this script.

The Filter application is a grep-like search filter that examines a file and returns (to standard output) all lines that meet a user specified set of criteria. Filter has been optimized for the search of the SCA Ordinary and Armorial, a database of heraldic registrations.


2. Installation

The use of this TCL script presupposes that TCL is installed on the system. TCL can be downloaded from www.scriptics.com.

Since this is a graphical front end for Filter that program must be available on your system as well. The default set up of the front end assumes that Filter is accessible in the system command path; this can be modified after the GUI starts, however.

The purpose for which Filter and this script were written is searching the SCA Armorial. Electronic copies of this are available from several sources, a (partial) list of which is maintained at the SCA Heraldry web page.

If these conditions are met, it will be most convenient to put the TCL script in a convenient working directory—probably the same directory the Armorial is stored in. By default, the script will look for the Armorial in a file named oanda.txt in the same directory the script is started from. As with the name of the Filter executable, this can be changed once the script is running.


2.1. Creating the TCL Script

This document is a literate program; it contains both the documentation and computer code of the application.

To create the TCL-usable script from this document, you will need to use DBLP, which is available from my DBLP web site. Or you can use the version found here, which just the undocumented TCL script.


2.2. Getting Filter

Filter is freely distributable software according to its author; the version I make available is a literate programming version of the basic program. The only substantive modification I have made to the program itself is to lengthen the maximum length of a line Filter will process to account for the maximum length of lines found in the SCA Armorial.

The Windows executable can be downloaded here. The Nuweb source is available as filter.w, and the PDF version of the documentation is in filter.pdf.


3. Implementation

This program section establishes the overall order in which the various actions are taken. The details are shown in later section.

The basic concept is to establish a top level window, into which are packed frames that contain the major groups of widgets.

<SearchOandA.tcl (ID: SEARCHOANDA)>=

  1 <Initialize Constants>
  2 <Top Level Frames>
  3 <Set Up Configuration Buttons>
  4 <Set Up Include Entries>
  5 <Set Up Exclude Entries>
  6 <Set Up Action Buttons>
  7 <Common Procedures>

3.1. Initialization

Several variables need intitial values. The variables controlling the actual behavior of the system can be reset by the user (see Section 3.3).

<Initialize Constants (ID: INIT)>=

  1 set filter_name "filter.exe"
  2 set oanda_name  "oanda.txt"
  3 set debug 0

3.2. Top Level Layout

Laying out the various sections of the GUI in top level frames both provides a convenient organization, and allows the individual wigets to be packed into them later in the program.

<Top Level Frames (ID: TOP-FRAMES)>=

  1 frame .config
  2 frame .includes
  3 frame .excludes
  4 frame .actions
  5 pack .config .includes .excludes .actions -side top 

3.3. Configuration Buttons

The configuration buttons allow the user to reset the filter that will be used and the name of the Armorial file that will be searched.

<Set Up Configuration Buttons (ID: CONFIG-BUTTONS)>=

  1 button .exe -textvariable filter_name -command {
  2  set filter_name [tk_getOpenFile -filetypes {
  3       {{Executable files} {.exe}}
  4       {{All files} {""}}
  5       }]
  6 }
  7 
  8 button .target -textvariable oanda_name -command {
  9  set oanda_name [tk_getOpenFile -filetypes {
 10       {{Text files} {.txt}}
 11       {{All files} {""}}
 12       }]
 13 }
 14 pack .exe .target -side top -in .config -fill x

3.4. Include Term Entry Fields

The include entry fields allow the user to specify strings to be included in the search. More specifically, the strings entered here will be matched against lines in the target file, and lines that contain a match will be considered successful.

<Set Up Include Entries (ID: INCLUDE-ENTRIES)>=

  1 frame .inc1 
  2 label .findlabel1 -text "Search for:" -width 15
  3 entry .search1    -width 20 -relief sunken -textvariable search1term
  4 checkbutton .case1inc -variable inc1yes -text "Case Sensitive" 
  5 pack .findlabel1 .search1 .case1inc -side left -padx 1m -pady 2m -in .inc1
  6 pack .inc1 -in .includes
  7 
  8 frame .inc2
  9 label .findlabel2 -text "Or:" -width 15
 10 entry .search2    -width 20 -relief sunken -textvariable search2term
 11 checkbutton .case2inc -variable inc2yes -text "Case Sensitive" 
 12 pack .findlabel2 .search2 .case2inc -side left -padx 1m -pady 2m -in .inc2
 13 pack .inc2 -in .includes
 14 

3.5. Exclude Term Entry Fields

Similarly, filter allows us to exclude lines that match a certain string. This entry field allows the user to specifiy such a string.

<Set Up Exclude Entries (ID: EXCLUDE-ENTRIES)>=

  1 frame .ex1
  2 label .excludelabel1 -text "But not:" -width 15
  3 entry .exclude1    -width 20 -relief sunken -textvariable x1term
  4 checkbutton .excl1inc -variable exc1yes -text "Case Sensitive" 
  5 pack .excludelabel1 .exclude1 .excl1inc -side left -padx 1m -pady 2m \
  6   -in .actions -in .ex1
  7 pack .ex1 -in .excludes

3.6. Action Buttons

The real meat of the program is in the "Find It" action button. It constructs a command string based on the user's selections, and then runs that command string, capturing the results in a disk file. The program then displays that disk file using a common procedure (an empty file provides the user feedback that the search found nothing).

Most of the complexity in the definition of the command string results form two logical tests; is the entry string empty, and should the search be case sensitive or not?

<Set Up Action Buttons (ID: ACTION-BUTTONS)>=

  1 button .go -text "Find It" -command {
  2     set cmd ""
  3     append cmd "\"$filter_name\" \"$oanda_name\" " 
  4     if $inc1yes {
  5       append cmd " \"="
  6       } else {
  7       append cmd " \"+" } 
  8     append cmd [string trim $search1term] "\""
  9     if [string length [string trim $search2term]]>0 then {
 10       if $inc2yes {
 11         append cmd " \"="
 12         } else {
 13         append cmd " \"+" } 
 14       append cmd [string trim $search2term] "\""
 15     }
 16     if [string length [string trim $x1term]]>0 then {
 17       if $exc1yes {
 18         append cmp " \"_"
 19         } else {
 20         append cmd " \"-" } 
 21       append cmd [string trim $x1term] "\""
 22     }
 23     if $debug {tk_messageBox -message $cmd}
 24     eval exec $cmd > results.txt
 25     showFile results.txt
 26 }
 27 
 28 button .exit -text "EXIT"  -command {exit}
 29 
 30 pack .go .exit -side left -fill x -in .actions

3.7. Common Procedures

These two procedures are basically common procedures to list a file. The basics of them are taken from the user documentation for TCL.

<Common Procedures (ID: COMMON-PROCS)>=

  1 proc showFile {thefile} {
  2      set w .text
  3      catch {destroy $w}
  4      toplevel $w
  5      wm title $w "$thefile"
  6      wm iconname $w "text"
  7 
  8      frame $w.buttons
  9      pack $w.buttons -side bottom -fill x -pady 2m
 10      button $w.buttons.dismiss -text Dismiss -command "destroy $w"
 11      pack $w.buttons.dismiss  -side bottom -expand 1
 12 
 13      text $w.text -relief sunken -bd 2 -yscrollcommand "$w.scroll set" \
 14         -setgrid 1 -height 30
 15      scrollbar $w.scroll -command "$w.text yview"
 16      pack $w.scroll -side right -fill y
 17      pack $w.text -expand yes -fill both
 18      textLoadFile $w.text $thefile
 19      $w.text mark set insert 0.0 } 
 20 
 21 proc textLoadFile {w file} {
 22     if [file exists $file] {
 23       set f [open $file]
 24       $w delete 1.0 end
 25       while {![eof $f]} {
 26          $w insert end [read $f 10000]
 27       }
 28       close $f
 29   }
 30 }

A. Command Line Use of Filter

A.1. Identification

This description is for Filter revision 2.0 dated 28 February 1994, a grep-like text searcher for multiple simultaneous keyword tests.

The Filter program itself is copyright © 1994 by Joel Polowin, Department of Chemistry, Queen's University, Kingston, Ontario, Canada, who granted permission for free use and distribution while asking for credit/blame for writing it. His email addresses at that time were , , or .

The literate programming (from which this documentation is adapted) version of this file should be blamed on Mark Wroth, . Joel was foolish enough to allow me to port his program to nuweb on the Amiga, a permission which I have extended to a Windows NT version. Kudos for the program goes to him; blame for the port belongs to me.

Literate programming documentation copyright © 1995, 2000 by Mark Wroth. Permission is granted for use and distribution under the terms set above by the code author.

If you see something wrong with it or it fails to work, please let one of us know!


A.2. Syntax

filter [filename] [filename ...] string [string ...]

where each string (default max. of 2000) is a term to be searched for in lines (default max. 600 chars) in file(s) filename, prefixed by one of the following characters:

  • + to show lines which contain string.

  • - to show lines which do not contain string.

  • = to show lines which contain string, case sensitive.

  • _ (underscore) to show lines which do not contain string, case sensitive.

A string as above may be further prefixed with the letter o to print the line if the current OR the preceding condition is true.

A string including blanks and the prefix may be enclosed in double quotes on most systems. Your operating system may have other ways of dealing with special characters.

Filter determines the first string which is a search term instead of a file name by its beginning with one of the characters +-=_. For files whose names begin with one of these characters, see below. Otherwise, the first search term must begin with one of these, as that first term cannot be or-linked to a preceding term.

For strings that begin with $ or & (usually names of files of search terms), see below.

Example A-1. Basic Searching

filter * +hawk +handsaw o+hound searches all files in the current directory and prints lines that contain the string hawk and at least one of handsaw and hound. This assumes that the operating system and compiler accept wild-carded file names; else Filter will be looking for a file named *. For DOS, one would use *.*.

Example A-2. Case Sensitive Combined Search

filter armorial =Vert +argent -gules _Or -azure -purpur +foil > tempfile.txt searches the file armorial for lines that contain the string Vert (case-sensitive) and argent (upper or lower case) but not gules (upper or lower case) and not Or (case-sensitive) and not azure or purpur (upper or lower case) and do contain foil (upper or lower case); the resulting lines are saved in file tempfile.txt.

Example A-3. Quoted Search Strings

type temp1.txt | filter +aardvark "o+winged pig" o+wombat "_| B|" The file temp1.txt is fed through the Filter program, which passes lines that contain aardvark (upper or lower case) or the string winged pig or wombat (upper or lower case) and do not contain | B|. The result is printed on the screen. Note use of quotation marks in the command line to include the space in winged pig and the special character | in | B|.


A.2.1. File names beginning with one of +-=_

If you absolutely must use text file names that begin with one of these characters, use the character twice when specifying the file name to Filter. Thus, the file name -stdev.c would be written --stdev.c; ++junk.c would be written ++++junk.c.

What Filter does is to go through each term in the command line and count the number of identical flag characters beginning each; that number is reduced by half, rounded down. An even number specifies a text file name; an odd number designates a search term. +junk has one flag character, is not changed (shortened by 1/2 → 0 characters), and is a search term: print lines containing junk. ++junk has two flag characters, is shortened to +junk, and is read as a text file name. +++junk is shortened to ++junk, and is a search term: print lines containing the string +junk. +=junk has one flag character and is a search term: print lines containing the string =junk.

This means that wild-carded file names that match files whose names begin with one of "+-=_" will cause trouble. I'm sorry; the telepathic monitors of most computer systems are not software-addressable. A compulsive urge to use files whose names begin with punctuation or mathematical symbols can now be treated successfully in a majority of cases.

Search terms specified in files (see below) are not themselves in the command line, and if they begin with one of "+-=_" those characters should not be doubled. Search-term file expansion takes place after Filter determines which command-line strings are file names.


A.2.2. $ and & usually flag search-term file names

A string which begins with \$} or \str{\& will be expanded as the name of a file containing a list of search terms. For example, the string +$critters tells {\filter} to look for a file \filename{critters}; lines from that file are taken as search terms. Filter adds the prefix to each term, depending on whether the file name is specified with $ or & and which of +-=_ precedes it. With $, + and = give or-linked terms, so that text file lines will be printed if any search-term file line is matched; - and _ are not or-linked, so that text file lines are printed only if no search-term file line is matched. With &, + and = are not or-linked, so that all search-term lines must be matched to print a text line; while - and _ are or-linked, so that any search-term line not matched will allow text-line printing.

To search for actual text strings beginning with $ or &, double the flag characters. Thus, to search for the string $100, use the search string +$$100. To expand file names beginning with those characters, use three of them: search-term file $&junk would be specified with something like -$$$&junk. In general, when a search term begins with a flag character, double each flag character of that kind beginning the term, and if the term is a file name, add an extra flag character.

Search-term files may contain file names, which will be expanded in turn. For this reason, initial $ and & characters must be doubled even in nested search-term files.

Note that the or-linking logic can get seriously messed up when terms beginning with - or _ are expanded carelessly, as Filter has no good sense of logical precedence. If file human contains man and woman, then o-$human would expand as o-man -woman.