<!DOCTYPE article
 PUBLIC "-//Mark Wroth//DTD DocBook V4.1-Based Extension Literate Programming 1.1//EN"[
<!ENTITY filter "<application>Filter</application>">
]>
<!-- # $Id: FilterTCL.sgm,v 1.4 2001/07/06 04:54:30 Mark Exp Mark $ -->
<article id="JadeTCL">
  <articleinfo>
    <author>
      <firstname>Mark</firstname>
      <surname>Wroth</surname>
    </author>
    <title>A Tcl/Tk Front End for the &filter; Processor</title>
    <revhistory>
      <revision>
	<revnumber>0</revnumber>
	<date>30 June 2001</date>
	<revremark>Initial implementation.</revremark>
      </revision>
    </revhistory>
  </articleinfo>
  <section>
    <title>Background</title>
    <para>This <application>Tcl/Tk</application> script provides a
    simple front end for &filter;.  The purpose
    is to provide a somewhat configurable graphical front end for
    repetitive invocation of the program. </para>

    <para><application>Filter</application> is a search program
    written by Joel Polowin, then of the Department of Chemistry,
    Queen's University, Kingston, Ontario
    (<email>polowin@silicon.chem.queensu.ca</email>), with some minor
    modifications by the author of this script.</para>

    <para>The &filter; application is a
    <application>grep</application>-like search filter that examines a
    file and returns (to standard output) all lines that meet a user
    specified set of criteria.  <application>Filter</application> has
    been optimized for the search of the <acronym>SCA</acronym>
    <citetitle>Ordinary and Armorial</citetitle>, a database of
    heraldic registrations.</para>
  </section>
  <section>
    <title>Installation</title>
    <para>The use of this <application>TCL</application> script
    presupposes that <application>TCL</application> is installed on
    the system.  <application>TCL</application> can be downloaded from
    <ulink
    url="http://www.scriptics.com/">www.scriptics.com</ulink>.</para> 

    <para>Since this is a graphical front end for &filter; that
    program must be available on your system as well.  The default set
    up of the front end assumes that &filter; is accessible in the
    system command path; this can be modified after the
    <acronym>GUI</acronym> starts, however.</para>

    <para>The purpose for which &filter; and this script were written
    is searching the <acronym>SCA</acronym>
    <citetitle>Armorial</citetitle>. Electronic copies of this are
    available from several sources, a (partial) list of which is
    maintained at <ulink url="http://sca.org/heraldry/OandA/">the SCA
    Heraldry web page</ulink>.</para>

    <para>If these conditions are met, it will be most convenient to
    put the <application>TCL</application> script in a convenient
    working directory&mdash;probably the same directory the
    <citetitle>Armorial</citetitle> is stored in.  By default, the
    script will look for the <citetitle>Armorial</citetitle> in a file
    named <filename>oanda.txt</filename> in the same directory the
    script is started from.  As with the name of the &filter;
    executable, this can be changed once the script is running.</para>
    <section>
      <title>Creating the <application>TCL</application> Script</title>
      <para>This document is a <firstterm>literate
      program</firstterm>; it contains both the documentation and
      computer code of the application.</para>
      <para>To create the <application>TCL</application>-usable script
      from this document, you will need to use
      <acronym>DBLP</acronym>, which is available from <ulink
      url="http://www.west-point.org/users/usma1978/36200/LitProg/SGMLWEB/index.htm">my
      DBLP web site</ulink>. Or you can use the version found <ulink
      url="./SearchOandA.tcl">here</ulink>,
      which just the undocumented <application>TCL</application> script.</para>
    </section>
    <section>
      <title>Getting &filter;</title> 

      <para>&filter; is freely distributable software according to its
      author; the version I make available is a literate programming
      version of the basic program.  The only substantive modification
      I have made to the program itself is to lengthen the maximum
      length of a line &filter; will process to account for the
      maximum length of lines found in the <acronym>SCA</acronym>
      <citetitle>Armorial</citetitle>.</para>

      <para>The <application>Windows</application> executable can be
      downloaded <ulink url="./filter.exe">here</ulink>.  The
      <application>Nuweb</application> source is available as <ulink
      url="./Filter/filter.w"><filename>filter.w</filename></ulink>,
      and the <acronym>PDF</acronym> version of the documentation is
      in <ulink
      url="./Filter/FILTER.PDF"><filename>filter.pdf</filename></ulink>.
      </para>
    </section>
  </section>

  <section>
    <title>Implementation</title>

    <para>This program section establishes the overall order in which
    the various actions are taken.  The details are shown in later
    section.</para>

    <para>The basic concept is to establish a top level window, into
    which are packed frames that contain the major groups of
    widgets.</para> 

<programlisting 
 id   = "SearchOandA"
 file = "SearchOandA.tcl"
>
<xref linkend="init">
<xref linkend="top-frames">
<xref linkend="config-buttons">
<xref linkend="include-entries">
<xref linkend="exclude-entries">
<xref linkend="action-buttons">
<xref linkend="common-procs">
</programlisting>
    <section>
      <title>Initialization</title>

      <para>Several variables need intitial values.  The variables
      controlling the actual behavior of the system can be reset by
      the user (see <xref linkend="sec-config">).</para>

<programlisting
 id="init"
 xreflabel = "Initialize Constants"
>
set filter_name "filter.exe"
set oanda_name  "oanda.txt"
set debug 0
</programlisting>
    </section>
    <section>
      <title>Top Level Layout</title>

      <para>Laying out the various sections of the
      <acronym>GUI</acronym> in top level frames both provides a
      convenient organization, and allows the individual wigets to be
      packed into them later in the program.</para>

<programlisting
 id="top-frames"
 xreflabel = "Top Level Frames"
>
frame .config
frame .includes
frame .excludes
frame .actions
pack .config .includes .excludes .actions -side top 
</programlisting>
    </section>
    <section id="sec-config">
      <title>Configuration Buttons</title>

      <para>The configuration buttons allow the user to reset the
      <application>filter</application> that will be used and the name
      of the Armorial file that will be searched.</para>

<programlisting
 id        = "config-buttons"
 xreflabel = "Set Up Configuration Buttons"
>
button .exe -textvariable filter_name -command {
 set filter_name [tk_getOpenFile -filetypes {
      {{Executable files} {.exe}}
      {{All files} {""}}
      }]
}

button .target -textvariable oanda_name -command {
 set oanda_name [tk_getOpenFile -filetypes {
      {{Text files} {.txt}}
      {{All files} {""}}
      }]
}
pack .exe .target -side top -in .config -fill x
</programlisting>
    </section>
    <section>
      <title>Include Term Entry Fields</title>

      <para>The include entry fields allow the user to specify strings
      to be included in the search.  More specifically, the strings
      entered here will be matched against lines in the target file,
      and lines that contain a match will be considered
      successful.</para> 

<programlisting
 id        = "include-entries"
 xreflabel = "Set Up Include Entries"
>
frame .inc1 
label .findlabel1 -text "Search for:" -width 15
entry .search1    -width 20 -relief sunken -textvariable search1term
checkbutton .case1inc -variable inc1yes -text "Case Sensitive" 
pack .findlabel1 .search1 .case1inc -side left -padx 1m -pady 2m -in .inc1
pack .inc1 -in .includes

frame .inc2
label .findlabel2 -text "Or:" -width 15
entry .search2    -width 20 -relief sunken -textvariable search2term
checkbutton .case2inc -variable inc2yes -text "Case Sensitive" 
pack .findlabel2 .search2 .case2inc -side left -padx 1m -pady 2m -in .inc2
pack .inc2 -in .includes

</programlisting>
    </section>
    <section>
      <title>Exclude Term Entry Fields</title>

      <para>Similarly, <application>filter</application> allows us to
      exclude lines that match a certain string.  This entry field
      allows the user to specifiy such a string.</para>

<programlisting
 id        = "exclude-entries"
 xreflabel = "Set Up Exclude Entries"
>
frame .ex1
label .excludelabel1 -text "But not:" -width 15
entry .exclude1    -width 20 -relief sunken -textvariable x1term
checkbutton .excl1inc -variable exc1yes -text "Case Sensitive" 
pack .excludelabel1 .exclude1 .excl1inc -side left -padx 1m -pady 2m \
  -in .actions -in .ex1
pack .ex1 -in .excludes
</programlisting>
    </section>
    <section>
      <title>Action Buttons</title>

      <para>The real meat of the program is in the <quote>Find
      It</quote> action button.  It constructs a command string
      based on the user's selections, and then runs that command
      string, capturing the results in a disk file.  The program then
      displays that disk file using a common procedure (an empty file
      provides the user feedback that the search found
      nothing).</para>

      <para>Most of the complexity in the definition of the command
      string results form two logical tests; is the entry string
      empty, and should the search be case sensitive or not?</para> 

<programlisting
 id        = "action-buttons"
 xreflabel = "Set Up Action Buttons"
>
button .go -text "Find It" -command {
    set cmd ""
    append cmd "\"$filter_name\" \"$oanda_name\" " 
    if $inc1yes {
      append cmd " \"="
      } else {
      append cmd " \"+" } 
    append cmd [string trim $search1term] "\""
    if [string length [string trim $search2term]]&greaterthan;0 then {
      if $inc2yes {
        append cmd " \"="
        } else {
        append cmd " \"+" } 
      append cmd [string trim $search2term] "\""
    }
    if [string length [string trim $x1term]]&greaterthan;0 then {
      if $exc1yes {
        append cmp " \"_"
        } else {
        append cmd " \"-" } 
      append cmd [string trim $x1term] "\""
    }
    if $debug {tk_messageBox -message $cmd}
    eval exec $cmd &greaterthan; results.txt
    showFile results.txt
}

button .exit -text "EXIT"  -command {exit}

pack .go .exit -side left -fill x -in .actions
</programlisting>
    </section>
    <section>
       <title>Common Procedures</title>

       <para>These two procedures are basically common procedures to
       list a file.  The basics of them are taken from the user
       documentation for <application>TCL</application>.</para>

<programlisting
 id        = "common-procs"
 xreflabel = "Common Procedures"
>
proc showFile {thefile} {
     set w .text
     catch {destroy $w}
     toplevel $w
     wm title $w "$thefile"
     wm iconname $w "text"

     frame $w.buttons
     pack $w.buttons -side bottom -fill x -pady 2m
     button $w.buttons.dismiss -text Dismiss -command "destroy $w"
     pack $w.buttons.dismiss  -side bottom -expand 1

     text $w.text -relief sunken -bd 2 -yscrollcommand "$w.scroll set" \
        -setgrid 1 -height 30
     scrollbar $w.scroll -command "$w.text yview"
     pack $w.scroll -side right -fill y
     pack $w.text -expand yes -fill both
     textLoadFile $w.text $thefile
     $w.text mark set insert 0.0 } 

proc textLoadFile {w file} {
    if [file exists $file] {
      set f [open $file]
      $w delete 1.0 end
      while {![eof $f]} {
         $w insert end [read $f 10000]
      }
      close $f
  }
}</programlisting>
    </section> 
  </section>
  <appendix>
    <title>Command Line Use of &filter;</title>

    <section>
      <title>Identification</title>

      <para>This description is for &filter; revision 2.0 dated 28
      February 1994, a <application>grep</application>-like text
      searcher for multiple simultaneous keyword tests.</para>

      <para>The &filter; program itself is copyright &copy; 1994 by
      Joel Polowin, Department of Chemistry, Queen's University,
      Kingston, Ontario, Canada, who granted permission for free use
      and distribution while asking for credit/blame for writing it.
      His email addresses at that time were
      <email>polowin@silicon.chem.queensu.ca</email>,
      <email>polowinj@qucdn.queensu.ca</email>, or
      <email>Joel.Polowin@p4.f107.n249.z1.fidonet.org</email>.</para> 

      <para>The literate programming (from which this documentation is
      adapted) version of this file should be blamed on Mark Wroth,
      <email>mark@astrid.upland.ca.us</email>.  Joel was foolish
      enough to allow me to port his program to
      <application>nuweb</application> on the Amiga, a permission
      which I have extended to a <application>Windows NT</application>
      version.  Kudos for the program goes to him; blame for the port
      belongs to me.</para>

      <para>Literate programming documentation copyright &copy; 1995,
      2000 by Mark Wroth.  Permission is granted for use and
      distribution under the terms set above by the code
      author.</para>

      <para>If you see something wrong with it or it fails to work,
      <emphasis>please</emphasis> let one of us know!</para>

    </section>

    <section>
      <title>Syntax</title>

      <synopsis>filter <optional><parameter>filename</parameter></optional> <optional><parameter>filename ...</parameter></optional> <parameter>string</parameter> <optional><parameter>string ...</parameter></optional></synopsis>
    
      <para>where each <parameter>string</parameter> (default max. of
      2000) is a term to be searched for in lines (default max. 600
      chars) in file(s) <parameter>filename</parameter>, prefixed by
      one of the following characters:<itemizedlist>
	  <listitem>
	    <para><keycap>+</keycap> to show lines which contain string.</para>
	  </listitem>
	  <listitem>
	    <para><keycap>-</keycap> to show lines which do not
	    contain string.</para>
	  </listitem>
	  <listitem>
	    <para><keycap>=</keycap> to show lines which contain
	    string, case sensitive.</para>
	  </listitem>
	  <listitem>
	    <para><keycap>_</keycap> (underscore) to show lines which
	    do not contain string, case sensitive.</para>
	  </listitem>
	</itemizedlist>
      </para>

      <para>A string as above may be further prefixed with the letter
      <userinput>o</userinput> to print the line if the current OR the
      preceding condition is true.</para>

      <para>A string including blanks and the prefix may be enclosed
      in double quotes on most systems.  Your operating system may
      have other ways of dealing with special characters.</para>

      <para>&filter; determines the first string which is a search
      term instead of a file name by its beginning with one of the
      characters <userinput>+-=_</userinput>.  For files whose names
      begin with one of these characters, see below.  Otherwise, the
      first search term must begin with one of these, as that first
      term cannot be <userinput>or</userinput>-linked to a preceding
      term.</para>

      <para>For strings that begin with <keycap>$</keycap> or
      <keycap>&</keycap> (usually names of files of search terms), see
      below.</para>

      <example>
        <title>Basic Searching</title>
        <para><userinput>filter * +hawk +handsaw o+hound</userinput>
        searches all files in the current directory and prints lines
        that contain the string <userinput>hawk</userinput> and at
        least one of <userinput>handsaw</userinput> and
        <userinput>hound</userinput>.  This assumes that the operating
        system and compiler accept wild-carded file names; else
        &filter; will be looking for a file named
        <userinput>*</userinput>.  For DOS, one would use
        <userinput>*.*</userinput>.</para>
      </example>
      <example>
	<title>Case Sensitive Combined Search</title>

        <para><userinput>filter armorial =Vert +argent -gules _Or
-azure -purpur +foil > tempfile.txt</userinput> searches the file
<filename>armorial</filename> for lines that contain the string
<userinput>Vert</userinput> (case-sensitive) and
<userinput>argent</userinput> (upper or lower case) but not
<userinput>gules</userinput> (upper or lower case) and not
<userinput>Or</userinput> (case-sensitive) and not
<userinput>azure</userinput> or <userinput>purpur</userinput> (upper
or lower case) and <emphasis>do</emphasis> contain
<userinput>foil</userinput> (upper or lower case); the resulting lines
are saved in file <filename>tempfile.txt</filename>.</para>
      </example>
  
	<example>
	  <title>Quoted Search Strings</title>

	  <para><userinput>type temp1.txt | filter +aardvark "o+winged
          pig" o+wombat "_| B|"</userinput> The file
          <filename>temp1.txt</filename> is fed through the &filter;
          program, which passes lines that contain
          <userinput>aardvark</userinput> (upper or lower case) or the
          string <userinput>winged pig</userinput> or
          <userinput>wombat</userinput> (upper or lower case) and do
          <emphasis>not</emphasis> contain <userinput>|
          B|</userinput>.  The result is printed on the screen.  Note
          use of quotation marks in the command line to include the
          space in <userinput>winged pig</userinput> and the special
          character <userinput>|</userinput> in <userinput>|
          B|</userinput>.</para>
	</example>
        
      <section>
        <title>File names beginning with one of
        <literal>+-=_</literal></title> 

        <para>If you absolutely <emphasis>must</emphasis> use text
        file names that begin with one of these characters, use the
        character twice when specifying the file name to &filter;.
        Thus, the file name <filename>-stdev.c</filename> would be
        written <filename>--stdev.c</filename>;
        <filename>++junk.c</filename> would be written
        <filename>++++junk.c</filename>.</para>

        <para>What &filter; does is to go through each term in the
        command line and count the number of identical flag characters
        beginning each; that number is reduced by half, rounded down.
        An even number specifies a text file name; an odd number
        designates a search term.  <literal>+junk</literal> has one
        flag character, is not changed (shortened by 1/2 &rarr; 0
        characters), and is a search term: print lines containing
        <userinput>junk</userinput>.  <userinput>++junk</userinput>
        has two flag characters, is shortened to
        <literal>+junk</literal>, and is read as a text file name.
        <userinput>+++junk</userinput> is shortened to
        <literal>++junk</literal>, and is a search term: print lines
        containing the string <literal>+junk</literal>.
        <userinput>+=junk</userinput> has one flag character and is a
        search term: print lines containing the string
        <literal>=junk</literal>.</para>

        <para>This means that wild-carded file names that match files
        whose names begin with one of
        <quote><literal>+-=_</literal></quote> will cause
        trouble.  I'm sorry; the telepathic monitors of most computer
        systems are not software-addressable.  A compulsive urge to
        use files whose names begin with punctuation or mathematical
        symbols can now be treated successfully in a majority of
        cases.</para>

        <para>Search terms specified in files (see below) are not
        themselves in the command line, and if they begin with one of
        <quote><literal>+-=_</literal></quote> those characters should
        not be doubled.  Search-term file expansion takes place after
        &filter; determines which command-line strings are file
        names.</para>
      </section>
      <section>
	<title><literal>$</literal> and <literal>&</literal> usually
	flag search-term file names</title>

        <para>A string which begins with <literal>\$} or
        \str{\&</literal> will be expanded as the name of a file
        containing a list of search terms.  For example, the string
        <literal>+$critters</literal> tells {\filter} to look for a
        file \filename{critters}; lines from that file are taken as
        search terms.  &filter; adds the prefix to each term,
        depending on whether the file name is specified with
        <literal>$</literal> or <literal>&</literal> and which of
        <literal>+-=_</literal> precedes it.  With
        <literal>$</literal>, <literal>+</literal> and
        <literal>=</literal> give or-linked terms, so that text file
        lines will be printed if any search-term file line is matched;
        <literal>-</literal> and <literal>_</literal> are not
        or-linked, so that text file lines are printed only if no
        search-term file line is matched.  With <literal>&</literal>,
        <literal>+</literal> and <literal>=</literal> are not
        or-linked, so that <emphasis>all</emphasis> search-term lines must be
        matched to print a text line; while <literal>-</literal> and
        <literal>_</literal> <emphasis>are</emphasis> or-linked, so
        that any search-term line <emphasis>not</emphasis> matched
        will allow text-line printing.</para>

        <para>To search for actual text strings beginning with
        <literal>$</literal> or <literal>&</literal>, double the flag
        characters.  Thus, to search for the string
        <literal>$100</literal>, use the search string
        <literal>+$$100</literal>.  To expand file names beginning
        with those characters, use three of them: search-term file
        <literal>$&amp;junk</literal> would be specified with
        something like <literal>-$$$&amp;junk</literal>.  In general,
        when a search term begins with a flag character, double each
        flag character of that kind beginning the term, and if the
        term is a file name, add an extra flag character.</para>

        <para>Search-term files may contain file names, which will be
        expanded in turn.  For this reason, initial
        <literal>$</literal> and <literal>&</literal> characters must
        be doubled even in nested search-term files.</para>

        <para>Note that the or-linking logic can get seriously messed
        up when terms beginning with <literal>-</literal> or
        <literal>_</literal> are expanded carelessly, as &filter; has
        no good sense of logical precedence.  If file
        <literal>human</literal> contains <literal>man</literal> and
        <literal>woman</literal>, then <literal>o-$human</literal>
        would expand as <literal>o-man -woman</literal>.</para>

<![ IGNORE [ 

Examples:

\texttt{filter armorial +\$animal o+\$vegetable \_\$mineral}

   If file \filename{animal}reads
\begin{verbatim}
   \$human
   reptile
   amphibian
\end{verbatim}

   and file \filename{human} reads
\begin{verbatim}
   man
   woman
\end{verbatim}
   
   and file \filename{vegetable} reads
\begin{verbatim}
   tree
   grain
\end{verbatim}

   and file \filename{mineral} reads
\begin{verbatim}
   rock
   dirt
\end{verbatim}

   then the above will be expanded to:
   
   \texttt{filter armorial +man o+woman o+reptile o+amphibian o+tree
     o+grain \_rock \_dirt}
   
   \texttt{filter armorial +\${\&}beastie +{\&}{\&}{\&}doggie
     -\$dragon}

   If file \filename{\&beastie} reads
\begin{verbatim}
   unicorn
   \$dragon
   manticore
\end{verbatim}

   and file \filename{\&doggie} reads
\begin{verbatim}
   terrier
   hound
   \$\$paniel
\end{verbatim}

   and file \filename{dragon} reads
\begin{verbatim}
   wyvern
   dragon
   lizard
\end{verbatim}

   then the above will be expanded to
   
   \texttt{filter armorial +unicorn o+wyvern o+dragon o+lizard
     o+manticore +terrier +hound +\$\$paniel -wyvern -dragon -lizard}
   
   which will print lines from file \str{armorial} that contain:
   any of: \str{unicorn}, \str{wyvern}, \str{dragon},
   `\str{lizard}, \str{manticore}; and ALL of: \str{terrier},
   \str{hound}, \str{\$paniel}; and none of: \str{wyvern},
   `\str{dragon}, \str{lizard}.
]]>
      </section>
    </section>
  </appendix>
</article>
