
% based upon cvs/hibou/hibou/doc/foundation/presentations/*-2002/lire.tex

% Copyright (C) 2002 Stichting LogReport Foundation LogReport@LogReport.org

%
% This presentation is free software; you can redistribute it and/or
% modify it under the terms of the GNU General Public License as
% published by the Free Software Foundation; either version 2 of
% the License, or (at your option) any later version.
%
% This is distributed in the hope that it will be useful,
% but WITHOUT ANY WARRANTY; without even the implied warranty of
% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
% GNU General Public License for more details.
%
% You should have received a copy of the GNU General Public License
% along with this manual (see COPYING); if not, check with
% http://www.gnu.org/copyleft/gpl.html or write to the Free Software
% Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA.
%

% general overview of various TeX based (like pdfslide, prosper, ifmslide, 
% powertex (and other) presentation tools
%  http://me.in-berlin.de/~miwie/presentations/html/index.html

% /usr/share/doc/texmf/latex/graphics/grfguide.ps

% other presentation packages:
% - magicpoint (mpg .deb)
% - http://docbook.sourceforge.net/projects/slides/index.html

% seminar.sty is an improved slides class.
%  see /usr/share/doc/texmf/latex/seminar/
%  and Denis Girou's page at http://www.tug.org/applications/Seminar/index.html
%  http://www.fz-rossendorf.de/FVTK/latex/doc/latex/fzr_local/sembsp.txt
% the 'slides' class used to be implemented by SliTeX in the LaTeX 2.09 days
% see http://www.colorado.edu/ITS/docs/latex/Unix/examples/sampleC.html
% for an annoted example.
% the beta TeXPower package by Stephan Lempke might become better then seminar

% www.tug.org/pipermail/pdftex/2000-March/006431.html 

% displaying slides: gnome-gv --fullscreen
% http://bugs.debian.org/cgi-bin/bugreport.cgi?archive=no&bug=99305

% graphs grabbed from http://fruit.eu.org/report/ .

% documentclass settings:
% slidesonly,%  Try notes and article instead.
% notes,%      Use instead of slidesonly to typeset notes and slides.
% article,%    For printing notes

\documentclass[%
  @class@,%     expanded by Makefile
  a4,%          a4 (sem-a4, as suggested by semsamp2.tex, seems to be missing)
  semlcmss%     font : use SliTeX style sans serif fonts for slides, Computer
                % Modern for notes
]{seminar}

\input{seminar.bug}
\input{seminar.bg2} % See the Seminar bugs list

\usepackage{url}
\urlstyle{sf}


% knappen fonts instead of old-school knuth stuff: we want small caps in 
% our slides, for e.g. `rpm'
% \renewcommand\encodingdefault{T1}
% nah, pdf gets borken

% \usepackage{charter,helvet}

\slideframe{@frame@}     % expanded to either plain or none by Makefile

% http://www-h.eng.cam.ac.uk/help/tpl/textprocessing/latex_maths+pix/
% http://www.math.nwu.edu/comp-help/including_graphics.html
% http://ls1-www.cs.uni-dortmund.de/~lehmke/TeXPower/00readme.txt

% i want slides with sections, defines \slideheading
\usepackage{slidesec}

% seminar.cls + pdftex: slide dimensions (partial solution)
% Ignasi Furió Caldentey ignasi@ipc4.uib.es
% Fri, 31 Mar 2000 10:36:44 +0200
%
% http://www.google.com/search?q=cache:0J_BW9KYsygC:www.tug.org/pipermail/pdftex/2000-March/006427.html+%22seminar.cls+%2B+pdftex:+slide+dimensions+(partial+solution)%22&hl=en
%
% on pdftex list
%
\newif\ifpdf
\ifx\pdfoutput\undefined
\pdffalse    % we are not running PDFLaTeX
\else
\pdfoutput=1 % we are running PDFLaTeX
\pdftrue \fi

\ifpdf
\usepackage[pdftex]{graphicx}
%when typesettting slides in pdf format
%\pdfpagewidth=297truemm
%\pdfpageheight=210truemm
\pdfhorigin=1truein
\pdfvorigin=1truein
\newcommand\eps[2][]{\includegraphics[#1]{#2.pdf}}
\newcommand\png[2][]{\includegraphics[#1]{#2.png}}
\else
\usepackage{graphicx}
\newcommand\eps[2][]{\includegraphics[#1]{#2.eps}}
\newcommand\png[2][]{\includegraphics[#1]{#2.eps}}
\fi

% the \png command is not used for now.  we might, one day...

% MAX 28 slides!

\title{LogReport's Lire: Integrated Analysis of all your Internet Services' Log
Files}

\author{Joost van Baal \url{joostvb@logreport.org}}
\date{11 October 2002}

\begin{document}

    \maketitle

    \tableofcontents

    \pagebreak

    \section*{Introduction}

        \begin{slide}
            \slideheading{LogReport's Lire: Integrated Analysis of all your
            Internet Services' Log Files}
            \begin{center}
Joost van Baal \url{joostvb@logreport.org} \\
LogReport \url{http://www.logreport.org/}
            \end{center}

        \end{slide}

This paper gives an introduction to \textsf{Lire}, LogReport's tool for
performing an integrated analysis of all ones Internet Services.

It is based upon presentations given at FOSDEM, the Free and Open Source
Software Developers Meeting, Februari 2002 in Brussels, Belgium, and at ne2000,
the Open Air Networking Event, July 2002, Nuenen, The Netherlands.

Joost van Baal and Wessel Dankers are employed as software developers by the
Stichting LogReport Foundation; together with two other developers they work on
maintaining and promoting \textsf{Lire}, LogReport's flagship product. 

Next to working for LogReport, Joost van Baal is doing volunteer work for the
Debian project.  His main interests are programming in Perl and deploying
Internet services on Unix platforms.

Wessel Dankers is a Computer Science student at the Universiteit Utrecht, next
to this he his involved in several Free Software development projects.

        \begin{slide}
            \slideheading{Table of Contents}
            \begin{itemize}
                \item Log file analysis
                \item Lire Overview
                \item Lire's Architecture
                \item Lire's Future
                \item The LogReport Project
                \item More information, contact, questions
            \end{itemize}

This talk will discuss the technical aspects of Lire as well as the
organisational aspects of LogReport as an Open Source project.

        \end{slide}

    \section{Log file analysis}

        \begin{slide}
            \slideheading{Log file analysis}

        Log file analysis is
            \begin{itemize}
                \item too often neglected, but
                \item giving access to invaluable information; however
                \item tedious and time-consuming, so
                \item in need for both flexible and generic software.
            \end{itemize}
        \end{slide}

Log files are often treated like a wasteful by-product of IT activity: they sit
somewhere in a dark corner of a computer system and are only examined
occasionally, usually in case of after-the-fact reactive problem solving.  The
infamous \textit{rotate} is the only application dealing with them.  This is
unfortunate. Log files contain the traces of computer activity, and by
intelligently analyzing these traces one can learn a lot about the behavior of
a system and its users.

Log file analysis is both an essential and tedious part of system
administration. It is essential because it's the best way of profiling the
usage of the service installed on the network.  It's tedious because programs
generate a lot of data and tools to report on this data are unavailable or
incomplete.  When such tools exist, they are specific to one product, which
means that you can't compare your qmail and Exim mail servers.

The Stichting LogReport Foundation detected this flaw in system administration
and chose to serve a dual purpose: developing and maintaining \textsf{Lire}, our
Open Source reporting and analysis software, and serving as a nexus of
documentation, ideas, and thought on the topic of log files and their potential
applications.

    \section{Lire Overview}

        \subsection{Lire benefits}

        \begin{slide}
            \slideheading{Why use \textsf{Lire}?}

\textsf{Lire} is
            \begin{itemize}
                \item generic
                \item flexible
                \item free, in both senses of the word
                \item actively maintained, in an open environment
                \item highly configurable
                \item very portable
                \item secure
                \item commercially supported
            \end{itemize}
        \end{slide}

The LogReport project tries to tackle the problems as outlined above by
developing \textsf{Lire}.  \textsf{Lire} is a software package to generate
useful reports from raw log files of various network programs.

\textsf{Lire} is flexible.  The tool can be accessed via a command line
interface, but can also be run from cron, and can even get accessed via an
email interface.

\textsf{Lire} is Free Software.  When using \textsf{Lire}, you'll have all the
benefits of Open Source software.  \textsf{Lire} is available at no cost, from
our website on \url{http://www.logreport.org/}, and is licenced under the GNU
General Public License.

\textsf{Lire} is actively being maintained by the LogReport team, which
currently consists of five experienced software developers.  The development
can be followed live on our CVS on SourceForge.  A new release gets shipped
almost monthly.

\textsf{Lire} is highly configurable.  All configuration files are in a very
simple syntax.  Of course, a userfriendly interface to write the configuration
is shipped with \textsf{Lire}.

\textsf{Lire} is very portable.  It runs on at least four different Unixen,
GNU/Linux included.  Since it's written in Perl, porting to different platforms
is easy.

\textsf{Lire} is secure.  It is run under a dedicated user account.  No
processes running as root are involved.  Care is taken when installing Lire as
an online responder.  (Of course, this does not exempt the system administrator
from defining and implementing her own security measures.)

\textsf{Lire} is commercially supported: the LogReport team offers commercial
consultancy and tailormade extensions on demand.

        \subsection{Which problems does Lire solve?}

        \begin{slide}
            \slideheading{Lire's users}
\textsf{Lire} is valuable for both
            \begin{itemize}
                \item system administrators, and
                \item business managers
            \end{itemize}
        \end{slide}

It enables one to schedule hardware upgrades, detect anomalities in usage from
services.  It can be used as a tool in building a traffic-based accounting
system for external customers.  It gives insight in who's talking to who, which
is valuable for marketing and business planners.

        \subsection{How does Lire do this?}

\textsf{Lire} converts stuff like

        \begin{tiny}
            \begin{verbatim}
1.example.com - - [03/Feb/2002:06:25:27 +0100] "GET /contact/lists/commit/msg01057.php HTTP/1.0" 200 11193 "-" "Googlebot/2.1
  (+http://www.googlebot.com/bot.html)"
2.example.com - - [03/Feb/2002:06:25:46 +0100] "GET /robots.txt HTTP/1.0" 404 5126 "-" "htdig/3.1.5 (webmaster@logreport.org)"
2.example.com - - [03/Feb/2002:06:25:46 +0100] "GET / HTTP/1.0" 200 12745 "-" "htdig/3.1.5 (webmaster@logreport.org)"
1157.example.com - - [10/Feb/2002:05:23:38 +0100] "GET /css.php HTTP/1.1" 200 3682 "http://logreport.org/doc/gen/dns/" "Mozilla/4.0 (compatible;
  MSIE 6.0; Windows NT 5.0; Q312461)"
1158.example.com - - [10/Feb/2002:06:22:44 +0100] "GET /lire/ex/plain.php HTTP/1.1" 200 14253
  "http://www.google.com/search?hl=en\&q=ascii+text+pics\&spell=1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90)"
            \end{verbatim}
        \end{tiny}

to a graph like the one below, in Figure 1.

        \begin{figure}[htb]
            \begin{center}
                \eps{tarball-downloads-from-hibou}
                \caption{Lire tarball downloads from LogReport webserver,
per week, May - Sept 2002}
            \end{center}
        \end{figure}

        \subsection{Lire supported log files and output formats}

        \begin{slide}
            \slideheading{\textsf{Lire} supported log files}

\textsf{Lire} currently supports log files from
            \begin{itemize}
                \item www and proxy (apache, IIS, squid, WELF, MS ISA)
                \item dns and dnszone (bind query and named logs)
                \item firewall (cisco IOS and PIX, Linux, IP Filter ...)
                \item email and msgstore (Exim, Postfix, qmail, sendmail, NMS, 
                   various POP and IMAP servers)
                \item ftp (ProFTPD, WU-FTPD, MS IIS)
                \item print (CUPS, LPRng)
                \item database (MySQL, PostgreSQL)
                \item dialup (isdn4linux) and syslog (generic syslog parser)
             \end{itemize}
        \end{slide}

Multiple programs are supported for various types of network services:

        \begin{itemize}

            \item \textit{www} log files in various formats from various
            webservers (Apache, IIS, Boa);

            \item \textit{proxy} log files from squid and from WELF proxies, as
            well as from MS ISA servers;

            \item Bind version 8 and version 9 \textit{dns} query logs and
            named logs;

            \item logs from \textit{firewalls} such as IOS and PIX CISCO router,
            Linux ipchains, ipfilter and iptables, BSD IP Filter, WatchGuard,
            as well as logs in the WELF format as used by a lot of commercial
            firewall products;

            \item \textit{email} logfiles from Exim, Postfix, qmail, sendmail,
            ArgoSoft and Netscape Messaging Server;

            \item POP and IMAP logs from various Netscape \textit{msgstore}
            servers, as well as the DBMAIL server.

            \item log files from \textit{ftp} servers in the xferlog format, as
            used by e.g. ProFTPD and WU-FTPD as well as logs from the MS IIS
            ftpserver;

            \item \textit{print} logfiles (CUPS and LPRng);

            \item \textit{database} transaction log files from MySQL servers;

            \item \textit{dialup} logs from isdn4linux's isdnlog tool;

            \item \textit{syslog} log files, with any mix of services.

        \end{itemize}

\textsf{Lire} also supports various output formats for the generated reports:
HTML, XHTML, XML, PDF, Excel 95, and plain ascii. Some of these formats support
graphical representation of the data.

        \subsection{Lire installation}

        \begin{slide}
            \slideheading{Ease of installation}

% sf doesnt have a small caps.  hack is ourselves (don't use textsc)
Lire comes as a tarball (``autoconfiscated''), as an {\small RPM} and as a
Debian package.  A FreeBSD package is shipped with the FreeBSD ports
collection.  Written in Perl and shell, so runs on any Unix-like OS.

        \end{slide}

A \textsf{Lire} tarball is available for download from the LogReport website at
\url{http://www.logreport.org/pub/}. A binary package for Debian GNU/Linux as
well as an \textsc{rpm} is also available.  Furthermore \textsf{Lire} is
shipped with the FreeBSD ports collection.

\textsf{Lire} is written in Perl and shell code.  Supported platforms are
GNU/Linux, the BSD's and Sun Solaris, but since it's written in Perl, it very
likely runs fine on a lot of other platforms too.

    \section{Lire's Architecture}

        \begin{slide}
            \slideheading{Table of Contents}
            \begin{itemize}
                \item Log file analysis
                \item Lire Overview
                \item Lire's Architecture
                \item Lire's Future
                \item The LogReport Project
                \item More information, contact, questions
            \end{itemize}
        \end{slide}

        \subsection{Overview}

            \begin{slide}
                \slideheading{services, superservices and {\small DLF}'s}
                \begin{description}
                    \item[{\small DLF}] ``distilled log format'' space
                    separated, line oriented, fixed fields
                    \item[service] raw log file format
                    \item[superservice] a class of services, sharing same
                    {\small DLF} and report
                \end{description}
            \end{slide}

Internally, \textsf{Lire} represents the log file in a \textsc{dlf} file (for
Distilled Log Format).  This is a simple space-separated line-oriented ascii
file.  Each logged event is represented by one fixed-fields line.

A service coincides with one well-defined log file format.  So, a service
generally coincides with one application: the \texttt{sendmail} service handles
sendmail log files.  However, a lot of webservers use W3C defined formats, and
a lot of commercial firewalls use the WELF format.  Therefore,
\texttt{w3c\_extended} and \texttt{welf} are services.  Each service has its
``2dlf''-convertor.  We ship e.g. \texttt{sendmail2dlf} and
\texttt{w3c\_extended2dlf}.

            \begin{slide}
                \slideheading{Lire's Architecture}
                \begin{center}

                    % 1em is about 10bp (a `bp' is `big point' 
                    % `postscript point'). 72bp = 1 inch
                    \vspace*{6mm}
                    \eps[width=25em]{lire-design}
                    %\eps[width=14em]{lire-design}
                \end{center}
            \end{slide}

A superservice is a class of applications which share the same \textsc{dlf}
format, and which will give the same reports.

        \subsection{Receiving the log file}

            \begin{slide}
                \slideheading{Running Lire}

One can run Lire:
                \begin{itemize}
                    \item as online responder
                    \item as client
                    \item from cron
                    \item from command line
                \end{itemize}
            \end{slide}

\textsf{Lire} can run in an online responder setup, as a client, as a cron
driven system, or as a command line driven system. In an online responder
setup, the \textsf{Lire} system receives log files from other hosts, and sends
generated reports back by email.  The log files can be received via email or
via HTTP file upload.  In a client setup, the system sends log files by email
to another \textsf{Lire} system, and receives reports back.  Optionally, the
logs can be anonymized before being sent. A cron driven setup reads and
processes log files after they're rotated, on the local host.  A userfriendly
script (\texttt{lr\_config}) is supplied which sets up the cronjob to your
taste.  In a command line driven system, users run the \textsf{Lire} scripts on
an ad-hoc basis.  One can use e.g. the \texttt{lr\_log2report} script.

% can't have \textsc{dlf} in \subsection

        \subsection{Converting to DLF}

            \begin{slide}
                \slideheading{{\small DLF} convertors}
                \begin{center}
				\vspace*{-6mm}
                    \eps[width=17em]{lire-dlf-converters}
                    %\eps[width=8em]{lire-dlf-converters}
				\vspace*{-6mm}
                \end{center}
            \end{slide}

By invoking the right \textsc{dlf} convertor (e.g. \texttt{sendmail2dlf},
\texttt{cups\_pagelog2dlf} or \texttt{squid\-2\-dlf}), the log file is
converted to the \textsc{dlf} format, suited for the superservice involved.
The \textsc{dlf} convertor coincides with a service.

            \begin{slide}
                \slideheading{Services and Superservices}

A service:
                \begin{itemize}
                    \item one raw log file format
                    \item generally one service is one application
                    \item belongs to one superservice
                    \item example: the postfix service
                \end{itemize}

A superservice:
                \begin{itemize}
                    \item one {\small DLF} format
                    \item a set of supported subreports
                    \item a set of services
                    \item example: the email superservice.
                \end{itemize}
            \end{slide}

The \textsc{dlf} format for a superservice is defined in a Lire defined XML
format.  E.g., for the email superservice, this features:

            \begin{small}
                \begin{verbatim}

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE lire:dlf-schema PUBLIC
  "-//LogReport.ORG//DTD Lire DLF Schema Markup Language V1.0//EN"
  "http://www.logreport.org/LDSML/1.0/ldsml.dtd">
<lire:dlf-schema superservice="email" timestamp="time"
 xmlns:lire="http://www.logreport.org/LDSML/">

<!-- snip -->

 <lire:field name="time"        type="timestamp"    default="0"/>
 <lire:field name="logrelay"    type="string"       default="-"/>
 <lire:field name="queueid"     type="string"       default="-"/>
 <lire:field name="msgid"       type="string"       default="-"/>
 <lire:field name="from_user"   type="string"       default="-"/>
 <lire:field name="from_domain" type="hostname"     default="-"/>
 <lire:field name="from_relay_host" type="hostname" default="-"/>
 <lire:field name="from_relay_ip"   type="ip"       default="-"/>
 <lire:field name="size"        type="bytes"        default="0"/>
 <lire:field name="delay"       type="duration"     default="0"/>

<!-- snip -->

</lire:dlf-schema>

                \end{verbatim}
            \end{small}

            \begin{slide}
                \slideheading{email {\small DLF} format}
                \begin{small}
                    \begin{verbatim}
[...]
<lire:field name="time"        type="timestamp" default="0"/>
<lire:field name="queueid"     type="string"    default="-"/>
<lire:field name="from_user"   type="string"    default="-"/>
<lire:field name="from_domain" type="hostname"  default="-"/>
[...]
                    \end{verbatim}
                \end{small}
            \end{slide}

Please note that Lire defines its own datatypes.  These are used later
in the report generating mechanisms.

        \subsection{Generating an XML Report}

A Lire report consists of several subreports, which can be displayed in
graphical form, or as a table.  A lot of subreports (196, as of September 2002)
come with Lire predefined, but of course one can define ones own reports.  A
report definition is written in the Lire Report Specification Markup Language;
it looks like e.g.

            \begin{small}
                \begin{verbatim}

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE lire:report-spec PUBLIC
  "-//LogReport.ORG//DTD Lire Report Specification Markup Language V1.0//EN"
  "http://www.logreport.org/LRSML/1.0/lrsml.dtd">
<lire:report-spec xmlns:lire="http://www.logreport.org/LRSML/"
 superservice="email" id="top-volume-to-domain" charttype="bars">

 <lire:title>Largest Volume Sent To Domain Email Report</lire:title>
 <lire:description>
  <para>This report lists the domains to which the
   largest volume of mail was sent.</para>
 </lire:description>

 <lire:param-spec>
  <lire:param name="domain_to_show" type="int" default="10">
  <!--- snip -->
  </lire:param>
 </lire:param-spec>

 <lire:display-spec>
  <lire:title>Largest Volume Sent To Domain, Top $domain_to_show</lire:title>
  <lire:description>
   <para>Volume is in bytes</para>
  </lire:description>
 </lire:display-spec>

 <lire:filter-spec>
  <lire:eq arg1="$stat" arg2="sent"/>
 </lire:filter-spec>

 <lire:report-calc-spec>
  <lire:group sort="-mail_volume" limit="$domain_to_show">
   <lire:field name="to_domain"/>
   <lire:sum name="mail_volume" field="size"/>
  </lire:group>
 </lire:report-calc-spec>

</lire:report-spec>

                \end{verbatim}
            \end{small}

            \begin{slide}
                \slideheading{Report Specification}
                \begin{verbatim}
 [...]
 <lire:report-calc-spec>
  <lire:group sort="-mail_volume" limit="$domain_to_show">
   <lire:field name="to_domain"/>
   <lire:sum name="mail_volume" field="size"/>
  </lire:group>
 </lire:report-calc-spec>
 [...]
                \end{verbatim}
            \end{slide}

We reuse the \texttt{stat}, \texttt{to\_domain}, and \texttt{size} fields from
the email \textsc{dlf} specification.  The \texttt{mail\_volume} field is
internal to this report; it's used only within this report calculation.
Operators like \texttt{lire:group}, \texttt{lire:timegroup},
\texttt{lire:\-rangegroup}, \texttt{lire:timeslot}, \texttt{lire:\-field},
\texttt{lire:sum}, \texttt{lire:avg}, \texttt{lire:min}, \texttt{lire:max},
\texttt{lire:count} can be used in the \texttt{report\-calc-\-spec}.

The \texttt{domain\_to\_show} variable is offered as a hook for user
configuration.  Users can set this variable in a report configuration file.

Per superservice, there is one configuration file, for all subreports.  Such a
file looks like e.g.

            \begin{small}
                \begin{verbatim}
# Report configuration for the email superservice

deliveries-by-period                    period=1d
volume-by-period                        period=1d
top-volume-to-domain                    domain_to_show=10
top-to-email-by-domain                  domain_to_show=30 user_to_show=5
deliveries-by-size                      size=1k
deliveries-by-delay                     delay-size=1s
tracked-recipients                      tracked_email_re="root@example\.com"
[...]
                \end{verbatim}
            \end{small}

The configuration file defines which subreports we want to show up in our
report, and defines the settings of the variables for these subreports.  For
the proxy superservice, the report configuration file looks like

            \begin{slide}
                \slideheading{Report configuration file}
                \begin{verbatim}
# Report configuration for the proxy superservice

=section General
requests-summary

=section Denied Sites Reports
|select-cache_result result=TCP_DENIED
top-destinations          dsts_to_show=50
top-users-by-destinations users_to_show=8 dsts_to_show=50
[...]
                \end{verbatim}
            \end{slide}

Note the \verb+|+ line: this defines a filter, shared among the reports
below.

The \texttt{lr\_config} script, which comes with \textsf{Lire}, gives a
userfriendly interface to set these variables.

        \subsection{Typesetting and publishing the Report}

Graph generation is done by either the GD::Graph perl module or ploticus, a
GPL-ed graphical data display engine, available from SourceForge.  We show some
more examples of graphs, from the \texttt{www} and \texttt{dns} supervices.
Similar graphs are generated out-of-the box for the six other superservices.

            \begin{figure}[htb]
                \eps[height=12em]{www-requests-by-browser}
                \eps[height=12em]{www-size_by_directory}
                \eps[height=12em]{www-user_session-visit-times}
                \caption{Reports from the www superservice: requests by
browser, size by directory, user session visit times}
            \end{figure}

            \begin{slide}
                \slideheading{Example Graphical Reports}

Some reports from the www superservice
                    \eps[height=9em]{www-bytes-by-period}
                    \eps[height=5em]{www-requests-by-version}

            \end{slide}

            \begin{slide}
Some reports from the www and dns superservices
                    \eps[height=9em]{www-top-requests}
                    \eps[height=9em]{dns-top-requested-names}

            \end{slide}

    \subsection{Implementation}

For small and medium-size logs Lire is fast.  Although some heavyweight
external programs are used to process XML files (jade or the fo stylesheet with
PassiveTex, for typesetting PDF and RTF), performance is above expectations.
When very high speed is needed, plain ascii reports can get produced.  Due to
its modular design, it's fairly easy to reimplement performance bottlenecks in
e.g. C, to fulfill extreme performance demands.

We do have a knob (\texttt{LR\_MAX\_MEMORY}) to tweak the amount of memory one
is willing to dedicate to \textsf{Lire}.  This enables one to exchange disk i/o
for memoryaccess.

    \section{Lire's future}

        \begin{slide}
            \slideheading{Table of Contents}
            \begin{itemize}
                \item Log file analysis
                \item Lire Overview
                \item Lire's Architecture
                \item Lire's Future
                \item The LogReport Project
                \item More information, contact, questions
            \end{itemize}
        \end{slide}

        \subsection{Releases}

On August 19, 2002, \textsf{Lire} 1.1 was released, which - again - has
support for more log files.

            \begin{slide}
                \slideheading{Release, Roadmap}

Lire 1.1 is the current release (we've published 23 releases since September
2000).  But we have more plans, for Lire 1.2 (late october 2002) and on.

                \begin{itemize}
                    \item better chart generation tool (1.2)
                    \item better reports (1.2)
                    \item configuration api (1.2)
                    \item expand developers documentation (1.2)
                    \item better support for merging
                    \item cross-superservice reporting
                    \item more services
                \end{itemize}
            \end{slide}

        \subsection{Roadmap}

We have an ambitious roadmap.

            \begin{description}
                \item[better chart generation tool]

In \textsf{Lire} 1.2, we will offer support for ploticus, next to GD::Graph.
Ploticus is far more flexible, and allows for sexier graphics, with more
user-configurable knobs.  This code has already been committed to CVS now.

                \item[better reports]

It should be able to typeset multiple column reports, and generate column
labels.  The HTML layout should get improved.

                \item[configuration api]

Lire's framework should contain a configuration API that should be used by all
of its components.  The current mix of Perl and Shell configuration is
suboptimal.  Once this is reimplemented properly, we can think about a better
configuration management tool (which can replace \texttt{lr\_config}).

                \item[expand developers documentation]

We love code contributions!  Therefore, it should be easy to get to learn the
\textsf{Lire} internals: new features will get properly documented, for both
users and developers.

                \item[merging and splitting of reports and log files]

We would like \textsf{Lire} to offer an easy interface to merge and split
reports and log files.  Currently, by default, one log file gets converted to
one report.  It should be possible to combine different reports after the fact,
e.g. for generating reports about longer timeframes, or to combine reports from
different servers.  It should be easy to generate handcrafted reports after the
fact from already processed log files.  We do offer preliminary support for
this since Lire 1.0, but the default configuration doesn't yet support this.
We need to implement a storage API to get this going: we need a datawarehouse
infrastructure.  Scheduled for \textsf{Lire} 1.3, december 2002.

                \item[cross-superservice reporting]

It will be possible to combine e.g. POP server's reports with firewall reports.
This way, one can e.g.  track IP's which failed to authenticate in the POP log,
and see wether they triggered some firewall rules too.  This feature is
scheduled for \textsf{Lire} 2.0, late february 2003.

                \item[even more services]

We plan to add some more firewall convertors.  Furthermore, LDAP log file
support should get added.  Since this task is relatively easy to do, we'd
prefer to have someone not in the LogReport Developers team doing this.  (The
team is focusing mainly on the \textsf{Lire} core code.)

            \end{description}


\section{The LogReport Project}

    \begin{slide}
        \slideheading{LogReport people}

LogReport developers

        \begin{itemize}
            \item Joost van Baal
            \item Wessel Dankers
            \item Josh Koenig
            \item Francis Lacoste
        \end{itemize}

LogReport board

        \begin{itemize}
            \item Teus Hagen (chairman)
            \item Wytze van der Raay (treasurer)
            \item Jakob Schripsema (secretary)
        \end{itemize}

    \end{slide}

    \subsection{People working for LogReport}

Four people are working for LogReport:

        \begin{itemize}
            \item Joost van Baal
            \item Wessel Dankers
            \item Josh Koenig
            \item Francis Lacoste
        \end{itemize}

Furthermore, Joost Bekkers, Arnaud Gaillard, Edwin Groothuis and Egon
Willighagen regularly contribute code to the project.

These people live and work from The Netherlands, Canada, Australia, Switserland
and the USA, all part time.  Communication is done using IRC (\url{#logreport}
on \url{freenode.net}) and email.

Next to a website, the project offers two mailinglists:
\url{development@logreport.org} and \url{questions@logreport.org}.  Both
\textsf{Lire} and our website are maintained via CVS, hosted on SourceForge.
We have our own server which hosts our website as well as the lists.

Furthermore this server hosts the LogReport Online Responder.  One can sent
(compressed) logfiles in email messages to dedicated addresses, like e.g.
\url{log@qmail.logreport.org}, \url{log@bind9.logreport.org}, and get a report
back as a response.  Optionally, one can anonymize the log before submitting
it, using a simple script which comes with \textsf{Lire}.

    \subsection{The LogReport Foundation}

Stichting LogReport Foundation is a non-profit organization; it got a legal
status as a foundation, and funding by the NLnet Foundation
(\url{http://www.nlnet.nl/}), in August 2000.

        \begin{itemize}
            \item Teus Hagen (chairman)
            \item Wytze van der Raay (treasurer)
            \item Jakob Schripsema (secretary)
        \end{itemize}

    \subsection{How to help}

        \begin{slide}
            \slideheading{How to help}
            \begin{itemize}
                \item Use our Online Responder
                \item Sent (anonymized) log files
                \item Download Lire, and use it
                \item Give feedback on our mailinglists: feature requests, bug
reports, help other people
                \item Even better: send patches and add support for other
services
                \item Promote Lire: via webpages and mailinglists
                \item Fund us; buy commercial support
            \end{itemize}
        \end{slide}

We need log files to test our code, and to be able to add support for more
services.  We especially lack log files from expensive commercial services,
like the WELF and Microsoft ones.  (We don't run these ourselves...)

If you use the Debian package, you can use the Debian Bug Tracking System to
report bugs.

If you intend to write code, be sure to use our current CVS, of course.  Our
CVS, hosted on SourceForge, is readable for anyone.  Code contributions of
non-trivial size are accepted if -- of course -- they meet our quality
standards, and they're offered under a GPL compatible license, like the GPL
itself, or e.g. the modified BSD license.  People who are willing to contribute
to the \textsf{Lire} project during a longer time, can get write access to our
CVS tree.  (Of course, since Lire is free software, you can keep your
modifications private and secret too.  Just be sure not to distribute them, in
such a case.) Contact us for more information.

Promote Lire: link to us from your webpage, suggest using Lire on mailinglists.
Join the Lire community: help other users on our lists.

Fund us: Funding from Stichting NLnet which currently enables us to spend a lot
of time on Lire will run out in the future. Financial contributions to the
LogReport foundation are tax-deductable under Dutch law, because LogReport is
recognized as a charitable goal.  Hire a LogReporter: the LogReport team is
available for commercial consultancy about log file and Lire issues, see the
LogReport.com website at \url{http://www.logreport.com/}.  Contact us if you're
interested.

\section{More information, contact info}

    \begin{slide}
        \slideheading{More information, contact info}

        \begin{description}
            \item[website] \url{http://www.logreport.org/}
            \item[mailing lists] (archived) \url{questions@logreport.org},
            \url{development@logreport.org}
            \item[irc] \url{#logreport} on \url{freenode.net}
            \item[announcements] \url{announcement@logreport.org}
        \end{description}

    Questions?

    \end{slide}

    \subsection{More information}

More information is on our website on \url{http://www.logreport.org/}.  We have
several mailing lists, which are archived.  A lot of documentation and manpages
come with \textsf{Lire}.

We sent newsitems via our \url{announcement@logreport.org} list.  Subscribe to
it if you wanna be kept informed.

    \subsection{Contact}

Contact us via our lists \url{questions@logreport.org} and
\url{development@logreport.org} (see our website for subscription info), or
privately on \url{logreport@logreport.org}.  To have an informal chat with the
LogReport developers, join the \url{#logreport} IRC channel on the Open
Projects Network.

\end{document}



