.\" -*- mode: troff; coding: utf-8 -*- .\" Automatically generated by Pod::Man v6.0.2 (Pod::Simple 3.45) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. .ie n \{\ . ds C` "" . ds C' "" 'br\} .el\{\ . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" .\" Required to disable full justification in groff 1.23.0. .if n .ds AD l .\" ======================================================================== .\" .IX Title "LR_ANONYMIZE.IN 1" .TH LR_ANONYMIZE.IN 1 2025-09-28 "Lire 2.1.1" "LogReport's Lire Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH NAME lr_anonymize \- anonymize a logfile .SH SYNOPSIS .IX Header "SYNOPSIS" \&\fBlr_anonymize\fR \fIdumpfilestem\fR .SH DESCRIPTION .IX Header "DESCRIPTION" \&\fBlr_anonymize\fR is typically used when sending logs to a responder. See the section on "Sending Anonymized Log Files To A Responder" in the chapter on "Using A Responder" in the Lire User Manual for usage examples. .PP \&\fBlr_anonymize\fR reads a file containing emailaddresses, ipnumbers, and hostnames (typically a logfile from an internet service) from stdin, and prints an "anonymized" version of this file to stdout. It dumps the information to "deanonymize" the file, using \fBlr_deanonymize\fR\|(1), to 5 Berkeley DB databases; the names of the files holding these databases is contructed by concatenating the strings \*(Aqinaddr\*(Aq, \*(Aqmaildomain\*(Aq, \*(Aqemail\*(Aq, \*(Aqip\*(Aq and \*(Aqdomain\*(Aq to \&\fIdumpfilestem\fR. \fIdumpfilestem\fR can be e.g. /tmp/dump, ../../var/dump or dump. The db files created silently overwrite stale ones, if found. .PP The script builds temporary files. By default, these are created in /tmp. One can override this by setting the TMPDIR environment variable. .SH "CONNECTION WITH lr_deanonymize" .IX Header "CONNECTION WITH lr_deanonymize" When running .PP .Vb 2 \& $ lr_anonymize dump < log > log.anon \& $ lr_deanonymize dump < log.anon > log.new .Ve .PP then log and log.new have the same content (except for case, check it with \&\fBdiff \-i\fR). .SH NOTES .IX Header "NOTES" We tried to optimize this script for memoryusage. This has the drawback the script will run for quite some time when anonymizing a big logfile. .PP We\*(Aqve run the script on a 25 M sendmail logfile. .PP Typical values in such a case are 2500 K for total amount of physical memory used, and 15m real, 8m user and 22s systime spent on a 64 MB system with a 300 MHz Pentium II processor. de_anonymizing this file took 11m real, 8m user and 17s system time. .PP We store maildomains in the dumpfile. These are used by \fBlr_deanonymize\fR\|(1), in case email addresses in our input file reoccur in the file read by \fBlr_deanonymize\fR\|(1) in split form, i.e. joe.user@example.com occurs as both joe.user@example.com and example.com in the to be deanonymized file. .PP All dumped objects are casted to lowercase. .SH BACKGROUND .IX Header "BACKGROUND" For your convenience, we quote a bit of rfc822: .PP .Vb 10 \& SPACE = ; ( 40, 32.) \& CTL = ; ( 177, 127.) \& specials = "(" / ")" / "<" / ">" / "@" ; Must be in quoted\- \& / "," / ";" / ":" / "\e" / <"> ; string, to use \& / "." / "[" / "]" ; within a word. \& atom = 1* \& quoted\-string = <"> *(qtext/quoted\-pair) <">; Regular qtext or \& ; quoted chars. \& word = atom / quoted\-string \& domain\-ref = atom ; symbolic reference \& domain\-literal = "[" *(dtext / quoted\-pair) "]" \& sub\-domain = domain\-ref / domain\-literal \& domain = sub\-domain *("." sub\-domain) \& local\-part = word *("." word) ; uninterpreted \& ; case\-preserved \& addr\-spec = local\-part "@" domain ; global address .Ve .PP and of rfc 2181 .PP .Vb 5 \& The DNS itself places only one restriction on the particular labels \& that can be used to identify resource records. That one restriction \& relates to the length of the label and the full name. The length of \& any one label is limited to between 1 and 63 octets. A full domain \& name is limited to 255 octets (including the separators). .Ve .PP rfc1123 .PP .Vb 3 \& However, a valid host name can never \& have the dotted\-decimal form #.#.#.#, since at least the \& highest\-level component label will be alphabetic. .Ve .PP rfc819 .PP .Vb 9 \& ::= | "." \& ::= |
\& ::= \& ::= | \& ::= | \& ::= | | "\-" \& ::= any one of the 52 alphabetic characters A through Z in upper \& case and a through z in lower case \& ::= any one of the ten digits 0 through 9 .Ve .SH EXAMPLE .IX Header "EXAMPLE" A \*(Aqlogfile\*(Aq like e.g. .PP .Vb 8 \& blaat fkrf 1.2.3.4.in\-addr.arpa] pietje@bigcompany.com bla 1 2 3 lj;agas;gag \& blaat 1.2.3.4 fkrf 3.2.3.4.in\-addr.arpa] bla 1 www.hotsex.com 2 3 lj;agas;gag \& jan@blaat.frut.com agagag \& blaat fkrf 4.2.3.4.in\-addr.arpa] bla pietje@bigcompany.com www.hotsex.com \& 234.34.2.0 jan@blaat.frut.com 4.2.3.4.in\-addr.arpa1 2 3 lj;agas;gag \& blaat fkrf tweede 3.2.3.4.in\-addr.arpa] bla 1.2.3.4 1 blablabla.com \& 2 mdcc.cx \& 3 lj;agas;gag .Ve .PP wil get anonymized to .PP .Vb 8 \& blaat fkrf 1.0.0.10.in\-addr.arpa] john.doe.1@example.com bla 1 2 3 lj;agas;gag \& blaat 10.0.0.1 fkrf 2.0.0.10.in\-addr.arpa] bla 1 1.example.com 2 3 lj;agas;gag \& john.doe.2@example.com agagag \& blaat fkrf 3.0.0.10.in\-addr.arpa] bla john.doe.1@example.com 1.example.com \& 10.0.0.2 john.doe.2@example.com 3.0.0.10.in\-addr.arpa1 2 3 lj;agas;gag \& blaat fkrf tweede 2.0.0.10.in\-addr.arpa] bla 10.0.0.1 1 2.example.com \& 2 3.example.com \& 3 lj;agas;gag .Ve .PP The dumps will represent something like .PP .Vb 10 \& ip 234.34.2.0 10.0.0.2 \& ip 1.2.3.4 10.0.0.1 \& inaddr 3.2.3.4.in\-addr.arpa 2.0.0.10.in\-addr.arpa \& inaddr 1.2.3.4.in\-addr.arpa 1.0.0.10.in\-addr.arpa \& inaddr 4.2.3.4.in\-addr.arpa 3.0.0.10.in\-addr.arpa \& domain mdcc.cx 3.example.com \& domain blablabla.com 2.example.com \& domain www.hotsex.com 1.example.com \& email jan@blaat.frut.com john.doe.2@example.com \& email pietje@bigcompany.com john.doe.1@example.com .Ve .SH BUGS .IX Header "BUGS" We can\*(Aqt handle files containing hostnames or email addresses in the example.com domain, usernames of the form john.doe. or ipnumbers in the rfc 1918 private network 10.0.0.0 \- 10.255.255.255 (10/8 prefix). .PP We don\*(Aqt handle quoted\-string\*(Aqs in email addresses. We don\*(Aqt handle domain\-literals in email addresses\*(Aq domain. .PP We regard 999.999.999.999 as an IP address: we don\*(Aqt mind the 255 limit. .PP We don\*(Aqt treat network ipaddresses like 100.10.3 as ipaddresses. These will \fInot\fR get anonymized. .SH "SEE ALSO" .IX Header "SEE ALSO" \&\fBlr_deanonymize\fR\|(1), the README.lire\-client file in the Lire distribution. .SH VERSION .IX Header "VERSION" \&\f(CW$Id:\fR lr_anonymize.in,v 1.5 2006/07/23 13:16:32 vanbaal Exp $ .SH COPYRIGHT .IX Header "COPYRIGHT" Copyright (C) 2000\-2001 Stichting LogReport Foundation LogReport@LogReport.org .PP This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. .PP This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. .PP You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with http://www.gnu.org/copyleft/gpl.html. .SH AUTHOR .IX Header "AUTHOR" Joost van Baal