PMIE(1) General Commands Manual PMIE(1) (NAME) pmie - (SYNOPSIS) pmie [-bCdeFfPqvVWxXz?] [-a archive] [-A align] [-c config] [-D debug] [-h host] [-l logfile] [-m note] [-j stompfile] [-n pmnsfile] [-o format] [-O offset] [-S starttime] [-t interval] [-T endtime] [-U username] [-Z timezone] [filename ...] (DESCRIPTION) pmie . (PMCD) Performance Co-Pilot (PCP) . pmie ( ) . . filename . filename . pmie : stdout (verbose) -v print. stderr . (OPTIONS) : -a archive, --archive=archive archive pmlogger(1) . -a . . pmie . . -A align, --align=align align . align PCPIntro(1) . -b, --buffer . -l . -b pmie pmie_check(1) . -c config, --config=config filename . -C, --check () . . -d, --interact pmie . -d . . -e, --timestamp -V -v -W . ctime(3) : expr_1 (Tue Feb 6 19:55:10 2001): 12 -f, --foreground -l -a ( ) pmie ( ). -f ( -F ) pmie . -F, --systemd -f -F pmie ( pid systemd(1) pmie ). pmie systemd(1) . -f -F . -h host, --host=host ( ) ( ) . host . . host pmNewContext pmcd . %h %c ..TP -j file STOMP stompfile . stomp $PCP_SYSCONF_DIR/pmie/config/stomp . -l logfile, --logfile=logfile logfile . -m note, --note=note pmie pmie_check(1) pmie_daily(1) -m pmie_check pmie PMCD HOSTNAME CHANGES . -n pmnsfile, --namespace=pmnsfile (PMNS) pmnsfile . -o format, --format=format -o format () . format DIFFERENCES IN HOST AND ARCHIVE MODES . -O origin, --origin=origin (origin) . PCPIntro(1) . -P, --primary pmie . <> . -q, --quiet "evaluator exiting" . -S starttime, --start=starttime (starttime) . PCPIntro(1) . -t interval, --interval=interval interval PCPIntro(1) ( ). pmie delta ( ) . 10.0 . -T endtime, --finish=endtime (endtime) . PCPIntro(1) . -U username, --username=username pmie . . PCP "pcp" ("root") . -v -V -v -W . -v . (canonical) <<>> <<>> <<>>. pmLookupDesc(3) . . -V, --verbose -v ( ) . -W -V (true) . %h %i %c %v . -x, --secret-agent (domain agent). Performance Co-Pilot : pmdasummary(1). ( ). -X, --secret-applet ( ). -z, --hostzone -h ( -a ) . -Z timezone, --timezone=timezone timezone TZ environ(7) . -?, --help ..SH " (EXAMPLES)" . $PCP_DEMOS_DIR/pmie pmie . delta . : delta = 1 sec; (context switch) : kernel.all.pswitch / hinv.ncpu > 10000 count/sec -> alarm "high context switch rate %v"; top(1) xterm(1) : all_sample ( kernel.all.pswitch @0..9 > 10 Kcount/sec * hinv.ncpu ) -> shell 5 min "xterm -e 'top'"; : delta = 20 sec; I/O dkvis(1) : some_inst ( disk.dev.total > 60 count/sec ) -> print "busy disks:" " %i" & shell 5 min "dkvis"; : $hour >= 9 && $hour <= 17 && some_inst ( 75 %_sample ( disk.dev.total @0..3 > 60 count/sec ) ) -> print "disks busy for 20 sec:" " [%h]%i"; : delta = 10 min; / /usr : filesys.free #'/dev/root' / filesys.capacity #'/dev/root' < 0.05 -> alarm 4 hour "root filesystem (almost) full"; filesys.free #'/dev/usr' / filesys.capacity #'/dev/usr' < 0.05 -> alarm 4 hour "/usr filesystem (almost) full"; lmsensors . : lmsensors.coretemp_isa.temp1 @0 - lmsensors.coretemp_isa.temp1 @1 > 2 -> alarm "temperature rising fast" & syslog "machine room temperature rise alarm"; Oracle : // back to 30sec evaluations delta = 30 sec; sid = "ptg1"; # $ORACLE_SID setting lid = "223"; # latch ID from v$latch lru = "#'$sid/$lid cache buffers lru chain'"; host = ":moomba.melbourne.sgi.com"; gets = "oracle.latch.gets $host $lru"; total = "oracle.latch.gets $host $lru + oracle.latch.misses $host $lru + oracle.latch.immisses $host $lru"; $total > 100 && $gets / $total < 0.2 -> alarm "high lru latch contention in database $sid"; ruleset (load average) . delta = 1 minute; ruleset kernel.all.load #'1 minute' > 10 * hinv.ncpu -> print "extreme load average %v" else kernel.all.load #'1 minute' > 2 * hinv.ncpu -> print "moderate load average %v" unknown -> print "load average unavailable" otherwise -> print "load average OK" ; . some_inst ( 100 * filesys.used / filesys.capacity > 75 && filesys.used + 30min * (rate filesys.used) > filesys.capacity ) -> print "filesystem will be full within 30 mins:" " %i"; mypmda.errors . mypmda.errors > 1 && instant mypmda.errors < 100 -> print "high error rate: %v"; (QUICK START) pmie . pmie pmieconf(1) pmie pmie . . Performance Co-Pilot User's and Administrator's Guide pmie . (EXPRESSION SYNTAX) . Performance Co-Pilot User's and Administrator's Guide . pmie . <> (comment) . C . . ( PMNS(5)) . hosts instances time . . kernel.percpu.cpu.user :foo :bar #cpu0 @0..2 (user mode) CPU 0 <> <> . : () # () @ () PCP . ( - ) . << >> . (counter) pmie (rate) . (double precision) <> <> <> . <> <> . rule () (->) . pmie : +----------------------+----------------------------------------------------------------------------------------------------------+ | | | +----------------------+----------------------------------------------------------------------------------------------------------+ |+ - * / | | |< <= == >= > != | ( ) | |! && || | | |-> | (Rule) | |rising | | |falling | | |rate | ( ) | |instant | ( ) | +----------------------+----------------------------------------------------------------------------------------------------------+ . (PM_TYPE_STRING) == != . rate instant expr rate instant expr . rate instant . (aggregate) . . +-------------------------+-----------------+--------------------------+ | | | | +-------------------------+-----------------+--------------------------+ |some_inst some_host | | | |some_sample | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <<>> | | | | | +-------------------------+-----------------+--------------------------+ |all_inst all_host | | | |all_sample | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <<>> | | | | | +-------------------------+-----------------+--------------------------+ |N%_inst N%_host | | | |N%_sample | | N | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <<>> | | | | | +-------------------------+-----------------+--------------------------+ (instantial) . egrep(1) regcomp(3) . +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+ | | | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+ |match_inst | | | | | | | <<>> | | | | | | | | | | | | | | | | | | <<>> | | | | | | | | | <<>> | | | . | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+ |nomatch_inst | | | | | | | <<>> | | | | | | | | | | | | | | | | | | <<>> | | | | | | | | | <<>> | | | . | | | | | "" (true) : | | |+0.5i | | |match_inst "^dks[23]d" disk.dev.total > 20; | | | | | | | | | (aggregate) . | | | | | |box,center; | | |cw(2.4i) | c | cw(2.4i) | | |lf(CB) | l | l. | | | | | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+ |min_inst min_host min_sample | | | | | |T} | | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+ |max_inst max_host max_sample | | | | | |T} | | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+ |sum_inst sum_host sum_sample | | | | | |T} | | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+ |avg_inst avg_host avg_sample | | | | | |T} | | +----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------+ count_inst count_host count_sample (true) . (action rules) : +----------------------+--------------------------------------------------------------------------------------+ | | | +----------------------+--------------------------------------------------------------------------------------+ |alarm | xconfirm(1) | |print | | |shell | sh(1) | |stomp | STOMP JMS | |syslog | | +----------------------+--------------------------------------------------------------------------------------+ & | ( ) ( ) . (suppression time) ( ). : %h () (true) . %c () PCP (true) . %i () (true) . %v (true) . . mumble grunt snort fumble puff ... -> shell myscript "Warning: %h:%i busy "; myscript "Warning: mumble:grunt busy Warning: mumble:snort busy Warning: fumble:puff busy" . ... -> shell myscript "Warning! busy:" " %h:%i"; myscript "Warning! busy: mumble:grunt mumble:snort fumble:puff" . ( ) ( ). ( myscript) . . pmie ( -W) ..SH " (BOOLEAN EXPRESSIONS)" pmie foo.bar > 10 some_inst ( my.table < 0 ) true false unknown . unknown pmcd(1) PCP . unknown ( ) : +------------+---------------------------+ | A and B | B | | +-------- | | | true | false | unknown | +--+---------+---------+-------+---------+ |A | true | true | false | unknown | | +---------+---------+-------+---------+ | | false | false | false | false | | +---------+---------+-------+---------+ | | unknown | unknown | false | unknown | +--+---------+---------+-------+---------+ +------------+--------------------------+ | A or B | B | | +----- | | | true | false | unknown | +--+---------+------+---------+---------+ |A | true | true | true | true | | +---------+------+---------+---------+ | | false | true | false | unknown | | +---------+------+---------+---------+ | | unknown | true | unknown | unknown | +--+---------+------+---------+---------+ +--------+---------+ | A | not A | +--------+---------+ | true | false | +--------+---------+ | false | true | +--------+---------+ |unknown | unknown | +--------+---------+ (RULESETS) ruleset ruleset . else . ( ) ruleset unknown -> action otherwise -> action . unknown unknown unknown . true unknown otherwise otherwise . (SCALE FACTORS) . : nanosecond nanosec nsec microsecond microsec usec millisecond millisec msec second sec minute min hour byte Kbyte Mbyte Gbyte Tbyte count Kcount Mcount / "Kbytes / hour". "temperature" () "voltage" () pmie : this.temp.celsius >= (other.temp.fahrenheit - 32) * 5 / 9 (MACROS) : name = constexpr; name - . constexpr ( ) . ($) constexpr . : minute . hour 0 23. day 1 31. month 0 () 11 (). year . day_of_week 0 () 6 (). delta . ( -Z -z )..SH " (AUTOMATIC RESTART)" pmie () ( ) . pmie_check(1) . pmcd(1) <<>> (primary) pmie . pmie $PCP_RC_DIR/pmie $PCP_SYSCONF_DIR/pmie/control $PCP_SYSCONF_DIR/pmie/control.d ( pmie chkconfig(8) systemctl(1) ) $PCP_VAR_DIR/config/pmie/config.default ( pmie ) . pmie -P . <<>> pmie . pmie ( ) pmcd(1) ( ) -h -P . (EVENT MONITORING) . UNIX - logger(1) syslogd(1). Windows (event log) pcp-eventlog(1) . syslog pmie . -p () / -t () logger(1) pcp-eventlog(1) . pmie . shell . stomp ( Streaming Text Oriented Messaging Protocol) pmie JMS ( ) PMIE . ( ) . stomp stomp JMS / . : host=messages.sgi.com # this is the JMS server (required) port=61616 # and its listening here (required) timeout=2 # seconds to wait for server (optional) username=joe # (required) password=j03ST0MP # (required) topic=PMIE # JMS topic for pmie messages (optional) timeout ( ) pmie JMS ( STOMP ). pmie . pmie JMS pmie stomp . . STOMP JMS stomp ..SH " (DIFFERENCES IN HOST AND ARCHIVE MODES)" delta (real- time) pmie . pmie delta "" . true (action) syslog alarm shell stomp pmie true . : delta = 10 sec; kernel.all.nprocs > 10 * hinv.ncpu -> print "lotsaprocs:" " %v"; : print Mon Sep 4 00:10:21 2017: lotsaprocs: 1292 print Mon Sep 4 00:10:31 2017: lotsaprocs: 1294 print Mon Sep 4 00:10:41 2017: lotsaprocs: 1291 ... ( ) ( pmie ). syslog STOMP pmie << >> . . -o <<>> (meta-field) : %a print syslog . %d ctime(3) . %f . %l () . %m . %u ctime(3) . %% (%). format %a %d: %m . (SIGNALS) SIGHUP pmie logfile (unlink) . pmie_daily(1) . pmie . SIGUSR1 pmie logfile . ..SH " (HOSTNAME CHANGES)" PMCD pmie . PMCD %h . pmie PMCD pmie_check(1) pmie_daily(1) ( systemd(1) cron(8) ) pmie $PCP_LOG_DIR/pmie/. PMCD . pmie pmie pmie_check(1) pmie_daily(1) pmie . systemd(1) cron(8) pmie pmie PMCD . (BUGS) . - (;) - . pmie . (FILES) $PCP_DEMOS_DIR/pmie/* $PCP_VAR_DIR/pmns/* PMNS $PCP_TMP_DIR/pmie pmie pmie - pmcd.pmie $PCP_PMIECONTROL_PATH pmie - pmie_check(1) PCP (PCP ENVIRONMENT) PCP_ PCP . /etc/pcp.conf . $PCP_CONF pcp.conf(5) . pmie IFS PATH . IFS "\t\n" . PATH ("/usr/sbin:/sbin:/usr/bin:/bin") . ( ) : $PCP_BIN_DIR $PCP_BINADM_DIR $PCP_PLATFORM_PATHS. pmie $PCP_XCONFIRM_PROG . pmconfirm(1) . (UNIX SEE ALSO) logger(1). (WINDOWS SEE ALSO) pcp-eventlog(1). (DEBUGGING OPTIONS) -D --debug stderr . debug pmdbg(1) -l . pmie : +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ | | | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ |appl0 | | | | | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ |appl1 | | | | | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ |appl2 | | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ |appl3 | | |" (SEE ALSO)" | | |PCPIntro (1), | | |pmcd (1), | | |pmconfirm (1), | | |pmie_check (1), | | |pmieconf (1), | | |pmie_daily (1), | | |pminfo (1), | | |pmlogdump (1), | | |pmlogger (1), | | |pmval (1), | | |systemd (1), | | |ctime (3), | | |PMAPI (3), | | |pcp.conf (5), | | |pcp.env (5) | | | | | |PMNS (5). | | |" (USER GUIDE)" | | | | | |pmie | | | | | |"Performance Co-Pilot Users and Administrators Guide" | | | . : | | |+4n | | |https://pcp.readthedocs.io/en/latest/UAG/PerformanceMetricsInferenceEngine.html | | |-4n | | | | | +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ Performance Co-Pilot PCP PMIE(1)