LOGARCHIVE(5) File Formats Manual LOGARCHIVE(5) (NAME) LOGARCHIVE - Performance Co- Pilot (DESCRIPTION) Performance Co-Pilot (PCP) . - (off-line) (off-site) . PMAPI(3) . . PCP PCP PCP PCP . . PCP . PCP -a/--archive NAME pmlogdump(1) (dump) . pmlogger(1) LOGIMPORT(3) . pmlogreduce(1) pmlogsummary(1) pmlogrewrite(1) pmlogextract(1) . PCP << >> (archive folios) mkaf(1) pmafm(1) . myarchive. myarchive.0, myarchive.1, ... (data volume) . pmlogger(1) . myarchive.meta PMAPI pmLookupName(3) pmLookupDesc(3) pmLookupLabels(3) pmLookupInDom(3). . myarchive.index (temporal index) . (COMMON FEATURES) (big-endian) / (padding) . ASCII . __pmLog* src/include/pcp/libpcp.h . (RECORD FRAMING) .meta - . +-------+--------+-----------------------------------------------------+ |Offset | Length | Name | +-------+--------+-----------------------------------------------------+ | 0 | 4 | N | | | | | | | | | | 4 | N-8 | (payload) | | | | | | | | | | | | | | N-4 | 4 | N () | +-------+--------+-----------------------------------------------------+ (ARCHIVE LABEL) << >> (archive label) ( pmlogger(1) ). << >> ( ) . +----------------------------------------------------------------------+ | Version 2 | +-------+--------+-----------------------------------------------------+ |Offset | Length | Name | +-------+--------+-----------------------------------------------------+ | 0 | 4 | PM_LOG_MAGIC | PM_LOG_VERS02=0x50052602 | | 4 | 4 | (PID) | | | | pmlogger | | | | | | 8 | 4 | | | | | ( | | | | ) | | 12 | 4 | | | | | | | 16 | 4 | | | | | ( -1=.meta | | | | -2=.index) | | 20 | 64 | | | | | | | 80 | 40 | | | | | | | | | | | | | ( $TZ) | +-------+--------+-----------------------------------------------------+ +----------------------------------------------------------------------+ | Version 3 | +-------+--------+-----------------------------------------------------+ |Offset | Length | Name | +-------+--------+-----------------------------------------------------+ | 0 | 4 | PM_LOG_MAGIC | PM_LOG_VERS03=0x50052603 | | 4 | 4 | PID pmlogger | | | | | | 8 | 8 | | | | | ( | | | | ) | | 16 | 4 | | | | | | | 20 | 4 | | | | | ( -1=.meta | | | | -2=.index) | | 24 | 4 | | | | | | | 28 | 4 | | | | | | | 32 | 256 | | | | | | | 288 | 256 | | | | | | | | | | | | | ( $TZ) | | | | AEDT-11 | | 544 | 256 | zoneinfo | | | | | | | | | | | | :Australia/Melbourne | +-------+--------+-----------------------------------------------------+ << >> . PM_LOG_FEATURE_XXX . << >> PCP . (ARCHIVE VOLUME (.0, .1, ...) RECORDS) (pmResult) pmResult pmFetch(3) . . (2GiB) . (8191PiB) . pmResult ( ). +--------------------------------------------------------------------------------------------------------------------------------------------------+ | Version 2 | +--------+--------+--------------------------------------------------------------------------------------------------------------------------------+ |Offset | Length | Name | +--------+--------+--------------------------------------------------------------------------------------------------------------------------------+ | 0 | 4 | ( ) | | 4 | 4 | | | 8 | 4 | | | 12 | M | pmValueSet #0 | | 12+M | N | pmValueSet #1 | | 12+M+N | ... | ... | | NOP | X | pmValueBlock #0 | | NOP+X | Y | pmValueBlock #1 | |NOP+X+Y | ... | ... | +--------+--------+--------------------------------------------------------------------------------------------------------------------------------+ +--------------------------------------------------------------------------------------------------------------------------------------------------+ | Version 3 | +--------+--------+--------------------------------------------------------------------------------------------------------------------------------+ |Offset | Length | Name | +--------+--------+--------------------------------------------------------------------------------------------------------------------------------+ | 0 | 8 | ( ) | | 8 | 4 | | | 12 | 4 | | | 16 | M | pmValueSet #0 | | 16+M | N | pmValueSet #1 | | 16+M+N | ... | ... | | NOP | X | pmValueBlock #0 | | NOP+X | Y | pmValueBlock #1 | |NOP+X+Y | ... | ... | +--------+--------+--------------------------------------------------------------------------------------------------------------------------------+ << >> << >> (mark records) . (pmValueSet) . +-------+--------+----------------------------------------------------------------------+ |Offset | Length | Name | +-------+--------+----------------------------------------------------------------------+ | 0 | 4 | (PMID) | | 4 | 4 | | | 8 | 4 | PM_VAL_INSITU=0 PM_VAL_DPTR=1 | | 12 | M | pmValue #0 | | 12+M | N | pmValue #1 | |12+M+N | ... | ... | +-------+--------+----------------------------------------------------------------------+ .meta . (pmValue) . (variant) pmValueSet . pmResult pmValueBlock . +-------+--------+-----------------------------------------------------+ |Offset | Length | Name | +-------+--------+-----------------------------------------------------+ | 0 | 4 | | | | | ( PM_IN_NULL=-1 | | | | ) | | 4 | 4 | (INSITU) | | | | pmResult pmValueBlock | | | | (DPTR) | +-------+--------+-----------------------------------------------------+ .meta . . T T' <= T . (pmValueBlock) pmValueSet pmValue . ( ) (pad) . +-------+--------+------------------------------------------------------------------------------------------------------------------------+ |Offset | Length | Name | +-------+--------+------------------------------------------------------------------------------------------------------------------------+ | 0 | 1 | ( pmDesc.type) | | 1 | 3 | + N | | 4 | N | | | 4+N | 0-3 | ( + N ) | +-------+--------+------------------------------------------------------------------------------------------------------------------------+ PM_TYPE_STRING NULL . PM_TYPE_EVENT . pmResult PMDAEVENTARRAY(3) . (.meta) (METADATA FILE (.meta) RECORDS) ( ) () . . (2GiB) . (8191PiB) . libpcp/src/logmeta.c . (Metric Descriptions) (pmDesc) PCP . (PMID) ( PM_INDOM_NULL ) ( ) . +-------+--------+-----------------------------------------------------+ |Offset | Length | Name | +-------+--------+-----------------------------------------------------+ | 0 | 4 | TYPE_DESC=1 | | 4 | 4 | PMID | | 8 | 4 | (PM_TYPE_*) | | 12 | 4 | | | 16 | 4 | | | | | (PM_SEM_*) | | 20 | 4 | : pmUnits | | | | | | 4 | 4 | | | | | PMID | | 28 | 4 | N: | | | | | | 32 | N | | | | | NULL | | | | | | 32+N | 4 | N2: | | | | | | 36+N | N2 | | | | | NULL | | | | | | ... | ... | ... | +-------+--------+-----------------------------------------------------+ (Instance Domains) () ( pmDesc ) ( ) () . .meta . pmResult . . ( ) <<>> (delta) . +-------------------------------------------------------------------------------------------------------------------------------------------+ | Full Instance Domain - Version 2 | +---------+--------+------------------------------------------------------------------------------------------------------------------------+ | Offset | Length | Name | +---------+--------+------------------------------------------------------------------------------------------------------------------------+ | 0 | 4 | TYPE_INDOM_V2=2 | | 4 | 4 | ( ) | | 8 | 4 | | | 12 | 4 | | | 16 | 4 | N: >0 | | 20 | 4 | | | 24 | 4 | ( ) | | ... | ... | ... | | 20+4*N | 4 | ( ) | |20+4*N+4 | 4 | ( ) | | ... | ... | ... | | 20+8*N | M | | | | | NULL | +---------+--------+------------------------------------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------+ | Full Instance Domain - Version 3 | +---------+--------+------------------------------------------------------------------------------------------------------------------------+ | Offset | Length | Name | +---------+--------+------------------------------------------------------------------------------------------------------------------------+ | 0 | 4 | TYPE_INDOM=5 | | 4 | 8 | ( ) | | 12 | 4 | | | 16 | 4 | | | 20 | 4 | N: >0 | | 24 | 4 | | | 28 | 4 | ( ) | | ... | ... | ... | | 24+4*N | 4 | ( ) | |24+4*N+4 | 4 | ( ) | | ... | ... | ... | | 24+8*N | M | | | | | NULL | +---------+--------+------------------------------------------------------------------------------------------------------------------------+ <<>> <<>> : * TYPE_INDOM_DELTA=6 . * << >> . * -1 . * . <<>> pmdaproc(1) . <<>> : . <<>> . <<>> <<>> . <<>> . (Labels for Contexts, Instance Domains and Metrics) (pmLogLabelSet) -:- - pmLookupLabels(3) . pmDesc . pmResult . pmLogLabelSet ( ) . +----------------------------------------------------------------------+ | Version 2 | +------------+--------+------------------------------------------------+ | Offset | Length | Name | +------------+--------+------------------------------------------------+ | 0 | 4 | TYPE_LABEL_V2=3 | | 4 | 4 | | | | | ( | | | | ) | | 8 | 4 | | | | | | | 12 | 4 | | | | | ( | | | | PM_LABEL_*) | | 16 | 4 | - | | | | PMID | | | | PM_IN_NULL=-1 | | | | | | 20 | 4 | N: | | | | | | | | | | | | | | | | | | 24 | 4 | | | | | JSONB | | 28 | L1 | | | | | labelset ( | | | | ) | | ... | ... | ... | | 28+L1 | LN | N- | | | | labelset ( | | | | ) | | ... | ... | ... | |28+L1+...LN | M | | | | | JSONB | | | | | | | | | +------------+--------+------------------------------------------------+ +----------------------------------------------------------------------+ | Version 3 | +------------+--------+------------------------------------------------+ | Offset | Length | Name | +------------+--------+------------------------------------------------+ | 0 | 4 | TYPE_LABEL=7 | | 4 | 8 | | | | | ( | | | | ) | | 12 | 4 | | | | | | | 16 | 4 | | | | | ( | | | | PM_LABEL_*) | | 20 | 4 | - | | | | PMID | | | | PM_IN_NULL=-1 | | | | | | 24 | 4 | N: | | | | | | | | | | | | | | | | | | 28 | 4 | | | | | JSONB | | 32 | L1 | | | | | labelset ( | | | | ) | | ... | ... | ... | | 32+L1 | LN | N- | | | | labelset ( | | | | ) | | ... | ... | ... | |32+L1+...LN | M | | | | | JSONB | | | | | | | | | +------------+--------+------------------------------------------------+ : . labelset . ( PM_LABEL_INSTANCES) . +-------+--------+------------------------------------------------------------------------------------------------------------------+ |Offset | Length | Name | +-------+--------+------------------------------------------------------------------------------------------------------------------+ | 0 | 4 | ( PM_IN_NULL=-1) | | 4 | 4 | JSONB | | 8 | 4 | N: labelset | | 12 | 2 | | | 14 | 1 | | | 15 | 1 | ( ) | | 16 | 2 | | | 18 | 2 | | | 20 | 2 | ( ) | | ... | ... | ... | +-------+--------+------------------------------------------------------------------------------------------------------------------+ (Help Text) (pmLogText) - pmLookupText(3) pmLookupInDomText(3) - . pmDesc . +-------+--------+------------------------------------------------------------------------------+ |Offset | Length | Name | +-------+--------+------------------------------------------------------------------------------+ | 0 | 4 | TYPE_TEXT=4 | | 4 | 4 | ( PM_TEXT_*) | | 8 | 4 | - PMID | | 12 | M | | +-------+--------+------------------------------------------------------------------------------+ (.index) (INDEX FILE (.index) RECORDS) (tuples) .meta . <<//>> : .index . . libpcp/src/logutil.c . ( ) . +---------------------------------------------------------------------------------------------------------------------------------+ | Version 2 | +-------+--------+----------------------------------------------------------------------------------------------------------------+ |Offset | Length | Name | +-------+--------+----------------------------------------------------------------------------------------------------------------+ | 0 | 4 | ( ) | | 4 | 4 | | | 8 | 4 | (0...N) | | 12 | 4 | .meta | | 16 | 4 | | +-------+--------+----------------------------------------------------------------------------------------------------------------+ +---------------------------------------------------------------------------------------------------------------------------------+ | Version 3 | +-------+--------+----------------------------------------------------------------------------------------------------------------+ |Offset | Length | Name | +-------+--------+----------------------------------------------------------------------------------------------------------------+ | 0 | 8 | ( ) | | 8 | 4 | | | 12 | 4 | (0...N) | | 16 | 8 | .meta | | 24 | 8 | | +-------+--------+----------------------------------------------------------------------------------------------------------------+ . . . . (FILES) PCP : $HOME/.pcp/pmlogger pmchart(1) $PCP_LOG_DIR/pmlogger pmlogger_daily(1) pmlogger_check(1) (SEE ALSO) mkaf(1) PCPIntro(1) pmafm(1) pmchart(1) pmdaproc(1) pmlogdump(1) pmlogger(1) pmlogger_check(1) pmlogger_daily(1) pmlogreduce(1) pmlogrewrite(1) pmlogsummary(1) LOGIMPORT(3) PMAPI(3) pmLookupDesc(3) pmLookupInDom(3) pmLookupInDomText(3) pmLookupLabels(3) pmLookupName(3) pmLookupText(3) pcp.conf(5) pcp.env(5). Performance Co-Pilot LOGARCHIVE(5)