sdiag(1) Slurm Commands sdiag(1) (NAME) sdiag - Slurm (SYNOPSIS) sdiag (DESCRIPTION) sdiag slurmctld : (threads) (agents) (jobs) . slurmctld . Slurm (throughput) . . --all --reset . (UTC) . slurmctld : Server thread count slurmctld. (job submissions) (jobs dispatching) (jobs completing) . MAX_SERVER_THREADS (bottleneck) . Agent queue size Slurm . (agent) slurmctld slurmd (best effort) . RPC . Agent count (agent threads). 2 + AGENT_THREAD_COUNT . Agent thread count . DBD Agent queue size Slurm SlurmDBD . SlurmDBD . slurm.conf MaxDBDMsgs . slurmdbd . Jobs submitted . Jobs started . (backfilled) . Jobs completed . Jobs canceled . Jobs failed slurmd . Job states ts: (timestamp) . Jobs pending: . Jobs running: . . job_write_lock . . (accounts limits) . Last cycle . Max cycle . Total cycles . ( ) . Mean cycle . Mean depth cycle . . Cycles per minute . Last queue length . (backfilling) . backfilling . . . . backfilling . . Total backfilled jobs (since last slurm start) backfilling slurm. Total backfilled jobs (since last stats cycle start) backfilling . UTC . Total backfilled heterogeneous job components (heterogeneous job components) backfilling Slurm. Total cycles backfill . Last cycle when backfill "weekday Month MonthDay hour:minute.seconds year" Last cycle backfill . (sleep time) . . Max cycle backfill . (sleep time) . ..IP Mean cycle backfilling . Last depth cycle backfilling. . Last depth cycle (try sched) backfilling. . . Depth Mean backfilling . backfill ( QOS ). Depth Mean (try sched) Depth Mean backfill . Last queue length backfilling. . (job array) ( / ). Queue length Mean backfilling. . ( / ). Last table size (time slots) backfill . Mean table size backfill . backfill . : bf_min_age_reserve bf_min_prio_reserve bf_resolution bf_window. Latency for 1000 calls to gettimeofday() gettimeofday() . (RPC) Slurmctld . RPC . RPC Slurm src/common/slurm_protocol_defs.h . RPC RPC RPC . RPC (user ID) RPC RPC RPC . RPC slurmctld --reset . Pending RPC Statistics RPC (agent) slurmctld . RPC . RPC . (cache) . (OPTIONS) -a, --all . . -M, --cluster= . . slurmdbd FederationParameters=fed_display . -h, --help . --json, --json=list, --json= JSON data_parser data_parser . . --no-trunc . -r, --reset RPC 0. Slurm . -i, --sort-by-id (RPC) (user ID). -t, --sort-by-time (RPC) . -T, --sort-by-time2 (RPC) . --usage . -V, --version . --yaml, --yaml=list, --yaml= YAML data_parser data_parser . . sdiag (RPC) slurmctld . sdiag Slurm slurmctld slurmctld (denial of service) . sdiag Slurm slurmctld . sdiag . (ENVIRONMENT VARIABLES) sdiag . . (: .) SLURM_CLUSTERS --cluster SLURM_CONF Slurm. SLURM_JSON JSON: compact JSON . pretty JSON . SLURM_YAML YAML: compact YAML . pretty YAML . (COPYING) Copyright (C) 2010-2011 Barcelona Supercomputing Center. Copyright (C) 2010-2022 SchedMD LLC. Slurm << >> (GNU General Public License) ( ) / . Slurm . << >> . (SEE ALSO) sinfo(1), squeue(1), scontrol(1), slurm.conf(5), Slurm 26.05 Slurm Commands sdiag(1)