CLI Reference

In the AppScope CLI, the scope command takes a Linux command as an argument. That's called "scoping" the Linux command. For example, if you run scope top, we say you have "scoped" the top command.

The AppScope CLI also has subcommands, which pair with scope to do many things. For example, if you run scope dash, AppScope displays its dashboard.

This Reference explains how to use scope and its arsenal of subcommands.

Subcommand Syntax

To execute CLI subcommands, the basic syntax is:

./scope <subcommand> [flags] [options]

Subcommands Available

To see a list of available subcommands, enter ./scope alone, or ./scope -h, or ./scope --help. This displays the basic help listing below.

Cribl AppScope Command Line Interface

AppScope is a general-purpose observable application telemetry system.

Running `scope` with no subcommands will execute the `scope run` command.

Usage:
  scope [command]

Available Commands:
  attach      Scope a currently-running process
  completion  Generates completion code for specified shell
  daemon      Run the scope daemon
  dash        Display scope dashboard for a previous or active session
  detach      Unscope a currently-running process
  events      Outputs events for a session
  extract     Output instrumentary library files to <dir>
  rules       View or modify system-wide AppScope rules
  flows       Observed flows from the session, potentially including payloads
  help        Help about any command
  history     List scope session history
  inspect     Returns information about scoped process
  k8s         Install scope in kubernetes
  logs        Display scope logs
  metrics     Outputs metrics for a session
  prom        Run the Prometheus Target
  prune       Prune deletes session history
  ps          List processes currently being scoped
  run         Executes a scoped command
  service     Configure a systemd/OpenRC service to be scoped
  snapshot    Create a snapshot for a process
  start       Install the AppScope library
  stop        Stop scoping all scoped processes and services
  update      Updates the configuration of a scoped process
  version     Display scope version
  watch       Executes a scoped command on an interval

Flags:
  -h, --help          help for scope
  -z, --passthrough   Scope an application with current environment & no config.

Use "scope [command] --help" for more information about a command.

As noted just above, to see a specific subcommand's help or its required parameters, enter: ./scope <subcommand> -h

…or: ./scope help <subcommand>.

attach

Scopes a currently-running process identified by PID or ProcessName.

The --*dest flags accept file names like /tmp/scope.log or URLs like file:///tmp/scope.log. They may also be set to sockets with unix:///var/run/mysock, tcp://hostname:port, udp://hostname:port, or tls://hostname:port.

Usage

scope attach [flags] PID | <process_name>

Examples

scope attach 1000
scope attach firefox 
scope attach top < scope.yml
scope attach --rootdir /path/to/host firefox 
scope attach --rootdir /path/to/host/mount/proc/<hostpid>/root 1000
scope attach --payloads 2000

Flags

  -a, --authtoken string      Set AuthToken for Cribl
  -b, --backtrace             Enable backtrace file generation when an application crashes.
  -d, --coredump              Enable core dump file generation when an application crashes.
  -c, --cribldest string      Set Cribl destination for metrics & events (host:port defaults to tls://)
  -e, --eventdest string      Set destination for events (host:port defaults to tls://)
  -h, --help                  help for attach
  -i, --inspect               Inspect the process after attach is complete
  -j, --json                  Output as newline delimited JSON
  -l, --librarypath string    Set path for dynamic libraries
      --loglevel string       Set scope library log level (debug, warning, info, error, none)
  -m, --metricdest string     Set destination for metrics (host:port defaults to tls://)
      --metricformat string   Set format of metrics output (statsd|ndjson) (default "ndjson")
      --metricprefix string   Set prefix for StatsD metrics, ignored if metric format isn't statsd
  -n, --nobreaker             Set Cribl to not break streams into events.
  -p, --payloads              Capture payloads of network transactions
  -R, --rootdir               Path to root filesystem of target namespace
  -u, --userconfig string     Scope an application with a user specified config file; overrides all other settings.
  -v, --verbosity int         Set scope metric verbosity (default 4)

completion

Generates completion code for specified shell.

Usage

scope completion [flags] [bash|zsh]

Examples

scope completion bash > /etc/bash_completion.d/scope # Generate and install scope autocompletion for bash
source <(scope completion bash)                      # Generate and load scope autocompletion for bash

Flags

  -h, --help   help for completion

dash

Displays an interactive dashboard with an overview of what's happening with the selected session.

Usage

scope dash [flags]

Examples

scope dash

Flags

  -h, --help     help for dash
  -i, --id int   Display info from specific from session ID (default -1)

detach

Unscopes a currently-running process identified by PID or process name.

Usage

scope detach [flags] PID | <process_name>

Examples

scope detach 1000
scope detach firefox
scope detach --all
scope detach 1000 --rootdir /path/to/host/mount
scope detach --rootdir /path/to/host/mount
scope detach --all --rootdir /path/to/host/mount/proc/<hostpid>/root

Flags

  -a, --all                   Detach from all processes
  -h, --help                  Help for detach
  -R, --rootdir               Path to root filesystem of target namespace
  -w, --wait                  Wait for detach to complete

events

Outputs events for a session. You can obtain detailed information about each event by inputting the Event ID as a positional parameter. (By default, the Event ID appears in blue, in []'s at the left.) You can provide filters to narrow down by name (e.g., http, net, fs, or console), or by field (e.g., fs.open, stdout, or net.open). You can use JavaScript expressions to further refine the query, and to express logic.

Usage

scope events [flags] ([eventId])

Examples

Examples:
scope events
scope events m61
scope events --sourcetype http
scope events --source stderr
scope events --match file
scope events --fields net_bytes_sent,net_bytes_recv --match net_bytes
scope events --follow
scope events --all
scope events --allfields
scope events --id 4
scope events --sort _time --reverse
scope events --eval 'sourcetype!="net"'
scope events -n 1000 -e 'sourcetype!="console" && source.indexOf("cribl.log") == -1 && (data["file.name"] || "").indexOf("/proc") == -1'

Flags

  -a, --all                  Show all events
      --allfields            Displaying hidden fields
      --color                Force color on (if tty detection fails or piping)
  -e, --eval string          Evaluate JavaScript expression against event. Must return truthy to print event.
                             Note: Post-processes after matching, not guaranteed to return last <n> events.
      --fields strings       Display the names and values for specified fields only, for each event (look at JSON output for field names)
  -f, --follow               Follow a file, like tail -f
  -h, --help                 help for events
  -i, --id int               Display info from specific from session ID (default -1)
  -j, --json                 Output as newline delimited JSON
  -n, --last int             Show last <n> events (default 20)
  -m, --match string         Display events containing supplied string
  -r, --reverse              Reverse sort to ascending. Must be combined with --sort
      --sort string          Sort descending by field (look at JSON output for field names)
  -s, --source strings       Display events matching supplied sources
  -t, --sourcetype strings   Display events matching supplied sourcetypes

extract

Outputs libscope.so and scope.yml to the provided directory. You can configure these files to instrument any application, and to output the data to any existing tool using simple TCP protocols.

Usage

scope extract [flags] (<dir>)

Aliases

extract, excrete, expunge, extricate, exorcise

Examples

scope extract
scope extract /opt/libscope
scope extract --metricdest tcp://some.host:8125 --eventdest tcp://other.host:10070 .

Flags

  -a, --authtoken string      Set AuthToken for Cribl
  -c, --cribldest string      Set Cribl destination for metrics & events (host:port defaults to tls://)
  -e, --eventdest string      Set destination for events (host:port defaults to tls://)
  -h, --help                  Help for extract
  -m, --metricdest string     Set destination for metrics (host:port defaults to tls://)
      --metricformat string   Set format of metrics output (statsd|ndjson); default is "ndjson"
      --metricprefix string   Set prefix for StatsD metrics, ignored if metric format isn't statsd
  -n, --nobreaker             Set Cribl to not break streams into events
  -p, --parents               Create any missing intermediate pathname components in provided directory parameter

rules

View or modify system-wide AppScope rules to automatically scope a set of processes. You can add or remove a single process at a time.

Usage

scope rules [flags]

Examples

scope rules
scope rules --rootdir /path/to/host/root --json
scope rules --add nginx
scope rules --add nginx < scope.yml
scope rules --add java --arg myServer 
scope rules --add firefox --rootdir /path/to/host/root
scope rules --remove chromium

Flags

      --add string            Add an entry to the global rules
      --arg string            Argument to the command to be added to the rules
  -a, --authtoken string      Set AuthToken for Cribl
  -b, --backtrace             Enable backtrace file generation when an application crashes.
  -d, --coredump              Enable core dump file generation when an application crashes.
  -c, --cribldest string      Set Cribl destination for metrics & events (host:port defaults to tls://)
  -e, --eventdest string      Set destination for events (host:port defaults to tls://)
  -h, --help                  help for rules
  -j, --json                  Output as newline delimited JSON
  -l, --librarypath string    Set path for dynamic libraries
      --loglevel string       Set scope library log level (debug, warning, info, error, none)
  -m, --metricdest string     Set destination for metrics (host:port defaults to tls://)
      --metricformat string   Set format of metrics output (statsd|ndjson|prometheus) (default "ndjson")
  -n, --nobreaker             Set Cribl to not break streams into events.
  -p, --payloads              Capture payloads of network transactions
      --remove string         Remove an entry from the global rules
  -R, --rootdir string        Path to root filesystem of target namespace
      --source string         Source identifier for a rules entry
      --unixpath string       Path to the unix socket
  -u, --userconfig string     Scope an application with a user specified config file; overrides all other settings.
  -v, --verbosity int         Set scope metric verbosity (default 4)

flows

Displays observed flows from the given session. If run with payload capture on, outputs full payloads from the flow.

Usage

scope flows [flags] <sessionId>

Examples

scope flows                # Displays all flows
scope flows 124x3c         # Displays more info about the flow
scope flows --in 124x3c    # Displays the inbound payload of that flow
scope flows --out 124x3c   # Displays the outbound payload of that flow
scope flows -p 0.0.0.0/24  # Displays flows in that subnet range
scope flows --sort net_host_port --reverse  # Sort flows by ascending host port

Flags

  -a, --all           Show all flows
  -h, --help          Help for flows
  -i, --id int        Display flows from specific from session ID (default -1)
      --in            Output contents of the inbound payload. Requires flow ID specified.
  -j, --json          Output as newline-delimited JSON
  -n, --last int      Show last <n> flows (default 20)
      --out           Output contents of the outbound payload. Requires flow ID specified.
  -p, --peer ipNet    Filter to peers in the given network
  -r, --reverse       Reverse sort to ascending
  -s, --sort string   Sort descending by field (look at JSON output for field names)

help

Displays help content for any AppScope subcommand. Just type scope help [subcommand] for full details.

Usage

scope help [subcommand] [flags]

Examples

scope help run

history

Prints information about sessions. Every time you scope a command, that is called an AppScope session. Each session has a directory which is referenced by a session ID. By default, the AppScope CLI stores all the information it collects during a given session in that session's directory. When you run history, you see a listing of sessions, one session per scoped command, along with information about when the session started, how many events were output during the session, and so on.

Usage

scope history [flags]

Aliases

history, hist

Examples

scope history                    # Displays session history
scope hist                       # Shortcut for scope history
scope hist -r                    # Displays running sessions
scope hist --id 2                # Displays detailed information for session 2
scope hist -n 50                 # Displays last 50 sessions
scope hist -d                    # Displays directory for the last session
cat $(scope hist -d)/args.json   # Outputs contents of args.json in the scope history directory for the current session

Flags

  -a, --all        List all sessions
  -d, --dir        Output just directory (with -i)
  -h, --help       Help for history
  -i, --id int     Display info from specific from session ID (default -1)
  -n, --last int   Show last <n> sessions (default 20)
  -r, --running    List running sessions

inspect

Returns information on scoped process identified by PID.

Usage

scope inspect [flags]

Examples

scope inspect
scope inspect 1000
scope inspect --all --json
scope inspect 1000 --rootdir /path/to/host/mount
scope inspect --all --rootdir /path/to/host/mount
scope inspect --all --rootdir /path/to/host/mount/proc/<hostpid>/root

Flags

  -a, --all             Inspect all processes
  -h, --help            Help for inspect
  -j, --json            Output as newline delimited JSON without pretty printing
  -R, --rootdir         Path to root filesystem of target namespace

k8s

Prints configurations to pass to kubectl, which then automatically instruments newly-launched containers. This installs a mutating admission webhook, which adds an initContainer to each pod. The webhook also sets environment variables that install AppScope for all processes in that container.

The --*dest flags accept file names like /tmp/scope.log; URLs like file:///tmp/scope.log; or sockets specified with the pattern unix:///var/run/mysock, tcp://hostname:port, udp://hostname:port, or tls://hostname:port.

Usage

scope k8s [flags]

Examples

scope k8s --metricdest tcp://some.host:8125 --eventdest tcp://other.host:10070 | kubectl apply -f -
kubectl label namespace default scope=enabled

Flags

      --app string            Name of the app in Kubernetes (default "scope")
  -a, --authtoken string      Set AuthToken for Cribl Stream
      --certfile string       Certificate file for TLS in the container (mounted secret) (default "/etc/certs/tls.crt")
  -c, --cribldest string      Set Cribl Stream destination for metrics & events (host:port defaults to tls://)
      --debug                 Turn on debug logging in the scope webhook container
  -e, --eventdest string      Set destination for events (host:port defaults to tls://)
  -h, --help                  Help for k8s
      --keyfile string        Private key file for TLS in the container (mounted secret) (default "/etc/certs/tls.key")
  -m, --metricdest string     Set destination for metrics (host:port defaults to tls://)
      --metricformat string   Set format of metrics output (statsd|ndjson); default is "ndjson"
      --metricprefix string   Set prefix for StatsD metrics, ignored if metric format isn't statsd
      --namespace string      Name of the namespace in which to install; default is "default"
  -n, --nobreaker             Set Cribl Stream to not break streams into events
      --noexporter            Disable StatsD to Prometheus Exporter deployment
      --port int              Port to listen on (default 4443)
      --promport int          Specify StatsD to Prometheus Exporter port for Prometheus HTTP metrics requests (default 9090)
      --server                Run Webhook server
      --signername string     Name of the signer used to sign the certificate request for the AppScope Admission Webhook (default "kubernetes.io/kubelet-serving")
      --version string        Version of scope to deploy

logs

Displays internal AppScope logs for troubleshooting AppScope itself.

Usage

scope logs [flags]

Examples

scope logs

Flags

  -h, --help             Help for logs
  -i, --id int           Display logs from specific from session ID (default -1)
  -n, --last int         Show last <n> lines (default 20)
  -s, --scope            Show scope.log (from CLI) instead of ldscope.log (from library)
  -S, --service string   Display logs from a systemd service instead of a session

metrics

Outputs metrics for a session.

Usage

scope metrics [flags]

Examples

scope metrics
scope metrics -m net.error,fs.error
scope metrics -m net.tx -g

Flags

  -c, --cols             Display metrics as columns. Must be combined with -m
  -g, --graph            Graph this metric. Must be combined with -m
  -h, --help             Help for metrics
  -i, --id int           Display info from specific from session ID (default -1)
  -m, --metric strings   Display for specified metrics only (comma-separated)
  -u, --uniq             Display first instance of each unique metric

prune

Prunes (deletes) one or more sessions from the history.

Usage

scope prune [flags]

Examples

scope prune -k 20
scope prune -a
scope prune -d 1

Flags

Negative arguments are not allowed.

  -a, --all          Delete all sessions
  -d, --delete int   Delete last <n> sessions
  -f, --force        Do not prompt for confirmation
  -h, --help         Help for prune
  -k, --keep int     Keep last <n> sessions, delete all others

ps

Lists all scoped processes. This means processes whose functions AppScope is interposing (which means that the AppScope library was loaded, and the AppScope reporting thread is running, in those processes, too).

Usage

scope ps

Examples

scope ps
scope ps --json
scope ps --rootdir /path/to/host/mount
scope ps --rootdir /path/to/host/mount/proc/<hostpid>/root`,

Flags

  -j, --json            Output as newline delimited JSON without pretty printing
  -R, --rootdir         Path to root filesystem of target namespace

run

Executes a scoped command. By default, calling scope with no subcommands will run the executables you pass as arguments to scope. However, scope allows for additional arguments to be passed to run, to capture payloads or to increase metrics' verbosity. Must be called with the -- flag, e.g., scope run -- <command>, to prevent AppScope from attempting to parse flags passed to the executed command.

Usage

scope run [flags] [command]

Examples

scope run -- /bin/echo "foo"
scope run -- perl -e 'print "foo\n"'
scope run --payloads -- nc -lp 10001
scope run -- curl https://wttr.in/94105
scope run -c tcp://127.0.0.1:10091 -- curl https://wttr.in/94105
scope run -c edge -- top

Flags

  -a, --authtoken string      Set AuthToken for Cribl
  -b, --backtrace             Enable backtrace file generation when an application crashes.
  -d, --coredump              Enable core dump file generation when an application crashes.
  -c, --cribldest string      Set Cribl destination for metrics & events (host:port defaults to tls://)
  -e, --eventdest string      Set destination for events (host:port defaults to tls://)
  -h, --help                  help for run
  -l, --librarypath string    Set path for dynamic libraries
      --loglevel string       Set scope library log level (debug, warning, info, error, none)
  -m, --metricdest string     Set destination for metrics (host:port defaults to tls://)
      --metricformat string   Set format of metrics output (statsd|ndjson) (default "ndjson")
      --metricprefix string   Set prefix for StatsD metrics, ignored if metric format isn't statsd
  -n, --nobreaker             Set Cribl to not break streams into events.
  -p, --payloads              Capture payloads of network transactions
  -u, --userconfig string     Scope an application with a user specified config file; overrides all other settings.
  -v, --verbosity int         Set scope metric verbosity (default 4)

service

Configures the specified systemd/OpenRC service to be scoped upon starting.

Usage

scope service SERVICE [flags]

Examples

scope service cribl -c tls://in.my-instance.cribl.cloud:10090

Flags

  -a, --authtoken string      Set AuthToken for Cribl Stream
  -c, --cribldest string      Set Cribl Stream destination for metrics & events (host:port defaults to tls://)
  -e, --eventdest string      Set destination for events (host:port defaults to tls://)
      --force                 Bypass confirmation prompt
  -h, --help                  Help for service
  -m, --metricdest string     Set destination for metrics (host:port defaults to tls://)
      --metricformat string   Set format of metrics output (statsd|ndjson); default is "ndjson"
      --metricprefix string   Set prefix for StatsD metrics, ignored if metric format isn't statsd
  -n, --nobreaker             Set Cribl Stream to not break streams into events
  -u, --user string           Specify owner username

snapshot

Create a snapshot for a process. Snapshot file/s will be created in /tmp/appscope/[PID]/.

Usage

scope snapshot [PID] [flags]

Flags

-h, --help help for snapshot

start

Install the AppScope library to: /usr/lib/appscope// with admin privileges, or /tmp/appscope// otherwise

Usage

scope start [flags]

Examples

scope start
scope start --rootdir /hostfs

Flags

  -h, --help             help for start
  -p, --rootdir string   Path to root filesystem of target namespace

stop

Performs the following actions: - Removal of /etc/ld.so.preload contents - Removal of the rules file from /usr/lib/appscope/scope_rules - Detach from all currently scoped processes

The command does not uninstall scope or libscope from /usr/lib/appscope or /tmp/appscope or remove any service configurations.

Usage

scope stop [flags]

Examples

scope stop

Flags

  -f, --force      Use this flag when you're sure you want to run scope stop
  -R, --rootdir    Path to root filesystem of target namespace
  -h, --help       help for stop

update

Updates configuration of scoped process identified by PID.

Usage

scope update [flags]

Examples

scope update 1000 --config scope_cfg.yml
scope update 1000 < scope_cfg.yml
scope update 1000 --json < scope_cfg.yml
scope update 1000 --rootdir /path/to/host/mount --config scope_cfg.yml
scope update 1000 --rootdir /path/to/host/mount/proc/<hostpid>/root < scope_cfg.yml

Flags

Flags:
  -i, --inspect         Inspect the process after the update is complete
  -c, --config string   Path to configuration file
  -h, --help            help for update
  -j, --json            Output as newline delimited JSON without pretty printing
  -R, --rootdir         Path to root filesystem of target namespace

version

Outputs version info.

Usage

scope version [flags]

Examples

scope version
scope version --date
scope version --summary
scope version --tag

Flags

      --date      Output just the date
  -h, --help      Help for version
      --summary   Output just the summary
      --tag       Output just the tag

watch

Executes a scoped command on an interval. Must be called with the -- flag, e.g., scope watch -- <command>, to prevent AppScope from attempting to parse flags passed to the executed command.

Usage

scope watch [flags]

Examples

scope watch -i 5s -- /bin/echo "foo"
scope watch --interval=1m-- perl -e 'print "foo\n"'
scope watch --interval=5s --payloads -- nc -lp 10001
scope watch -i 1h -- curl https://wttr.in/94105
scope watch --interval=10s -- curl https://wttr.in/94105

Flags

  -a, --authtoken string      Set AuthToken for Cribl
  -b, --backtrace             Enable backtrace file generation when an application crashes.
  -d, --coredump              Enable core dump file generation when an application crashes.
  -c, --cribldest string      Set Cribl destination for metrics & events (host:port defaults to tls://)
  -e, --eventdest string      Set destination for events (host:port defaults to tls://)
  -h, --help                  help for watch
  -i, --interval string       Run every <x>(s|m|h)
  -l, --librarypath string    Set path for dynamic libraries
      --loglevel string       Set scope library log level (debug, warning, info, error, none)
  -m, --metricdest string     Set destination for metrics (host:port defaults to tls://)
      --metricformat string   Set format of metrics output (statsd|ndjson) (default "ndjson")
      --metricprefix string   Set prefix for StatsD metrics, ignored if metric format isn't statsd
  -n, --nobreaker             Set Cribl to not break streams into events.
  -p, --payloads              Capture payloads of network transactions
  -u, --userconfig string     Scope an application with a user specified config file; overrides all other settings.
  -v, --verbosity int         Set scope metric verbosity (default 4)

Menu

Dark

CLI Reference

Subcommand Syntax

Subcommands Available

attach

Usage

Examples

Flags

completion

Usage

Examples

Flags

dash

Usage

Examples

Flags

detach

Usage

Examples

Flags

events

Usage

Examples

Flags

extract

Usage

Aliases

Examples

Flags

rules

Usage

Examples

Flags

flows

Usage

Examples

Flags

help

Usage

Examples

history

Usage

Aliases

Examples

Flags

inspect

Usage

Examples

Flags

k8s

Usage

Examples

Flags

logs

Usage

Examples

Flags

metrics

Usage

Examples

Flags

prune

Usage

Examples

Flags

ps

Usage

Examples

Flags

run

Usage

Examples

Flags

service

Usage

Examples

Flags

snapshot

Usage