cat APIedit

If you work from the command line often, the cat APIs will be helpful to you. Named after the linux cat command, these APIs are designed to work like *nix command-line tools.

They provide statistics that are identical to all the previously discussed APIs (Health, node-stats, and so forth), but present the output in tabular form instead of JSON. This is very convenient for a system administrator, and you just want to glance over your cluster or find nodes with high memory usage.

Executing a plain GET against the cat endpoint will show you all available APIs:

GET /_cat

=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}

Many of these APIs should look familiar to you (and yes, that’s a cat at the top :) ). Let’s take a look at the Cat Health API:

GET /_cat/health

1408723713 12:08:33 elasticsearch_zach yellow 1 1 114 114 0 0 114

The first thing you’ll notice is that the response is plain text in tabular form, not JSON. The second thing you’ll notice is that there are no column headers enabled by default. This is designed to emulate *nix tools, since it is assumed that once you become familiar with the output, you no longer want to see the headers.

To enable headers, add the ?v parameter:

GET /_cat/health?v

epoch   time    cluster status node.total node.data shards pri relo init
1408[..] 12[..] el[..]  1         1         114 114    0    0     114
unassign

Ah, much better. We now see the timestamp, cluster name, status, the number of nodes in the cluster, and more—​all the same information as the cluster-health API.

Let’s look at node-stats in the cat API:

GET /_cat/nodes?v

host         ip            heap.percent ram.percent load node.role master name
zacharys-air 192.168.1.131           45          72 1.85 d         *      Zach

We see some stats about the nodes in our cluster, but the output is basic compared to the full node-stats output. You can include many additional metrics, but rather than consulting the documentation, let’s just ask the cat API what is available.

You can do this by adding ?help to any API:

GET /_cat/nodes?help

id               | id,nodeId               | unique node id
pid              | p                       | process id
host             | h                       | host name
ip               | i                       | ip address
port             | po                      | bound transport port
version          | v                       | es version
build            | b                       | es build hash
jdk              | j                       | jdk version
disk.avail       | d,disk,diskAvail        | available disk space
heap.percent     | hp,heapPercent          | used heap ratio
heap.max         | hm,heapMax              | max configured heap
ram.percent      | rp,ramPercent           | used machine memory ratio
ram.max          | rm,ramMax               | total machine memory
load             | l                       | most recent load avg
uptime           | u                       | node uptime
node.role        | r,role,dc,nodeRole      | d:data node, c:client node
master           | m                       | m:master-eligible, *:current master
...
...

(Note that the output has been truncated for brevity).

The first column shows the full name, the second column shows the short name, and the third column offers a brief description about the parameter. Now that we know some column names, we can ask for those explicitly by using the ?h parameter:

GET /_cat/nodes?v&h=ip,port,heapPercent,heapMax

ip            port heapPercent heapMax
192.168.1.131 9300          53 990.7mb

Because the cat API tries to behave like *nix utilities, you can pipe the output to other tools such as sort grep or awk. For example, we can find the largest index in our cluster by using the following:

% curl 'localhost:9200/_cat/indices?bytes=b' | sort -rnk8

yellow test_names         5 1 3476004 0 376324705 376324705
yellow .marvel-2014.08.19 1 1  263878 0 160777194 160777194
yellow .marvel-2014.08.15 1 1  234482 0 143020770 143020770
yellow .marvel-2014.08.09 1 1  222532 0 138177271 138177271
yellow .marvel-2014.08.18 1 1  225921 0 138116185 138116185
yellow .marvel-2014.07.26 1 1  173423 0 132031505 132031505
yellow .marvel-2014.08.21 1 1  219857 0 128414798 128414798
yellow .marvel-2014.07.27 1 1   75202 0  56320862  56320862
yellow wavelet            5 1    5979 0  54815185  54815185
yellow .marvel-2014.07.28 1 1   57483 0  43006141  43006141
yellow .marvel-2014.07.21 1 1   31134 0  27558507  27558507
yellow .marvel-2014.08.01 1 1   41100 0  27000476  27000476
yellow kibana-int         5 1       2 0     17791     17791
yellow t                  5 1       7 0     15280     15280
yellow website            5 1      12 0     12631     12631
yellow agg_analysis       5 1       5 0      5804      5804
yellow v2                 5 1       2 0      5410      5410
yellow v1                 5 1       2 0      5367      5367
yellow bank               1 1      16 0      4303      4303
yellow v                  5 1       1 0      2954      2954
yellow p                  5 1       2 0      2939      2939
yellow b0001_072320141238 5 1       1 0      2923      2923
yellow ipaddr             5 1       1 0      2917      2917
yellow v2a                5 1       1 0      2895      2895
yellow movies             5 1       1 0      2738      2738
yellow cars               5 1       0 0      1249      1249
yellow wavelet2           5 1       0 0       615       615

By adding ?bytes=b, we disable the human-readable formatting on numbers and force them to be listed as bytes. This output is then piped into sort so that our indices are ranked according to size (the eighth column).

Unfortunately, you’ll notice that the Marvel indices are clogging up the results, and we don’t really care about those indices right now. Let’s pipe the output through grep and remove anything mentioning Marvel:

% curl 'localhost:9200/_cat/indices?bytes=b' | sort -rnk8 | grep -v marvel

yellow test_names         5 1 3476004 0 376324705 376324705
yellow wavelet            5 1    5979 0  54815185  54815185
yellow kibana-int         5 1       2 0     17791     17791
yellow t                  5 1       7 0     15280     15280
yellow website            5 1      12 0     12631     12631
yellow agg_analysis       5 1       5 0      5804      5804
yellow v2                 5 1       2 0      5410      5410
yellow v1                 5 1       2 0      5367      5367
yellow bank               1 1      16 0      4303      4303
yellow v                  5 1       1 0      2954      2954
yellow p                  5 1       2 0      2939      2939
yellow b0001_072320141238 5 1       1 0      2923      2923
yellow ipaddr             5 1       1 0      2917      2917
yellow v2a                5 1       1 0      2895      2895
yellow movies             5 1       1 0      2738      2738
yellow cars               5 1       0 0      1249      1249
yellow wavelet2           5 1       0 0       615       615

Voila! After piping through grep (with -v to invert the matches), we get a sorted list of indices without Marvel cluttering it up.

This is just a simple example of the flexibility of cat at the command line. Once you get used to using cat, you’ll see it like any other *nix tool and start going crazy with piping, sorting, and grepping. If you are a system admin and spend any time SSH’d into boxes, definitely spend some time getting familiar with the cat API.