jq

Filter on Object key/value

Select the array in .items, loop through and select the items which have a .name equal to "thisname", then return the .name and .status

cat file.json | jq '.items | .[] | select(.name == "thisname") | {name: .name, status: .status}'

Sort and diff JSON filed

Use the --sort-keys flag to sort the files and then compare using diff

diff <(cat /tmp/A.json | jq --sort-keys) <(cat /tmp/B.json | jq --sort-keys)

Walk an object tree looking for a specific key

Walk through an entire object and return all instances of a key (in the below example .target)

echo '{"target": ["a","b","c"], "nested": {"nothingHere": {"somethingElse": ["z", "y", "x"]}, "checkHere": {"target": ["d","e","f"], "anotherLevel": {"not-target": ["w", "v", "u"], "target": ["g", "h", "i"]}}}}' | jq '..|.target?'

To select an object based on whether the key exists:

cat /tmp/file.json | jq -r '.items | .[] | select( (.some.object | ..|.target? != null) )'

Subtitute Text

echo '{"Hello": ",[World,]}"}' | jq '.Hello | sub(",\\[";"") | sub(",]}";"")'

Filter on length of array

echo '{"items": [{"name": "one", "arr": [1,2,3]}, {"name": "two", "arr": []}, {"name": "three", "arr": [1]}, {"name": "four", "arr": [1,2]}]}' | jq -r '.items | .[] | select((.arr|length) > 2)'

Parse string as JSON

$ echo '{"a": 1, "b": 2, "c": "{\"id\":\"9ee ...\",\"parent\":\"abc...\"}"}' | jq -r '.c | fromjson'
{
  "id": "9ee ...",
  "parent": "abc..."
}

Filter on Object key/value (contains)

Loop through an array and select the items which have a .name which contains "someText", then return the .name and .status

cat file.json | jq '.[] | select( .name | contains("someText") ) | {name: .name, status: .status}'

Filter on Object key existing

Select the array in .items, loop through and select the items which have the specified key, then return the .name

cat file.json | jq -r '.items | .[] | select(.a.path.to.an.array[]?.key != null) | .name'

Filter with multiple conditions

Use or in a select statement, to filter on multiple conditions

cat file.json | jq -r '.items | .[] | select((.status == "Running") or (.status == "Scheduled") or .status == "ConfirmationNeeded") | .name'

Filter on Object key/value (not contains)

Loop through an array and select the items which don't have a .name which contains "some-text", then return the .name.

cat file.json | jq -r '.items | .[] | select(.name | contains("some-text") | not) | .name'

Filter Key with startswith

echo '{"items": [{"hostname": "aThing"}, {"hostname": "aThing2"}, {"hostname": "bThing"}]}' | jq -r '.items | .[] | select(.hostname | startswith("aT"))'

Get the length of an array, sort by the highest count, and then output with the corresponding name

cat << EOF > /tmp/data.json
{
  "items": [
    {
      "name": "name-one",
      "non-needed-object": {
        "foo": "bar",
        "hello": "world",
        "bar": "baz"
      },
      "apps": ["abc", "def", "123"]
    },
    {
      "name": "name-two",
      "non-needed-object": {
        "foo": "bar"
      },
      "apps": ["abc"]
    },
    {
      "name": "name-three",
      "non-needed-object": {
        "foo": "bar"
      }
    },
    {
      "name": "name-four",
      "non-needed-object": {
        "foo": "bar"
      },
      "apps": ["abc", "def", "123", "456", "xyz"]
    },
    {
      "name": "name-five",
      "non-needed-object": {
        "foo": "bar"
      },
      "apps": ["abc", "123"]
    }
  ]
}
EOF

jq -c '.items | .[] |= . + {length: .apps|length} | sort_by(.length) | reverse | .[] | {name: .name, app_count: .length}' /tmp/data.json
{"name":"name-four","app_count":5}
{"name":"name-one","app_count":3}
{"name":"name-five","app_count":2}
{"name":"name-two","app_count":1}
{"name":"name-three","app_count":0}

.[] |= . + {length: .apps|length} - adds the length key onto each object
sort_by(.length) | reverse - sort by highest count -> lowest

Use if statements

cat << EOF > data.json
{
  "items": [
    {
      "name": "nameone",
      "nonimportantfields": {
        "foo": "bar",
        "hello": "world"
      },
      "importantfields": {
        "superimportant": "foo"
      }
    },
    {
      "name": "nametwo",
      "nonimportantfields": {
        "foo": "bar",
        "hello": "world"
      },
      "importantfields": {
        "superimportant": "bar"
      }
    },
    {
      "name": "namethree",
      "superimportant": true,
      "nonimportantfields": {
        "foo": "bar",
        "hello": "world"
      }
    }
  ]
}
EOF

cat data.json | jq '.items | .[] | if .importantfields and .nonimportantfields.foo == "bar" then .name else empty end'
 "nameone"
 "nametwo"

Filter Key with endswith

echo '{"items": [{"hostname": "aThing1"}, {"hostname": "aThing2"}, {"hostname": "bThing1"}]}' | jq -r '.items | .[] | select(.hostname | endswith("ing1"))'

Get length of an array

cat file.json | jq -r '.items | length'

Check if a key or value exists

cat << EOF > data.json
{
  "name": "name-field",
  "skills": {
    "cloud": "aws",
    "devops": "git"
  }
}
EOF

## Check if a key exists
cat data.json | jq 'has("notexists")'                 ## returns false
cat data.json | jq 'has("skills")'                    ## returns true
cat data.json | jq '.skills | has("devops")'          ## returns true

## Check if a value exists
cat data.json | jq '.name | contains("notexists")'    ## returns false
cat data.json | jq '.name | contains("debjeet")'      ## returns true
cat data.json | jq '.skills.cloud | contains("aws")'  ## returns true

Combine has with select to extract the objects which are true

cat << EOF > data.json
{
  "items": [
    {
      "name": "name-one",
      "non-important-fields": {
        "foo": "bar",
        "hello": "world"
      },
      "important-fields": {
        "super-important": "foo"
      }
    },
    {
      "name": "name-two",
      "non-important-fields": {
        "foo": "bar",
        "hello": "world"
      },
      "important-fields": {
        "super-important": "bar"
      }
    },
    {
      "name": "name-three",
      "super-important": true,
      "non-important-fields": {
        "foo": "bar",
        "hello": "world"
      }
    }
  ]
}
EOF

cat data.json | jq -r '.items | .[] | {name: .name, exists: .["important-fields"]? | has("super-important")}' | jq -r 'select(.exists == true)'
{
  "name": "name-one",
  "exists": true
}
{
  "name": "name-two",
  "exists": true
}

Group by and count

echo '{"items": [ {"Priority": "High"},{"Priority": "High"},{"Priority": "High"},{"Priority": "High"},{"Priority": "Low"},{"Priority": "Low"},{"Priority": "Low"} ]}' | jq -r '.items | group_by(.Priority)[] | {Priority: .[0].Priority, Count: length}'
{
  "Priority": "High",
  "Count": 4
}
{
  "Priority": "Low",
  "Count": 3
}

Select, Group by and count

echo '{"items": [ {"Priority": "High"},{"Priority": "High"},{"Priority": "High"},{"Priority": "High"},{"Priority": "Medium"},{"Priority": "Medium"},{"Priority": "Low"},{"Priority": "Low"},{"Priority": "Low"} ]}' | jq -r '.items | .[] | select((.Priority == "High") or (.Priority == "Medium"))' | jq -s 'group_by(.Priority)[] | {Priority: .[0].Priority, Count: length}'
{
  "Priority": "High",
  "Count": 4
}
{
  "Priority": "Medium",
  "Count": 2
}

Select fields starting with

Select all fields that start with prefixA, from within the anotherField object

echo '{"aField": {}, "anotherField": { "prefixAOne": "foo", "prefixATwo": "foo", "prefixAThree": "foo", "prefixBOne": "bar", "prefixBTwo": "bar" }, "lastField": true}' | jq -r '.anotherField | to_entries[] | select(.key|startswith("prefixA"))'
{
  "key": "prefixAOne",
  "value": "foo"
}
{
  "key": "prefixATwo",
  "value": "foo"
}
{
  "key": "prefixAThree",
  "value": "foo"
}

Add a field to an object

Assign bar to argument foo and then add the "foo": "bar" key/value to the object

echo '{"hello": "world"}' | jq --arg foo bar '. + {foo: $foo}'
{
  "hello": "world",
  "foo": "bar"
}

Preserve nested objects with *

echo '{"hello": {"value": "world"}}' | jq '. * {"hello": {foo: ("bar")}}'
{
  "hello": {
    "value": "world",
    "foo": "bar"
  }
}

Using + will not preserve nested objects

echo '{"hello": {"value": "world"}}' | jq '. + {"hello": {foo: ("bar")}}'
{
  "hello": {
    "foo": "bar"
  }
}

Concat strings

echo '{"hello": {"value": "world"}}' | jq --arg foo bar '. * {"hello": {foo: ("not" + $foo)}}'
{
  "hello": {
    "value": "world",
    "foo": "notbar"
  }
}